2024-09-10T20:18:19.709 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T20:18:19.725 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T20:18:19.804 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/skanta-2024-09-10_15:30:27-rados-wip-bharath7-testing-2024-09-10-1409-distro-default-smithi/7899571 branch: wip-bharath7-testing-2024-09-10-1409 description: rados/singleton/{all/lost-unfound-delete mon_election/connectivity msgr-failures/few msgr/async objectstore/{bluestore-options/write$/{write_v2} bluestore/bluestore-comp-zstd} rados supported-random-distro$/{centos_latest}} email: skanta@redhat.com first_in_suite: false job_id: '7899571' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: skanta-2024-09-10_15:30:27-rados-wip-bharath7-testing-2024-09-10-1409-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-bharath7-testing-2024-09-10-1409 ceph: conf: global: mon client directed command retry: 5 mon election default strategy: 3 ms inject socket failures: 5000 ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon scrub interval: 300 osd: bluestore block size: 96636764160 bluestore compression algorithm: zstd bluestore compression mode: aggressive bluestore fsck on mount: true bluestore write v2: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - \(MON_DOWN\) sha1: bad1f558c327094897e4a32e99dd5cb4ae6ccd46 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: bad1f558c327094897e4a32e99dd5cb4ae6ccd46 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-bharath7-testing-2024-09-10-1409 sha1: bad1f558c327094897e4a32e99dd5cb4ae6ccd46 owner: scheduled_skanta@teuthology priority: 99 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.b - mon.c - mgr.x - osd.0 - osd.1 - osd.2 seed: 5245 sha1: bad1f558c327094897e4a32e99dd5cb4ae6ccd46 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-bharath7-testing-2024-09-10-1409 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_bad1f558c327094897e4a32e99dd5cb4ae6ccd46/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: bad1f558c327094897e4a32e99dd5cb4ae6ccd46 targets: smithi131.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBASFrE86gn+M380wKFO1yYVeF0gCfQGbWHo109TMLSHZjuVGd0n1B9nlXOYvY70Pvxc1v6O6rq2RM5Dp2E+x+Iw= tasks: - install: null - ceph: log-ignorelist: - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(OBJECT_ - \(SLOW_OPS\) - slow request - \(POOL_APP_NOT_ENABLED\) pre-mgr-commands: - sudo ceph config set mgr mgr_pool false --force - rep_lost_unfound_delete: null teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_15:30:27 tube: smithi user: skanta verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T20:18:19.805 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_bad1f558c327094897e4a32e99dd5cb4ae6ccd46/qa; will attempt to use it 2024-09-10T20:18:19.805 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_bad1f558c327094897e4a32e99dd5cb4ae6ccd46/qa/tasks 2024-09-10T20:18:19.805 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T20:18:19.807 INFO:teuthology.task.internal:Checking packages... 2024-09-10T20:18:19.827 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'bad1f558c327094897e4a32e99dd5cb4ae6ccd46' 2024-09-10T20:18:19.827 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T20:18:19.827 INFO:teuthology.packaging:ref: None 2024-09-10T20:18:19.827 INFO:teuthology.packaging:tag: None 2024-09-10T20:18:19.827 INFO:teuthology.packaging:branch: wip-bharath7-testing-2024-09-10-1409 2024-09-10T20:18:19.828 INFO:teuthology.packaging:sha1: bad1f558c327094897e4a32e99dd5cb4ae6ccd46 2024-09-10T20:18:19.828 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-bharath7-testing-2024-09-10-1409 2024-09-10T20:18:19.973 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4858.gbad1f558 2024-09-10T20:18:19.975 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T20:18:19.982 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T20:18:19.982 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T20:18:20.003 INFO:teuthology.task.internal:Saving configuration 2024-09-10T20:18:20.014 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T20:18:20.021 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T20:18:20.046 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi131.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/skanta-2024-09-10_15:30:27-rados-wip-bharath7-testing-2024-09-10-1409-distro-default-smithi/7899571', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 20:13:29.724254', 'locked_by': 'scheduled_skanta@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBASFrE86gn+M380wKFO1yYVeF0gCfQGbWHo109TMLSHZjuVGd0n1B9nlXOYvY70Pvxc1v6O6rq2RM5Dp2E+x+Iw='} 2024-09-10T20:18:20.046 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T20:18:20.055 INFO:teuthology.task.internal:roles: ubuntu@smithi131.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'osd.0', 'osd.1', 'osd.2'] 2024-09-10T20:18:20.056 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T20:18:20.094 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2c92d9fbe0>, signals=[15]) 2024-09-10T20:18:20.094 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T20:18:20.101 INFO:teuthology.task.internal:Opening connections... 2024-09-10T20:18:20.101 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi131.front.sepia.ceph.com 2024-09-10T20:18:20.103 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T20:18:20.184 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T20:18:20.190 DEBUG:teuthology.orchestra.run.smithi131:> uname -m 2024-09-10T20:18:20.274 INFO:teuthology.orchestra.run.smithi131.stdout:x86_64 2024-09-10T20:18:20.275 DEBUG:teuthology.orchestra.run.smithi131:> cat /etc/os-release 2024-09-10T20:18:20.333 INFO:teuthology.orchestra.run.smithi131.stdout:NAME="CentOS Stream" 2024-09-10T20:18:20.333 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION="9" 2024-09-10T20:18:20.333 INFO:teuthology.orchestra.run.smithi131.stdout:ID="centos" 2024-09-10T20:18:20.333 INFO:teuthology.orchestra.run.smithi131.stdout:ID_LIKE="rhel fedora" 2024-09-10T20:18:20.334 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION_ID="9" 2024-09-10T20:18:20.334 INFO:teuthology.orchestra.run.smithi131.stdout:PLATFORM_ID="platform:el9" 2024-09-10T20:18:20.334 INFO:teuthology.orchestra.run.smithi131.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T20:18:20.334 INFO:teuthology.orchestra.run.smithi131.stdout:ANSI_COLOR="0;31" 2024-09-10T20:18:20.334 INFO:teuthology.orchestra.run.smithi131.stdout:LOGO="fedora-logo-icon" 2024-09-10T20:18:20.334 INFO:teuthology.orchestra.run.smithi131.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T20:18:20.334 INFO:teuthology.orchestra.run.smithi131.stdout:HOME_URL="https://centos.org/" 2024-09-10T20:18:20.334 INFO:teuthology.orchestra.run.smithi131.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T20:18:20.334 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T20:18:20.334 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T20:18:20.335 INFO:teuthology.lock.ops:Updating smithi131.front.sepia.ceph.com on lock server 2024-09-10T20:18:20.349 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T20:18:20.358 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T20:18:20.363 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T20:18:20.363 DEBUG:teuthology.orchestra.run.smithi131:> test '!' -e /home/ubuntu/cephtest 2024-09-10T20:18:20.390 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T20:18:20.396 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T20:18:20.396 DEBUG:teuthology.orchestra.run.smithi131:> test -z $(ls -A /var/lib/ceph) 2024-09-10T20:18:20.527 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T20:18:20.557 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T20:18:20.573 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-10T20:18:20.573 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-10T20:18:20.574 DEBUG:teuthology.orchestra.run.smithi131:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T20:18:20.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:18:20.591 DEBUG:teuthology.orchestra.run.smithi131:> uname -r 2024-09-10T20:18:20.649 INFO:teuthology.orchestra.run.smithi131.stdout:5.14.0-503.el9.x86_64 2024-09-10T20:18:20.649 INFO:teuthology.task.kernel:Running kernel on smithi131: 5.14.0-503.el9.x86_64 2024-09-10T20:18:20.649 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum install -y kernel 2024-09-10T20:18:23.867 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - BaseOS 5.7 MB/s | 8.2 MB 00:01 2024-09-10T20:18:25.743 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - AppStream 27 MB/s | 20 MB 00:00 2024-09-10T20:18:30.294 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - CRB 9.4 MB/s | 6.5 MB 00:00 2024-09-10T20:18:32.469 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - Extras packages 19 kB/s | 19 kB 00:00 2024-09-10T20:18:33.906 INFO:teuthology.orchestra.run.smithi131.stdout:Extra Packages for Enterprise Linux 17 MB/s | 23 MB 00:01 2024-09-10T20:18:39.421 INFO:teuthology.orchestra.run.smithi131.stdout:lab-extras 22 kB/s | 1.7 kB 00:00 2024-09-10T20:18:41.477 INFO:teuthology.orchestra.run.smithi131.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T20:18:41.478 INFO:teuthology.orchestra.run.smithi131.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T20:18:41.569 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T20:18:41.586 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:18:41.586 INFO:teuthology.orchestra.run.smithi131.stdout: Package Architecture Version Repository Size 2024-09-10T20:18:41.586 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:18:41.586 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-10T20:18:41.586 INFO:teuthology.orchestra.run.smithi131.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T20:18:41.586 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-09-10T20:18:41.586 INFO:teuthology.orchestra.run.smithi131.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T20:18:41.586 INFO:teuthology.orchestra.run.smithi131.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T20:18:41.586 INFO:teuthology.orchestra.run.smithi131.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T20:18:41.586 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:18:41.586 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-10T20:18:41.587 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:18:41.587 INFO:teuthology.orchestra.run.smithi131.stdout:Install 4 Packages 2024-09-10T20:18:41.587 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:18:41.587 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 79 M 2024-09-10T20:18:41.587 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 126 M 2024-09-10T20:18:41.587 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-10T20:18:41.822 INFO:teuthology.orchestra.run.smithi131.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 203 kB/s | 23 kB 00:00 2024-09-10T20:18:42.297 INFO:teuthology.orchestra.run.smithi131.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-09-10T20:18:42.623 INFO:teuthology.orchestra.run.smithi131.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 38 MB/s | 35 MB 00:00 2024-09-10T20:18:42.898 INFO:teuthology.orchestra.run.smithi131.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 27 MB/s | 29 MB 00:01 2024-09-10T20:18:42.898 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-10T20:18:42.898 INFO:teuthology.orchestra.run.smithi131.stdout:Total 60 MB/s | 79 MB 00:01 2024-09-10T20:18:43.575 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-10T20:18:43.948 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-10T20:18:43.949 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-10T20:18:44.890 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-10T20:18:44.890 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-10T20:18:47.343 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-10T20:18:48.025 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T20:18:48.115 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T20:18:50.906 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T20:18:51.041 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T20:18:55.843 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T20:18:55.966 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T20:19:00.799 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T20:19:38.771 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T20:19:38.811 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T20:19:39.086 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T20:19:39.086 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T20:19:39.086 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T20:19:39.086 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T20:19:39.944 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T20:19:39.944 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:19:39.945 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-10T20:19:39.945 INFO:teuthology.orchestra.run.smithi131.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T20:19:39.945 INFO:teuthology.orchestra.run.smithi131.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T20:19:39.945 INFO:teuthology.orchestra.run.smithi131.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T20:19:39.945 INFO:teuthology.orchestra.run.smithi131.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T20:19:39.945 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:19:39.945 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T20:19:40.329 DEBUG:teuthology.orchestra.run.smithi131:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T20:19:40.434 INFO:teuthology.orchestra.run.smithi131.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T20:19:40.435 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T20:19:40.435 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T20:19:40.436 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T20:19:40.436 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T20:19:40.436 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-10T20:19:40.436 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi131.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T20:19:40.436 DEBUG:teuthology.orchestra.run.smithi131:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T20:19:40.495 INFO:teuthology.orchestra.run.smithi131.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T20:19:40.495 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T20:19:40.495 DEBUG:teuthology.orchestra.run.smithi131:> sudo rpm -qi grub2-tools 2024-09-10T20:19:40.555 INFO:teuthology.orchestra.run.smithi131.stdout:Name : grub2-tools 2024-09-10T20:19:40.555 INFO:teuthology.orchestra.run.smithi131.stdout:Epoch : 1 2024-09-10T20:19:40.555 INFO:teuthology.orchestra.run.smithi131.stdout:Version : 2.06 2024-09-10T20:19:40.555 INFO:teuthology.orchestra.run.smithi131.stdout:Release : 82.el9 2024-09-10T20:19:40.555 INFO:teuthology.orchestra.run.smithi131.stdout:Architecture: x86_64 2024-09-10T20:19:40.555 INFO:teuthology.orchestra.run.smithi131.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T20:19:40.555 INFO:teuthology.orchestra.run.smithi131.stdout:Group : Unspecified 2024-09-10T20:19:40.555 INFO:teuthology.orchestra.run.smithi131.stdout:Size : 8274670 2024-09-10T20:19:40.555 INFO:teuthology.orchestra.run.smithi131.stdout:License : GPLv3+ 2024-09-10T20:19:40.555 INFO:teuthology.orchestra.run.smithi131.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T20:19:40.556 INFO:teuthology.orchestra.run.smithi131.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T20:19:40.556 INFO:teuthology.orchestra.run.smithi131.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T20:19:40.556 INFO:teuthology.orchestra.run.smithi131.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T20:19:40.556 INFO:teuthology.orchestra.run.smithi131.stdout:Packager : builder@centos.org 2024-09-10T20:19:40.556 INFO:teuthology.orchestra.run.smithi131.stdout:Vendor : CentOS 2024-09-10T20:19:40.556 INFO:teuthology.orchestra.run.smithi131.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T20:19:40.556 INFO:teuthology.orchestra.run.smithi131.stdout:Summary : Support tools for GRUB. 2024-09-10T20:19:40.556 INFO:teuthology.orchestra.run.smithi131.stdout:Description : 2024-09-10T20:19:40.556 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:19:40.556 INFO:teuthology.orchestra.run.smithi131.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T20:19:40.556 INFO:teuthology.orchestra.run.smithi131.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T20:19:40.556 INFO:teuthology.orchestra.run.smithi131.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T20:19:40.556 INFO:teuthology.orchestra.run.smithi131.stdout:hardware devices. 2024-09-10T20:19:40.556 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:19:40.557 INFO:teuthology.orchestra.run.smithi131.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T20:19:40.557 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T20:19:40.557 INFO:teuthology.task.kernel:Updating grub on smithi131 to boot 5.14.0-505.el9.x86_64 2024-09-10T20:19:40.558 DEBUG:teuthology.orchestra.run.smithi131:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T20:19:41.194 INFO:teuthology.orchestra.run.smithi131.stderr:Generating grub configuration file ... 2024-09-10T20:19:42.473 INFO:teuthology.orchestra.run.smithi131.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T20:19:42.506 INFO:teuthology.orchestra.run.smithi131.stderr:done 2024-09-10T20:19:42.508 DEBUG:teuthology.orchestra.run.smithi131:> mktemp 2024-09-10T20:19:42.524 INFO:teuthology.orchestra.run.smithi131.stdout:/tmp/tmp.gJytzcnuQu 2024-09-10T20:19:42.525 DEBUG:teuthology.orchestra.run.smithi131:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.gJytzcnuQu 2024-09-10T20:19:42.590 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 0666 /tmp/tmp.gJytzcnuQu 2024-09-10T20:19:42.723 DEBUG:teuthology.orchestra.remote:smithi131:/tmp/tmp.gJytzcnuQu is 6KB 2024-09-10T20:19:42.772 DEBUG:teuthology.orchestra.run.smithi131:> rm -fr /tmp/tmp.gJytzcnuQu 2024-09-10T20:19:42.787 DEBUG:teuthology.orchestra.run.smithi131:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T20:19:42.850 INFO:teuthology.orchestra.run.smithi131.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T20:19:42.850 INFO:teuthology.orchestra.run.smithi131.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T20:19:42.850 INFO:teuthology.orchestra.run.smithi131.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T20:19:42.850 INFO:teuthology.orchestra.run.smithi131.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T20:19:42.852 DEBUG:teuthology.orchestra.run.smithi131:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T20:19:43.016 DEBUG:teuthology.orchestra.run.smithi131:> sudo shutdown -r now 2024-09-10T20:20:13.045 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-10T20:20:13.046 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-09-10T20:20:13.047 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T20:20:31.425 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.131 2024-09-10T20:20:40.431 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-09-10T20:20:40.432 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T20:21:40.495 DEBUG:teuthology.orchestra.remote:timed out 2024-09-10T20:21:52.508 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-09-10T20:21:52.510 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T20:21:52.902 DEBUG:teuthology.orchestra.run.smithi131:> true 2024-09-10T20:21:53.289 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-09-10T20:21:53.289 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-10T20:21:53.290 DEBUG:teuthology.orchestra.run.smithi131:> uname -r 2024-09-10T20:21:53.306 INFO:teuthology.orchestra.run.smithi131.stdout:5.14.0-505.el9.x86_64 2024-09-10T20:21:53.307 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T20:21:53.307 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T20:21:53.307 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T20:21:54.308 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-10T20:21:54.308 DEBUG:teuthology.orchestra.run.smithi131:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T20:21:54.373 INFO:teuthology.orchestra.run.smithi131.stdout:ttyS1 2024-09-10T20:21:54.396 DEBUG:teuthology.parallel:result is None 2024-09-10T20:21:54.396 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T20:21:54.403 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T20:21:54.404 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T20:21:54.422 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T20:21:54.430 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T20:21:54.436 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T20:21:54.436 DEBUG:teuthology.orchestra.run.smithi131:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T20:21:54.487 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T20:21:54.494 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T20:21:54.494 DEBUG:teuthology.orchestra.run.smithi131:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T20:21:54.555 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T20:21:54.567 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T20:21:54.568 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T20:21:54.575 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T20:21:54.575 DEBUG:teuthology.orchestra.run.smithi131:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T20:21:54.636 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T20:21:54.646 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T20:21:54.646 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T20:21:54.693 DEBUG:teuthology.orchestra.run.smithi131:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T20:21:54.779 DEBUG:teuthology.orchestra.run.smithi131:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T20:21:54.823 DEBUG:teuthology.orchestra.run.smithi131:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T20:21:54.906 DEBUG:teuthology.orchestra.run.smithi131:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T20:21:54.947 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-10T20:21:54.947 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T20:21:55.012 DEBUG:teuthology.orchestra.run.smithi131:> sudo service rsyslog restart 2024-09-10T20:21:55.103 INFO:teuthology.orchestra.run.smithi131.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T20:21:55.345 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T20:21:55.354 INFO:teuthology.task.internal:Starting timer... 2024-09-10T20:21:55.354 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T20:21:55.364 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T20:21:55.373 DEBUG:teuthology.orchestra.run.smithi131:> sudo service auditd rotate 2024-09-10T20:21:55.424 INFO:teuthology.orchestra.run.smithi131.stdout:Rotating logs: 2024-09-10T20:21:55.425 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T20:21:55.426 DEBUG:teuthology.orchestra.run.smithi131:> /usr/sbin/getenforce 2024-09-10T20:21:55.463 INFO:teuthology.orchestra.run.smithi131.stdout:Permissive 2024-09-10T20:21:55.463 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi131.front.sepia.ceph.com': 'permissive'} 2024-09-10T20:21:55.463 DEBUG:teuthology.orchestra.run.smithi131:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T20:21:55.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:21:55.528 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T20:21:55.528 DEBUG:teuthology.orchestra.run.smithi131:> sudo /usr/sbin/setenforce permissive 2024-09-10T20:21:55.598 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T20:21:55.611 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-10T20:21:55.614 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-10T20:21:55.727 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T20:21:55.738 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T20:21:55.739 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi131.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T20:25:34.024 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi131.front.sepia.ceph.com')] 2024-09-10T20:25:34.026 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-09-10T20:25:34.027 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T20:25:34.103 DEBUG:teuthology.orchestra.run.smithi131:> true 2024-09-10T20:25:34.180 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-09-10T20:25:34.181 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T20:25:34.194 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T20:25:34.195 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T20:25:34.195 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T20:25:34.250 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T20:25:34.264 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T20:25:34.289 INFO:teuthology.orchestra.run.smithi131.stderr:sudo: ntpd: command not found 2024-09-10T20:25:34.300 INFO:teuthology.orchestra.run.smithi131.stdout:506 Cannot talk to daemon 2024-09-10T20:25:34.314 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T20:25:34.328 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T20:25:34.384 INFO:teuthology.orchestra.run.smithi131.stderr:bash: line 1: ntpq: command not found 2024-09-10T20:25:34.387 INFO:teuthology.orchestra.run.smithi131.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T20:25:34.387 INFO:teuthology.orchestra.run.smithi131.stdout:=============================================================================== 2024-09-10T20:25:34.387 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T20:25:34.387 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T20:25:34.387 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T20:25:34.387 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T20:25:34.388 INFO:teuthology.run_tasks:Running task install... 2024-09-10T20:25:34.401 DEBUG:teuthology.task.install:project ceph 2024-09-10T20:25:34.401 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bad1f558c327094897e4a32e99dd5cb4ae6ccd46'}} 2024-09-10T20:25:34.401 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'bad1f558c327094897e4a32e99dd5cb4ae6ccd46'} 2024-09-10T20:25:34.401 INFO:teuthology.task.install:Using flavor: default 2024-09-10T20:25:34.408 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-10T20:25:34.408 INFO:teuthology.task.install:extra packages: [] 2024-09-10T20:25:34.408 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bad1f558c327094897e4a32e99dd5cb4ae6ccd46', 'tag': None, 'wait_for_package': False} 2024-09-10T20:25:34.409 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bad1f558c327094897e4a32e99dd5cb4ae6ccd46 2024-09-10T20:25:34.554 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-bharath7-testing-2024-09-10-1409/bad1f558c327094897e4a32e99dd5cb4ae6ccd46/centos/9/flavors/default/ 2024-09-10T20:25:34.554 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4858.gbad1f558 2024-09-10T20:25:35.696 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-bharath7-testing-2024-09-10-1409/bad1f558c327094897e4a32e99dd5cb4ae6ccd46/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-bharath7-testing-2024-09-10-1409/bad1f558c327094897e4a32e99dd5cb4ae6ccd46/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-bharath7-testing-2024-09-10-1409/bad1f558c327094897e4a32e99dd5cb4ae6ccd46/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T20:25:35.697 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-10T20:25:35.697 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T20:25:35.728 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-10T20:25:35.728 DEBUG:teuthology.orchestra.run.smithi131:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bad1f558c327094897e4a32e99dd5cb4ae6ccd46/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T20:25:35.797 DEBUG:teuthology.orchestra.run.smithi131:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T20:25:35.876 DEBUG:teuthology.orchestra.run.smithi131:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T20:25:35.942 INFO:teuthology.orchestra.run.smithi131.stdout:check_obsoletes = 1 2024-09-10T20:25:35.944 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum clean all 2024-09-10T20:25:36.172 INFO:teuthology.orchestra.run.smithi131.stdout:45 files removed 2024-09-10T20:25:36.199 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-radosgw 2024-09-10T20:25:36.765 INFO:teuthology.orchestra.run.smithi131.stdout:ceph packages for x86_64 300 kB/s | 84 kB 00:00 2024-09-10T20:25:37.065 INFO:teuthology.orchestra.run.smithi131.stdout:ceph noarch packages 94 kB/s | 24 kB 00:00 2024-09-10T20:25:37.349 INFO:teuthology.orchestra.run.smithi131.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-10T20:25:37.825 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - BaseOS 19 MB/s | 8.2 MB 00:00 2024-09-10T20:25:43.326 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - AppStream 4.6 MB/s | 20 MB 00:04 2024-09-10T20:25:47.843 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - CRB 9.3 MB/s | 6.5 MB 00:00 2024-09-10T20:25:49.319 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - Extras packages 77 kB/s | 19 kB 00:00 2024-09-10T20:25:51.073 INFO:teuthology.orchestra.run.smithi131.stdout:Extra Packages for Enterprise Linux 14 MB/s | 23 MB 00:01 2024-09-10T20:25:56.546 INFO:teuthology.orchestra.run.smithi131.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-10T20:25:58.939 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T20:25:58.950 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:25:58.950 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-09-10T20:25:58.950 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:25:58.950 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-10T20:25:58.951 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-radosgw x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 11 M 2024-09-10T20:25:58.951 INFO:teuthology.orchestra.run.smithi131.stdout:Upgrading: 2024-09-10T20:25:58.951 INFO:teuthology.orchestra.run.smithi131.stdout: librados2 x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 3.5 M 2024-09-10T20:25:58.951 INFO:teuthology.orchestra.run.smithi131.stdout: librbd1 x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 3.2 M 2024-09-10T20:25:58.951 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-09-10T20:25:58.951 INFO:teuthology.orchestra.run.smithi131.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T20:25:58.951 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-base x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 5.5 M 2024-09-10T20:25:58.951 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-common x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 22 M 2024-09-10T20:25:58.951 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-selinux x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 25 k 2024-09-10T20:25:58.951 INFO:teuthology.orchestra.run.smithi131.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T20:25:58.951 INFO:teuthology.orchestra.run.smithi131.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T20:25:58.951 INFO:teuthology.orchestra.run.smithi131.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T20:25:58.952 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs2 x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 755 k 2024-09-10T20:25:58.952 INFO:teuthology.orchestra.run.smithi131.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-10T20:25:58.952 INFO:teuthology.orchestra.run.smithi131.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T20:25:58.952 INFO:teuthology.orchestra.run.smithi131.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T20:25:58.952 INFO:teuthology.orchestra.run.smithi131.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T20:25:58.952 INFO:teuthology.orchestra.run.smithi131.stdout: libradosstriper1 x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 509 k 2024-09-10T20:25:58.952 INFO:teuthology.orchestra.run.smithi131.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T20:25:58.952 INFO:teuthology.orchestra.run.smithi131.stdout: librgw2 x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 5.5 M 2024-09-10T20:25:58.952 INFO:teuthology.orchestra.run.smithi131.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T20:25:58.952 INFO:teuthology.orchestra.run.smithi131.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T20:25:58.952 INFO:teuthology.orchestra.run.smithi131.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T20:25:58.953 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-argparse x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 45 k 2024-09-10T20:25:58.953 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-common x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 143 k 2024-09-10T20:25:58.953 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cephfs x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 161 k 2024-09-10T20:25:58.953 INFO:teuthology.orchestra.run.smithi131.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T20:25:58.953 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rados x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 321 k 2024-09-10T20:25:58.953 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rbd x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 306 k 2024-09-10T20:25:58.953 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rgw x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 99 k 2024-09-10T20:25:58.953 INFO:teuthology.orchestra.run.smithi131.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T20:25:58.953 INFO:teuthology.orchestra.run.smithi131.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T20:25:58.953 INFO:teuthology.orchestra.run.smithi131.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T20:25:58.953 INFO:teuthology.orchestra.run.smithi131.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T20:25:58.953 INFO:teuthology.orchestra.run.smithi131.stdout:Installing weak dependencies: 2024-09-10T20:25:58.953 INFO:teuthology.orchestra.run.smithi131.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T20:25:58.953 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:25:58.953 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-10T20:25:58.954 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:25:58.954 INFO:teuthology.orchestra.run.smithi131.stdout:Install 31 Packages 2024-09-10T20:25:58.954 INFO:teuthology.orchestra.run.smithi131.stdout:Upgrade 2 Packages 2024-09-10T20:25:58.954 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:25:58.954 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 62 M 2024-09-10T20:25:58.954 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-10T20:25:59.656 INFO:teuthology.orchestra.run.smithi131.stdout:(1/33): ceph-base-19.3.0-4858.gbad1f558.el9.x86 11 MB/s | 5.5 MB 00:00 2024-09-10T20:25:59.697 INFO:teuthology.orchestra.run.smithi131.stdout:(2/33): ceph-selinux-19.3.0-4858.gbad1f558.el9. 608 kB/s | 25 kB 00:00 2024-09-10T20:25:59.798 INFO:teuthology.orchestra.run.smithi131.stdout:(3/33): libcephfs2-19.3.0-4858.gbad1f558.el9.x8 7.4 MB/s | 755 kB 00:00 2024-09-10T20:25:59.898 INFO:teuthology.orchestra.run.smithi131.stdout:(4/33): libradosstriper1-19.3.0-4858.gbad1f558. 5.0 MB/s | 509 kB 00:00 2024-09-10T20:26:00.032 INFO:teuthology.orchestra.run.smithi131.stdout:(5/33): ceph-radosgw-19.3.0-4858.gbad1f558.el9. 12 MB/s | 11 MB 00:00 2024-09-10T20:26:00.082 INFO:teuthology.orchestra.run.smithi131.stdout:(6/33): python3-ceph-argparse-19.3.0-4858.gbad1 905 kB/s | 45 kB 00:00 2024-09-10T20:26:00.191 INFO:teuthology.orchestra.run.smithi131.stdout:(7/33): librgw2-19.3.0-4858.gbad1f558.el9.x86_6 19 MB/s | 5.5 MB 00:00 2024-09-10T20:26:00.216 INFO:teuthology.orchestra.run.smithi131.stdout:(8/33): python3-ceph-common-19.3.0-4858.gbad1f5 1.0 MB/s | 143 kB 00:00 2024-09-10T20:26:00.433 INFO:teuthology.orchestra.run.smithi131.stdout:(9/33): ceph-common-19.3.0-4858.gbad1f558.el9.x 17 MB/s | 22 MB 00:01 2024-09-10T20:26:00.458 INFO:teuthology.orchestra.run.smithi131.stdout:(10/33): python3-cephfs-19.3.0-4858.gbad1f558.e 604 kB/s | 161 kB 00:00 2024-09-10T20:26:00.483 INFO:teuthology.orchestra.run.smithi131.stdout:(11/33): python3-rados-19.3.0-4858.gbad1f558.el 1.2 MB/s | 321 kB 00:00 2024-09-10T20:26:00.509 INFO:teuthology.orchestra.run.smithi131.stdout:(12/33): python3-rbd-19.3.0-4858.gbad1f558.el9. 4.0 MB/s | 306 kB 00:00 2024-09-10T20:26:00.534 INFO:teuthology.orchestra.run.smithi131.stdout:(13/33): python3-rgw-19.3.0-4858.gbad1f558.el9. 1.3 MB/s | 99 kB 00:00 2024-09-10T20:26:00.726 INFO:teuthology.orchestra.run.smithi131.stdout:(14/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 836 kB/s | 160 kB 00:00 2024-09-10T20:26:00.760 INFO:teuthology.orchestra.run.smithi131.stdout:(15/33): libnbd-1.20.2-2.el9.x86_64.rpm 680 kB/s | 171 kB 00:00 2024-09-10T20:26:00.785 INFO:teuthology.orchestra.run.smithi131.stdout:(16/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 771 kB/s | 45 kB 00:00 2024-09-10T20:26:00.810 INFO:teuthology.orchestra.run.smithi131.stdout:(17/33): boost-program-options-1.75.0-8.el9.x86 327 kB/s | 107 kB 00:00 2024-09-10T20:26:00.861 INFO:teuthology.orchestra.run.smithi131.stdout:(18/33): python3-prettytable-0.7.2-27.el9.noarc 843 kB/s | 42 kB 00:00 2024-09-10T20:26:00.944 INFO:teuthology.orchestra.run.smithi131.stdout:(19/33): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-09-10T20:26:00.978 INFO:teuthology.orchestra.run.smithi131.stdout:(20/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.5 MB/s | 292 kB 00:00 2024-09-10T20:26:01.003 INFO:teuthology.orchestra.run.smithi131.stdout:(21/33): qatlib-service-24.02.0-1.el9.x86_64.rp 613 kB/s | 36 kB 00:00 2024-09-10T20:26:01.037 INFO:teuthology.orchestra.run.smithi131.stdout:(22/33): librdkafka-1.6.1-102.el9.x86_64.rpm 2.3 MB/s | 662 kB 00:00 2024-09-10T20:26:01.054 INFO:teuthology.orchestra.run.smithi131.stdout:(23/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 628 kB/s | 47 kB 00:00 2024-09-10T20:26:01.112 INFO:teuthology.orchestra.run.smithi131.stdout:(24/33): gperftools-libs-2.9.1-2.el9.x86_64.rpm 2.8 MB/s | 309 kB 00:00 2024-09-10T20:26:01.129 INFO:teuthology.orchestra.run.smithi131.stdout:(25/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 433 kB/s | 26 kB 00:00 2024-09-10T20:26:01.146 INFO:teuthology.orchestra.run.smithi131.stdout:(26/33): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-09-10T20:26:01.171 INFO:teuthology.orchestra.run.smithi131.stdout:(27/33): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-09-10T20:26:01.205 INFO:teuthology.orchestra.run.smithi131.stdout:(28/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 14 MB/s | 839 kB 00:00 2024-09-10T20:26:01.230 INFO:teuthology.orchestra.run.smithi131.stdout:(29/33): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-10T20:26:01.298 INFO:teuthology.orchestra.run.smithi131.stdout:(30/33): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-10T20:26:01.423 INFO:teuthology.orchestra.run.smithi131.stdout:(31/33): librados2-19.3.0-4858.gbad1f558.el9.x8 18 MB/s | 3.5 MB 00:00 2024-09-10T20:26:01.498 INFO:teuthology.orchestra.run.smithi131.stdout:(32/33): libarrow-9.0.0-11.el9.x86_64.rpm 9.6 MB/s | 4.4 MB 00:00 2024-09-10T20:26:01.582 INFO:teuthology.orchestra.run.smithi131.stdout:(33/33): librbd1-19.3.0-4858.gbad1f558.el9.x86_ 11 MB/s | 3.2 MB 00:00 2024-09-10T20:26:01.586 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-10T20:26:01.586 INFO:teuthology.orchestra.run.smithi131.stdout:Total 23 MB/s | 62 MB 00:02 2024-09-10T20:26:01.698 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-10T20:26:01.734 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-10T20:26:01.734 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-10T20:26:02.423 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-10T20:26:02.423 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-10T20:26:03.960 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-10T20:26:04.054 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-10T20:26:04.086 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-10T20:26:04.112 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-10T20:26:04.135 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-10T20:26:04.160 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-10T20:26:04.345 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-10T20:26:04.398 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-10T20:26:04.732 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-10T20:26:04.856 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : librados2-2:19.3.0-4858.gbad1f558.el9.x86_64 7/35 2024-09-10T20:26:04.931 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librados2-2:19.3.0-4858.gbad1f558.el9.x86_64 7/35 2024-09-10T20:26:05.022 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rados-2:19.3.0-4858.gbad1f558.el9.x86_64 8/35 2024-09-10T20:26:05.049 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephfs2-2:19.3.0-4858.gbad1f558.el9.x86_64 9/35 2024-09-10T20:26:05.112 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libcephfs2-2:19.3.0-4858.gbad1f558.el9.x86_64 9/35 2024-09-10T20:26:05.149 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-10T20:26:05.193 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-ceph-argparse-2:19.3.0-4858.gbad1f558.el9. 11/35 2024-09-10T20:26:05.272 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cephfs-2:19.3.0-4858.gbad1f558.el9.x86_64 12/35 2024-09-10T20:26:05.309 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libradosstriper1-2:19.3.0-4858.gbad1f558.el9.x86_6 13/35 2024-09-10T20:26:05.374 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4858.gbad1f558.el9.x86_6 13/35 2024-09-10T20:26:05.408 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-10T20:26:05.519 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-10T20:26:05.559 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-10T20:26:05.600 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-10T20:26:05.633 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/35 2024-09-10T20:26:05.662 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-10T20:26:05.694 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-10T20:26:05.732 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-10T20:26:06.057 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-10T20:26:06.125 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : librbd1-2:19.3.0-4858.gbad1f558.el9.x86_64 23/35 2024-09-10T20:26:06.217 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librbd1-2:19.3.0-4858.gbad1f558.el9.x86_64 23/35 2024-09-10T20:26:06.246 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rbd-2:19.3.0-4858.gbad1f558.el9.x86_64 24/35 2024-09-10T20:26:06.282 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-10T20:26:06.772 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-10T20:26:06.803 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librgw2-2:19.3.0-4858.gbad1f558.el9.x86_64 27/35 2024-09-10T20:26:06.869 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librgw2-2:19.3.0-4858.gbad1f558.el9.x86_64 27/35 2024-09-10T20:26:06.944 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rgw-2:19.3.0-4858.gbad1f558.el9.x86_64 28/35 2024-09-10T20:26:06.980 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-ceph-common-2:19.3.0-4858.gbad1f558.el9.x8 29/35 2024-09-10T20:26:09.332 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-common-2:19.3.0-4858.gbad1f558.el9.x86_64 30/35 2024-09-10T20:26:09.368 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-common-2:19.3.0-4858.gbad1f558.el9.x86_64 30/35 2024-09-10T20:26:09.942 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-common-2:19.3.0-4858.gbad1f558.el9.x86_64 30/35 2024-09-10T20:26:09.980 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-base-2:19.3.0-4858.gbad1f558.el9.x86_64 31/35 2024-09-10T20:26:10.063 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-base-2:19.3.0-4858.gbad1f558.el9.x86_64 31/35 2024-09-10T20:26:10.063 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T20:26:10.063 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T20:26:10.063 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:26:10.098 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-selinux-2:19.3.0-4858.gbad1f558.el9.x86_64 32/35 2024-09-10T20:26:20.129 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4858.gbad1f558.el9.x86_64 32/35 2024-09-10T20:26:20.129 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /sys 2024-09-10T20:26:20.130 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /proc 2024-09-10T20:26:20.130 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /mnt 2024-09-10T20:26:20.130 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /var/tmp 2024-09-10T20:26:20.130 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /home 2024-09-10T20:26:20.130 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /root 2024-09-10T20:26:20.130 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /tmp 2024-09-10T20:26:20.130 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:26:21.154 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-radosgw-2:19.3.0-4858.gbad1f558.el9.x86_64 33/35 2024-09-10T20:26:21.179 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4858.gbad1f558.el9.x86_64 33/35 2024-09-10T20:26:21.179 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:26:21.179 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T20:26:21.180 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T20:26:21.180 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T20:26:21.180 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:26:21.181 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-10T20:26:21.264 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-10T20:26:21.264 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-10T20:26:22.463 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-10T20:26:22.463 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-base-2:19.3.0-4858.gbad1f558.el9.x86_64 1/35 2024-09-10T20:26:22.463 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-common-2:19.3.0-4858.gbad1f558.el9.x86_64 2/35 2024-09-10T20:26:22.463 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-radosgw-2:19.3.0-4858.gbad1f558.el9.x86_64 3/35 2024-09-10T20:26:22.463 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-selinux-2:19.3.0-4858.gbad1f558.el9.x86_64 4/35 2024-09-10T20:26:22.463 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephfs2-2:19.3.0-4858.gbad1f558.el9.x86_64 5/35 2024-09-10T20:26:22.463 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libradosstriper1-2:19.3.0-4858.gbad1f558.el9.x86_6 6/35 2024-09-10T20:26:22.463 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librgw2-2:19.3.0-4858.gbad1f558.el9.x86_64 7/35 2024-09-10T20:26:22.464 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4858.gbad1f558.el9. 8/35 2024-09-10T20:26:22.464 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-ceph-common-2:19.3.0-4858.gbad1f558.el9.x8 9/35 2024-09-10T20:26:22.464 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cephfs-2:19.3.0-4858.gbad1f558.el9.x86_64 10/35 2024-09-10T20:26:22.464 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rados-2:19.3.0-4858.gbad1f558.el9.x86_64 11/35 2024-09-10T20:26:22.464 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rbd-2:19.3.0-4858.gbad1f558.el9.x86_64 12/35 2024-09-10T20:26:22.464 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rgw-2:19.3.0-4858.gbad1f558.el9.x86_64 13/35 2024-09-10T20:26:22.464 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-10T20:26:22.464 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-10T20:26:22.464 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-10T20:26:22.464 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-10T20:26:22.464 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-10T20:26:22.464 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-10T20:26:22.464 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-10T20:26:22.465 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-10T20:26:22.465 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-10T20:26:22.465 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-10T20:26:22.465 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 24/35 2024-09-10T20:26:22.465 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-10T20:26:22.465 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-10T20:26:22.465 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-10T20:26:22.465 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-10T20:26:22.465 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-10T20:26:22.465 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-10T20:26:22.466 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-10T20:26:22.466 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados2-2:19.3.0-4858.gbad1f558.el9.x86_64 32/35 2024-09-10T20:26:22.466 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-10T20:26:22.466 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librbd1-2:19.3.0-4858.gbad1f558.el9.x86_64 34/35 2024-09-10T20:26:23.321 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-10T20:26:23.321 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:26:23.321 INFO:teuthology.orchestra.run.smithi131.stdout:Upgraded: 2024-09-10T20:26:23.321 INFO:teuthology.orchestra.run.smithi131.stdout: librados2-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T20:26:23.322 INFO:teuthology.orchestra.run.smithi131.stdout: librbd1-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T20:26:23.322 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-10T20:26:23.322 INFO:teuthology.orchestra.run.smithi131.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T20:26:23.322 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-base-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T20:26:23.322 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-common-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T20:26:23.322 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-radosgw-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T20:26:23.322 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-selinux-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T20:26:23.322 INFO:teuthology.orchestra.run.smithi131.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T20:26:23.322 INFO:teuthology.orchestra.run.smithi131.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T20:26:23.322 INFO:teuthology.orchestra.run.smithi131.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T20:26:23.322 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs2-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T20:26:23.322 INFO:teuthology.orchestra.run.smithi131.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-10T20:26:23.322 INFO:teuthology.orchestra.run.smithi131.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T20:26:23.323 INFO:teuthology.orchestra.run.smithi131.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T20:26:23.323 INFO:teuthology.orchestra.run.smithi131.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T20:26:23.323 INFO:teuthology.orchestra.run.smithi131.stdout: libradosstriper1-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T20:26:23.323 INFO:teuthology.orchestra.run.smithi131.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T20:26:23.323 INFO:teuthology.orchestra.run.smithi131.stdout: librgw2-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T20:26:23.323 INFO:teuthology.orchestra.run.smithi131.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T20:26:23.323 INFO:teuthology.orchestra.run.smithi131.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T20:26:23.323 INFO:teuthology.orchestra.run.smithi131.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T20:26:23.323 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-argparse-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T20:26:23.323 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-common-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T20:26:23.323 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cephfs-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T20:26:23.323 INFO:teuthology.orchestra.run.smithi131.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T20:26:23.323 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rados-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T20:26:23.323 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rbd-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T20:26:23.324 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rgw-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T20:26:23.324 INFO:teuthology.orchestra.run.smithi131.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T20:26:23.324 INFO:teuthology.orchestra.run.smithi131.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T20:26:23.324 INFO:teuthology.orchestra.run.smithi131.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T20:26:23.324 INFO:teuthology.orchestra.run.smithi131.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T20:26:23.324 INFO:teuthology.orchestra.run.smithi131.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T20:26:23.324 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:26:23.324 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T20:26:23.667 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-test 2024-09-10T20:26:24.229 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:00:28 ago on Tue 10 Sep 2024 08:25:56 PM UTC. 2024-09-10T20:26:24.360 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T20:26:24.361 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:26:24.361 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-09-10T20:26:24.361 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:26:24.361 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-10T20:26:24.361 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-test x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 50 M 2024-09-10T20:26:24.361 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-09-10T20:26:24.362 INFO:teuthology.orchestra.run.smithi131.stdout: libcephsqlite x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 161 k 2024-09-10T20:26:24.362 INFO:teuthology.orchestra.run.smithi131.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T20:26:24.362 INFO:teuthology.orchestra.run.smithi131.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T20:26:24.362 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:26:24.362 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-10T20:26:24.362 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:26:24.362 INFO:teuthology.orchestra.run.smithi131.stdout:Install 4 Packages 2024-09-10T20:26:24.362 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:26:24.363 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 50 M 2024-09-10T20:26:24.363 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 215 M 2024-09-10T20:26:24.363 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-10T20:26:24.595 INFO:teuthology.orchestra.run.smithi131.stdout:(1/4): libcephsqlite-19.3.0-4858.gbad1f558.el9. 764 kB/s | 161 kB 00:00 2024-09-10T20:26:24.970 INFO:teuthology.orchestra.run.smithi131.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 170 kB/s | 64 kB 00:00 2024-09-10T20:26:25.012 INFO:teuthology.orchestra.run.smithi131.stdout:(3/4): socat-1.7.4.1-6.el9.x86_64.rpm 484 kB/s | 304 kB 00:00 2024-09-10T20:26:25.904 INFO:teuthology.orchestra.run.smithi131.stdout:(4/4): ceph-test-19.3.0-4858.gbad1f558.el9.x86_ 33 MB/s | 50 MB 00:01 2024-09-10T20:26:25.906 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-10T20:26:25.907 INFO:teuthology.orchestra.run.smithi131.stdout:Total 33 MB/s | 50 MB 00:01 2024-09-10T20:26:25.934 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-10T20:26:25.955 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-10T20:26:25.955 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-10T20:26:26.330 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-10T20:26:26.330 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-10T20:26:26.879 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-10T20:26:26.960 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-10T20:26:27.003 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-10T20:26:27.031 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephsqlite-2:19.3.0-4858.gbad1f558.el9.x86_64 3/4 2024-09-10T20:26:31.726 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4858.gbad1f558.el9.x86_64 3/4 2024-09-10T20:26:31.804 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-test-2:19.3.0-4858.gbad1f558.el9.x86_64 4/4 2024-09-10T20:26:34.298 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-test-2:19.3.0-4858.gbad1f558.el9.x86_64 4/4 2024-09-10T20:26:34.298 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-test-2:19.3.0-4858.gbad1f558.el9.x86_64 1/4 2024-09-10T20:26:34.298 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephsqlite-2:19.3.0-4858.gbad1f558.el9.x86_64 2/4 2024-09-10T20:26:34.298 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-10T20:26:34.642 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-10T20:26:34.643 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:26:34.643 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-10T20:26:34.643 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-test-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T20:26:34.643 INFO:teuthology.orchestra.run.smithi131.stdout: libcephsqlite-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T20:26:34.643 INFO:teuthology.orchestra.run.smithi131.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T20:26:34.643 INFO:teuthology.orchestra.run.smithi131.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T20:26:34.643 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:26:34.643 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T20:26:34.926 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph 2024-09-10T20:26:35.511 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:00:39 ago on Tue 10 Sep 2024 08:25:56 PM UTC. 2024-09-10T20:26:35.648 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T20:26:35.650 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:26:35.650 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-09-10T20:26:35.650 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:26:35.650 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-10T20:26:35.650 INFO:teuthology.orchestra.run.smithi131.stdout: ceph x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 6.6 k 2024-09-10T20:26:35.650 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-09-10T20:26:35.651 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mds x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 2.4 M 2024-09-10T20:26:35.651 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 1.5 M 2024-09-10T20:26:35.651 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4858.gbad1f558.el9 ceph-noarch 291 k 2024-09-10T20:26:35.651 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mon x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 4.8 M 2024-09-10T20:26:35.651 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-osd x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 17 M 2024-09-10T20:26:35.651 INFO:teuthology.orchestra.run.smithi131.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T20:26:35.651 INFO:teuthology.orchestra.run.smithi131.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T20:26:35.651 INFO:teuthology.orchestra.run.smithi131.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T20:26:35.651 INFO:teuthology.orchestra.run.smithi131.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T20:26:35.651 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T20:26:35.651 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T20:26:35.651 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T20:26:35.651 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T20:26:35.652 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T20:26:35.652 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T20:26:35.652 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-collections 2024-09-10T20:26:35.652 INFO:teuthology.orchestra.run.smithi131.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-10T20:26:35.652 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T20:26:35.652 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T20:26:35.652 INFO:teuthology.orchestra.run.smithi131.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T20:26:35.652 INFO:teuthology.orchestra.run.smithi131.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T20:26:35.652 INFO:teuthology.orchestra.run.smithi131.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T20:26:35.652 INFO:teuthology.orchestra.run.smithi131.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T20:26:35.652 INFO:teuthology.orchestra.run.smithi131.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T20:26:35.652 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T20:26:35.652 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T20:26:35.653 INFO:teuthology.orchestra.run.smithi131.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T20:26:35.653 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T20:26:35.653 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T20:26:35.653 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T20:26:35.653 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T20:26:35.653 INFO:teuthology.orchestra.run.smithi131.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T20:26:35.653 INFO:teuthology.orchestra.run.smithi131.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T20:26:35.653 INFO:teuthology.orchestra.run.smithi131.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T20:26:35.653 INFO:teuthology.orchestra.run.smithi131.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T20:26:35.653 INFO:teuthology.orchestra.run.smithi131.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T20:26:35.653 INFO:teuthology.orchestra.run.smithi131.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T20:26:35.654 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:26:35.654 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-10T20:26:35.654 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:26:35.654 INFO:teuthology.orchestra.run.smithi131.stdout:Install 37 Packages 2024-09-10T20:26:35.654 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:26:35.654 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 31 M 2024-09-10T20:26:35.655 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 109 M 2024-09-10T20:26:35.655 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-10T20:26:36.023 INFO:teuthology.orchestra.run.smithi131.stdout:(1/37): ceph-19.3.0-4858.gbad1f558.el9.x86_64.r 48 kB/s | 6.6 kB 00:00 2024-09-10T20:26:36.231 INFO:teuthology.orchestra.run.smithi131.stdout:(2/37): ceph-mds-19.3.0-4858.gbad1f558.el9.x86_ 6.9 MB/s | 2.4 MB 00:00 2024-09-10T20:26:36.274 INFO:teuthology.orchestra.run.smithi131.stdout:(3/37): ceph-mgr-19.3.0-4858.gbad1f558.el9.x86_ 3.9 MB/s | 1.5 MB 00:00 2024-09-10T20:26:36.366 INFO:teuthology.orchestra.run.smithi131.stdout:(4/37): ceph-mgr-modules-core-19.3.0-4858.gbad1 3.1 MB/s | 291 kB 00:00 2024-09-10T20:26:36.470 INFO:teuthology.orchestra.run.smithi131.stdout:(5/37): ceph-mon-19.3.0-4858.gbad1f558.el9.x86_ 11 MB/s | 4.8 MB 00:00 2024-09-10T20:26:36.808 INFO:teuthology.orchestra.run.smithi131.stdout:(6/37): ceph-osd-19.3.0-4858.gbad1f558.el9.x86_ 30 MB/s | 17 MB 00:00 2024-09-10T20:26:37.066 INFO:teuthology.orchestra.run.smithi131.stdout:(7/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 362 kB/s | 253 kB 00:00 2024-09-10T20:26:37.183 INFO:teuthology.orchestra.run.smithi131.stdout:(8/37): python3-pycparser-2.20-6.el9.noarch.rpm 1.1 MB/s | 135 kB 00:00 2024-09-10T20:26:37.267 INFO:teuthology.orchestra.run.smithi131.stdout:(9/37): python3-ply-3.11-14.el9.noarch.rpm 232 kB/s | 106 kB 00:00 2024-09-10T20:26:37.292 INFO:teuthology.orchestra.run.smithi131.stdout:(10/37): python3-requests-2.25.1-8.el9.noarch.r 1.1 MB/s | 125 kB 00:00 2024-09-10T20:26:37.417 INFO:teuthology.orchestra.run.smithi131.stdout:(11/37): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-09-10T20:26:37.476 INFO:teuthology.orchestra.run.smithi131.stdout:(12/37): python3-cryptography-36.0.1-4.el9.x86_ 1.2 MB/s | 1.2 MB 00:01 2024-09-10T20:26:37.501 INFO:teuthology.orchestra.run.smithi131.stdout:(13/37): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-09-10T20:26:37.518 INFO:teuthology.orchestra.run.smithi131.stdout:(14/37): python3-markupsafe-1.1.1-12.el9.x86_64 827 kB/s | 35 kB 00:00 2024-09-10T20:26:37.543 INFO:teuthology.orchestra.run.smithi131.stdout:(15/37): python3-urllib3-1.26.5-6.el9.noarch.rp 777 kB/s | 215 kB 00:00 2024-09-10T20:26:37.560 INFO:teuthology.orchestra.run.smithi131.stdout:(16/37): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-09-10T20:26:37.603 INFO:teuthology.orchestra.run.smithi131.stdout:(17/37): python3-packaging-20.9-5.el9.noarch.rp 764 kB/s | 77 kB 00:00 2024-09-10T20:26:37.653 INFO:teuthology.orchestra.run.smithi131.stdout:(18/37): luarocks-3.9.2-5.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-09-10T20:26:37.681 INFO:teuthology.orchestra.run.smithi131.stdout:(19/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.7 MB/s | 43 kB 00:00 2024-09-10T20:26:37.704 INFO:teuthology.orchestra.run.smithi131.stdout:(20/37): python3-toml-0.10.2-6.el9.noarch.rpm 261 kB/s | 42 kB 00:00 2024-09-10T20:26:37.729 INFO:teuthology.orchestra.run.smithi131.stdout:(21/37): lua-devel-5.4.4-4.el9.x86_64.rpm 132 kB/s | 22 kB 00:00 2024-09-10T20:26:37.754 INFO:teuthology.orchestra.run.smithi131.stdout:(22/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-09-10T20:26:37.796 INFO:teuthology.orchestra.run.smithi131.stdout:(23/37): python3-jaraco-classes-3.2.1-5.el9.noa 657 kB/s | 18 kB 00:00 2024-09-10T20:26:37.821 INFO:teuthology.orchestra.run.smithi131.stdout:(24/37): python3-jaraco-collections-3.0.0-8.el9 1.0 MB/s | 23 kB 00:00 2024-09-10T20:26:37.847 INFO:teuthology.orchestra.run.smithi131.stdout:(25/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 91 kB/s | 11 kB 00:00 2024-09-10T20:26:37.872 INFO:teuthology.orchestra.run.smithi131.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-09-10T20:26:37.897 INFO:teuthology.orchestra.run.smithi131.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 391 kB/s | 20 kB 00:00 2024-09-10T20:26:37.922 INFO:teuthology.orchestra.run.smithi131.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 927 kB/s | 46 kB 00:00 2024-09-10T20:26:37.956 INFO:teuthology.orchestra.run.smithi131.stdout:(29/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.5 MB/s | 358 kB 00:00 2024-09-10T20:26:37.981 INFO:teuthology.orchestra.run.smithi131.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 942 kB/s | 79 kB 00:00 2024-09-10T20:26:38.007 INFO:teuthology.orchestra.run.smithi131.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 328 kB/s | 16 kB 00:00 2024-09-10T20:26:38.032 INFO:teuthology.orchestra.run.smithi131.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-09-10T20:26:38.057 INFO:teuthology.orchestra.run.smithi131.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 714 kB/s | 36 kB 00:00 2024-09-10T20:26:38.082 INFO:teuthology.orchestra.run.smithi131.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-09-10T20:26:38.108 INFO:teuthology.orchestra.run.smithi131.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 798 kB/s | 20 kB 00:00 2024-09-10T20:26:38.133 INFO:teuthology.orchestra.run.smithi131.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-10T20:26:38.167 INFO:teuthology.orchestra.run.smithi131.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.8 MB/s | 427 kB 00:00 2024-09-10T20:26:38.171 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-10T20:26:38.171 INFO:teuthology.orchestra.run.smithi131.stdout:Total 12 MB/s | 31 MB 00:02 2024-09-10T20:26:38.324 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-10T20:26:38.363 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-10T20:26:38.363 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-10T20:26:38.844 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-10T20:26:38.844 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-10T20:26:40.238 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-10T20:26:40.311 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-10T20:26:40.353 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-10T20:26:40.389 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-10T20:26:40.478 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-10T20:26:40.538 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-10T20:26:40.571 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-10T20:26:40.664 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-10T20:26:40.750 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-10T20:26:40.791 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-10T20:26:40.837 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-10T20:26:40.876 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-10T20:26:40.910 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-10T20:26:40.944 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-10T20:26:40.984 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-10T20:26:41.019 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-10T20:26:41.053 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-10T20:26:41.102 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-10T20:26:41.134 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-10T20:26:41.208 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-10T20:26:41.355 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-10T20:26:41.442 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-10T20:26:41.496 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-10T20:26:41.534 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-10T20:26:41.584 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-10T20:26:41.647 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-10T20:26:41.842 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-10T20:26:41.897 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-10T20:26:41.975 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-10T20:26:42.072 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-10T20:26:42.127 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-10T20:26:42.260 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-10T20:26:42.429 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4858.gbad1f558.el9. 32/37 2024-09-10T20:26:42.470 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-2:19.3.0-4858.gbad1f558.el9.x86_64 33/37 2024-09-10T20:26:42.497 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4858.gbad1f558.el9.x86_64 33/37 2024-09-10T20:26:42.497 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:26:42.497 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T20:26:42.498 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T20:26:42.498 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T20:26:42.498 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:26:43.971 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-osd-2:19.3.0-4858.gbad1f558.el9.x86_64 34/37 2024-09-10T20:26:43.998 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-osd-2:19.3.0-4858.gbad1f558.el9.x86_64 34/37 2024-09-10T20:26:43.999 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:26:43.999 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T20:26:43.999 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T20:26:43.999 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T20:26:43.999 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:26:44.447 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mon-2:19.3.0-4858.gbad1f558.el9.x86_64 35/37 2024-09-10T20:26:44.471 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mon-2:19.3.0-4858.gbad1f558.el9.x86_64 35/37 2024-09-10T20:26:44.471 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:26:44.471 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T20:26:44.471 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T20:26:44.471 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T20:26:44.472 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:26:44.697 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mds-2:19.3.0-4858.gbad1f558.el9.x86_64 36/37 2024-09-10T20:26:44.721 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mds-2:19.3.0-4858.gbad1f558.el9.x86_64 36/37 2024-09-10T20:26:44.721 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:26:44.721 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T20:26:44.721 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T20:26:44.722 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T20:26:44.722 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:26:44.787 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-2:19.3.0-4858.gbad1f558.el9.x86_64 37/37 2024-09-10T20:26:46.381 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-2:19.3.0-4858.gbad1f558.el9.x86_64 37/37 2024-09-10T20:26:46.381 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-2:19.3.0-4858.gbad1f558.el9.x86_64 1/37 2024-09-10T20:26:46.381 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mds-2:19.3.0-4858.gbad1f558.el9.x86_64 2/37 2024-09-10T20:26:46.381 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-2:19.3.0-4858.gbad1f558.el9.x86_64 3/37 2024-09-10T20:26:46.381 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mon-2:19.3.0-4858.gbad1f558.el9.x86_64 4/37 2024-09-10T20:26:46.381 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-osd-2:19.3.0-4858.gbad1f558.el9.x86_64 5/37 2024-09-10T20:26:46.381 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4858.gbad1f558.el9. 6/37 2024-09-10T20:26:46.381 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-10T20:26:46.381 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-10T20:26:46.382 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-10T20:26:46.382 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-10T20:26:46.382 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-10T20:26:46.382 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-10T20:26:46.382 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-10T20:26:46.382 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-10T20:26:46.382 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-10T20:26:46.382 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-10T20:26:46.382 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-10T20:26:46.382 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-10T20:26:46.382 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-10T20:26:46.382 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-10T20:26:46.382 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-10T20:26:46.382 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-10T20:26:46.383 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-10T20:26:46.383 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-10T20:26:46.383 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-10T20:26:46.383 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-10T20:26:46.383 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-10T20:26:46.383 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-10T20:26:46.384 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-10T20:26:46.384 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-10T20:26:46.384 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-10T20:26:46.384 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-10T20:26:46.384 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-10T20:26:46.384 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-10T20:26:46.384 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-10T20:26:46.385 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-10T20:26:47.383 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-10T20:26:47.383 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:26:47.383 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-10T20:26:47.383 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T20:26:47.384 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mds-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T20:26:47.384 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T20:26:47.384 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-modules-core-2:19.3.0-4858.gbad1f558.el9.noarch 2024-09-10T20:26:47.384 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mon-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T20:26:47.384 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-osd-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T20:26:47.384 INFO:teuthology.orchestra.run.smithi131.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T20:26:47.384 INFO:teuthology.orchestra.run.smithi131.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T20:26:47.384 INFO:teuthology.orchestra.run.smithi131.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T20:26:47.384 INFO:teuthology.orchestra.run.smithi131.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T20:26:47.384 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T20:26:47.385 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T20:26:47.385 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T20:26:47.385 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T20:26:47.385 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T20:26:47.385 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T20:26:47.385 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T20:26:47.385 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T20:26:47.385 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T20:26:47.385 INFO:teuthology.orchestra.run.smithi131.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T20:26:47.385 INFO:teuthology.orchestra.run.smithi131.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T20:26:47.385 INFO:teuthology.orchestra.run.smithi131.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T20:26:47.386 INFO:teuthology.orchestra.run.smithi131.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T20:26:47.386 INFO:teuthology.orchestra.run.smithi131.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T20:26:47.386 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T20:26:47.386 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T20:26:47.386 INFO:teuthology.orchestra.run.smithi131.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T20:26:47.386 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T20:26:47.386 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T20:26:47.386 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T20:26:47.386 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T20:26:47.386 INFO:teuthology.orchestra.run.smithi131.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T20:26:47.386 INFO:teuthology.orchestra.run.smithi131.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T20:26:47.387 INFO:teuthology.orchestra.run.smithi131.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T20:26:47.387 INFO:teuthology.orchestra.run.smithi131.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T20:26:47.387 INFO:teuthology.orchestra.run.smithi131.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T20:26:47.387 INFO:teuthology.orchestra.run.smithi131.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T20:26:47.387 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:26:47.387 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T20:26:47.779 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-base 2024-09-10T20:26:48.348 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:00:52 ago on Tue 10 Sep 2024 08:25:56 PM UTC. 2024-09-10T20:26:48.439 INFO:teuthology.orchestra.run.smithi131.stdout:Package ceph-base-2:19.3.0-4858.gbad1f558.el9.x86_64 is already installed. 2024-09-10T20:26:48.481 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T20:26:48.482 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2024-09-10T20:26:48.482 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T20:26:48.547 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install cephadm 2024-09-10T20:26:49.105 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:00:53 ago on Tue 10 Sep 2024 08:25:56 PM UTC. 2024-09-10T20:26:49.237 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T20:26:49.237 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:26:49.237 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-09-10T20:26:49.238 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:26:49.238 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-10T20:26:49.238 INFO:teuthology.orchestra.run.smithi131.stdout: cephadm noarch 2:19.3.0-4858.gbad1f558.el9 ceph-noarch 784 k 2024-09-10T20:26:49.238 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:26:49.238 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-10T20:26:49.238 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:26:49.238 INFO:teuthology.orchestra.run.smithi131.stdout:Install 1 Package 2024-09-10T20:26:49.238 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:26:49.238 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 784 k 2024-09-10T20:26:49.238 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 790 k 2024-09-10T20:26:49.238 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-10T20:26:49.514 INFO:teuthology.orchestra.run.smithi131.stdout:cephadm-19.3.0-4858.gbad1f558.el9.noarch.rpm 2.8 MB/s | 784 kB 00:00 2024-09-10T20:26:49.515 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-10T20:26:49.515 INFO:teuthology.orchestra.run.smithi131.stdout:Total 2.8 MB/s | 784 kB 00:00 2024-09-10T20:26:49.515 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-10T20:26:49.520 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-10T20:26:49.520 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-10T20:26:49.526 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-10T20:26:49.527 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-10T20:26:49.602 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-10T20:26:50.043 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: cephadm-2:19.3.0-4858.gbad1f558.el9.noarch 1/1 2024-09-10T20:26:50.151 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephadm-2:19.3.0-4858.gbad1f558.el9.noarch 1/1 2024-09-10T20:26:50.558 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: cephadm-2:19.3.0-4858.gbad1f558.el9.noarch 1/1 2024-09-10T20:26:50.797 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephadm-2:19.3.0-4858.gbad1f558.el9.noarch 1/1 2024-09-10T20:26:50.798 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:26:50.798 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-10T20:26:50.798 INFO:teuthology.orchestra.run.smithi131.stdout: cephadm-2:19.3.0-4858.gbad1f558.el9.noarch 2024-09-10T20:26:50.798 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:26:50.798 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T20:26:51.010 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-immutable-object-cache 2024-09-10T20:26:51.590 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:00:55 ago on Tue 10 Sep 2024 08:25:56 PM UTC. 2024-09-10T20:26:51.728 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T20:26:51.728 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:26:51.728 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repo Size 2024-09-10T20:26:51.729 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:26:51.729 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-10T20:26:51.729 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 145 k 2024-09-10T20:26:51.729 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:26:51.729 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-10T20:26:51.729 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:26:51.729 INFO:teuthology.orchestra.run.smithi131.stdout:Install 1 Package 2024-09-10T20:26:51.729 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:26:51.729 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 145 k 2024-09-10T20:26:51.729 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 432 k 2024-09-10T20:26:51.729 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-10T20:26:51.946 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-immutable-object-cache-19.3.0-4858.gbad1f5 670 kB/s | 145 kB 00:00 2024-09-10T20:26:51.947 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-10T20:26:51.947 INFO:teuthology.orchestra.run.smithi131.stdout:Total 665 kB/s | 145 kB 00:00 2024-09-10T20:26:51.947 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-10T20:26:51.955 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-10T20:26:51.955 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-10T20:26:52.015 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-10T20:26:52.015 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-10T20:26:52.160 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-10T20:26:52.247 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4858.gbad1f558. 1/1 2024-09-10T20:26:52.272 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4858.gbad1f558. 1/1 2024-09-10T20:26:52.272 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:26:52.273 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T20:26:52.273 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:26:53.179 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4858.gbad1f558. 1/1 2024-09-10T20:26:53.179 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:26:53.179 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-10T20:26:53.179 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-immutable-object-cache-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T20:26:53.179 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:26:53.179 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T20:26:53.419 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-mgr 2024-09-10T20:26:53.997 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:00:57 ago on Tue 10 Sep 2024 08:25:56 PM UTC. 2024-09-10T20:26:54.086 INFO:teuthology.orchestra.run.smithi131.stdout:Package ceph-mgr-2:19.3.0-4858.gbad1f558.el9.x86_64 is already installed. 2024-09-10T20:26:54.128 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T20:26:54.129 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2024-09-10T20:26:54.129 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T20:26:54.188 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-mgr-dashboard 2024-09-10T20:26:54.751 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:00:58 ago on Tue 10 Sep 2024 08:25:56 PM UTC. 2024-09-10T20:26:54.885 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T20:26:54.886 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:26:54.886 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-09-10T20:26:54.886 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:26:54.886 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-10T20:26:54.886 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4858.gbad1f558.el9 ceph-noarch 14 M 2024-09-10T20:26:54.886 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-09-10T20:26:54.886 INFO:teuthology.orchestra.run.smithi131.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T20:26:54.886 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4858.gbad1f558.el9 ceph-noarch 34 k 2024-09-10T20:26:54.886 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4858.gbad1f558.el9 ceph-noarch 16 k 2024-09-10T20:26:54.886 INFO:teuthology.orchestra.run.smithi131.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T20:26:54.886 INFO:teuthology.orchestra.run.smithi131.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T20:26:54.886 INFO:teuthology.orchestra.run.smithi131.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T20:26:54.887 INFO:teuthology.orchestra.run.smithi131.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T20:26:54.887 INFO:teuthology.orchestra.run.smithi131.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T20:26:54.887 INFO:teuthology.orchestra.run.smithi131.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T20:26:54.887 INFO:teuthology.orchestra.run.smithi131.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T20:26:54.887 INFO:teuthology.orchestra.run.smithi131.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T20:26:54.887 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:26:54.887 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-10T20:26:54.887 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:26:54.887 INFO:teuthology.orchestra.run.smithi131.stdout:Install 12 Packages 2024-09-10T20:26:54.887 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:26:54.888 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 19 M 2024-09-10T20:26:54.888 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 159 M 2024-09-10T20:26:54.888 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-10T20:26:55.625 INFO:teuthology.orchestra.run.smithi131.stdout:(1/12): ceph-prometheus-alerts-19.3.0-4858.gbad 103 kB/s | 16 kB 00:00 2024-09-10T20:26:55.650 INFO:teuthology.orchestra.run.smithi131.stdout:(2/12): ceph-grafana-dashboards-19.3.0-4858.gba 185 kB/s | 34 kB 00:00 2024-09-10T20:26:55.842 INFO:teuthology.orchestra.run.smithi131.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.4 MB/s | 269 kB 00:00 2024-09-10T20:26:56.101 INFO:teuthology.orchestra.run.smithi131.stdout:(4/12): ceph-mgr-dashboard-19.3.0-4858.gbad1f55 22 MB/s | 14 MB 00:00 2024-09-10T20:26:56.143 INFO:teuthology.orchestra.run.smithi131.stdout:(5/12): protobuf-3.14.0-13.el9.x86_64.rpm 2.0 MB/s | 1.0 MB 00:00 2024-09-10T20:26:56.185 INFO:teuthology.orchestra.run.smithi131.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 486 kB/s | 19 kB 00:00 2024-09-10T20:26:56.227 INFO:teuthology.orchestra.run.smithi131.stdout:(7/12): protobuf-compiler-3.14.0-13.el9.x86_64. 2.2 MB/s | 863 kB 00:00 2024-09-10T20:26:56.252 INFO:teuthology.orchestra.run.smithi131.stdout:(8/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 3.6 MB/s | 548 kB 00:00 2024-09-10T20:26:56.294 INFO:teuthology.orchestra.run.smithi131.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 901 kB/s | 31 kB 00:00 2024-09-10T20:26:56.336 INFO:teuthology.orchestra.run.smithi131.stdout:(10/12): python3-grpcio-tools-1.46.7-10.el9.x86 1.3 MB/s | 144 kB 00:00 2024-09-10T20:26:56.361 INFO:teuthology.orchestra.run.smithi131.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 2.8 MB/s | 188 kB 00:00 2024-09-10T20:26:56.411 INFO:teuthology.orchestra.run.smithi131.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 9.0 MB/s | 2.0 MB 00:00 2024-09-10T20:26:56.415 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-10T20:26:56.415 INFO:teuthology.orchestra.run.smithi131.stdout:Total 13 MB/s | 19 MB 00:01 2024-09-10T20:26:56.461 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-10T20:26:56.479 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-10T20:26:56.479 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-10T20:26:56.757 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-10T20:26:56.757 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-10T20:26:57.355 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-10T20:26:57.437 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-10T20:26:57.481 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-10T20:26:57.527 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-10T20:26:57.558 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-10T20:26:57.663 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-10T20:26:57.793 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-10T20:26:57.886 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-10T20:26:57.929 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-10T20:26:57.958 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-10T20:26:57.990 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4858.gbad1f558.el9 10/12 2024-09-10T20:27:01.149 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4858.gbad1f558.el 11/12 2024-09-10T20:27:01.203 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4858.gbad1f558.el9.noa 12/12 2024-09-10T20:27:03.110 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4858.gbad1f558.el9.noa 12/12 2024-09-10T20:27:03.110 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4858.gbad1f558.el 1/12 2024-09-10T20:27:03.111 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4858.gbad1f558.el9.noa 2/12 2024-09-10T20:27:03.111 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4858.gbad1f558.el9 3/12 2024-09-10T20:27:03.111 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-10T20:27:03.111 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-10T20:27:03.111 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-10T20:27:03.111 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-10T20:27:03.111 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-10T20:27:03.111 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-10T20:27:03.111 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-10T20:27:03.111 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-10T20:27:03.454 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-10T20:27:03.454 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:03.454 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-10T20:27:03.454 INFO:teuthology.orchestra.run.smithi131.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T20:27:03.454 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-grafana-dashboards-2:19.3.0-4858.gbad1f558.el9.noarch 2024-09-10T20:27:03.454 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-dashboard-2:19.3.0-4858.gbad1f558.el9.noarch 2024-09-10T20:27:03.455 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-prometheus-alerts-2:19.3.0-4858.gbad1f558.el9.noarch 2024-09-10T20:27:03.455 INFO:teuthology.orchestra.run.smithi131.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T20:27:03.455 INFO:teuthology.orchestra.run.smithi131.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T20:27:03.455 INFO:teuthology.orchestra.run.smithi131.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T20:27:03.455 INFO:teuthology.orchestra.run.smithi131.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T20:27:03.455 INFO:teuthology.orchestra.run.smithi131.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T20:27:03.455 INFO:teuthology.orchestra.run.smithi131.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T20:27:03.455 INFO:teuthology.orchestra.run.smithi131.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T20:27:03.455 INFO:teuthology.orchestra.run.smithi131.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T20:27:03.455 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:03.455 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T20:27:03.713 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-10T20:27:04.275 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:08 ago on Tue 10 Sep 2024 08:25:56 PM UTC. 2024-09-10T20:27:04.409 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T20:27:04.410 INFO:teuthology.orchestra.run.smithi131.stdout:======================================================================================= 2024-09-10T20:27:04.410 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-09-10T20:27:04.410 INFO:teuthology.orchestra.run.smithi131.stdout:======================================================================================= 2024-09-10T20:27:04.410 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-10T20:27:04.410 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4858.gbad1f558.el9 ceph-noarch 7.4 M 2024-09-10T20:27:04.410 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-09-10T20:27:04.410 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T20:27:04.411 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T20:27:04.411 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T20:27:04.411 INFO:teuthology.orchestra.run.smithi131.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T20:27:04.411 INFO:teuthology.orchestra.run.smithi131.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T20:27:04.411 INFO:teuthology.orchestra.run.smithi131.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T20:27:04.411 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T20:27:04.411 INFO:teuthology.orchestra.run.smithi131.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T20:27:04.411 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T20:27:04.411 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T20:27:04.411 INFO:teuthology.orchestra.run.smithi131.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T20:27:04.411 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:04.412 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-10T20:27:04.412 INFO:teuthology.orchestra.run.smithi131.stdout:======================================================================================= 2024-09-10T20:27:04.412 INFO:teuthology.orchestra.run.smithi131.stdout:Install 12 Packages 2024-09-10T20:27:04.412 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:04.413 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 38 M 2024-09-10T20:27:04.413 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 212 M 2024-09-10T20:27:04.413 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-10T20:27:04.952 INFO:teuthology.orchestra.run.smithi131.stdout:(1/12): ceph-mgr-diskprediction-local-19.3.0-48 16 MB/s | 7.4 MB 00:00 2024-09-10T20:27:04.977 INFO:teuthology.orchestra.run.smithi131.stdout:(2/12): libquadmath-11.5.0-2.el9.x86_64.rpm 402 kB/s | 191 kB 00:00 2024-09-10T20:27:05.044 INFO:teuthology.orchestra.run.smithi131.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 356 kB/s | 33 kB 00:00 2024-09-10T20:27:05.086 INFO:teuthology.orchestra.run.smithi131.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 437 kB/s | 18 kB 00:00 2024-09-10T20:27:05.145 INFO:teuthology.orchestra.run.smithi131.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 660 kB/s | 38 kB 00:00 2024-09-10T20:27:05.212 INFO:teuthology.orchestra.run.smithi131.stdout:(6/12): libgfortran-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 801 kB 00:00 2024-09-10T20:27:05.338 INFO:teuthology.orchestra.run.smithi131.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.9 MB/s | 245 kB 00:00 2024-09-10T20:27:05.530 INFO:teuthology.orchestra.run.smithi131.stdout:(8/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 13 MB/s | 4.9 MB 00:00 2024-09-10T20:27:05.597 INFO:teuthology.orchestra.run.smithi131.stdout:(9/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 4.8 MB/s | 3.0 MB 00:00 2024-09-10T20:27:05.622 INFO:teuthology.orchestra.run.smithi131.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.7 MB/s | 250 kB 00:00 2024-09-10T20:27:05.689 INFO:teuthology.orchestra.run.smithi131.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-09-10T20:27:06.523 INFO:teuthology.orchestra.run.smithi131.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 18 MB/s | 16 MB 00:00 2024-09-10T20:27:06.523 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-10T20:27:06.523 INFO:teuthology.orchestra.run.smithi131.stdout:Total 18 MB/s | 38 MB 00:02 2024-09-10T20:27:06.877 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-10T20:27:06.902 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-10T20:27:06.902 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-10T20:27:07.324 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-10T20:27:07.324 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-10T20:27:08.088 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-10T20:27:08.153 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-10T20:27:08.216 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-10T20:27:08.311 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-10T20:27:08.355 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-10T20:27:08.543 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-10T20:27:08.571 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-10T20:27:08.656 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-10T20:27:09.301 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-10T20:27:09.416 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-10T20:27:10.657 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-10T20:27:11.692 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-10T20:27:11.721 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4858.gbad1f 12/12 2024-09-10T20:27:14.288 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4858.gbad1f 12/12 2024-09-10T20:27:14.288 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4858.gbad1f 1/12 2024-09-10T20:27:14.288 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-10T20:27:14.288 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-10T20:27:14.288 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-10T20:27:14.288 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-10T20:27:14.288 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-10T20:27:14.289 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-10T20:27:14.289 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-10T20:27:14.289 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-10T20:27:14.289 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-10T20:27:14.289 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-10T20:27:14.729 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-10T20:27:14.729 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:14.730 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-10T20:27:14.730 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4858.gbad1f558.el9.noarch 2024-09-10T20:27:14.730 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T20:27:14.730 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T20:27:14.730 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T20:27:14.730 INFO:teuthology.orchestra.run.smithi131.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T20:27:14.730 INFO:teuthology.orchestra.run.smithi131.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T20:27:14.730 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T20:27:14.730 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T20:27:14.730 INFO:teuthology.orchestra.run.smithi131.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T20:27:14.730 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T20:27:14.730 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T20:27:14.730 INFO:teuthology.orchestra.run.smithi131.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T20:27:14.731 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:14.731 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T20:27:15.119 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-mgr-rook 2024-09-10T20:27:15.703 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:19 ago on Tue 10 Sep 2024 08:25:56 PM UTC. 2024-09-10T20:27:15.852 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T20:27:15.854 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:27:15.854 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-09-10T20:27:15.854 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:27:15.854 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-10T20:27:15.854 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-rook noarch 2:19.3.0-4858.gbad1f558.el9 ceph-noarch 49 k 2024-09-10T20:27:15.854 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-09-10T20:27:15.854 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T20:27:15.854 INFO:teuthology.orchestra.run.smithi131.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T20:27:15.854 INFO:teuthology.orchestra.run.smithi131.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T20:27:15.854 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T20:27:15.854 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T20:27:15.854 INFO:teuthology.orchestra.run.smithi131.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T20:27:15.854 INFO:teuthology.orchestra.run.smithi131.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T20:27:15.855 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T20:27:15.855 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T20:27:15.855 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T20:27:15.855 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T20:27:15.855 INFO:teuthology.orchestra.run.smithi131.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T20:27:15.855 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:15.855 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-10T20:27:15.855 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:27:15.855 INFO:teuthology.orchestra.run.smithi131.stdout:Install 13 Packages 2024-09-10T20:27:15.855 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:15.856 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 2.2 M 2024-09-10T20:27:15.856 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 26 M 2024-09-10T20:27:15.856 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-10T20:27:16.047 INFO:teuthology.orchestra.run.smithi131.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 233 kB/s | 26 kB 00:00 2024-09-10T20:27:16.072 INFO:teuthology.orchestra.run.smithi131.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 142 kB/s | 19 kB 00:00 2024-09-10T20:27:16.097 INFO:teuthology.orchestra.run.smithi131.stdout:(3/13): ceph-mgr-rook-19.3.0-4858.gbad1f558.el9 307 kB/s | 49 kB 00:00 2024-09-10T20:27:16.198 INFO:teuthology.orchestra.run.smithi131.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.7 MB/s | 222 kB 00:00 2024-09-10T20:27:16.223 INFO:teuthology.orchestra.run.smithi131.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 159 kB 00:00 2024-09-10T20:27:16.282 INFO:teuthology.orchestra.run.smithi131.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 641 kB/s | 54 kB 00:00 2024-09-10T20:27:16.315 INFO:teuthology.orchestra.run.smithi131.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 350 kB/s | 32 kB 00:00 2024-09-10T20:27:16.340 INFO:teuthology.orchestra.run.smithi131.stdout:(8/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.1 MB/s | 279 kB 00:00 2024-09-10T20:27:16.365 INFO:teuthology.orchestra.run.smithi131.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 172 kB/s | 14 kB 00:00 2024-09-10T20:27:16.399 INFO:teuthology.orchestra.run.smithi131.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.6 MB/s | 220 kB 00:00 2024-09-10T20:27:16.424 INFO:teuthology.orchestra.run.smithi131.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-10T20:27:16.450 INFO:teuthology.orchestra.run.smithi131.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.8 MB/s | 90 kB 00:00 2024-09-10T20:27:16.567 INFO:teuthology.orchestra.run.smithi131.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 4.5 MB/s | 1.0 MB 00:00 2024-09-10T20:27:16.570 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-10T20:27:16.570 INFO:teuthology.orchestra.run.smithi131.stdout:Total 3.1 MB/s | 2.2 MB 00:00 2024-09-10T20:27:16.632 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-10T20:27:16.648 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-10T20:27:16.648 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-10T20:27:16.738 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-10T20:27:16.738 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-10T20:27:17.247 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-10T20:27:17.408 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-10T20:27:17.460 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-10T20:27:17.524 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-10T20:27:17.569 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-10T20:27:17.611 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-10T20:27:17.759 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-10T20:27:17.900 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-10T20:27:17.955 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-10T20:27:18.695 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-10T20:27:18.743 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-10T20:27:18.777 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-10T20:27:18.825 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-10T20:27:18.872 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-rook-2:19.3.0-4858.gbad1f558.el9.noarch 13/13 2024-09-10T20:27:19.616 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4858.gbad1f558.el9.noarch 13/13 2024-09-10T20:27:19.617 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4858.gbad1f558.el9.noarch 1/13 2024-09-10T20:27:19.617 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-10T20:27:19.617 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-10T20:27:19.617 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-10T20:27:19.617 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-10T20:27:19.617 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-10T20:27:19.617 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-10T20:27:19.617 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-10T20:27:19.617 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-10T20:27:19.617 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-10T20:27:19.617 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-10T20:27:19.617 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-10T20:27:19.967 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-10T20:27:19.967 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:19.967 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-10T20:27:19.967 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-rook-2:19.3.0-4858.gbad1f558.el9.noarch 2024-09-10T20:27:19.968 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T20:27:19.968 INFO:teuthology.orchestra.run.smithi131.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T20:27:19.968 INFO:teuthology.orchestra.run.smithi131.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T20:27:19.968 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T20:27:19.968 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T20:27:19.968 INFO:teuthology.orchestra.run.smithi131.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T20:27:19.968 INFO:teuthology.orchestra.run.smithi131.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T20:27:19.968 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T20:27:19.968 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T20:27:19.968 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T20:27:19.968 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T20:27:19.968 INFO:teuthology.orchestra.run.smithi131.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T20:27:19.968 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:19.969 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T20:27:20.204 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-mgr-cephadm 2024-09-10T20:27:20.767 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:24 ago on Tue 10 Sep 2024 08:25:56 PM UTC. 2024-09-10T20:27:20.900 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T20:27:20.901 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:27:20.901 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-09-10T20:27:20.901 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:27:20.901 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-10T20:27:20.901 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4858.gbad1f558.el9 ceph-noarch 154 k 2024-09-10T20:27:20.902 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-09-10T20:27:20.902 INFO:teuthology.orchestra.run.smithi131.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T20:27:20.902 INFO:teuthology.orchestra.run.smithi131.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T20:27:20.902 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T20:27:20.902 INFO:teuthology.orchestra.run.smithi131.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T20:27:20.902 INFO:teuthology.orchestra.run.smithi131.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T20:27:20.902 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:20.902 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-10T20:27:20.902 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:27:20.902 INFO:teuthology.orchestra.run.smithi131.stdout:Install 6 Packages 2024-09-10T20:27:20.902 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:20.903 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 7.0 M 2024-09-10T20:27:20.903 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 33 M 2024-09-10T20:27:20.903 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-10T20:27:21.178 INFO:teuthology.orchestra.run.smithi131.stdout:(1/6): ceph-mgr-cephadm-19.3.0-4858.gbad1f558.e 806 kB/s | 154 kB 00:00 2024-09-10T20:27:21.237 INFO:teuthology.orchestra.run.smithi131.stdout:(2/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 993 kB/s | 248 kB 00:00 2024-09-10T20:27:21.279 INFO:teuthology.orchestra.run.smithi131.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.4 MB/s | 58 kB 00:00 2024-09-10T20:27:21.321 INFO:teuthology.orchestra.run.smithi131.stdout:(4/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 3.8 MB/s | 548 kB 00:00 2024-09-10T20:27:21.346 INFO:teuthology.orchestra.run.smithi131.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.1 MB/s | 74 kB 00:00 2024-09-10T20:27:21.488 INFO:teuthology.orchestra.run.smithi131.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 6.0 MB 00:00 2024-09-10T20:27:21.490 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-10T20:27:21.490 INFO:teuthology.orchestra.run.smithi131.stdout:Total 12 MB/s | 7.0 MB 00:00 2024-09-10T20:27:21.561 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-10T20:27:21.573 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-10T20:27:21.573 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-10T20:27:21.674 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-10T20:27:21.674 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-10T20:27:21.992 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-10T20:27:22.163 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-10T20:27:22.215 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-10T20:27:22.738 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-10T20:27:22.826 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-10T20:27:22.894 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-10T20:27:22.927 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4858.gbad1f558.el9.noarch 6/6 2024-09-10T20:27:23.725 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4858.gbad1f558.el9.noarch 6/6 2024-09-10T20:27:23.726 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4858.gbad1f558.el9.noarch 1/6 2024-09-10T20:27:23.726 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-10T20:27:23.726 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-10T20:27:23.726 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-10T20:27:23.726 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-10T20:27:23.999 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-10T20:27:23.999 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:23.999 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-10T20:27:23.999 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-cephadm-2:19.3.0-4858.gbad1f558.el9.noarch 2024-09-10T20:27:23.999 INFO:teuthology.orchestra.run.smithi131.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T20:27:23.999 INFO:teuthology.orchestra.run.smithi131.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T20:27:23.999 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T20:27:24.000 INFO:teuthology.orchestra.run.smithi131.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T20:27:24.000 INFO:teuthology.orchestra.run.smithi131.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T20:27:24.000 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:24.000 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T20:27:24.263 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-fuse 2024-09-10T20:27:24.829 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:28 ago on Tue 10 Sep 2024 08:25:56 PM UTC. 2024-09-10T20:27:24.962 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T20:27:24.962 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:27:24.963 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-09-10T20:27:24.963 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:27:24.963 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-10T20:27:24.963 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-fuse x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 901 k 2024-09-10T20:27:24.963 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-09-10T20:27:24.963 INFO:teuthology.orchestra.run.smithi131.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T20:27:24.963 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:24.963 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-10T20:27:24.963 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:27:24.963 INFO:teuthology.orchestra.run.smithi131.stdout:Install 2 Packages 2024-09-10T20:27:24.963 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:24.964 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 981 k 2024-09-10T20:27:24.964 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 2.9 M 2024-09-10T20:27:24.965 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-10T20:27:25.082 INFO:teuthology.orchestra.run.smithi131.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 843 kB/s | 80 kB 00:00 2024-09-10T20:27:25.282 INFO:teuthology.orchestra.run.smithi131.stdout:(2/2): ceph-fuse-19.3.0-4858.gbad1f558.el9.x86_ 3.0 MB/s | 901 kB 00:00 2024-09-10T20:27:25.283 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-10T20:27:25.283 INFO:teuthology.orchestra.run.smithi131.stdout:Total 3.0 MB/s | 981 kB 00:00 2024-09-10T20:27:25.295 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-10T20:27:25.317 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-10T20:27:25.317 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-10T20:27:25.397 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-10T20:27:25.398 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-10T20:27:25.594 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-10T20:27:25.736 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-10T20:27:25.807 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-fuse-2:19.3.0-4858.gbad1f558.el9.x86_64 2/2 2024-09-10T20:27:26.498 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-fuse-2:19.3.0-4858.gbad1f558.el9.x86_64 2/2 2024-09-10T20:27:26.498 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-fuse-2:19.3.0-4858.gbad1f558.el9.x86_64 1/2 2024-09-10T20:27:26.759 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-10T20:27:26.759 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:26.759 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-10T20:27:26.759 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-fuse-2:19.3.0-4858.gbad1f558.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-10T20:27:26.760 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:26.760 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T20:27:27.020 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-volume 2024-09-10T20:27:27.601 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:31 ago on Tue 10 Sep 2024 08:25:56 PM UTC. 2024-09-10T20:27:27.742 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T20:27:27.743 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:27:27.743 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-09-10T20:27:27.743 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:27:27.743 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-10T20:27:27.743 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-volume noarch 2:19.3.0-4858.gbad1f558.el9 ceph-noarch 285 k 2024-09-10T20:27:27.743 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:27.743 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-10T20:27:27.743 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:27:27.743 INFO:teuthology.orchestra.run.smithi131.stdout:Install 1 Package 2024-09-10T20:27:27.743 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:27.744 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 285 k 2024-09-10T20:27:27.744 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 1.3 M 2024-09-10T20:27:27.744 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-10T20:27:27.988 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-volume-19.3.0-4858.gbad1f558.el9.noarch.rp 1.1 MB/s | 285 kB 00:00 2024-09-10T20:27:27.989 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-10T20:27:27.989 INFO:teuthology.orchestra.run.smithi131.stdout:Total 1.1 MB/s | 285 kB 00:00 2024-09-10T20:27:27.989 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-10T20:27:27.998 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-10T20:27:27.998 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-10T20:27:28.029 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-10T20:27:28.029 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-10T20:27:28.251 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-10T20:27:28.332 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-volume-2:19.3.0-4858.gbad1f558.el9.noarch 1/1 2024-09-10T20:27:28.348 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-volume-2:19.3.0-4858.gbad1f558.el9.noarch 1/1 2024-09-10T20:27:28.348 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:27:28.348 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T20:27:28.348 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:29.399 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-volume-2:19.3.0-4858.gbad1f558.el9.noarch 1/1 2024-09-10T20:27:29.399 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:29.399 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-10T20:27:29.400 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-volume-2:19.3.0-4858.gbad1f558.el9.noarch 2024-09-10T20:27:29.400 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:29.400 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T20:27:29.716 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install librados-devel 2024-09-10T20:27:30.302 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:34 ago on Tue 10 Sep 2024 08:25:56 PM UTC. 2024-09-10T20:27:30.436 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T20:27:30.437 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:27:30.437 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repo Size 2024-09-10T20:27:30.437 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:27:30.437 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-10T20:27:30.437 INFO:teuthology.orchestra.run.smithi131.stdout: librados-devel x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 126 k 2024-09-10T20:27:30.437 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:30.438 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-10T20:27:30.438 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:27:30.438 INFO:teuthology.orchestra.run.smithi131.stdout:Install 1 Package 2024-09-10T20:27:30.438 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:30.438 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 126 k 2024-09-10T20:27:30.438 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 456 k 2024-09-10T20:27:30.438 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-10T20:27:30.653 INFO:teuthology.orchestra.run.smithi131.stdout:librados-devel-19.3.0-4858.gbad1f558.el9.x86_64 589 kB/s | 126 kB 00:00 2024-09-10T20:27:30.653 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-10T20:27:30.654 INFO:teuthology.orchestra.run.smithi131.stdout:Total 586 kB/s | 126 kB 00:00 2024-09-10T20:27:30.654 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-10T20:27:30.659 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-10T20:27:30.659 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-10T20:27:30.722 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-10T20:27:30.722 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-10T20:27:30.876 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-10T20:27:30.975 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librados-devel-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T20:27:31.451 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librados-devel-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T20:27:31.697 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados-devel-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T20:27:31.697 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:31.698 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-10T20:27:31.698 INFO:teuthology.orchestra.run.smithi131.stdout: librados-devel-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T20:27:31.698 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:31.698 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T20:27:31.937 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install libcephfs2 2024-09-10T20:27:32.513 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:36 ago on Tue 10 Sep 2024 08:25:56 PM UTC. 2024-09-10T20:27:32.605 INFO:teuthology.orchestra.run.smithi131.stdout:Package libcephfs2-2:19.3.0-4858.gbad1f558.el9.x86_64 is already installed. 2024-09-10T20:27:32.649 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T20:27:32.650 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2024-09-10T20:27:32.650 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T20:27:32.743 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install libcephfs-devel 2024-09-10T20:27:33.316 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:37 ago on Tue 10 Sep 2024 08:25:56 PM UTC. 2024-09-10T20:27:33.449 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T20:27:33.450 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:27:33.450 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repo Size 2024-09-10T20:27:33.450 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:27:33.450 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-10T20:27:33.450 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs-devel x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 32 k 2024-09-10T20:27:33.450 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:33.450 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-10T20:27:33.450 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:27:33.450 INFO:teuthology.orchestra.run.smithi131.stdout:Install 1 Package 2024-09-10T20:27:33.450 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:33.450 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 32 k 2024-09-10T20:27:33.450 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 142 k 2024-09-10T20:27:33.451 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-10T20:27:33.607 INFO:teuthology.orchestra.run.smithi131.stdout:libcephfs-devel-19.3.0-4858.gbad1f558.el9.x86_6 202 kB/s | 32 kB 00:00 2024-09-10T20:27:33.608 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-10T20:27:33.608 INFO:teuthology.orchestra.run.smithi131.stdout:Total 201 kB/s | 32 kB 00:00 2024-09-10T20:27:33.608 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-10T20:27:33.612 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-10T20:27:33.612 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-10T20:27:33.632 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-10T20:27:33.632 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-10T20:27:33.716 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-10T20:27:33.830 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephfs-devel-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T20:27:34.230 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T20:27:34.482 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephfs-devel-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T20:27:34.482 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:34.482 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-10T20:27:34.482 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs-devel-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T20:27:34.482 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:34.482 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T20:27:34.664 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install librados2 2024-09-10T20:27:35.259 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:39 ago on Tue 10 Sep 2024 08:25:56 PM UTC. 2024-09-10T20:27:35.352 INFO:teuthology.orchestra.run.smithi131.stdout:Package librados2-2:19.3.0-4858.gbad1f558.el9.x86_64 is already installed. 2024-09-10T20:27:35.396 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T20:27:35.397 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2024-09-10T20:27:35.397 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T20:27:35.458 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install librbd1 2024-09-10T20:27:36.048 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:40 ago on Tue 10 Sep 2024 08:25:56 PM UTC. 2024-09-10T20:27:36.139 INFO:teuthology.orchestra.run.smithi131.stdout:Package librbd1-2:19.3.0-4858.gbad1f558.el9.x86_64 is already installed. 2024-09-10T20:27:36.183 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T20:27:36.184 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2024-09-10T20:27:36.184 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T20:27:36.262 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install python3-rados 2024-09-10T20:27:36.833 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:40 ago on Tue 10 Sep 2024 08:25:56 PM UTC. 2024-09-10T20:27:36.923 INFO:teuthology.orchestra.run.smithi131.stdout:Package python3-rados-2:19.3.0-4858.gbad1f558.el9.x86_64 is already installed. 2024-09-10T20:27:36.966 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T20:27:36.967 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2024-09-10T20:27:36.967 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T20:27:37.043 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install python3-rgw 2024-09-10T20:27:37.606 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:41 ago on Tue 10 Sep 2024 08:25:56 PM UTC. 2024-09-10T20:27:37.697 INFO:teuthology.orchestra.run.smithi131.stdout:Package python3-rgw-2:19.3.0-4858.gbad1f558.el9.x86_64 is already installed. 2024-09-10T20:27:37.741 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T20:27:37.742 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2024-09-10T20:27:37.742 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T20:27:37.798 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install python3-cephfs 2024-09-10T20:27:38.365 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:42 ago on Tue 10 Sep 2024 08:25:56 PM UTC. 2024-09-10T20:27:38.455 INFO:teuthology.orchestra.run.smithi131.stdout:Package python3-cephfs-2:19.3.0-4858.gbad1f558.el9.x86_64 is already installed. 2024-09-10T20:27:38.498 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T20:27:38.499 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2024-09-10T20:27:38.499 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T20:27:38.553 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install python3-rbd 2024-09-10T20:27:39.117 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:43 ago on Tue 10 Sep 2024 08:25:56 PM UTC. 2024-09-10T20:27:39.207 INFO:teuthology.orchestra.run.smithi131.stdout:Package python3-rbd-2:19.3.0-4858.gbad1f558.el9.x86_64 is already installed. 2024-09-10T20:27:39.250 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T20:27:39.251 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2024-09-10T20:27:39.251 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T20:27:39.326 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install rbd-fuse 2024-09-10T20:27:39.890 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:43 ago on Tue 10 Sep 2024 08:25:56 PM UTC. 2024-09-10T20:27:40.024 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T20:27:40.025 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:27:40.025 INFO:teuthology.orchestra.run.smithi131.stdout: Package Architecture Version Repository Size 2024-09-10T20:27:40.025 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:27:40.025 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-10T20:27:40.025 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-fuse x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 85 k 2024-09-10T20:27:40.025 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:40.025 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-10T20:27:40.025 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:27:40.025 INFO:teuthology.orchestra.run.smithi131.stdout:Install 1 Package 2024-09-10T20:27:40.025 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:40.025 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 85 k 2024-09-10T20:27:40.025 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 231 k 2024-09-10T20:27:40.026 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-10T20:27:40.211 INFO:teuthology.orchestra.run.smithi131.stdout:rbd-fuse-19.3.0-4858.gbad1f558.el9.x86_64.rpm 459 kB/s | 85 kB 00:00 2024-09-10T20:27:40.212 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-10T20:27:40.212 INFO:teuthology.orchestra.run.smithi131.stdout:Total 455 kB/s | 85 kB 00:00 2024-09-10T20:27:40.212 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-10T20:27:40.219 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-10T20:27:40.219 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-10T20:27:40.283 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-10T20:27:40.283 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-10T20:27:40.427 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-10T20:27:40.517 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-fuse-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T20:27:40.954 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: rbd-fuse-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T20:27:41.215 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-fuse-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T20:27:41.215 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:41.215 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-10T20:27:41.216 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-fuse-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T20:27:41.216 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:41.216 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T20:27:41.449 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install rbd-mirror 2024-09-10T20:27:42.025 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:46 ago on Tue 10 Sep 2024 08:25:56 PM UTC. 2024-09-10T20:27:42.165 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T20:27:42.166 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:27:42.166 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repo Size 2024-09-10T20:27:42.166 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:27:42.166 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-10T20:27:42.166 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-mirror x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 3.2 M 2024-09-10T20:27:42.166 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:42.166 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-10T20:27:42.166 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:27:42.166 INFO:teuthology.orchestra.run.smithi131.stdout:Install 1 Package 2024-09-10T20:27:42.166 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:42.166 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 3.2 M 2024-09-10T20:27:42.166 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 14 M 2024-09-10T20:27:42.166 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-10T20:27:42.515 INFO:teuthology.orchestra.run.smithi131.stdout:rbd-mirror-19.3.0-4858.gbad1f558.el9.x86_64.rpm 9.1 MB/s | 3.2 MB 00:00 2024-09-10T20:27:42.516 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-10T20:27:42.516 INFO:teuthology.orchestra.run.smithi131.stdout:Total 9.1 MB/s | 3.2 MB 00:00 2024-09-10T20:27:42.516 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-10T20:27:42.525 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-10T20:27:42.525 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-10T20:27:42.611 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-10T20:27:42.611 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-10T20:27:43.046 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-10T20:27:43.108 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-mirror-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T20:27:43.134 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T20:27:43.134 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:27:43.134 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T20:27:43.134 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T20:27:43.134 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T20:27:43.134 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:44.271 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-mirror-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T20:27:44.271 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:44.271 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-10T20:27:44.271 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-mirror-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T20:27:44.272 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:44.272 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T20:27:44.494 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install rbd-nbd 2024-09-10T20:27:45.081 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:49 ago on Tue 10 Sep 2024 08:25:56 PM UTC. 2024-09-10T20:27:45.217 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T20:27:45.218 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:27:45.218 INFO:teuthology.orchestra.run.smithi131.stdout: Package Architecture Version Repository Size 2024-09-10T20:27:45.218 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:27:45.218 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-10T20:27:45.218 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-nbd x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 170 k 2024-09-10T20:27:45.218 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:45.218 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-10T20:27:45.218 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T20:27:45.218 INFO:teuthology.orchestra.run.smithi131.stdout:Install 1 Package 2024-09-10T20:27:45.218 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:45.219 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 170 k 2024-09-10T20:27:45.219 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 490 k 2024-09-10T20:27:45.219 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-10T20:27:45.428 INFO:teuthology.orchestra.run.smithi131.stdout:rbd-nbd-19.3.0-4858.gbad1f558.el9.x86_64.rpm 817 kB/s | 170 kB 00:00 2024-09-10T20:27:45.429 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-10T20:27:45.429 INFO:teuthology.orchestra.run.smithi131.stdout:Total 810 kB/s | 170 kB 00:00 2024-09-10T20:27:45.429 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-10T20:27:45.437 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-10T20:27:45.437 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-10T20:27:45.501 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-10T20:27:45.501 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-10T20:27:45.652 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-10T20:27:45.750 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-nbd-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T20:27:46.193 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: rbd-nbd-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T20:27:46.447 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-nbd-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T20:27:46.448 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:46.448 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-10T20:27:46.448 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-nbd-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T20:27:46.448 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:46.448 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T20:27:46.669 DEBUG:teuthology.parallel:result is None 2024-09-10T20:27:46.671 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bad1f558c327094897e4a32e99dd5cb4ae6ccd46 2024-09-10T20:27:46.823 DEBUG:teuthology.orchestra.run.smithi131:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T20:27:46.845 INFO:teuthology.orchestra.run.smithi131.stdout:19.3.0-4858.gbad1f558.el9 2024-09-10T20:27:46.845 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4858.gbad1f558.el9 2024-09-10T20:27:46.846 INFO:teuthology.task.install:The correct ceph version 19.3.0-4858.gbad1f558 is installed. 2024-09-10T20:27:46.847 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-10T20:27:46.848 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-10T20:27:46.848 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T20:27:46.911 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-10T20:27:46.911 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-10T20:27:46.911 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T20:27:46.976 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T20:27:47.041 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-10T20:27:47.041 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-10T20:27:47.041 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T20:27:47.106 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T20:27:47.171 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-10T20:27:47.172 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-10T20:27:47.172 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T20:27:47.236 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T20:27:47.301 INFO:teuthology.run_tasks:Running task ceph... 2024-09-10T20:27:47.384 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-10T20:27:47.385 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 777 /var/log/ceph 2024-09-10T20:27:47.409 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-10T20:27:47.409 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-10T20:27:47.474 INFO:tasks.ceph:Creating extra log directories... 2024-09-10T20:27:47.475 DEBUG:teuthology.orchestra.run.smithi131:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-10T20:27:47.546 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-10T20:27:47.547 INFO:tasks.ceph:config {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore write v2': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(OBJECT_', '\\(SLOW_OPS\\)', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', '\\(MON_DOWN\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-10T20:27:47.547 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/skanta-2024-09-10_15:30:27-rados-wip-bharath7-testing-2024-09-10-1409-distro-default-smithi/7899571', 'branch': 'wip-bharath7-testing-2024-09-10-1409', 'description': 'rados/singleton/{all/lost-unfound-delete mon_election/connectivity msgr-failures/few msgr/async objectstore/{bluestore-options/write$/{write_v2} bluestore/bluestore-comp-zstd} rados supported-random-distro$/{centos_latest}}', 'email': 'skanta@redhat.com', 'first_in_suite': False, 'job_id': '7899571', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'skanta-2024-09-10_15:30:27-rados-wip-bharath7-testing-2024-09-10-1409-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 3, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-bharath7-testing-2024-09-10-1409'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore write v2': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', '\\(MON_DOWN\\)'], 'sha1': 'bad1f558c327094897e4a32e99dd5cb4ae6ccd46'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'bad1f558c327094897e4a32e99dd5cb4ae6ccd46'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-bharath7-testing-2024-09-10-1409', 'sha1': 'bad1f558c327094897e4a32e99dd5cb4ae6ccd46'}}, 'owner': 'scheduled_skanta@teuthology', 'priority': 99, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'osd.0', 'osd.1', 'osd.2']], 'seed': 5245, 'sha1': 'bad1f558c327094897e4a32e99dd5cb4ae6ccd46', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'wip-bharath7-testing-2024-09-10-1409', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_bad1f558c327094897e4a32e99dd5cb4ae6ccd46/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'bad1f558c327094897e4a32e99dd5cb4ae6ccd46', 'targets': {'smithi131.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBASFrE86gn+M380wKFO1yYVeF0gCfQGbWHo109TMLSHZjuVGd0n1B9nlXOYvY70Pvxc1v6O6rq2RM5Dp2E+x+Iw='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'log-ignorelist': ['objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(OBJECT_', '\\(SLOW_OPS\\)', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', '\\(MON_DOWN\\)'], 'pre-mgr-commands': ['sudo ceph config set mgr mgr_pool false --force'], 'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore write v2': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'fs': 'xfs', 'sha1': 'bad1f558c327094897e4a32e99dd5cb4ae6ccd46', 'cluster': 'ceph'}}, {'rep_lost_unfound_delete': None}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-10_15:30:27', 'tube': 'smithi', 'user': 'skanta', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-10T20:27:47.547 DEBUG:teuthology.orchestra.run.smithi131:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-10T20:27:47.595 DEBUG:teuthology.orchestra.run.smithi131:> sudo install -d -m0777 -- /var/run/ceph 2024-09-10T20:27:47.660 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-10T20:27:47.661 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T20:27:47.718 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T20:27:47.718 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_1 2024-09-10T20:27:47.773 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T20:27:47.774 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T20:27:47.774 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-10T20:27:47.774 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T20:27:47.774 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T20:27:47.774 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-09-10 20:27:43.375647255 +0000 2024-09-10T20:27:47.774 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-09-10 20:25:15.279917965 +0000 2024-09-10T20:27:47.774 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-09-10 20:25:15.279917965 +0000 2024-09-10T20:27:47.774 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-09-10 20:25:15.279917965 +0000 2024-09-10T20:27:47.775 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T20:27:47.837 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-09-10T20:27:47.837 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-09-10T20:27:47.837 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000207163 s, 2.5 MB/s 2024-09-10T20:27:47.838 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T20:27:47.895 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_2 2024-09-10T20:27:47.949 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T20:27:47.950 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T20:27:47.950 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-10T20:27:47.950 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T20:27:47.950 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T20:27:47.950 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-09-10 20:27:43.376647234 +0000 2024-09-10T20:27:47.950 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-09-10 20:25:15.605911019 +0000 2024-09-10T20:27:47.950 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-09-10 20:25:15.605911019 +0000 2024-09-10T20:27:47.950 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-09-10 20:25:15.605911019 +0000 2024-09-10T20:27:47.950 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T20:27:48.013 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-09-10T20:27:48.013 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-09-10T20:27:48.013 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000215193 s, 2.4 MB/s 2024-09-10T20:27:48.014 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T20:27:48.070 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_3 2024-09-10T20:27:48.125 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T20:27:48.125 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T20:27:48.126 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-10T20:27:48.126 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T20:27:48.126 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T20:27:48.126 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-09-10 20:27:43.376647234 +0000 2024-09-10T20:27:48.126 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-09-10 20:25:15.918904351 +0000 2024-09-10T20:27:48.126 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-09-10 20:25:15.918904351 +0000 2024-09-10T20:27:48.126 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-09-10 20:25:15.918904351 +0000 2024-09-10T20:27:48.126 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T20:27:48.188 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-09-10T20:27:48.188 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-09-10T20:27:48.188 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000215563 s, 2.4 MB/s 2024-09-10T20:27:48.189 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T20:27:48.245 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_4 2024-09-10T20:27:48.301 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T20:27:48.301 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T20:27:48.301 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-10T20:27:48.302 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T20:27:48.302 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T20:27:48.302 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-09-10 20:27:43.377647213 +0000 2024-09-10T20:27:48.302 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-09-10 20:25:16.245897385 +0000 2024-09-10T20:27:48.302 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-09-10 20:25:16.245897385 +0000 2024-09-10T20:27:48.302 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-09-10 20:25:16.245897385 +0000 2024-09-10T20:27:48.302 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T20:27:48.365 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-09-10T20:27:48.365 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-09-10T20:27:48.365 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000199029 s, 2.6 MB/s 2024-09-10T20:27:48.366 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T20:27:48.423 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-10T20:27:48.423 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi131.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2024-09-10T20:27:48.423 INFO:tasks.ceph:Generating config... 2024-09-10T20:27:48.424 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-10T20:27:48.424 INFO:tasks.ceph:[global] mon election default strategy = 3 2024-09-10T20:27:48.424 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-10T20:27:48.424 INFO:tasks.ceph:[global] ms type = async 2024-09-10T20:27:48.425 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-10T20:27:48.425 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-10T20:27:48.425 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-10T20:27:48.425 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-10T20:27:48.425 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-10T20:27:48.425 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-09-10T20:27:48.425 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-10T20:27:48.425 INFO:tasks.ceph:[osd] bluestore compression algorithm = zstd 2024-09-10T20:27:48.425 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-10T20:27:48.425 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-10T20:27:48.425 INFO:tasks.ceph:[osd] bluestore write v2 = True 2024-09-10T20:27:48.425 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-09-10T20:27:48.425 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-10T20:27:48.425 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-10T20:27:48.426 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-10T20:27:48.426 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-10T20:27:48.426 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-10T20:27:48.426 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-10T20:27:48.426 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-10T20:27:48.426 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-10T20:27:48.426 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-09-10T20:27:48.426 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-09-10T20:27:48.426 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-10T20:27:48.426 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-09-10T20:27:48.426 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-09-10T20:27:48.426 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-10T20:27:48.426 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-09-10T20:27:48.426 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-09-10T20:27:48.427 INFO:tasks.ceph:Setting up mon.a... 2024-09-10T20:27:48.427 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-10T20:27:48.508 INFO:teuthology.orchestra.run.smithi131.stdout:creating /etc/ceph/ceph.keyring 2024-09-10T20:27:48.510 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-10T20:27:48.552 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T20:27:48.617 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.131'), ('mon.b', '[v2:172.21.15.131:3301,v1:172.21.15.131:6790]'), ('mon.c', '[v2:172.21.15.131:3302,v1:172.21.15.131:6791]')] 2024-09-10T20:27:48.618 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.131,[v2:172.21.15.131:3301,v1:172.21.15.131:6790],[v2:172.21.15.131:3302,v1:172.21.15.131:6791]', 'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore write v2': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon scrub interval': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}} 2024-09-10T20:27:48.619 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-10T20:27:48.619 DEBUG:teuthology.orchestra.run.smithi131:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-10T20:27:48.673 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.131 --addv b '[v2:172.21.15.131:3301,v1:172.21.15.131:6790]' --addv c '[v2:172.21.15.131:3302,v1:172.21.15.131:6791]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-10T20:27:48.759 INFO:teuthology.orchestra.run.smithi131.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-10T20:27:48.760 INFO:teuthology.orchestra.run.smithi131.stdout:monmaptool: generated fsid 3053cb46-0512-45f5-9707-4eb48df5a665 2024-09-10T20:27:48.760 INFO:teuthology.orchestra.run.smithi131.stdout:setting min_mon_release = quincy 2024-09-10T20:27:48.760 INFO:teuthology.orchestra.run.smithi131.stdout:epoch 0 2024-09-10T20:27:48.760 INFO:teuthology.orchestra.run.smithi131.stdout:fsid 3053cb46-0512-45f5-9707-4eb48df5a665 2024-09-10T20:27:48.760 INFO:teuthology.orchestra.run.smithi131.stdout:last_changed 2024-09-10T20:27:48.756932+0000 2024-09-10T20:27:48.760 INFO:teuthology.orchestra.run.smithi131.stdout:created 2024-09-10T20:27:48.756932+0000 2024-09-10T20:27:48.760 INFO:teuthology.orchestra.run.smithi131.stdout:min_mon_release 17 (quincy) 2024-09-10T20:27:48.760 INFO:teuthology.orchestra.run.smithi131.stdout:election_strategy: 3 2024-09-10T20:27:48.760 INFO:teuthology.orchestra.run.smithi131.stdout:0: [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon.a 2024-09-10T20:27:48.760 INFO:teuthology.orchestra.run.smithi131.stdout:1: [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] mon.b 2024-09-10T20:27:48.760 INFO:teuthology.orchestra.run.smithi131.stdout:2: [v2:172.21.15.131:3302/0,v1:172.21.15.131:6791/0] mon.c 2024-09-10T20:27:48.760 INFO:teuthology.orchestra.run.smithi131.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-10T20:27:48.762 DEBUG:teuthology.orchestra.run.smithi131:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-10T20:27:48.777 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 3053cb46-0512-45f5-9707-4eb48df5a665... 2024-09-10T20:27:48.777 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T20:27:48.860 INFO:teuthology.orchestra.run.smithi131.stdout:[global] 2024-09-10T20:27:48.860 INFO:teuthology.orchestra.run.smithi131.stdout: chdir = "" 2024-09-10T20:27:48.860 INFO:teuthology.orchestra.run.smithi131.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T20:27:48.860 INFO:teuthology.orchestra.run.smithi131.stdout: auth supported = cephx 2024-09-10T20:27:48.861 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:48.861 INFO:teuthology.orchestra.run.smithi131.stdout: filestore xattr use omap = true 2024-09-10T20:27:48.861 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:48.861 INFO:teuthology.orchestra.run.smithi131.stdout: mon clock drift allowed = 1.000 2024-09-10T20:27:48.861 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:48.861 INFO:teuthology.orchestra.run.smithi131.stdout: osd crush chooseleaf type = 0 2024-09-10T20:27:48.861 INFO:teuthology.orchestra.run.smithi131.stdout: auth debug = true 2024-09-10T20:27:48.861 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:48.861 INFO:teuthology.orchestra.run.smithi131.stdout: ms die on old message = true 2024-09-10T20:27:48.861 INFO:teuthology.orchestra.run.smithi131.stdout: ms die on bug = true 2024-09-10T20:27:48.861 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:48.861 INFO:teuthology.orchestra.run.smithi131.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T20:27:48.862 INFO:teuthology.orchestra.run.smithi131.stdout: mon pg warn max object skew = 0 2024-09-10T20:27:48.862 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:48.862 INFO:teuthology.orchestra.run.smithi131.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T20:27:48.862 INFO:teuthology.orchestra.run.smithi131.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T20:27:48.862 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:48.862 INFO:teuthology.orchestra.run.smithi131.stdout: osd pool default size = 2 2024-09-10T20:27:48.862 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:48.862 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd allow primary affinity = true 2024-09-10T20:27:48.862 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd allow pg remap = true 2024-09-10T20:27:48.862 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on legacy crush tunables = false 2024-09-10T20:27:48.862 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on crush straw calc version zero = false 2024-09-10T20:27:48.862 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on no sortbitwise = false 2024-09-10T20:27:48.862 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on osd down out interval zero = false 2024-09-10T20:27:48.862 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on too few osds = false 2024-09-10T20:27:48.863 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T20:27:48.863 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T20:27:48.863 INFO:teuthology.orchestra.run.smithi131.stdout: mon_allow_pool_size_one = true 2024-09-10T20:27:48.863 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:48.863 INFO:teuthology.orchestra.run.smithi131.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T20:27:48.863 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:48.863 INFO:teuthology.orchestra.run.smithi131.stdout: osd default data pool replay window = 5 2024-09-10T20:27:48.863 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:48.863 INFO:teuthology.orchestra.run.smithi131.stdout: mon allow pool delete = true 2024-09-10T20:27:48.864 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:48.864 INFO:teuthology.orchestra.run.smithi131.stdout: mon cluster log file level = debug 2024-09-10T20:27:48.864 INFO:teuthology.orchestra.run.smithi131.stdout: debug asserts on shutdown = true 2024-09-10T20:27:48.864 INFO:teuthology.orchestra.run.smithi131.stdout: mon health detail to clog = false 2024-09-10T20:27:48.864 INFO:teuthology.orchestra.run.smithi131.stdout: mon host = "172.21.15.131,[v2:172.21.15.131:3301,v1:172.21.15.131:6790],[v2:172.21.15.131:3302,v1:172.21.15.131:6791]" 2024-09-10T20:27:48.864 INFO:teuthology.orchestra.run.smithi131.stdout: mon client directed command retry = 5 2024-09-10T20:27:48.864 INFO:teuthology.orchestra.run.smithi131.stdout: mon election default strategy = 3 2024-09-10T20:27:48.865 INFO:teuthology.orchestra.run.smithi131.stdout: ms inject socket failures = 5000 2024-09-10T20:27:48.865 INFO:teuthology.orchestra.run.smithi131.stdout: ms type = async 2024-09-10T20:27:48.865 INFO:teuthology.orchestra.run.smithi131.stdout: fsid = 3053cb46-0512-45f5-9707-4eb48df5a665 2024-09-10T20:27:48.865 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:48.865 INFO:teuthology.orchestra.run.smithi131.stdout:[osd] 2024-09-10T20:27:48.865 INFO:teuthology.orchestra.run.smithi131.stdout: osd journal size = 100 2024-09-10T20:27:48.865 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:48.865 INFO:teuthology.orchestra.run.smithi131.stdout: osd scrub load threshold = 5.0 2024-09-10T20:27:48.865 INFO:teuthology.orchestra.run.smithi131.stdout: osd scrub max interval = 600 2024-09-10T20:27:48.865 INFO:teuthology.orchestra.run.smithi131.stdout: osd mclock profile = high_recovery_ops 2024-09-10T20:27:48.865 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:48.865 INFO:teuthology.orchestra.run.smithi131.stdout: osd recover clone overlap = true 2024-09-10T20:27:48.865 INFO:teuthology.orchestra.run.smithi131.stdout: osd recovery max chunk = 1048576 2024-09-10T20:27:48.865 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:48.865 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug shutdown = true 2024-09-10T20:27:48.866 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug op order = true 2024-09-10T20:27:48.866 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug verify stray on activate = true 2024-09-10T20:27:48.866 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug trim objects = true 2024-09-10T20:27:48.866 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:48.866 INFO:teuthology.orchestra.run.smithi131.stdout: osd open classes on start = true 2024-09-10T20:27:48.866 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug pg log writeout = true 2024-09-10T20:27:48.866 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:48.866 INFO:teuthology.orchestra.run.smithi131.stdout: osd deep scrub update digest min age = 30 2024-09-10T20:27:48.866 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:48.866 INFO:teuthology.orchestra.run.smithi131.stdout: osd map max advance = 10 2024-09-10T20:27:48.866 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:48.866 INFO:teuthology.orchestra.run.smithi131.stdout: journal zero on create = true 2024-09-10T20:27:48.866 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:48.867 INFO:teuthology.orchestra.run.smithi131.stdout: filestore ondisk finisher threads = 3 2024-09-10T20:27:48.867 INFO:teuthology.orchestra.run.smithi131.stdout: filestore apply finisher threads = 3 2024-09-10T20:27:48.867 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:48.867 INFO:teuthology.orchestra.run.smithi131.stdout: bdev debug aio = true 2024-09-10T20:27:48.867 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug misdirected ops = true 2024-09-10T20:27:48.867 INFO:teuthology.orchestra.run.smithi131.stdout: bluestore block size = 96636764160 2024-09-10T20:27:48.867 INFO:teuthology.orchestra.run.smithi131.stdout: bluestore compression algorithm = zstd 2024-09-10T20:27:48.868 INFO:teuthology.orchestra.run.smithi131.stdout: bluestore compression mode = aggressive 2024-09-10T20:27:48.868 INFO:teuthology.orchestra.run.smithi131.stdout: bluestore fsck on mount = True 2024-09-10T20:27:48.868 INFO:teuthology.orchestra.run.smithi131.stdout: bluestore write v2 = True 2024-09-10T20:27:48.868 INFO:teuthology.orchestra.run.smithi131.stdout: bluestore zero block detection = True 2024-09-10T20:27:48.868 INFO:teuthology.orchestra.run.smithi131.stdout: debug bluefs = 20 2024-09-10T20:27:48.868 INFO:teuthology.orchestra.run.smithi131.stdout: debug bluestore = 20 2024-09-10T20:27:48.868 INFO:teuthology.orchestra.run.smithi131.stdout: debug ms = 1 2024-09-10T20:27:48.868 INFO:teuthology.orchestra.run.smithi131.stdout: debug osd = 20 2024-09-10T20:27:48.868 INFO:teuthology.orchestra.run.smithi131.stdout: debug rocksdb = 10 2024-09-10T20:27:48.868 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T20:27:48.868 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd full ratio = 0.9 2024-09-10T20:27:48.868 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd nearfull ratio = 0.8 2024-09-10T20:27:48.868 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug verify cached snaps = True 2024-09-10T20:27:48.869 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug verify missing on start = True 2024-09-10T20:27:48.869 INFO:teuthology.orchestra.run.smithi131.stdout: osd failsafe full ratio = 0.95 2024-09-10T20:27:48.869 INFO:teuthology.orchestra.run.smithi131.stdout: osd mclock override recovery settings = True 2024-09-10T20:27:48.869 INFO:teuthology.orchestra.run.smithi131.stdout: osd objectstore = bluestore 2024-09-10T20:27:48.869 INFO:teuthology.orchestra.run.smithi131.stdout: osd op queue = debug_random 2024-09-10T20:27:48.869 INFO:teuthology.orchestra.run.smithi131.stdout: osd op queue cut off = debug_random 2024-09-10T20:27:48.869 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:48.869 INFO:teuthology.orchestra.run.smithi131.stdout:[mgr] 2024-09-10T20:27:48.869 INFO:teuthology.orchestra.run.smithi131.stdout: debug ms = 1 2024-09-10T20:27:48.869 INFO:teuthology.orchestra.run.smithi131.stdout: debug mgr = 20 2024-09-10T20:27:48.869 INFO:teuthology.orchestra.run.smithi131.stdout: debug mon = 20 2024-09-10T20:27:48.869 INFO:teuthology.orchestra.run.smithi131.stdout: debug auth = 20 2024-09-10T20:27:48.869 INFO:teuthology.orchestra.run.smithi131.stdout: mon reweight min pgs per osd = 4 2024-09-10T20:27:48.869 INFO:teuthology.orchestra.run.smithi131.stdout: mon reweight min bytes per osd = 10 2024-09-10T20:27:48.869 INFO:teuthology.orchestra.run.smithi131.stdout: mgr/telemetry/nag = false 2024-09-10T20:27:48.870 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:48.870 INFO:teuthology.orchestra.run.smithi131.stdout:[mon] 2024-09-10T20:27:48.870 INFO:teuthology.orchestra.run.smithi131.stdout: debug ms = 1 2024-09-10T20:27:48.870 INFO:teuthology.orchestra.run.smithi131.stdout: debug mon = 20 2024-09-10T20:27:48.870 INFO:teuthology.orchestra.run.smithi131.stdout: debug paxos = 20 2024-09-10T20:27:48.870 INFO:teuthology.orchestra.run.smithi131.stdout: debug auth = 20 2024-09-10T20:27:48.870 INFO:teuthology.orchestra.run.smithi131.stdout: mon data avail warn = 5 2024-09-10T20:27:48.870 INFO:teuthology.orchestra.run.smithi131.stdout: mon mgr mkfs grace = 240 2024-09-10T20:27:48.870 INFO:teuthology.orchestra.run.smithi131.stdout: mon reweight min pgs per osd = 4 2024-09-10T20:27:48.870 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd reporter subtree level = osd 2024-09-10T20:27:48.870 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd prime pg temp = true 2024-09-10T20:27:48.870 INFO:teuthology.orchestra.run.smithi131.stdout: mon reweight min bytes per osd = 10 2024-09-10T20:27:48.870 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:48.870 INFO:teuthology.orchestra.run.smithi131.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T20:27:48.871 INFO:teuthology.orchestra.run.smithi131.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T20:27:48.871 INFO:teuthology.orchestra.run.smithi131.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T20:27:48.871 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:48.871 INFO:teuthology.orchestra.run.smithi131.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T20:27:48.871 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T20:27:48.871 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T20:27:48.871 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:48.871 INFO:teuthology.orchestra.run.smithi131.stdout: # 1m isn't quite enough 2024-09-10T20:27:48.871 INFO:teuthology.orchestra.run.smithi131.stdout: mon_down_mkfs_grace = 2m 2024-09-10T20:27:48.871 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:48.871 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_filestore_osds = false 2024-09-10T20:27:48.871 INFO:teuthology.orchestra.run.smithi131.stdout: mon scrub interval = 300 2024-09-10T20:27:48.871 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:27:48.871 INFO:teuthology.orchestra.run.smithi131.stdout:[client] 2024-09-10T20:27:48.871 INFO:teuthology.orchestra.run.smithi131.stdout: rgw cache enabled = true 2024-09-10T20:27:48.872 INFO:teuthology.orchestra.run.smithi131.stdout: rgw enable ops log = true 2024-09-10T20:27:48.872 INFO:teuthology.orchestra.run.smithi131.stdout: rgw enable usage log = true 2024-09-10T20:27:48.872 INFO:teuthology.orchestra.run.smithi131.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T20:27:48.872 INFO:teuthology.orchestra.run.smithi131.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T20:27:48.872 INFO:teuthology.orchestra.run.smithi131.stdout:[mon.a] 2024-09-10T20:27:48.872 INFO:teuthology.orchestra.run.smithi131.stdout:[mon.b] 2024-09-10T20:27:48.872 INFO:teuthology.orchestra.run.smithi131.stdout:[mon.c] 2024-09-10T20:27:48.873 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-10T20:27:48.873 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-10T20:27:48.968 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-10T20:27:48.969 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-10T20:27:48.969 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-10T20:27:48.984 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-10T20:27:48.984 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-10T20:27:49.039 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi131.front.sepia.ceph.com 2024-09-10T20:27:49.039 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-10T20:27:49.039 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-10T20:27:49.040 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T20:27:49.114 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-10T20:27:49.114 DEBUG:teuthology.orchestra.run.smithi131:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-10T20:27:49.170 INFO:tasks.ceph:Setting up mon nodes... 2024-09-10T20:27:49.170 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-10T20:27:49.170 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-10T20:27:49.261 INFO:teuthology.orchestra.run.smithi131.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-10T20:27:49.263 INFO:tasks.ceph:Setting up mds nodes... 2024-09-10T20:27:49.263 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-10T20:27:49.264 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-10T20:27:49.264 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi131.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2024-09-10T20:27:49.264 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-10T20:27:49.289 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-10T20:27:49.289 INFO:tasks.ceph:role: osd.0 2024-09-10T20:27:49.289 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi131.front.sepia.ceph.com 2024-09-10T20:27:49.289 DEBUG:teuthology.orchestra.run.smithi131:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-10T20:27:49.361 INFO:teuthology.orchestra.run.smithi131.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T20:27:49.361 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T20:27:49.362 INFO:teuthology.orchestra.run.smithi131.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T20:27:49.362 INFO:teuthology.orchestra.run.smithi131.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T20:27:49.362 INFO:teuthology.orchestra.run.smithi131.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T20:27:49.362 INFO:teuthology.orchestra.run.smithi131.stdout: = sunit=0 swidth=0 blks 2024-09-10T20:27:49.362 INFO:teuthology.orchestra.run.smithi131.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T20:27:49.362 INFO:teuthology.orchestra.run.smithi131.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T20:27:49.362 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T20:27:49.362 INFO:teuthology.orchestra.run.smithi131.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T20:27:49.858 INFO:teuthology.orchestra.run.smithi131.stdout:Discarding blocks...Done. 2024-09-10T20:27:49.860 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi131.front.sepia.ceph.com -o noatime 2024-09-10T20:27:49.861 DEBUG:teuthology.orchestra.run.smithi131:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-10T20:27:49.969 DEBUG:teuthology.orchestra.run.smithi131:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-10T20:27:50.004 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-10T20:27:50.069 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-10T20:27:50.070 INFO:tasks.ceph:role: osd.1 2024-09-10T20:27:50.070 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi131.front.sepia.ceph.com 2024-09-10T20:27:50.070 DEBUG:teuthology.orchestra.run.smithi131:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-10T20:27:50.141 INFO:teuthology.orchestra.run.smithi131.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T20:27:50.142 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T20:27:50.142 INFO:teuthology.orchestra.run.smithi131.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T20:27:50.142 INFO:teuthology.orchestra.run.smithi131.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T20:27:50.142 INFO:teuthology.orchestra.run.smithi131.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T20:27:50.142 INFO:teuthology.orchestra.run.smithi131.stdout: = sunit=0 swidth=0 blks 2024-09-10T20:27:50.142 INFO:teuthology.orchestra.run.smithi131.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T20:27:50.142 INFO:teuthology.orchestra.run.smithi131.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T20:27:50.142 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T20:27:50.142 INFO:teuthology.orchestra.run.smithi131.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T20:27:50.608 INFO:teuthology.orchestra.run.smithi131.stdout:Discarding blocks...Done. 2024-09-10T20:27:50.610 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi131.front.sepia.ceph.com -o noatime 2024-09-10T20:27:50.610 DEBUG:teuthology.orchestra.run.smithi131:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-10T20:27:50.704 DEBUG:teuthology.orchestra.run.smithi131:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-10T20:27:50.733 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-10T20:27:50.798 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-10T20:27:50.799 INFO:tasks.ceph:role: osd.2 2024-09-10T20:27:50.799 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi131.front.sepia.ceph.com 2024-09-10T20:27:50.799 DEBUG:teuthology.orchestra.run.smithi131:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-10T20:27:50.870 INFO:teuthology.orchestra.run.smithi131.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T20:27:50.870 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T20:27:50.870 INFO:teuthology.orchestra.run.smithi131.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T20:27:50.871 INFO:teuthology.orchestra.run.smithi131.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T20:27:50.871 INFO:teuthology.orchestra.run.smithi131.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T20:27:50.871 INFO:teuthology.orchestra.run.smithi131.stdout: = sunit=0 swidth=0 blks 2024-09-10T20:27:50.871 INFO:teuthology.orchestra.run.smithi131.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T20:27:50.871 INFO:teuthology.orchestra.run.smithi131.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T20:27:50.871 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T20:27:50.871 INFO:teuthology.orchestra.run.smithi131.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T20:27:51.357 INFO:teuthology.orchestra.run.smithi131.stdout:Discarding blocks...Done. 2024-09-10T20:27:51.359 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi131.front.sepia.ceph.com -o noatime 2024-09-10T20:27:51.359 DEBUG:teuthology.orchestra.run.smithi131:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-10T20:27:51.499 DEBUG:teuthology.orchestra.run.smithi131:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-10T20:27:51.530 DEBUG:teuthology.orchestra.run.smithi131:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T20:27:51.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T20:27:51.620+0000 7f0ddc9bf640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-10T20:27:51.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T20:27:51.622+0000 7f0ddc9bf640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-10T20:27:51.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T20:27:51.622+0000 7f0ddc9bf640 -1 bdev(0x55b258229800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-10T20:27:51.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T20:27:51.622+0000 7f0ddc9bf640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-10T20:27:54.536 DEBUG:teuthology.orchestra.run.smithi131:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-10T20:27:54.565 DEBUG:teuthology.orchestra.run.smithi131:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T20:27:54.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T20:27:54.652+0000 7f7511351640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-10T20:27:54.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T20:27:54.653+0000 7f7511351640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-10T20:27:54.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T20:27:54.653+0000 7f7511351640 -1 bdev(0x55bc45c49800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-10T20:27:54.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T20:27:54.653+0000 7f7511351640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-10T20:27:57.817 DEBUG:teuthology.orchestra.run.smithi131:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-10T20:27:57.846 DEBUG:teuthology.orchestra.run.smithi131:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T20:27:57.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T20:27:57.933+0000 7f175e94f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-10T20:27:57.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T20:27:57.934+0000 7f175e94f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-10T20:27:57.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T20:27:57.934+0000 7f175e94f640 -1 bdev(0x558c589af800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-10T20:27:57.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T20:27:57.934+0000 7f175e94f640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-10T20:28:00.854 DEBUG:teuthology.orchestra.run.smithi131:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-10T20:28:00.881 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-10T20:28:00.881 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-10T20:28:00.881 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-10T20:28:00.947 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-10T20:28:00.948 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-10T20:28:01.013 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-10T20:28:01.013 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-10T20:28:01.078 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-10T20:28:01.078 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-10T20:28:01.143 INFO:tasks.ceph:Adding keys to all mons... 2024-09-10T20:28:01.143 DEBUG:teuthology.orchestra.run.smithi131:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-10T20:28:01.207 INFO:teuthology.orchestra.run.smithi131.stdout:[mgr.x] 2024-09-10T20:28:01.207 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDFq+BmOXlmDxAA3e/zh02NhIClHx8rwujF3Q== 2024-09-10T20:28:01.207 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.0] 2024-09-10T20:28:01.207 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDHq+BmjV4EJRAAWV++/dCc43pzvKt9YMbvGw== 2024-09-10T20:28:01.207 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.1] 2024-09-10T20:28:01.207 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDKq+BmF23zJhAAXqPGrl1cUUp1MjF/k3TJEA== 2024-09-10T20:28:01.207 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.2] 2024-09-10T20:28:01.207 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDNq+BmpcKyNxAAejckfjGyDh59vzAFdoqOaw== 2024-09-10T20:28:01.208 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T20:28:01.301 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T20:28:01.351 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T20:28:01.404 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T20:28:01.455 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-10T20:28:01.455 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-10T20:28:01.480 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T20:28:01.576 DEBUG:teuthology.orchestra.run.smithi131:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-10T20:28:01.601 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-10T20:28:01.665 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T20:28:01.762 DEBUG:teuthology.orchestra.run.smithi131:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-10T20:28:01.787 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-10T20:28:01.852 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T20:28:01.948 DEBUG:teuthology.orchestra.run.smithi131:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-10T20:28:01.972 DEBUG:teuthology.orchestra.run.smithi131:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-10T20:28:02.027 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-10T20:28:02.027 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-10T20:28:02.027 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-10T20:28:02.070 INFO:tasks.ceph.mon.a:Started 2024-09-10T20:28:02.070 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-10T20:28:02.070 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-10T20:28:02.072 INFO:tasks.ceph.mon.b:Started 2024-09-10T20:28:02.073 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-10T20:28:02.073 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-10T20:28:02.075 INFO:tasks.ceph.mon.c:Started 2024-09-10T20:28:02.075 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph config set mgr mgr_pool false --force 2024-09-10T20:28:08.476 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-10T20:28:08.476 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-10T20:28:08.476 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-10T20:28:08.479 INFO:tasks.ceph.mgr.x:Started 2024-09-10T20:28:08.479 DEBUG:tasks.ceph:set 0 configs 2024-09-10T20:28:08.479 INFO:tasks.ceph:Setting crush tunables to default 2024-09-10T20:28:08.479 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph osd crush tunables default 2024-09-10T20:28:08.659 INFO:teuthology.orchestra.run.smithi131.stderr:adjusted tunables profile to default 2024-09-10T20:28:08.671 INFO:tasks.ceph:check_enable_crimson: False 2024-09-10T20:28:08.671 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-10T20:28:08.671 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-10T20:28:08.671 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-10T20:28:08.696 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-10T20:28:08.696 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-10T20:28:08.760 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-10T20:28:08.760 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-10T20:28:08.825 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph osd new d2261461-51c1-40ff-9ad5-8f4b10800e7e 0 2024-09-10T20:28:09.063 INFO:teuthology.orchestra.run.smithi131.stdout:0 2024-09-10T20:28:09.074 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph osd new 9cb327f8-9ea4-489b-9cdf-1007a983d385 1 2024-09-10T20:28:09.256 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-09-10T20:28:09.268 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph osd new 2a1df90e-2933-4052-a62a-ffa454708bb6 2 2024-09-10T20:28:09.477 INFO:teuthology.orchestra.run.smithi131.stdout:2 2024-09-10T20:28:09.489 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-10T20:28:09.489 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-10T20:28:09.491 INFO:tasks.ceph.osd.0:Started 2024-09-10T20:28:09.491 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-10T20:28:09.492 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-10T20:28:09.493 INFO:tasks.ceph.osd.1:Started 2024-09-10T20:28:09.494 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-10T20:28:09.494 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-10T20:28:09.496 INFO:tasks.ceph.osd.2:Started 2024-09-10T20:28:09.496 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T20:28:09.687 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:28:09.687 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":5,"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","created":"2024-09-10T20:28:08.371455+0000","modified":"2024-09-10T20:28:09.473799+0000","last_up_change":"0.000000","last_in_change":"2024-09-10T20:28:09.473799+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d2261461-51c1-40ff-9ad5-8f4b10800e7e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9cb327f8-9ea4-489b-9cdf-1007a983d385","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2a1df90e-2933-4052-a62a-ffa454708bb6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T20:28:09.698 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-10T20:28:09.698 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-10T20:28:10.000 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T20:28:10.158 INFO:teuthology.misc.health.smithi131.stdout: 2024-09-10T20:28:10.158 INFO:teuthology.misc.health.smithi131.stdout:{"epoch":5,"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","created":"2024-09-10T20:28:08.371455+0000","modified":"2024-09-10T20:28:09.473799+0000","last_up_change":"0.000000","last_in_change":"2024-09-10T20:28:09.473799+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d2261461-51c1-40ff-9ad5-8f4b10800e7e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9cb327f8-9ea4-489b-9cdf-1007a983d385","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2a1df90e-2933-4052-a62a-ffa454708bb6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T20:28:10.168 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-09-10T20:28:10.735 INFO:tasks.ceph.osd.2.smithi131.stderr:2024-09-10T20:28:10.731+0000 7f69d740d640 -1 Falling back to public interface 2024-09-10T20:28:10.967 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:28:10.963+0000 7fbe6e420640 -1 Falling back to public interface 2024-09-10T20:28:10.967 INFO:tasks.ceph.osd.1.smithi131.stderr:2024-09-10T20:28:10.964+0000 7fe14ae18640 -1 Falling back to public interface 2024-09-10T20:28:10.968 INFO:tasks.ceph.osd.1.smithi131.stderr:2024-09-10T20:28:10.964+0000 7fe149793640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6813/0: (98) Address already in use 2024-09-10T20:28:13.821 INFO:tasks.ceph.mgr.x.smithi131.stderr:2024-09-10T20:28:13.816+0000 7fb977c0e640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-10T20:28:13.822 INFO:tasks.ceph.mgr.x.smithi131.stderr:2024-09-10T20:28:13.817+0000 7fb977c0e640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-10T20:28:14.578 INFO:tasks.ceph.osd.1.smithi131.stderr:2024-09-10T20:28:14.573+0000 7fe14ae18640 -1 osd.1 0 log_to_monitors true 2024-09-10T20:28:14.615 INFO:tasks.ceph.osd.2.smithi131.stderr:2024-09-10T20:28:14.611+0000 7f69d740d640 -1 osd.2 0 log_to_monitors true 2024-09-10T20:28:15.290 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:28:15.285+0000 7fbe6e420640 -1 osd.0 0 log_to_monitors true 2024-09-10T20:28:16.471 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T20:28:16.593 INFO:tasks.ceph.osd.2.smithi131.stderr:2024-09-10T20:28:16.588+0000 7f69d2d1e640 -1 osd.2 0 waiting for initial osdmap 2024-09-10T20:28:16.611 INFO:tasks.ceph.osd.1.smithi131.stderr:2024-09-10T20:28:16.606+0000 7fe146f8e640 -1 osd.1 0 waiting for initial osdmap 2024-09-10T20:28:16.614 INFO:tasks.ceph.osd.2.smithi131.stderr:2024-09-10T20:28:16.610+0000 7f69cab3d640 -1 osd.2 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T20:28:16.628 INFO:tasks.ceph.osd.1.smithi131.stderr:2024-09-10T20:28:16.624+0000 7fe13e599640 -1 osd.1 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T20:28:16.773 INFO:teuthology.misc.health.smithi131.stdout: 2024-09-10T20:28:16.773 INFO:teuthology.misc.health.smithi131.stdout:{"epoch":7,"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","created":"2024-09-10T20:28:08.371455+0000","modified":"2024-09-10T20:28:15.861686+0000","last_up_change":"0.000000","last_in_change":"2024-09-10T20:28:09.473799+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d2261461-51c1-40ff-9ad5-8f4b10800e7e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9cb327f8-9ea4-489b-9cdf-1007a983d385","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2a1df90e-2933-4052-a62a-ffa454708bb6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T20:28:16.784 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-09-10T20:28:17.469 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:28:17.465+0000 7fbe69d11640 -1 osd.0 0 waiting for initial osdmap 2024-09-10T20:28:17.479 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:28:17.475+0000 7fbe61b30640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T20:28:23.087 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T20:28:23.372 INFO:teuthology.misc.health.smithi131.stdout: 2024-09-10T20:28:23.372 INFO:teuthology.misc.health.smithi131.stdout:{"epoch":10,"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","created":"2024-09-10T20:28:08.371455+0000","modified":"2024-09-10T20:28:19.876766+0000","last_up_change":"2024-09-10T20:28:18.467829+0000","last_in_change":"2024-09-10T20:28:09.473799+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d2261461-51c1-40ff-9ad5-8f4b10800e7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":571540218},{"type":"v1","addr":"172.21.15.131:6809","nonce":571540218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":571540218},{"type":"v1","addr":"172.21.15.131:6811","nonce":571540218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6815","nonce":571540218},{"type":"v1","addr":"172.21.15.131:6817","nonce":571540218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":571540218},{"type":"v1","addr":"172.21.15.131:6813","nonce":571540218}]},"public_addr":"172.21.15.131:6809/571540218","cluster_addr":"172.21.15.131:6811/571540218","heartbeat_back_addr":"172.21.15.131:6817/571540218","heartbeat_front_addr":"172.21.15.131:6813/571540218","state":["exists","up"]},{"osd":1,"uuid":"9cb327f8-9ea4-489b-9cdf-1007a983d385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":454060533},{"type":"v1","addr":"172.21.15.131:6816","nonce":454060533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":454060533},{"type":"v1","addr":"172.21.15.131:6819","nonce":454060533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":454060533},{"type":"v1","addr":"172.21.15.131:6823","nonce":454060533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":454060533},{"type":"v1","addr":"172.21.15.131:6821","nonce":454060533}]},"public_addr":"172.21.15.131:6816/454060533","cluster_addr":"172.21.15.131:6819/454060533","heartbeat_back_addr":"172.21.15.131:6823/454060533","heartbeat_front_addr":"172.21.15.131:6821/454060533","state":["exists","up"]},{"osd":2,"uuid":"2a1df90e-2933-4052-a62a-ffa454708bb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2084549604},{"type":"v1","addr":"172.21.15.131:6801","nonce":2084549604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2084549604},{"type":"v1","addr":"172.21.15.131:6803","nonce":2084549604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2084549604},{"type":"v1","addr":"172.21.15.131:6807","nonce":2084549604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2084549604},{"type":"v1","addr":"172.21.15.131:6805","nonce":2084549604}]},"public_addr":"172.21.15.131:6801/2084549604","cluster_addr":"172.21.15.131:6803/2084549604","heartbeat_back_addr":"172.21.15.131:6807/2084549604","heartbeat_front_addr":"172.21.15.131:6805/2084549604","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:28:16.289792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:28:15.574958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:28:15.599924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T20:28:23.383 DEBUG:teuthology.misc:3 of 3 OSDs are up 2024-09-10T20:28:23.383 INFO:tasks.ceph:Creating RBD pool 2024-09-10T20:28:23.383 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-10T20:28:24.486 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'rbd' created 2024-09-10T20:28:24.504 DEBUG:teuthology.orchestra.run.smithi131:> rbd --cluster ceph pool init rbd 2024-09-10T20:28:27.516 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-10T20:28:27.517 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-10T20:28:27.518 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-10T20:28:27.807 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:28:27.820 INFO:teuthology.orchestra.run.smithi131.stdout:[{"version":5,"timestamp":"2024-09-10T20:28:17.467007+0000","name":"","changes":[{"name":"osd.0/osd_mclock_max_capacity_iops_ssd","new_value":"8722.211307"}]}] 2024-09-10T20:28:27.820 INFO:tasks.ceph_manager:config epoch is 5 2024-09-10T20:28:27.820 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T20:28:27.820 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-10T20:28:27.821 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-10T20:28:28.153 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:28:28.164 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":4,"flags":0,"active_gid":4107,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1261510591},{"type":"v1","addr":"172.21.15.131:6825","nonce":1261510591}]},"active_addr":"172.21.15.131:6825/1261510591","active_change":"2024-09-10T20:28:12.803631+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":1777840032}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":1104818044}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":3053975191}]}]} 2024-09-10T20:28:28.166 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-10T20:28:28.167 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-10T20:28:28.167 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T20:28:28.450 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:28:28.451 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":14,"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","created":"2024-09-10T20:28:08.371455+0000","modified":"2024-09-10T20:28:27.481449+0000","last_up_change":"2024-09-10T20:28:18.467829+0000","last_in_change":"2024-09-10T20:28:09.473799+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-10T20:28:23.669739+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d2261461-51c1-40ff-9ad5-8f4b10800e7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":571540218},{"type":"v1","addr":"172.21.15.131:6809","nonce":571540218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":571540218},{"type":"v1","addr":"172.21.15.131:6811","nonce":571540218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6815","nonce":571540218},{"type":"v1","addr":"172.21.15.131:6817","nonce":571540218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":571540218},{"type":"v1","addr":"172.21.15.131:6813","nonce":571540218}]},"public_addr":"172.21.15.131:6809/571540218","cluster_addr":"172.21.15.131:6811/571540218","heartbeat_back_addr":"172.21.15.131:6817/571540218","heartbeat_front_addr":"172.21.15.131:6813/571540218","state":["exists","up"]},{"osd":1,"uuid":"9cb327f8-9ea4-489b-9cdf-1007a983d385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":454060533},{"type":"v1","addr":"172.21.15.131:6816","nonce":454060533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":454060533},{"type":"v1","addr":"172.21.15.131:6819","nonce":454060533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":454060533},{"type":"v1","addr":"172.21.15.131:6823","nonce":454060533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":454060533},{"type":"v1","addr":"172.21.15.131:6821","nonce":454060533}]},"public_addr":"172.21.15.131:6816/454060533","cluster_addr":"172.21.15.131:6819/454060533","heartbeat_back_addr":"172.21.15.131:6823/454060533","heartbeat_front_addr":"172.21.15.131:6821/454060533","state":["exists","up"]},{"osd":2,"uuid":"2a1df90e-2933-4052-a62a-ffa454708bb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2084549604},{"type":"v1","addr":"172.21.15.131:6801","nonce":2084549604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2084549604},{"type":"v1","addr":"172.21.15.131:6803","nonce":2084549604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2084549604},{"type":"v1","addr":"172.21.15.131:6807","nonce":2084549604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2084549604},{"type":"v1","addr":"172.21.15.131:6805","nonce":2084549604}]},"public_addr":"172.21.15.131:6801/2084549604","cluster_addr":"172.21.15.131:6803/2084549604","heartbeat_back_addr":"172.21.15.131:6807/2084549604","heartbeat_front_addr":"172.21.15.131:6805/2084549604","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:28:16.289792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:28:15.574958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:28:15.599924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T20:28:28.463 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-10T20:28:28.463 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T20:28:28.747 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:28:28.748 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":14,"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","created":"2024-09-10T20:28:08.371455+0000","modified":"2024-09-10T20:28:27.481449+0000","last_up_change":"2024-09-10T20:28:18.467829+0000","last_in_change":"2024-09-10T20:28:09.473799+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-10T20:28:23.669739+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d2261461-51c1-40ff-9ad5-8f4b10800e7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":571540218},{"type":"v1","addr":"172.21.15.131:6809","nonce":571540218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":571540218},{"type":"v1","addr":"172.21.15.131:6811","nonce":571540218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6815","nonce":571540218},{"type":"v1","addr":"172.21.15.131:6817","nonce":571540218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":571540218},{"type":"v1","addr":"172.21.15.131:6813","nonce":571540218}]},"public_addr":"172.21.15.131:6809/571540218","cluster_addr":"172.21.15.131:6811/571540218","heartbeat_back_addr":"172.21.15.131:6817/571540218","heartbeat_front_addr":"172.21.15.131:6813/571540218","state":["exists","up"]},{"osd":1,"uuid":"9cb327f8-9ea4-489b-9cdf-1007a983d385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":454060533},{"type":"v1","addr":"172.21.15.131:6816","nonce":454060533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":454060533},{"type":"v1","addr":"172.21.15.131:6819","nonce":454060533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":454060533},{"type":"v1","addr":"172.21.15.131:6823","nonce":454060533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":454060533},{"type":"v1","addr":"172.21.15.131:6821","nonce":454060533}]},"public_addr":"172.21.15.131:6816/454060533","cluster_addr":"172.21.15.131:6819/454060533","heartbeat_back_addr":"172.21.15.131:6823/454060533","heartbeat_front_addr":"172.21.15.131:6821/454060533","state":["exists","up"]},{"osd":2,"uuid":"2a1df90e-2933-4052-a62a-ffa454708bb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2084549604},{"type":"v1","addr":"172.21.15.131:6801","nonce":2084549604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2084549604},{"type":"v1","addr":"172.21.15.131:6803","nonce":2084549604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2084549604},{"type":"v1","addr":"172.21.15.131:6807","nonce":2084549604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2084549604},{"type":"v1","addr":"172.21.15.131:6805","nonce":2084549604}]},"public_addr":"172.21.15.131:6801/2084549604","cluster_addr":"172.21.15.131:6803/2084549604","heartbeat_back_addr":"172.21.15.131:6807/2084549604","heartbeat_front_addr":"172.21.15.131:6805/2084549604","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:28:16.289792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:28:15.574958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:28:15.599924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T20:28:28.761 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-10T20:28:28.761 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-10T20:28:28.762 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-10T20:28:28.899 INFO:teuthology.orchestra.run.smithi131.stdout:34359738373 2024-09-10T20:28:28.899 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T20:28:28.903 INFO:teuthology.orchestra.run.smithi131.stdout:38654705670 2024-09-10T20:28:28.903 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T20:28:28.903 INFO:teuthology.orchestra.run.smithi131.stdout:34359738373 2024-09-10T20:28:28.904 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T20:28:29.198 INFO:teuthology.orchestra.run.smithi131.stdout:34359738371 2024-09-10T20:28:29.211 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738373 got 34359738371 for osd.2 2024-09-10T20:28:29.244 INFO:teuthology.orchestra.run.smithi131.stdout:34359738371 2024-09-10T20:28:29.253 INFO:teuthology.orchestra.run.smithi131.stdout:38654705668 2024-09-10T20:28:29.257 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738373 got 34359738371 for osd.1 2024-09-10T20:28:29.267 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705668 for osd.0 2024-09-10T20:28:30.213 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T20:28:30.258 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T20:28:30.268 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T20:28:30.525 INFO:teuthology.orchestra.run.smithi131.stdout:34359738372 2024-09-10T20:28:30.538 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738373 got 34359738372 for osd.2 2024-09-10T20:28:30.565 INFO:teuthology.orchestra.run.smithi131.stdout:34359738372 2024-09-10T20:28:30.578 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738373 got 34359738372 for osd.1 2024-09-10T20:28:30.601 INFO:teuthology.orchestra.run.smithi131.stdout:38654705668 2024-09-10T20:28:30.614 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705668 for osd.0 2024-09-10T20:28:31.539 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T20:28:31.579 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T20:28:31.615 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T20:28:31.854 INFO:teuthology.orchestra.run.smithi131.stdout:34359738373 2024-09-10T20:28:31.867 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738373 got 34359738373 for osd.2 2024-09-10T20:28:31.867 DEBUG:teuthology.parallel:result is None 2024-09-10T20:28:31.897 INFO:teuthology.orchestra.run.smithi131.stdout:34359738373 2024-09-10T20:28:31.909 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738373 got 34359738373 for osd.1 2024-09-10T20:28:31.910 DEBUG:teuthology.parallel:result is None 2024-09-10T20:28:31.928 INFO:teuthology.orchestra.run.smithi131.stdout:38654705670 2024-09-10T20:28:31.941 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705670 for osd.0 2024-09-10T20:28:31.941 DEBUG:teuthology.parallel:result is None 2024-09-10T20:28:31.941 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T20:28:31.941 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:28:32.267 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:28:32.267 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:28:32.279 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-09-10T20:28:32.155846+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81464,"kb_used_data":368,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034056,"statfs":{"total":289910292480,"available":289826873344,"internally_reserved":0,"allocated":376832,"data_stored":97409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.344321"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T20:28:27.488224+0000","last_change":"2024-09-10T20:28:27.488224+0000","last_active":"2024-09-10T20:28:27.488224+0000","last_peered":"2024-09-10T20:28:27.488224+0000","last_clean":"2024-09-10T20:28:27.488224+0000","last_became_active":"2024-09-10T20:28:25.485918+0000","last_became_peered":"2024-09-10T20:28:25.485918+0000","last_unstale":"2024-09-10T20:28:27.488224+0000","last_undegraded":"2024-09-10T20:28:27.488224+0000","last_fullsized":"2024-09-10T20:28:27.488224+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:14:12.771955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T20:28:27.487827+0000","last_change":"2024-09-10T20:28:27.487827+0000","last_active":"2024-09-10T20:28:27.487827+0000","last_peered":"2024-09-10T20:28:27.487827+0000","last_clean":"2024-09-10T20:28:27.487827+0000","last_became_active":"2024-09-10T20:28:25.489162+0000","last_became_peered":"2024-09-10T20:28:25.489162+0000","last_unstale":"2024-09-10T20:28:27.487827+0000","last_undegraded":"2024-09-10T20:28:27.487827+0000","last_fullsized":"2024-09-10T20:28:27.487827+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:01:18.730084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T20:28:27.526743+0000","last_change":"2024-09-10T20:28:27.526743+0000","last_active":"2024-09-10T20:28:27.526743+0000","last_peered":"2024-09-10T20:28:27.526743+0000","last_clean":"2024-09-10T20:28:27.526743+0000","last_became_active":"2024-09-10T20:28:25.490897+0000","last_became_peered":"2024-09-10T20:28:25.490897+0000","last_unstale":"2024-09-10T20:28:27.526743+0000","last_undegraded":"2024-09-10T20:28:27.526743+0000","last_fullsized":"2024-09-10T20:28:27.526743+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:54:36.134110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T20:28:27.487945+0000","last_change":"2024-09-10T20:28:27.487945+0000","last_active":"2024-09-10T20:28:27.487945+0000","last_peered":"2024-09-10T20:28:27.487945+0000","last_clean":"2024-09-10T20:28:27.487945+0000","last_became_active":"2024-09-10T20:28:25.489104+0000","last_became_peered":"2024-09-10T20:28:25.489104+0000","last_unstale":"2024-09-10T20:28:27.487945+0000","last_undegraded":"2024-09-10T20:28:27.487945+0000","last_fullsized":"2024-09-10T20:28:27.487945+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:30.886798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"12'1","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T20:28:27.487624+0000","last_change":"2024-09-10T20:28:27.487624+0000","last_active":"2024-09-10T20:28:27.487624+0000","last_peered":"2024-09-10T20:28:27.487624+0000","last_clean":"2024-09-10T20:28:27.487624+0000","last_became_active":"2024-09-10T20:28:25.486015+0000","last_became_peered":"2024-09-10T20:28:25.486015+0000","last_unstale":"2024-09-10T20:28:27.487624+0000","last_undegraded":"2024-09-10T20:28:27.487624+0000","last_fullsized":"2024-09-10T20:28:27.487624+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:03:12.564455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"14'2","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T20:28:27.491676+0000","last_change":"2024-09-10T20:28:27.487461+0000","last_active":"2024-09-10T20:28:27.491676+0000","last_peered":"2024-09-10T20:28:27.491676+0000","last_clean":"2024-09-10T20:28:27.491676+0000","last_became_active":"2024-09-10T20:28:25.491325+0000","last_became_peered":"2024-09-10T20:28:25.491325+0000","last_unstale":"2024-09-10T20:28:27.491676+0000","last_undegraded":"2024-09-10T20:28:27.491676+0000","last_fullsized":"2024-09-10T20:28:27.491676+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:53.879060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T20:28:27.526736+0000","last_change":"2024-09-10T20:28:27.526736+0000","last_active":"2024-09-10T20:28:27.526736+0000","last_peered":"2024-09-10T20:28:27.526736+0000","last_clean":"2024-09-10T20:28:27.526736+0000","last_became_active":"2024-09-10T20:28:25.489180+0000","last_became_peered":"2024-09-10T20:28:25.489180+0000","last_unstale":"2024-09-10T20:28:27.526736+0000","last_undegraded":"2024-09-10T20:28:27.526736+0000","last_fullsized":"2024-09-10T20:28:27.526736+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:59:00.063761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T20:28:27.487579+0000","last_change":"2024-09-10T20:28:27.487579+0000","last_active":"2024-09-10T20:28:27.487579+0000","last_peered":"2024-09-10T20:28:27.487579+0000","last_clean":"2024-09-10T20:28:27.487579+0000","last_became_active":"2024-09-10T20:28:25.489055+0000","last_became_peered":"2024-09-10T20:28:25.489055+0000","last_unstale":"2024-09-10T20:28:27.487579+0000","last_undegraded":"2024-09-10T20:28:27.487579+0000","last_fullsized":"2024-09-10T20:28:27.487579+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:12:00.009762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":9,"seq":38654705670,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27156,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344684,"statfs":{"total":96636764160,"available":96608956416,"internally_reserved":0,"allocated":126976,"data_stored":32476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":8,"seq":34359738373,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":122880,"data_stored":32457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738373,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27156,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344684,"statfs":{"total":96636764160,"available":96608956416,"internally_reserved":0,"allocated":126976,"data_stored":32476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:28:32.280 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:28:32.565 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:28:32.566 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:28:32.578 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-09-10T20:28:32.155846+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81464,"kb_used_data":368,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034056,"statfs":{"total":289910292480,"available":289826873344,"internally_reserved":0,"allocated":376832,"data_stored":97409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.344321"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T20:28:27.488224+0000","last_change":"2024-09-10T20:28:27.488224+0000","last_active":"2024-09-10T20:28:27.488224+0000","last_peered":"2024-09-10T20:28:27.488224+0000","last_clean":"2024-09-10T20:28:27.488224+0000","last_became_active":"2024-09-10T20:28:25.485918+0000","last_became_peered":"2024-09-10T20:28:25.485918+0000","last_unstale":"2024-09-10T20:28:27.488224+0000","last_undegraded":"2024-09-10T20:28:27.488224+0000","last_fullsized":"2024-09-10T20:28:27.488224+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:14:12.771955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T20:28:27.487827+0000","last_change":"2024-09-10T20:28:27.487827+0000","last_active":"2024-09-10T20:28:27.487827+0000","last_peered":"2024-09-10T20:28:27.487827+0000","last_clean":"2024-09-10T20:28:27.487827+0000","last_became_active":"2024-09-10T20:28:25.489162+0000","last_became_peered":"2024-09-10T20:28:25.489162+0000","last_unstale":"2024-09-10T20:28:27.487827+0000","last_undegraded":"2024-09-10T20:28:27.487827+0000","last_fullsized":"2024-09-10T20:28:27.487827+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:01:18.730084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T20:28:27.526743+0000","last_change":"2024-09-10T20:28:27.526743+0000","last_active":"2024-09-10T20:28:27.526743+0000","last_peered":"2024-09-10T20:28:27.526743+0000","last_clean":"2024-09-10T20:28:27.526743+0000","last_became_active":"2024-09-10T20:28:25.490897+0000","last_became_peered":"2024-09-10T20:28:25.490897+0000","last_unstale":"2024-09-10T20:28:27.526743+0000","last_undegraded":"2024-09-10T20:28:27.526743+0000","last_fullsized":"2024-09-10T20:28:27.526743+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:54:36.134110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T20:28:27.487945+0000","last_change":"2024-09-10T20:28:27.487945+0000","last_active":"2024-09-10T20:28:27.487945+0000","last_peered":"2024-09-10T20:28:27.487945+0000","last_clean":"2024-09-10T20:28:27.487945+0000","last_became_active":"2024-09-10T20:28:25.489104+0000","last_became_peered":"2024-09-10T20:28:25.489104+0000","last_unstale":"2024-09-10T20:28:27.487945+0000","last_undegraded":"2024-09-10T20:28:27.487945+0000","last_fullsized":"2024-09-10T20:28:27.487945+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:30.886798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"12'1","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T20:28:27.487624+0000","last_change":"2024-09-10T20:28:27.487624+0000","last_active":"2024-09-10T20:28:27.487624+0000","last_peered":"2024-09-10T20:28:27.487624+0000","last_clean":"2024-09-10T20:28:27.487624+0000","last_became_active":"2024-09-10T20:28:25.486015+0000","last_became_peered":"2024-09-10T20:28:25.486015+0000","last_unstale":"2024-09-10T20:28:27.487624+0000","last_undegraded":"2024-09-10T20:28:27.487624+0000","last_fullsized":"2024-09-10T20:28:27.487624+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:03:12.564455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"14'2","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T20:28:27.491676+0000","last_change":"2024-09-10T20:28:27.487461+0000","last_active":"2024-09-10T20:28:27.491676+0000","last_peered":"2024-09-10T20:28:27.491676+0000","last_clean":"2024-09-10T20:28:27.491676+0000","last_became_active":"2024-09-10T20:28:25.491325+0000","last_became_peered":"2024-09-10T20:28:25.491325+0000","last_unstale":"2024-09-10T20:28:27.491676+0000","last_undegraded":"2024-09-10T20:28:27.491676+0000","last_fullsized":"2024-09-10T20:28:27.491676+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:53.879060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T20:28:27.526736+0000","last_change":"2024-09-10T20:28:27.526736+0000","last_active":"2024-09-10T20:28:27.526736+0000","last_peered":"2024-09-10T20:28:27.526736+0000","last_clean":"2024-09-10T20:28:27.526736+0000","last_became_active":"2024-09-10T20:28:25.489180+0000","last_became_peered":"2024-09-10T20:28:25.489180+0000","last_unstale":"2024-09-10T20:28:27.526736+0000","last_undegraded":"2024-09-10T20:28:27.526736+0000","last_fullsized":"2024-09-10T20:28:27.526736+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:59:00.063761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T20:28:27.487579+0000","last_change":"2024-09-10T20:28:27.487579+0000","last_active":"2024-09-10T20:28:27.487579+0000","last_peered":"2024-09-10T20:28:27.487579+0000","last_clean":"2024-09-10T20:28:27.487579+0000","last_became_active":"2024-09-10T20:28:25.489055+0000","last_became_peered":"2024-09-10T20:28:25.489055+0000","last_unstale":"2024-09-10T20:28:27.487579+0000","last_undegraded":"2024-09-10T20:28:27.487579+0000","last_fullsized":"2024-09-10T20:28:27.487579+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:12:00.009762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":9,"seq":38654705670,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27156,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344684,"statfs":{"total":96636764160,"available":96608956416,"internally_reserved":0,"allocated":126976,"data_stored":32476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":8,"seq":34359738373,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":122880,"data_stored":32457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738373,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27156,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344684,"statfs":{"total":96636764160,"available":96608956416,"internally_reserved":0,"allocated":126976,"data_stored":32476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:28:32.578 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T20:28:32.578 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T20:28:32.578 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-10T20:28:32.579 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T20:28:32.894 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:28:32.894 INFO:teuthology.orchestra.run.smithi131.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T20:28:32.907 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-10T20:28:32.907 INFO:teuthology.run_tasks:Running task rep_lost_unfound_delete... 2024-09-10T20:28:32.943 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T20:28:33.228 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:28:33.228 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":15,"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","created":"2024-09-10T20:28:08.371455+0000","modified":"2024-09-10T20:28:32.152163+0000","last_up_change":"2024-09-10T20:28:18.467829+0000","last_in_change":"2024-09-10T20:28:09.473799+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-10T20:28:23.669739+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d2261461-51c1-40ff-9ad5-8f4b10800e7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":571540218},{"type":"v1","addr":"172.21.15.131:6809","nonce":571540218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":571540218},{"type":"v1","addr":"172.21.15.131:6811","nonce":571540218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6815","nonce":571540218},{"type":"v1","addr":"172.21.15.131:6817","nonce":571540218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":571540218},{"type":"v1","addr":"172.21.15.131:6813","nonce":571540218}]},"public_addr":"172.21.15.131:6809/571540218","cluster_addr":"172.21.15.131:6811/571540218","heartbeat_back_addr":"172.21.15.131:6817/571540218","heartbeat_front_addr":"172.21.15.131:6813/571540218","state":["exists","up"]},{"osd":1,"uuid":"9cb327f8-9ea4-489b-9cdf-1007a983d385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":454060533},{"type":"v1","addr":"172.21.15.131:6816","nonce":454060533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":454060533},{"type":"v1","addr":"172.21.15.131:6819","nonce":454060533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":454060533},{"type":"v1","addr":"172.21.15.131:6823","nonce":454060533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":454060533},{"type":"v1","addr":"172.21.15.131:6821","nonce":454060533}]},"public_addr":"172.21.15.131:6816/454060533","cluster_addr":"172.21.15.131:6819/454060533","heartbeat_back_addr":"172.21.15.131:6823/454060533","heartbeat_front_addr":"172.21.15.131:6821/454060533","state":["exists","up"]},{"osd":2,"uuid":"2a1df90e-2933-4052-a62a-ffa454708bb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2084549604},{"type":"v1","addr":"172.21.15.131:6801","nonce":2084549604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2084549604},{"type":"v1","addr":"172.21.15.131:6803","nonce":2084549604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2084549604},{"type":"v1","addr":"172.21.15.131:6807","nonce":2084549604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2084549604},{"type":"v1","addr":"172.21.15.131:6805","nonce":2084549604}]},"public_addr":"172.21.15.131:6801/2084549604","cluster_addr":"172.21.15.131:6803/2084549604","heartbeat_back_addr":"172.21.15.131:6807/2084549604","heartbeat_front_addr":"172.21.15.131:6805/2084549604","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:28:16.289792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:28:15.574958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:28:15.599924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T20:28:33.241 INFO:tasks.rep_lost_unfound_delete.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-10T20:28:23.669739+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 14, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T20:28:33.241 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-09-10T20:28:33.528 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 8 2024-09-10T20:28:33.541 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-10T20:28:33.824 INFO:teuthology.orchestra.run.smithi131.stdout:epoch 15 2024-09-10T20:28:33.825 INFO:teuthology.orchestra.run.smithi131.stdout:fsid 3053cb46-0512-45f5-9707-4eb48df5a665 2024-09-10T20:28:33.825 INFO:teuthology.orchestra.run.smithi131.stdout:created 2024-09-10T20:28:08.371455+0000 2024-09-10T20:28:33.825 INFO:teuthology.orchestra.run.smithi131.stdout:modified 2024-09-10T20:28:32.152163+0000 2024-09-10T20:28:33.825 INFO:teuthology.orchestra.run.smithi131.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-10T20:28:33.825 INFO:teuthology.orchestra.run.smithi131.stdout:crush_version 5 2024-09-10T20:28:33.825 INFO:teuthology.orchestra.run.smithi131.stdout:full_ratio 0.95 2024-09-10T20:28:33.825 INFO:teuthology.orchestra.run.smithi131.stdout:backfillfull_ratio 0.9 2024-09-10T20:28:33.825 INFO:teuthology.orchestra.run.smithi131.stdout:nearfull_ratio 0.85 2024-09-10T20:28:33.825 INFO:teuthology.orchestra.run.smithi131.stdout:require_min_compat_client luminous 2024-09-10T20:28:33.825 INFO:teuthology.orchestra.run.smithi131.stdout:min_compat_client jewel 2024-09-10T20:28:33.825 INFO:teuthology.orchestra.run.smithi131.stdout:require_osd_release squid 2024-09-10T20:28:33.826 INFO:teuthology.orchestra.run.smithi131.stdout:stretch_mode_enabled false 2024-09-10T20:28:33.826 INFO:teuthology.orchestra.run.smithi131.stdout:pool 1 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 14 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 1.88 2024-09-10T20:28:33.826 INFO:teuthology.orchestra.run.smithi131.stdout:max_osd 3 2024-09-10T20:28:33.826 INFO:teuthology.orchestra.run.smithi131.stdout:osd.0 up in weight 1 up_from 9 up_thru 11 down_at 0 last_clean_interval [0,0) [v2:172.21.15.131:6808/571540218,v1:172.21.15.131:6809/571540218] [v2:172.21.15.131:6810/571540218,v1:172.21.15.131:6811/571540218] exists,up d2261461-51c1-40ff-9ad5-8f4b10800e7e 2024-09-10T20:28:33.826 INFO:teuthology.orchestra.run.smithi131.stdout:osd.1 up in weight 1 up_from 8 up_thru 11 down_at 0 last_clean_interval [0,0) [v2:172.21.15.131:6814/454060533,v1:172.21.15.131:6816/454060533] [v2:172.21.15.131:6818/454060533,v1:172.21.15.131:6819/454060533] exists,up 9cb327f8-9ea4-489b-9cdf-1007a983d385 2024-09-10T20:28:33.826 INFO:teuthology.orchestra.run.smithi131.stdout:osd.2 up in weight 1 up_from 8 up_thru 11 down_at 0 last_clean_interval [0,0) [v2:172.21.15.131:6800/2084549604,v1:172.21.15.131:6801/2084549604] [v2:172.21.15.131:6802/2084549604,v1:172.21.15.131:6803/2084549604] exists,up 2a1df90e-2933-4052-a62a-ffa454708bb6 2024-09-10T20:28:33.837 INFO:tasks.rep_lost_unfound_delete.ceph_manager:['osd.0 up in weight 1 up_from 9 up_thru 11 down_at 0 last_clean_interval [0,0) [v2:172.21.15.131:6808/571540218,v1:172.21.15.131:6809/571540218] [v2:172.21.15.131:6810/571540218,v1:172.21.15.131:6811/571540218] exists,up d2261461-51c1-40ff-9ad5-8f4b10800e7e', 'osd.1 up in weight 1 up_from 8 up_thru 11 down_at 0 last_clean_interval [0,0) [v2:172.21.15.131:6814/454060533,v1:172.21.15.131:6816/454060533] [v2:172.21.15.131:6818/454060533,v1:172.21.15.131:6819/454060533] exists,up 9cb327f8-9ea4-489b-9cdf-1007a983d385', 'osd.2 up in weight 1 up_from 8 up_thru 11 down_at 0 last_clean_interval [0,0) [v2:172.21.15.131:6800/2084549604,v1:172.21.15.131:6801/2084549604] [v2:172.21.15.131:6802/2084549604,v1:172.21.15.131:6803/2084549604] exists,up 2a1df90e-2933-4052-a62a-ffa454708bb6'] 2024-09-10T20:28:33.838 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-10T20:28:33.838 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-10T20:28:33.839 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-10T20:28:33.977 INFO:teuthology.orchestra.run.smithi131.stdout:38654705672 2024-09-10T20:28:33.978 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T20:28:33.979 INFO:teuthology.orchestra.run.smithi131.stdout:34359738375 2024-09-10T20:28:33.979 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T20:28:33.980 INFO:teuthology.orchestra.run.smithi131.stdout:34359738375 2024-09-10T20:28:33.980 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T20:28:34.323 INFO:teuthology.orchestra.run.smithi131.stdout:38654705670 2024-09-10T20:28:34.339 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 38654705672 got 38654705670 for osd.0 2024-09-10T20:28:34.351 INFO:teuthology.orchestra.run.smithi131.stdout:34359738373 2024-09-10T20:28:34.354 INFO:teuthology.orchestra.run.smithi131.stdout:34359738373 2024-09-10T20:28:34.365 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738375 got 34359738373 for osd.2 2024-09-10T20:28:34.367 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738375 got 34359738373 for osd.1 2024-09-10T20:28:35.340 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T20:28:35.365 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T20:28:35.368 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T20:28:35.653 INFO:teuthology.orchestra.run.smithi131.stdout:38654705672 2024-09-10T20:28:35.667 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 38654705672 got 38654705672 for osd.0 2024-09-10T20:28:35.667 DEBUG:teuthology.parallel:result is None 2024-09-10T20:28:35.689 INFO:teuthology.orchestra.run.smithi131.stdout:34359738375 2024-09-10T20:28:35.691 INFO:teuthology.orchestra.run.smithi131.stdout:34359738375 2024-09-10T20:28:35.704 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738375 got 34359738375 for osd.2 2024-09-10T20:28:35.704 DEBUG:teuthology.parallel:result is None 2024-09-10T20:28:35.704 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738375 got 34359738375 for osd.1 2024-09-10T20:28:35.705 DEBUG:teuthology.parallel:result is None 2024-09-10T20:28:35.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for clean 2024-09-10T20:28:35.705 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:28:36.023 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:28:36.023 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:28:36.035 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-10T20:28:34.812519+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81476,"kb_used_data":392,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034044,"statfs":{"total":289910292480,"available":289826861056,"internally_reserved":0,"allocated":401408,"data_stored":105839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000994"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T20:28:32.157539+0000","last_change":"2024-09-10T20:28:27.488224+0000","last_active":"2024-09-10T20:28:32.157539+0000","last_peered":"2024-09-10T20:28:32.157539+0000","last_clean":"2024-09-10T20:28:32.157539+0000","last_became_active":"2024-09-10T20:28:25.485918+0000","last_became_peered":"2024-09-10T20:28:25.485918+0000","last_unstale":"2024-09-10T20:28:32.157539+0000","last_undegraded":"2024-09-10T20:28:32.157539+0000","last_fullsized":"2024-09-10T20:28:32.157539+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:14:12.771955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T20:28:32.157274+0000","last_change":"2024-09-10T20:28:27.487827+0000","last_active":"2024-09-10T20:28:32.157274+0000","last_peered":"2024-09-10T20:28:32.157274+0000","last_clean":"2024-09-10T20:28:32.157274+0000","last_became_active":"2024-09-10T20:28:25.489162+0000","last_became_peered":"2024-09-10T20:28:25.489162+0000","last_unstale":"2024-09-10T20:28:32.157274+0000","last_undegraded":"2024-09-10T20:28:32.157274+0000","last_fullsized":"2024-09-10T20:28:32.157274+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:01:18.730084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T20:28:32.157898+0000","last_change":"2024-09-10T20:28:27.526743+0000","last_active":"2024-09-10T20:28:32.157898+0000","last_peered":"2024-09-10T20:28:32.157898+0000","last_clean":"2024-09-10T20:28:32.157898+0000","last_became_active":"2024-09-10T20:28:25.490897+0000","last_became_peered":"2024-09-10T20:28:25.490897+0000","last_unstale":"2024-09-10T20:28:32.157898+0000","last_undegraded":"2024-09-10T20:28:32.157898+0000","last_fullsized":"2024-09-10T20:28:32.157898+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:54:36.134110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T20:28:32.157330+0000","last_change":"2024-09-10T20:28:27.487945+0000","last_active":"2024-09-10T20:28:32.157330+0000","last_peered":"2024-09-10T20:28:32.157330+0000","last_clean":"2024-09-10T20:28:32.157330+0000","last_became_active":"2024-09-10T20:28:25.489104+0000","last_became_peered":"2024-09-10T20:28:25.489104+0000","last_unstale":"2024-09-10T20:28:32.157330+0000","last_undegraded":"2024-09-10T20:28:32.157330+0000","last_fullsized":"2024-09-10T20:28:32.157330+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:30.886798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":23,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T20:28:32.157436+0000","last_change":"2024-09-10T20:28:27.487624+0000","last_active":"2024-09-10T20:28:32.157436+0000","last_peered":"2024-09-10T20:28:32.157436+0000","last_clean":"2024-09-10T20:28:32.157436+0000","last_became_active":"2024-09-10T20:28:25.486015+0000","last_became_peered":"2024-09-10T20:28:25.486015+0000","last_unstale":"2024-09-10T20:28:32.157436+0000","last_undegraded":"2024-09-10T20:28:32.157436+0000","last_fullsized":"2024-09-10T20:28:32.157436+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:03:12.564455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":24,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T20:28:32.244851+0000","last_change":"2024-09-10T20:28:27.487461+0000","last_active":"2024-09-10T20:28:32.244851+0000","last_peered":"2024-09-10T20:28:32.244851+0000","last_clean":"2024-09-10T20:28:32.244851+0000","last_became_active":"2024-09-10T20:28:25.491325+0000","last_became_peered":"2024-09-10T20:28:25.491325+0000","last_unstale":"2024-09-10T20:28:32.244851+0000","last_undegraded":"2024-09-10T20:28:32.244851+0000","last_fullsized":"2024-09-10T20:28:32.244851+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:53.879060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T20:28:32.157389+0000","last_change":"2024-09-10T20:28:27.526736+0000","last_active":"2024-09-10T20:28:32.157389+0000","last_peered":"2024-09-10T20:28:32.157389+0000","last_clean":"2024-09-10T20:28:32.157389+0000","last_became_active":"2024-09-10T20:28:25.489180+0000","last_became_peered":"2024-09-10T20:28:25.489180+0000","last_unstale":"2024-09-10T20:28:32.157389+0000","last_undegraded":"2024-09-10T20:28:32.157389+0000","last_fullsized":"2024-09-10T20:28:32.157389+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:59:00.063761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T20:28:32.157242+0000","last_change":"2024-09-10T20:28:27.487579+0000","last_active":"2024-09-10T20:28:32.157242+0000","last_peered":"2024-09-10T20:28:32.157242+0000","last_clean":"2024-09-10T20:28:32.157242+0000","last_became_active":"2024-09-10T20:28:25.489055+0000","last_became_peered":"2024-09-10T20:28:25.489055+0000","last_unstale":"2024-09-10T20:28:32.157242+0000","last_undegraded":"2024-09-10T20:28:32.157242+0000","last_fullsized":"2024-09-10T20:28:32.157242+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:12:00.009762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":9,"seq":38654705672,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":135168,"data_stored":35286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":8,"seq":34359738375,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27156,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344684,"statfs":{"total":96636764160,"available":96608956416,"internally_reserved":0,"allocated":131072,"data_stored":35267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738375,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":135168,"data_stored":35286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:28:36.036 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:28:36.317 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:28:36.318 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:28:36.329 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-10T20:28:34.812519+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81476,"kb_used_data":392,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034044,"statfs":{"total":289910292480,"available":289826861056,"internally_reserved":0,"allocated":401408,"data_stored":105839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000994"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T20:28:32.157539+0000","last_change":"2024-09-10T20:28:27.488224+0000","last_active":"2024-09-10T20:28:32.157539+0000","last_peered":"2024-09-10T20:28:32.157539+0000","last_clean":"2024-09-10T20:28:32.157539+0000","last_became_active":"2024-09-10T20:28:25.485918+0000","last_became_peered":"2024-09-10T20:28:25.485918+0000","last_unstale":"2024-09-10T20:28:32.157539+0000","last_undegraded":"2024-09-10T20:28:32.157539+0000","last_fullsized":"2024-09-10T20:28:32.157539+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:14:12.771955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T20:28:32.157274+0000","last_change":"2024-09-10T20:28:27.487827+0000","last_active":"2024-09-10T20:28:32.157274+0000","last_peered":"2024-09-10T20:28:32.157274+0000","last_clean":"2024-09-10T20:28:32.157274+0000","last_became_active":"2024-09-10T20:28:25.489162+0000","last_became_peered":"2024-09-10T20:28:25.489162+0000","last_unstale":"2024-09-10T20:28:32.157274+0000","last_undegraded":"2024-09-10T20:28:32.157274+0000","last_fullsized":"2024-09-10T20:28:32.157274+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:01:18.730084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T20:28:32.157898+0000","last_change":"2024-09-10T20:28:27.526743+0000","last_active":"2024-09-10T20:28:32.157898+0000","last_peered":"2024-09-10T20:28:32.157898+0000","last_clean":"2024-09-10T20:28:32.157898+0000","last_became_active":"2024-09-10T20:28:25.490897+0000","last_became_peered":"2024-09-10T20:28:25.490897+0000","last_unstale":"2024-09-10T20:28:32.157898+0000","last_undegraded":"2024-09-10T20:28:32.157898+0000","last_fullsized":"2024-09-10T20:28:32.157898+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:54:36.134110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T20:28:32.157330+0000","last_change":"2024-09-10T20:28:27.487945+0000","last_active":"2024-09-10T20:28:32.157330+0000","last_peered":"2024-09-10T20:28:32.157330+0000","last_clean":"2024-09-10T20:28:32.157330+0000","last_became_active":"2024-09-10T20:28:25.489104+0000","last_became_peered":"2024-09-10T20:28:25.489104+0000","last_unstale":"2024-09-10T20:28:32.157330+0000","last_undegraded":"2024-09-10T20:28:32.157330+0000","last_fullsized":"2024-09-10T20:28:32.157330+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:30.886798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":23,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T20:28:32.157436+0000","last_change":"2024-09-10T20:28:27.487624+0000","last_active":"2024-09-10T20:28:32.157436+0000","last_peered":"2024-09-10T20:28:32.157436+0000","last_clean":"2024-09-10T20:28:32.157436+0000","last_became_active":"2024-09-10T20:28:25.486015+0000","last_became_peered":"2024-09-10T20:28:25.486015+0000","last_unstale":"2024-09-10T20:28:32.157436+0000","last_undegraded":"2024-09-10T20:28:32.157436+0000","last_fullsized":"2024-09-10T20:28:32.157436+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:03:12.564455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":24,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T20:28:32.244851+0000","last_change":"2024-09-10T20:28:27.487461+0000","last_active":"2024-09-10T20:28:32.244851+0000","last_peered":"2024-09-10T20:28:32.244851+0000","last_clean":"2024-09-10T20:28:32.244851+0000","last_became_active":"2024-09-10T20:28:25.491325+0000","last_became_peered":"2024-09-10T20:28:25.491325+0000","last_unstale":"2024-09-10T20:28:32.244851+0000","last_undegraded":"2024-09-10T20:28:32.244851+0000","last_fullsized":"2024-09-10T20:28:32.244851+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:53.879060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T20:28:32.157389+0000","last_change":"2024-09-10T20:28:27.526736+0000","last_active":"2024-09-10T20:28:32.157389+0000","last_peered":"2024-09-10T20:28:32.157389+0000","last_clean":"2024-09-10T20:28:32.157389+0000","last_became_active":"2024-09-10T20:28:25.489180+0000","last_became_peered":"2024-09-10T20:28:25.489180+0000","last_unstale":"2024-09-10T20:28:32.157389+0000","last_undegraded":"2024-09-10T20:28:32.157389+0000","last_fullsized":"2024-09-10T20:28:32.157389+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:59:00.063761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T20:28:32.157242+0000","last_change":"2024-09-10T20:28:27.487579+0000","last_active":"2024-09-10T20:28:32.157242+0000","last_peered":"2024-09-10T20:28:32.157242+0000","last_clean":"2024-09-10T20:28:32.157242+0000","last_became_active":"2024-09-10T20:28:25.489055+0000","last_became_peered":"2024-09-10T20:28:25.489055+0000","last_unstale":"2024-09-10T20:28:32.157242+0000","last_undegraded":"2024-09-10T20:28:32.157242+0000","last_fullsized":"2024-09-10T20:28:32.157242+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:12:00.009762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":9,"seq":38654705672,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":135168,"data_stored":35286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":8,"seq":34359738375,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27156,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344684,"statfs":{"total":96636764160,"available":96608956416,"internally_reserved":0,"allocated":131072,"data_stored":35267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738375,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":135168,"data_stored":35286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:28:36.330 INFO:tasks.rep_lost_unfound_delete.ceph_manager:clean! 2024-09-10T20:28:36.330 INFO:tasks.rep_lost_unfound_delete.ceph_manager:creating pool_name unfounddel_pool 2024-09-10T20:28:36.330 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unfounddel_pool 16 2024-09-10T20:28:37.517 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'unfounddel_pool' created 2024-09-10T20:28:37.537 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unfounddel_pool rados --yes-i-really-mean-it || true 2024-09-10T20:28:38.527 INFO:teuthology.orchestra.run.smithi131.stderr:enabled application 'rados' on pool 'unfounddel_pool' 2024-09-10T20:28:39.551 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-10T20:28:39.551 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T20:28:39.582 INFO:tasks.ceph.osd.2:Stopped 2024-09-10T20:28:39.583 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 2 2024-09-10T20:28:39.919 INFO:tasks.ceph.mon.a.smithi131.stderr:2024-09-10T20:28:39.916+0000 7f1e634ff640 -1 mon.a@0(leader).osd e18 definitely_dead 0 2024-09-10T20:28:40.539 INFO:teuthology.orchestra.run.smithi131.stderr:marked down osd.2. 2024-09-10T20:28:40.554 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-09-10T20:28:40.847 INFO:tasks.ceph.mon.a.smithi131.stderr:2024-09-10T20:28:40.844+0000 7f1e634ff640 -1 mon.a@0(leader).osd e19 definitely_dead 0 2024-09-10T20:28:41.545 INFO:teuthology.orchestra.run.smithi131.stderr:marked out osd.2. 2024-09-10T20:28:41.563 INFO:tasks.util.rados:rados -p unfounddel_pool put dummy /etc/fstab 2024-09-10T20:28:41.563 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put dummy /etc/fstab 2024-09-10T20:28:41.615 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-10T20:28:41.615 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-10T20:28:41.749 INFO:teuthology.orchestra.run.smithi131.stdout:38654705674 2024-09-10T20:28:41.750 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T20:28:41.751 INFO:teuthology.orchestra.run.smithi131.stdout:34359738377 2024-09-10T20:28:41.751 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T20:28:42.082 INFO:teuthology.orchestra.run.smithi131.stdout:34359738376 2024-09-10T20:28:42.085 INFO:teuthology.orchestra.run.smithi131.stdout:38654705673 2024-09-10T20:28:42.097 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738377 got 34359738376 for osd.1 2024-09-10T20:28:42.098 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 38654705674 got 38654705673 for osd.0 2024-09-10T20:28:43.097 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T20:28:43.099 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T20:28:43.405 INFO:teuthology.orchestra.run.smithi131.stdout:38654705673 2024-09-10T20:28:43.406 INFO:teuthology.orchestra.run.smithi131.stdout:34359738376 2024-09-10T20:28:43.419 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 38654705674 got 38654705673 for osd.0 2024-09-10T20:28:43.420 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738377 got 34359738376 for osd.1 2024-09-10T20:28:44.421 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T20:28:44.421 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T20:28:44.724 INFO:teuthology.orchestra.run.smithi131.stdout:34359738377 2024-09-10T20:28:44.730 INFO:teuthology.orchestra.run.smithi131.stdout:38654705674 2024-09-10T20:28:44.737 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738377 got 34359738377 for osd.1 2024-09-10T20:28:44.738 DEBUG:teuthology.parallel:result is None 2024-09-10T20:28:44.743 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 38654705674 got 38654705674 for osd.0 2024-09-10T20:28:44.743 DEBUG:teuthology.parallel:result is None 2024-09-10T20:28:44.743 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for recovery to complete 2024-09-10T20:28:44.743 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:28:45.063 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:28:45.063 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:28:45.075 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-09-10T20:28:44.814404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54420,"kb_used_data":360,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689260,"statfs":{"total":193273528320,"available":193217802240,"internally_reserved":0,"allocated":368640,"data_stored":108783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":55309194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.283454"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.550109+0000","last_change":"2024-09-10T20:28:38.533153+0000","last_active":"2024-09-10T20:28:42.550109+0000","last_peered":"2024-09-10T20:28:42.550109+0000","last_clean":"2024-09-10T20:28:42.550109+0000","last_became_active":"2024-09-10T20:28:38.532869+0000","last_became_peered":"2024-09-10T20:28:38.532869+0000","last_unstale":"2024-09-10T20:28:42.550109+0000","last_undegraded":"2024-09-10T20:28:42.550109+0000","last_fullsized":"2024-09-10T20:28:42.550109+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:35:53.692741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.554905+0000","last_change":"2024-09-10T20:28:42.554807+0000","last_active":"2024-09-10T20:28:42.554905+0000","last_peered":"2024-09-10T20:28:42.554905+0000","last_clean":"2024-09-10T20:28:42.554905+0000","last_became_active":"2024-09-10T20:28:42.554548+0000","last_became_peered":"2024-09-10T20:28:42.554548+0000","last_unstale":"2024-09-10T20:28:42.554905+0000","last_undegraded":"2024-09-10T20:28:42.554905+0000","last_fullsized":"2024-09-10T20:28:42.554905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:42:55.733310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":24,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.556680+0000","last_change":"2024-09-10T20:28:42.556584+0000","last_active":"2024-09-10T20:28:42.556680+0000","last_peered":"2024-09-10T20:28:42.556680+0000","last_clean":"2024-09-10T20:28:42.556680+0000","last_became_active":"2024-09-10T20:28:42.556111+0000","last_became_peered":"2024-09-10T20:28:42.556111+0000","last_unstale":"2024-09-10T20:28:42.556680+0000","last_undegraded":"2024-09-10T20:28:42.556680+0000","last_fullsized":"2024-09-10T20:28:42.556680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:45:23.920515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.554801+0000","last_change":"2024-09-10T20:28:42.554717+0000","last_active":"2024-09-10T20:28:42.554801+0000","last_peered":"2024-09-10T20:28:42.554801+0000","last_clean":"2024-09-10T20:28:42.554801+0000","last_became_active":"2024-09-10T20:28:42.554446+0000","last_became_peered":"2024-09-10T20:28:42.554446+0000","last_unstale":"2024-09-10T20:28:42.554801+0000","last_undegraded":"2024-09-10T20:28:42.554801+0000","last_fullsized":"2024-09-10T20:28:42.554801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:05.256590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.554688+0000","last_change":"2024-09-10T20:28:42.554592+0000","last_active":"2024-09-10T20:28:42.554688+0000","last_peered":"2024-09-10T20:28:42.554688+0000","last_clean":"2024-09-10T20:28:42.554688+0000","last_became_active":"2024-09-10T20:28:42.554319+0000","last_became_peered":"2024-09-10T20:28:42.554319+0000","last_unstale":"2024-09-10T20:28:42.554688+0000","last_undegraded":"2024-09-10T20:28:42.554688+0000","last_fullsized":"2024-09-10T20:28:42.554688+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:44:47.134431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.552108+0000","last_change":"2024-09-10T20:28:38.529819+0000","last_active":"2024-09-10T20:28:42.552108+0000","last_peered":"2024-09-10T20:28:42.552108+0000","last_clean":"2024-09-10T20:28:42.552108+0000","last_became_active":"2024-09-10T20:28:38.529527+0000","last_became_peered":"2024-09-10T20:28:38.529527+0000","last_unstale":"2024-09-10T20:28:42.552108+0000","last_undegraded":"2024-09-10T20:28:42.552108+0000","last_fullsized":"2024-09-10T20:28:42.552108+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:53.366512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.552038+0000","last_change":"2024-09-10T20:28:38.531876+0000","last_active":"2024-09-10T20:28:42.552038+0000","last_peered":"2024-09-10T20:28:42.552038+0000","last_clean":"2024-09-10T20:28:42.552038+0000","last_became_active":"2024-09-10T20:28:38.531534+0000","last_became_peered":"2024-09-10T20:28:38.531534+0000","last_unstale":"2024-09-10T20:28:42.552038+0000","last_undegraded":"2024-09-10T20:28:42.552038+0000","last_fullsized":"2024-09-10T20:28:42.552038+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:14.045705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.554334+0000","last_change":"2024-09-10T20:28:42.554243+0000","last_active":"2024-09-10T20:28:42.554334+0000","last_peered":"2024-09-10T20:28:42.554334+0000","last_clean":"2024-09-10T20:28:42.554334+0000","last_became_active":"2024-09-10T20:28:42.553958+0000","last_became_peered":"2024-09-10T20:28:42.553958+0000","last_unstale":"2024-09-10T20:28:42.554334+0000","last_undegraded":"2024-09-10T20:28:42.554334+0000","last_fullsized":"2024-09-10T20:28:42.554334+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:14:41.171197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":34,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.551990+0000","last_change":"2024-09-10T20:28:27.487579+0000","last_active":"2024-09-10T20:28:42.551990+0000","last_peered":"2024-09-10T20:28:42.551990+0000","last_clean":"2024-09-10T20:28:42.551990+0000","last_became_active":"2024-09-10T20:28:25.489055+0000","last_became_peered":"2024-09-10T20:28:25.489055+0000","last_unstale":"2024-09-10T20:28:42.551990+0000","last_undegraded":"2024-09-10T20:28:42.551990+0000","last_fullsized":"2024-09-10T20:28:42.551990+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:12:00.009762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.556530+0000","last_change":"2024-09-10T20:28:42.556434+0000","last_active":"2024-09-10T20:28:42.556530+0000","last_peered":"2024-09-10T20:28:42.556530+0000","last_clean":"2024-09-10T20:28:42.556530+0000","last_became_active":"2024-09-10T20:28:42.556142+0000","last_became_peered":"2024-09-10T20:28:42.556142+0000","last_unstale":"2024-09-10T20:28:42.556530+0000","last_undegraded":"2024-09-10T20:28:42.556530+0000","last_fullsized":"2024-09-10T20:28:42.556530+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:26:51.220984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":32,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.554286+0000","last_change":"2024-09-10T20:28:42.554184+0000","last_active":"2024-09-10T20:28:42.554286+0000","last_peered":"2024-09-10T20:28:42.554286+0000","last_clean":"2024-09-10T20:28:42.554286+0000","last_became_active":"2024-09-10T20:28:42.553919+0000","last_became_peered":"2024-09-10T20:28:42.553919+0000","last_unstale":"2024-09-10T20:28:42.554286+0000","last_undegraded":"2024-09-10T20:28:42.554286+0000","last_fullsized":"2024-09-10T20:28:42.554286+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:50:32.916635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.550075+0000","last_change":"2024-09-10T20:28:38.532851+0000","last_active":"2024-09-10T20:28:42.550075+0000","last_peered":"2024-09-10T20:28:42.550075+0000","last_clean":"2024-09-10T20:28:42.550075+0000","last_became_active":"2024-09-10T20:28:38.532567+0000","last_became_peered":"2024-09-10T20:28:38.532567+0000","last_unstale":"2024-09-10T20:28:42.550075+0000","last_undegraded":"2024-09-10T20:28:42.550075+0000","last_fullsized":"2024-09-10T20:28:42.550075+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:33.078863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":35,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.550219+0000","last_change":"2024-09-10T20:28:27.487461+0000","last_active":"2024-09-10T20:28:42.550219+0000","last_peered":"2024-09-10T20:28:42.550219+0000","last_clean":"2024-09-10T20:28:42.550219+0000","last_became_active":"2024-09-10T20:28:25.491325+0000","last_became_peered":"2024-09-10T20:28:25.491325+0000","last_unstale":"2024-09-10T20:28:42.550219+0000","last_undegraded":"2024-09-10T20:28:42.550219+0000","last_fullsized":"2024-09-10T20:28:42.550219+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:53.879060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.556367+0000","last_change":"2024-09-10T20:28:42.556266+0000","last_active":"2024-09-10T20:28:42.556367+0000","last_peered":"2024-09-10T20:28:42.556367+0000","last_clean":"2024-09-10T20:28:42.556367+0000","last_became_active":"2024-09-10T20:28:42.555965+0000","last_became_peered":"2024-09-10T20:28:42.555965+0000","last_unstale":"2024-09-10T20:28:42.556367+0000","last_undegraded":"2024-09-10T20:28:42.556367+0000","last_fullsized":"2024-09-10T20:28:42.556367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:34.451914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":44,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.562791+0000","last_change":"2024-09-10T20:28:42.562731+0000","last_active":"2024-09-10T20:28:42.562791+0000","last_peered":"2024-09-10T20:28:42.562791+0000","last_clean":"2024-09-10T20:28:42.562791+0000","last_became_active":"2024-09-10T20:28:42.557538+0000","last_became_peered":"2024-09-10T20:28:42.557538+0000","last_unstale":"2024-09-10T20:28:42.562791+0000","last_undegraded":"2024-09-10T20:28:42.562791+0000","last_fullsized":"2024-09-10T20:28:42.562791+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:16:31.248113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.556376+0000","last_change":"2024-09-10T20:28:42.556288+0000","last_active":"2024-09-10T20:28:42.556376+0000","last_peered":"2024-09-10T20:28:42.556376+0000","last_clean":"2024-09-10T20:28:42.556376+0000","last_became_active":"2024-09-10T20:28:42.556008+0000","last_became_peered":"2024-09-10T20:28:42.556008+0000","last_unstale":"2024-09-10T20:28:42.556376+0000","last_undegraded":"2024-09-10T20:28:42.556376+0000","last_fullsized":"2024-09-10T20:28:42.556376+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:14:32.567759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.552255+0000","last_change":"2024-09-10T20:28:38.531900+0000","last_active":"2024-09-10T20:28:42.552255+0000","last_peered":"2024-09-10T20:28:42.552255+0000","last_clean":"2024-09-10T20:28:42.552255+0000","last_became_active":"2024-09-10T20:28:38.531428+0000","last_became_peered":"2024-09-10T20:28:38.531428+0000","last_unstale":"2024-09-10T20:28:42.552255+0000","last_undegraded":"2024-09-10T20:28:42.552255+0000","last_fullsized":"2024-09-10T20:28:42.552255+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:44:00.704033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.552375+0000","last_change":"2024-09-10T20:28:27.487945+0000","last_active":"2024-09-10T20:28:42.552375+0000","last_peered":"2024-09-10T20:28:42.552375+0000","last_clean":"2024-09-10T20:28:42.552375+0000","last_became_active":"2024-09-10T20:28:25.489104+0000","last_became_peered":"2024-09-10T20:28:25.489104+0000","last_unstale":"2024-09-10T20:28:42.552375+0000","last_undegraded":"2024-09-10T20:28:42.552375+0000","last_fullsized":"2024-09-10T20:28:42.552375+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:30.886798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.552245+0000","last_change":"2024-09-10T20:28:38.529840+0000","last_active":"2024-09-10T20:28:42.552245+0000","last_peered":"2024-09-10T20:28:42.552245+0000","last_clean":"2024-09-10T20:28:42.552245+0000","last_became_active":"2024-09-10T20:28:38.529540+0000","last_became_peered":"2024-09-10T20:28:38.529540+0000","last_unstale":"2024-09-10T20:28:42.552245+0000","last_undegraded":"2024-09-10T20:28:42.552245+0000","last_fullsized":"2024-09-10T20:28:42.552245+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:15.282053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.554294+0000","last_change":"2024-09-10T20:28:42.554202+0000","last_active":"2024-09-10T20:28:42.554294+0000","last_peered":"2024-09-10T20:28:42.554294+0000","last_clean":"2024-09-10T20:28:42.554294+0000","last_became_active":"2024-09-10T20:28:42.553892+0000","last_became_peered":"2024-09-10T20:28:42.553892+0000","last_unstale":"2024-09-10T20:28:42.554294+0000","last_undegraded":"2024-09-10T20:28:42.554294+0000","last_fullsized":"2024-09-10T20:28:42.554294+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:21:15.208796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.552381+0000","last_change":"2024-09-10T20:28:38.531076+0000","last_active":"2024-09-10T20:28:42.552381+0000","last_peered":"2024-09-10T20:28:42.552381+0000","last_clean":"2024-09-10T20:28:42.552381+0000","last_became_active":"2024-09-10T20:28:38.530836+0000","last_became_peered":"2024-09-10T20:28:38.530836+0000","last_unstale":"2024-09-10T20:28:42.552381+0000","last_undegraded":"2024-09-10T20:28:42.552381+0000","last_fullsized":"2024-09-10T20:28:42.552381+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:22.402269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.552281+0000","last_change":"2024-09-10T20:28:27.487827+0000","last_active":"2024-09-10T20:28:42.552281+0000","last_peered":"2024-09-10T20:28:42.552281+0000","last_clean":"2024-09-10T20:28:42.552281+0000","last_became_active":"2024-09-10T20:28:25.489162+0000","last_became_peered":"2024-09-10T20:28:25.489162+0000","last_unstale":"2024-09-10T20:28:42.552281+0000","last_undegraded":"2024-09-10T20:28:42.552281+0000","last_fullsized":"2024-09-10T20:28:42.552281+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:01:18.730084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":39,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.556616+0000","last_change":"2024-09-10T20:28:42.556528+0000","last_active":"2024-09-10T20:28:42.556616+0000","last_peered":"2024-09-10T20:28:42.556616+0000","last_clean":"2024-09-10T20:28:42.556616+0000","last_became_active":"2024-09-10T20:28:42.556068+0000","last_became_peered":"2024-09-10T20:28:42.556068+0000","last_unstale":"2024-09-10T20:28:42.556616+0000","last_undegraded":"2024-09-10T20:28:42.556616+0000","last_fullsized":"2024-09-10T20:28:42.556616+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:52:59.645158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.552140+0000","last_change":"2024-09-10T20:28:38.532441+0000","last_active":"2024-09-10T20:28:42.552140+0000","last_peered":"2024-09-10T20:28:42.552140+0000","last_clean":"2024-09-10T20:28:42.552140+0000","last_became_active":"2024-09-10T20:28:38.532149+0000","last_became_peered":"2024-09-10T20:28:38.532149+0000","last_unstale":"2024-09-10T20:28:42.552140+0000","last_undegraded":"2024-09-10T20:28:42.552140+0000","last_fullsized":"2024-09-10T20:28:42.552140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:12.589450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":9,"seq":38654705675,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":188416,"data_stored":55960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738378,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":180224,"data_stored":52823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:28:45.077 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:28:45.359 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:28:45.359 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:28:45.371 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-09-10T20:28:44.814404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54420,"kb_used_data":360,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689260,"statfs":{"total":193273528320,"available":193217802240,"internally_reserved":0,"allocated":368640,"data_stored":108783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":55309194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.283454"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.550109+0000","last_change":"2024-09-10T20:28:38.533153+0000","last_active":"2024-09-10T20:28:42.550109+0000","last_peered":"2024-09-10T20:28:42.550109+0000","last_clean":"2024-09-10T20:28:42.550109+0000","last_became_active":"2024-09-10T20:28:38.532869+0000","last_became_peered":"2024-09-10T20:28:38.532869+0000","last_unstale":"2024-09-10T20:28:42.550109+0000","last_undegraded":"2024-09-10T20:28:42.550109+0000","last_fullsized":"2024-09-10T20:28:42.550109+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:35:53.692741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.554905+0000","last_change":"2024-09-10T20:28:42.554807+0000","last_active":"2024-09-10T20:28:42.554905+0000","last_peered":"2024-09-10T20:28:42.554905+0000","last_clean":"2024-09-10T20:28:42.554905+0000","last_became_active":"2024-09-10T20:28:42.554548+0000","last_became_peered":"2024-09-10T20:28:42.554548+0000","last_unstale":"2024-09-10T20:28:42.554905+0000","last_undegraded":"2024-09-10T20:28:42.554905+0000","last_fullsized":"2024-09-10T20:28:42.554905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:42:55.733310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":24,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.556680+0000","last_change":"2024-09-10T20:28:42.556584+0000","last_active":"2024-09-10T20:28:42.556680+0000","last_peered":"2024-09-10T20:28:42.556680+0000","last_clean":"2024-09-10T20:28:42.556680+0000","last_became_active":"2024-09-10T20:28:42.556111+0000","last_became_peered":"2024-09-10T20:28:42.556111+0000","last_unstale":"2024-09-10T20:28:42.556680+0000","last_undegraded":"2024-09-10T20:28:42.556680+0000","last_fullsized":"2024-09-10T20:28:42.556680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:45:23.920515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.554801+0000","last_change":"2024-09-10T20:28:42.554717+0000","last_active":"2024-09-10T20:28:42.554801+0000","last_peered":"2024-09-10T20:28:42.554801+0000","last_clean":"2024-09-10T20:28:42.554801+0000","last_became_active":"2024-09-10T20:28:42.554446+0000","last_became_peered":"2024-09-10T20:28:42.554446+0000","last_unstale":"2024-09-10T20:28:42.554801+0000","last_undegraded":"2024-09-10T20:28:42.554801+0000","last_fullsized":"2024-09-10T20:28:42.554801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:05.256590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.554688+0000","last_change":"2024-09-10T20:28:42.554592+0000","last_active":"2024-09-10T20:28:42.554688+0000","last_peered":"2024-09-10T20:28:42.554688+0000","last_clean":"2024-09-10T20:28:42.554688+0000","last_became_active":"2024-09-10T20:28:42.554319+0000","last_became_peered":"2024-09-10T20:28:42.554319+0000","last_unstale":"2024-09-10T20:28:42.554688+0000","last_undegraded":"2024-09-10T20:28:42.554688+0000","last_fullsized":"2024-09-10T20:28:42.554688+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:44:47.134431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.552108+0000","last_change":"2024-09-10T20:28:38.529819+0000","last_active":"2024-09-10T20:28:42.552108+0000","last_peered":"2024-09-10T20:28:42.552108+0000","last_clean":"2024-09-10T20:28:42.552108+0000","last_became_active":"2024-09-10T20:28:38.529527+0000","last_became_peered":"2024-09-10T20:28:38.529527+0000","last_unstale":"2024-09-10T20:28:42.552108+0000","last_undegraded":"2024-09-10T20:28:42.552108+0000","last_fullsized":"2024-09-10T20:28:42.552108+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:53.366512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.552038+0000","last_change":"2024-09-10T20:28:38.531876+0000","last_active":"2024-09-10T20:28:42.552038+0000","last_peered":"2024-09-10T20:28:42.552038+0000","last_clean":"2024-09-10T20:28:42.552038+0000","last_became_active":"2024-09-10T20:28:38.531534+0000","last_became_peered":"2024-09-10T20:28:38.531534+0000","last_unstale":"2024-09-10T20:28:42.552038+0000","last_undegraded":"2024-09-10T20:28:42.552038+0000","last_fullsized":"2024-09-10T20:28:42.552038+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:14.045705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.554334+0000","last_change":"2024-09-10T20:28:42.554243+0000","last_active":"2024-09-10T20:28:42.554334+0000","last_peered":"2024-09-10T20:28:42.554334+0000","last_clean":"2024-09-10T20:28:42.554334+0000","last_became_active":"2024-09-10T20:28:42.553958+0000","last_became_peered":"2024-09-10T20:28:42.553958+0000","last_unstale":"2024-09-10T20:28:42.554334+0000","last_undegraded":"2024-09-10T20:28:42.554334+0000","last_fullsized":"2024-09-10T20:28:42.554334+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:14:41.171197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":34,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.551990+0000","last_change":"2024-09-10T20:28:27.487579+0000","last_active":"2024-09-10T20:28:42.551990+0000","last_peered":"2024-09-10T20:28:42.551990+0000","last_clean":"2024-09-10T20:28:42.551990+0000","last_became_active":"2024-09-10T20:28:25.489055+0000","last_became_peered":"2024-09-10T20:28:25.489055+0000","last_unstale":"2024-09-10T20:28:42.551990+0000","last_undegraded":"2024-09-10T20:28:42.551990+0000","last_fullsized":"2024-09-10T20:28:42.551990+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:12:00.009762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.556530+0000","last_change":"2024-09-10T20:28:42.556434+0000","last_active":"2024-09-10T20:28:42.556530+0000","last_peered":"2024-09-10T20:28:42.556530+0000","last_clean":"2024-09-10T20:28:42.556530+0000","last_became_active":"2024-09-10T20:28:42.556142+0000","last_became_peered":"2024-09-10T20:28:42.556142+0000","last_unstale":"2024-09-10T20:28:42.556530+0000","last_undegraded":"2024-09-10T20:28:42.556530+0000","last_fullsized":"2024-09-10T20:28:42.556530+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:26:51.220984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":32,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.554286+0000","last_change":"2024-09-10T20:28:42.554184+0000","last_active":"2024-09-10T20:28:42.554286+0000","last_peered":"2024-09-10T20:28:42.554286+0000","last_clean":"2024-09-10T20:28:42.554286+0000","last_became_active":"2024-09-10T20:28:42.553919+0000","last_became_peered":"2024-09-10T20:28:42.553919+0000","last_unstale":"2024-09-10T20:28:42.554286+0000","last_undegraded":"2024-09-10T20:28:42.554286+0000","last_fullsized":"2024-09-10T20:28:42.554286+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:50:32.916635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.550075+0000","last_change":"2024-09-10T20:28:38.532851+0000","last_active":"2024-09-10T20:28:42.550075+0000","last_peered":"2024-09-10T20:28:42.550075+0000","last_clean":"2024-09-10T20:28:42.550075+0000","last_became_active":"2024-09-10T20:28:38.532567+0000","last_became_peered":"2024-09-10T20:28:38.532567+0000","last_unstale":"2024-09-10T20:28:42.550075+0000","last_undegraded":"2024-09-10T20:28:42.550075+0000","last_fullsized":"2024-09-10T20:28:42.550075+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:33.078863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":35,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.550219+0000","last_change":"2024-09-10T20:28:27.487461+0000","last_active":"2024-09-10T20:28:42.550219+0000","last_peered":"2024-09-10T20:28:42.550219+0000","last_clean":"2024-09-10T20:28:42.550219+0000","last_became_active":"2024-09-10T20:28:25.491325+0000","last_became_peered":"2024-09-10T20:28:25.491325+0000","last_unstale":"2024-09-10T20:28:42.550219+0000","last_undegraded":"2024-09-10T20:28:42.550219+0000","last_fullsized":"2024-09-10T20:28:42.550219+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:53.879060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.556367+0000","last_change":"2024-09-10T20:28:42.556266+0000","last_active":"2024-09-10T20:28:42.556367+0000","last_peered":"2024-09-10T20:28:42.556367+0000","last_clean":"2024-09-10T20:28:42.556367+0000","last_became_active":"2024-09-10T20:28:42.555965+0000","last_became_peered":"2024-09-10T20:28:42.555965+0000","last_unstale":"2024-09-10T20:28:42.556367+0000","last_undegraded":"2024-09-10T20:28:42.556367+0000","last_fullsized":"2024-09-10T20:28:42.556367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:34.451914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":44,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.562791+0000","last_change":"2024-09-10T20:28:42.562731+0000","last_active":"2024-09-10T20:28:42.562791+0000","last_peered":"2024-09-10T20:28:42.562791+0000","last_clean":"2024-09-10T20:28:42.562791+0000","last_became_active":"2024-09-10T20:28:42.557538+0000","last_became_peered":"2024-09-10T20:28:42.557538+0000","last_unstale":"2024-09-10T20:28:42.562791+0000","last_undegraded":"2024-09-10T20:28:42.562791+0000","last_fullsized":"2024-09-10T20:28:42.562791+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:16:31.248113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.556376+0000","last_change":"2024-09-10T20:28:42.556288+0000","last_active":"2024-09-10T20:28:42.556376+0000","last_peered":"2024-09-10T20:28:42.556376+0000","last_clean":"2024-09-10T20:28:42.556376+0000","last_became_active":"2024-09-10T20:28:42.556008+0000","last_became_peered":"2024-09-10T20:28:42.556008+0000","last_unstale":"2024-09-10T20:28:42.556376+0000","last_undegraded":"2024-09-10T20:28:42.556376+0000","last_fullsized":"2024-09-10T20:28:42.556376+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:14:32.567759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.552255+0000","last_change":"2024-09-10T20:28:38.531900+0000","last_active":"2024-09-10T20:28:42.552255+0000","last_peered":"2024-09-10T20:28:42.552255+0000","last_clean":"2024-09-10T20:28:42.552255+0000","last_became_active":"2024-09-10T20:28:38.531428+0000","last_became_peered":"2024-09-10T20:28:38.531428+0000","last_unstale":"2024-09-10T20:28:42.552255+0000","last_undegraded":"2024-09-10T20:28:42.552255+0000","last_fullsized":"2024-09-10T20:28:42.552255+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:44:00.704033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.552375+0000","last_change":"2024-09-10T20:28:27.487945+0000","last_active":"2024-09-10T20:28:42.552375+0000","last_peered":"2024-09-10T20:28:42.552375+0000","last_clean":"2024-09-10T20:28:42.552375+0000","last_became_active":"2024-09-10T20:28:25.489104+0000","last_became_peered":"2024-09-10T20:28:25.489104+0000","last_unstale":"2024-09-10T20:28:42.552375+0000","last_undegraded":"2024-09-10T20:28:42.552375+0000","last_fullsized":"2024-09-10T20:28:42.552375+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:30.886798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.552245+0000","last_change":"2024-09-10T20:28:38.529840+0000","last_active":"2024-09-10T20:28:42.552245+0000","last_peered":"2024-09-10T20:28:42.552245+0000","last_clean":"2024-09-10T20:28:42.552245+0000","last_became_active":"2024-09-10T20:28:38.529540+0000","last_became_peered":"2024-09-10T20:28:38.529540+0000","last_unstale":"2024-09-10T20:28:42.552245+0000","last_undegraded":"2024-09-10T20:28:42.552245+0000","last_fullsized":"2024-09-10T20:28:42.552245+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:15.282053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.554294+0000","last_change":"2024-09-10T20:28:42.554202+0000","last_active":"2024-09-10T20:28:42.554294+0000","last_peered":"2024-09-10T20:28:42.554294+0000","last_clean":"2024-09-10T20:28:42.554294+0000","last_became_active":"2024-09-10T20:28:42.553892+0000","last_became_peered":"2024-09-10T20:28:42.553892+0000","last_unstale":"2024-09-10T20:28:42.554294+0000","last_undegraded":"2024-09-10T20:28:42.554294+0000","last_fullsized":"2024-09-10T20:28:42.554294+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:21:15.208796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.552381+0000","last_change":"2024-09-10T20:28:38.531076+0000","last_active":"2024-09-10T20:28:42.552381+0000","last_peered":"2024-09-10T20:28:42.552381+0000","last_clean":"2024-09-10T20:28:42.552381+0000","last_became_active":"2024-09-10T20:28:38.530836+0000","last_became_peered":"2024-09-10T20:28:38.530836+0000","last_unstale":"2024-09-10T20:28:42.552381+0000","last_undegraded":"2024-09-10T20:28:42.552381+0000","last_fullsized":"2024-09-10T20:28:42.552381+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:22.402269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.552281+0000","last_change":"2024-09-10T20:28:27.487827+0000","last_active":"2024-09-10T20:28:42.552281+0000","last_peered":"2024-09-10T20:28:42.552281+0000","last_clean":"2024-09-10T20:28:42.552281+0000","last_became_active":"2024-09-10T20:28:25.489162+0000","last_became_peered":"2024-09-10T20:28:25.489162+0000","last_unstale":"2024-09-10T20:28:42.552281+0000","last_undegraded":"2024-09-10T20:28:42.552281+0000","last_fullsized":"2024-09-10T20:28:42.552281+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:01:18.730084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":39,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.556616+0000","last_change":"2024-09-10T20:28:42.556528+0000","last_active":"2024-09-10T20:28:42.556616+0000","last_peered":"2024-09-10T20:28:42.556616+0000","last_clean":"2024-09-10T20:28:42.556616+0000","last_became_active":"2024-09-10T20:28:42.556068+0000","last_became_peered":"2024-09-10T20:28:42.556068+0000","last_unstale":"2024-09-10T20:28:42.556616+0000","last_undegraded":"2024-09-10T20:28:42.556616+0000","last_fullsized":"2024-09-10T20:28:42.556616+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:52:59.645158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T20:28:42.552140+0000","last_change":"2024-09-10T20:28:38.532441+0000","last_active":"2024-09-10T20:28:42.552140+0000","last_peered":"2024-09-10T20:28:42.552140+0000","last_clean":"2024-09-10T20:28:42.552140+0000","last_became_active":"2024-09-10T20:28:38.532149+0000","last_became_peered":"2024-09-10T20:28:38.532149+0000","last_unstale":"2024-09-10T20:28:42.552140+0000","last_undegraded":"2024-09-10T20:28:42.552140+0000","last_fullsized":"2024-09-10T20:28:42.552140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:12.589450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":9,"seq":38654705675,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":188416,"data_stored":55960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738378,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":180224,"data_stored":52823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:28:45.373 INFO:tasks.rep_lost_unfound_delete.ceph_manager:recovered! 2024-09-10T20:28:45.373 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_1 /etc/fstab 2024-09-10T20:28:45.373 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_1 /etc/fstab 2024-09-10T20:28:45.423 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_1 /etc/fstab 2024-09-10T20:28:45.424 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_1 /etc/fstab 2024-09-10T20:28:45.473 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_1 2024-09-10T20:28:45.474 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_1 2024-09-10T20:28:45.522 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_2 /etc/fstab 2024-09-10T20:28:45.522 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_2 /etc/fstab 2024-09-10T20:28:45.578 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_2 /etc/fstab 2024-09-10T20:28:45.578 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_2 /etc/fstab 2024-09-10T20:28:45.628 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_2 2024-09-10T20:28:45.628 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_2 2024-09-10T20:28:45.677 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_3 /etc/fstab 2024-09-10T20:28:45.677 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_3 /etc/fstab 2024-09-10T20:28:45.727 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_3 /etc/fstab 2024-09-10T20:28:45.728 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_3 /etc/fstab 2024-09-10T20:28:45.778 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_3 2024-09-10T20:28:45.778 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_3 2024-09-10T20:28:45.827 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_4 /etc/fstab 2024-09-10T20:28:45.827 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_4 /etc/fstab 2024-09-10T20:28:45.878 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_4 /etc/fstab 2024-09-10T20:28:45.878 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_4 /etc/fstab 2024-09-10T20:28:45.928 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_4 2024-09-10T20:28:45.928 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_4 2024-09-10T20:28:45.977 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_5 /etc/fstab 2024-09-10T20:28:45.977 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_5 /etc/fstab 2024-09-10T20:28:46.027 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_5 /etc/fstab 2024-09-10T20:28:46.027 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_5 /etc/fstab 2024-09-10T20:28:46.077 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_5 2024-09-10T20:28:46.077 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_5 2024-09-10T20:28:46.126 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_6 /etc/fstab 2024-09-10T20:28:46.126 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_6 /etc/fstab 2024-09-10T20:28:46.176 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_6 /etc/fstab 2024-09-10T20:28:46.176 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_6 /etc/fstab 2024-09-10T20:28:46.227 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_6 2024-09-10T20:28:46.227 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_6 2024-09-10T20:28:46.276 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_7 /etc/fstab 2024-09-10T20:28:46.276 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_7 /etc/fstab 2024-09-10T20:28:46.326 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_7 /etc/fstab 2024-09-10T20:28:46.327 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_7 /etc/fstab 2024-09-10T20:28:46.377 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_7 2024-09-10T20:28:46.377 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_7 2024-09-10T20:28:46.426 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_8 /etc/fstab 2024-09-10T20:28:46.426 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_8 /etc/fstab 2024-09-10T20:28:46.476 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_8 /etc/fstab 2024-09-10T20:28:46.476 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_8 /etc/fstab 2024-09-10T20:28:46.526 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_8 2024-09-10T20:28:46.526 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_8 2024-09-10T20:28:46.577 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_9 /etc/fstab 2024-09-10T20:28:46.577 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_9 /etc/fstab 2024-09-10T20:28:46.627 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_9 /etc/fstab 2024-09-10T20:28:46.627 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_9 /etc/fstab 2024-09-10T20:28:46.677 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_9 2024-09-10T20:28:46.677 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_9 2024-09-10T20:28:46.728 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 injectargs '--osd-recovery-delay-start 1000 --osd-min-pg-log-entries 100000000' 2024-09-10T20:28:46.849 INFO:teuthology.orchestra.run.smithi131.stdout:{} 2024-09-10T20:28:46.849 INFO:teuthology.orchestra.run.smithi131.stderr:osd_min_pg_log_entries = '' (not observed, change may require restart) osd_recovery_delay_start = '' 2024-09-10T20:28:46.859 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-10T20:28:46.860 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T20:28:46.898 INFO:tasks.ceph.osd.0:Stopped 2024-09-10T20:28:46.899 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 0 2024-09-10T20:28:47.232 INFO:tasks.ceph.mon.a.smithi131.stderr:2024-09-10T20:28:47.228+0000 7f1e634ff640 -1 mon.a@0(leader).osd e21 definitely_dead 0 2024-09-10T20:28:47.575 INFO:teuthology.orchestra.run.smithi131.stderr:marked down osd.0. 2024-09-10T20:28:47.590 INFO:tasks.util.rados:rados -p unfounddel_pool put new_1 /etc/fstab 2024-09-10T20:28:47.591 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_1 /etc/fstab 2024-09-10T20:28:48.589 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_1 /etc/fstab 2024-09-10T20:28:48.589 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_1 /etc/fstab 2024-09-10T20:28:48.645 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_1 /etc/fstab 2024-09-10T20:28:48.646 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_1 /etc/fstab 2024-09-10T20:28:48.698 INFO:tasks.util.rados:rados -p unfounddel_pool put new_2 /etc/fstab 2024-09-10T20:28:48.698 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_2 /etc/fstab 2024-09-10T20:28:48.748 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_2 /etc/fstab 2024-09-10T20:28:48.749 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_2 /etc/fstab 2024-09-10T20:28:48.798 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_2 /etc/fstab 2024-09-10T20:28:48.798 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_2 /etc/fstab 2024-09-10T20:28:48.851 INFO:tasks.util.rados:rados -p unfounddel_pool put new_3 /etc/fstab 2024-09-10T20:28:48.851 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_3 /etc/fstab 2024-09-10T20:28:48.901 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_3 /etc/fstab 2024-09-10T20:28:48.901 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_3 /etc/fstab 2024-09-10T20:28:48.951 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_3 /etc/fstab 2024-09-10T20:28:48.951 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_3 /etc/fstab 2024-09-10T20:28:49.001 INFO:tasks.util.rados:rados -p unfounddel_pool put new_4 /etc/fstab 2024-09-10T20:28:49.002 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_4 /etc/fstab 2024-09-10T20:28:49.051 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_4 /etc/fstab 2024-09-10T20:28:49.052 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_4 /etc/fstab 2024-09-10T20:28:49.101 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_4 /etc/fstab 2024-09-10T20:28:49.102 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_4 /etc/fstab 2024-09-10T20:28:49.151 INFO:tasks.util.rados:rados -p unfounddel_pool put new_5 /etc/fstab 2024-09-10T20:28:49.152 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_5 /etc/fstab 2024-09-10T20:28:49.202 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_5 /etc/fstab 2024-09-10T20:28:49.202 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_5 /etc/fstab 2024-09-10T20:28:49.251 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_5 /etc/fstab 2024-09-10T20:28:49.251 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_5 /etc/fstab 2024-09-10T20:28:49.301 INFO:tasks.util.rados:rados -p unfounddel_pool put new_6 /etc/fstab 2024-09-10T20:28:49.301 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_6 /etc/fstab 2024-09-10T20:28:49.351 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_6 /etc/fstab 2024-09-10T20:28:49.351 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_6 /etc/fstab 2024-09-10T20:28:49.402 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_6 /etc/fstab 2024-09-10T20:28:49.402 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_6 /etc/fstab 2024-09-10T20:28:49.452 INFO:tasks.util.rados:rados -p unfounddel_pool put new_7 /etc/fstab 2024-09-10T20:28:49.452 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_7 /etc/fstab 2024-09-10T20:28:49.502 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_7 /etc/fstab 2024-09-10T20:28:49.502 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_7 /etc/fstab 2024-09-10T20:28:49.553 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_7 /etc/fstab 2024-09-10T20:28:49.554 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_7 /etc/fstab 2024-09-10T20:28:49.604 INFO:tasks.util.rados:rados -p unfounddel_pool put new_8 /etc/fstab 2024-09-10T20:28:49.604 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_8 /etc/fstab 2024-09-10T20:28:49.654 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_8 /etc/fstab 2024-09-10T20:28:49.655 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_8 /etc/fstab 2024-09-10T20:28:49.704 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_8 /etc/fstab 2024-09-10T20:28:49.704 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_8 /etc/fstab 2024-09-10T20:28:49.754 INFO:tasks.util.rados:rados -p unfounddel_pool put new_9 /etc/fstab 2024-09-10T20:28:49.754 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_9 /etc/fstab 2024-09-10T20:28:49.803 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_9 /etc/fstab 2024-09-10T20:28:49.803 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_9 /etc/fstab 2024-09-10T20:28:49.852 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_9 /etc/fstab 2024-09-10T20:28:49.853 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_9 /etc/fstab 2024-09-10T20:28:49.902 INFO:tasks.rep_lost_unfound_delete:osd.0 command_args is foo 2024-09-10T20:28:49.902 INFO:tasks.rep_lost_unfound_delete:() 2024-09-10T20:28:49.902 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-10T20:28:49.902 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 --osd-recovery-delay-start 1000 2024-09-10T20:28:49.905 INFO:tasks.ceph.osd.0:Started 2024-09-10T20:28:49.905 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-10T20:28:50.002 INFO:teuthology.orchestra.run.smithi131.stderr:no valid command found; 10 closest matches: 2024-09-10T20:28:50.003 INFO:teuthology.orchestra.run.smithi131.stderr:0 2024-09-10T20:28:50.003 INFO:teuthology.orchestra.run.smithi131.stderr:1 2024-09-10T20:28:50.003 INFO:teuthology.orchestra.run.smithi131.stderr:2 2024-09-10T20:28:50.003 INFO:teuthology.orchestra.run.smithi131.stderr:abort 2024-09-10T20:28:50.003 INFO:teuthology.orchestra.run.smithi131.stderr:assert 2024-09-10T20:28:50.003 INFO:teuthology.orchestra.run.smithi131.stderr:config diff 2024-09-10T20:28:50.004 INFO:teuthology.orchestra.run.smithi131.stderr:config diff get 2024-09-10T20:28:50.004 INFO:teuthology.orchestra.run.smithi131.stderr:config get 2024-09-10T20:28:50.004 INFO:teuthology.orchestra.run.smithi131.stderr:config help [] 2024-09-10T20:28:50.004 INFO:teuthology.orchestra.run.smithi131.stderr:config set ... 2024-09-10T20:28:50.004 INFO:teuthology.orchestra.run.smithi131.stderr:admin_socket: invalid command 2024-09-10T20:28:50.005 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-10T20:28:50.006 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-10T20:28:50.833 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:28:50.829+0000 7fb3360f1640 -1 Falling back to public interface 2024-09-10T20:28:55.007 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-10T20:28:55.110 INFO:teuthology.orchestra.run.smithi131.stderr:no valid command found; 10 closest matches: 2024-09-10T20:28:55.110 INFO:teuthology.orchestra.run.smithi131.stderr:0 2024-09-10T20:28:55.110 INFO:teuthology.orchestra.run.smithi131.stderr:1 2024-09-10T20:28:55.110 INFO:teuthology.orchestra.run.smithi131.stderr:2 2024-09-10T20:28:55.110 INFO:teuthology.orchestra.run.smithi131.stderr:abort 2024-09-10T20:28:55.110 INFO:teuthology.orchestra.run.smithi131.stderr:assert 2024-09-10T20:28:55.110 INFO:teuthology.orchestra.run.smithi131.stderr:bluefs debug_inject_read_zeros 2024-09-10T20:28:55.110 INFO:teuthology.orchestra.run.smithi131.stderr:bluefs files list 2024-09-10T20:28:55.110 INFO:teuthology.orchestra.run.smithi131.stderr:bluefs stats 2024-09-10T20:28:55.110 INFO:teuthology.orchestra.run.smithi131.stderr:bluestore allocator dump block 2024-09-10T20:28:55.111 INFO:teuthology.orchestra.run.smithi131.stderr:bluestore allocator fragmentation block 2024-09-10T20:28:55.111 INFO:teuthology.orchestra.run.smithi131.stderr:admin_socket: invalid command 2024-09-10T20:28:55.112 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-10T20:28:55.112 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-10T20:28:55.128 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:28:55.124+0000 7fb3360f1640 -1 osd.0 21 log_to_monitors true 2024-09-10T20:28:55.953 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:28:55.949+0000 7fb329e9f640 -1 osd.0 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T20:29:00.113 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-10T20:29:00.229 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-10T20:29:00.229 INFO:teuthology.orchestra.run.smithi131.stdout: "ops": [], 2024-09-10T20:29:00.230 INFO:teuthology.orchestra.run.smithi131.stdout: "num_ops": 0 2024-09-10T20:29:00.230 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-10T20:29:00.240 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2024-09-10T20:29:00.532 INFO:tasks.ceph.mon.a.smithi131.stderr:2024-09-10T20:29:00.529+0000 7f1e634ff640 -1 mon.a@0(leader).osd e26 definitely_dead 0 2024-09-10T20:29:00.533 INFO:teuthology.orchestra.run.smithi131.stderr:osd.0 is already in. 2024-09-10T20:29:00.545 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for osd.0 to be up 2024-09-10T20:29:00.546 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T20:29:00.832 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:29:00.833 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":26,"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","created":"2024-09-10T20:28:08.371455+0000","modified":"2024-09-10T20:28:57.937223+0000","last_up_change":"2024-09-10T20:28:56.932886+0000","last_in_change":"2024-09-10T20:28:41.533865+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-10T20:28:23.669739+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8700000047683716,"score_stable":1.8700000047683716,"optimal_score":0.67000001668930054,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfounddel_pool","create_time":"2024-09-10T20:28:36.617409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8700000047683716,"score_stable":1.8700000047683716,"optimal_score":0.67000001668930054,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d2261461-51c1-40ff-9ad5-8f4b10800e7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":21,"up_from":25,"up_thru":25,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":567435338},{"type":"v1","addr":"172.21.15.131:6801","nonce":567435338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":567435338},{"type":"v1","addr":"172.21.15.131:6803","nonce":567435338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":567435338},{"type":"v1","addr":"172.21.15.131:6807","nonce":567435338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":567435338},{"type":"v1","addr":"172.21.15.131:6805","nonce":567435338}]},"public_addr":"172.21.15.131:6801/567435338","cluster_addr":"172.21.15.131:6803/567435338","heartbeat_back_addr":"172.21.15.131:6807/567435338","heartbeat_front_addr":"172.21.15.131:6805/567435338","state":["exists","up"]},{"osd":1,"uuid":"9cb327f8-9ea4-489b-9cdf-1007a983d385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":454060533},{"type":"v1","addr":"172.21.15.131:6816","nonce":454060533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":454060533},{"type":"v1","addr":"172.21.15.131:6819","nonce":454060533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":454060533},{"type":"v1","addr":"172.21.15.131:6823","nonce":454060533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":454060533},{"type":"v1","addr":"172.21.15.131:6821","nonce":454060533}]},"public_addr":"172.21.15.131:6816/454060533","cluster_addr":"172.21.15.131:6819/454060533","heartbeat_back_addr":"172.21.15.131:6823/454060533","heartbeat_front_addr":"172.21.15.131:6821/454060533","state":["exists","up"]},{"osd":2,"uuid":"2a1df90e-2933-4052-a62a-ffa454708bb6","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":16,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2084549604},{"type":"v1","addr":"172.21.15.131:6801","nonce":2084549604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2084549604},{"type":"v1","addr":"172.21.15.131:6803","nonce":2084549604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2084549604},{"type":"v1","addr":"172.21.15.131:6807","nonce":2084549604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2084549604},{"type":"v1","addr":"172.21.15.131:6805","nonce":2084549604}]},"public_addr":"172.21.15.131:6801/2084549604","cluster_addr":"172.21.15.131:6803/2084549604","heartbeat_back_addr":"172.21.15.131:6807/2084549604","heartbeat_front_addr":"172.21.15.131:6805/2084549604","state":["exists"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:28:16.289792+0000","dead_epoch":22},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:28:15.574958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":65536,"last_purged_snaps_scrub":"2024-09-10T20:28:15.599924+0000","dead_epoch":19}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T20:29:00.846 INFO:tasks.rep_lost_unfound_delete.ceph_manager:osd.0 is up 2024-09-10T20:29:00.846 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-10T20:29:00.847 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-10T20:29:01.000 INFO:teuthology.orchestra.run.smithi131.stdout:107374182403 2024-09-10T20:29:01.001 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T20:29:01.002 INFO:teuthology.orchestra.run.smithi131.stdout:34359738382 2024-09-10T20:29:01.002 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T20:29:01.335 INFO:teuthology.orchestra.run.smithi131.stdout:34359738381 2024-09-10T20:29:01.336 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-09-10T20:29:01.349 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738382 got 34359738381 for osd.1 2024-09-10T20:29:01.350 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 107374182403 got 1 for osd.0 2024-09-10T20:29:02.350 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T20:29:02.351 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T20:29:02.658 INFO:teuthology.orchestra.run.smithi131.stdout:34359738381 2024-09-10T20:29:02.661 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-09-10T20:29:02.672 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738382 got 34359738381 for osd.1 2024-09-10T20:29:02.675 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 107374182403 got 1 for osd.0 2024-09-10T20:29:03.673 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T20:29:03.676 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T20:29:03.980 INFO:teuthology.orchestra.run.smithi131.stdout:107374182403 2024-09-10T20:29:03.981 INFO:teuthology.orchestra.run.smithi131.stdout:34359738382 2024-09-10T20:29:03.998 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 107374182403 got 107374182403 for osd.0 2024-09-10T20:29:03.999 DEBUG:teuthology.parallel:result is None 2024-09-10T20:29:03.999 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738382 got 34359738382 for osd.1 2024-09-10T20:29:03.999 DEBUG:teuthology.parallel:result is None 2024-09-10T20:29:03.999 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting till active 2024-09-10T20:29:04.000 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:29:04.280 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:29:04.280 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:29:04.293 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":47,"stamp":"2024-09-10T20:29:02.817470+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13543,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":27,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58,"ondisk_log_size":58,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55112,"kb_used_data":592,"kb_used_omap":5,"kb_used_meta":54458,"kb_avail":188688568,"statfs":{"total":193273528320,"available":193217093632,"internally_reserved":0,"allocated":606208,"data_stored":164756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6033,"internal_metadata":55765103},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":-3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.882323"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T20:28:57.953040+0000","last_change":"2024-09-10T20:28:57.952946+0000","last_active":"2024-09-10T20:28:57.953040+0000","last_peered":"2024-09-10T20:28:57.953040+0000","last_clean":"2024-09-10T20:28:57.953040+0000","last_became_active":"2024-09-10T20:28:57.952628+0000","last_became_peered":"2024-09-10T20:28:57.952628+0000","last_unstale":"2024-09-10T20:28:57.953040+0000","last_undegraded":"2024-09-10T20:28:57.953040+0000","last_fullsized":"2024-09-10T20:28:57.953040+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:01:33.359983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"23'4","reported_seq":30,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:28:57.952942+0000","last_change":"2024-09-10T20:28:57.953308+0000","last_active":"2024-09-10T20:28:57.952942+0000","last_peered":"2024-09-10T20:28:57.952942+0000","last_clean":"2024-09-10T20:28:45.872406+0000","last_became_active":"2024-09-10T20:28:57.952848+0000","last_became_peered":"2024-09-10T20:28:57.952848+0000","last_unstale":"2024-09-10T20:28:57.952942+0000","last_undegraded":"2024-09-10T20:28:57.947354+0000","last_fullsized":"2024-09-10T20:28:57.952942+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"23'3","reported_seq":44,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:28:57.954788+0000","last_change":"2024-09-10T20:28:57.954966+0000","last_active":"2024-09-10T20:28:57.954788+0000","last_peered":"2024-09-10T20:28:57.954788+0000","last_clean":"2024-09-10T20:28:46.420618+0000","last_became_active":"2024-09-10T20:28:57.954689+0000","last_became_peered":"2024-09-10T20:28:57.954689+0000","last_unstale":"2024-09-10T20:28:57.954788+0000","last_undegraded":"2024-09-10T20:28:57.947570+0000","last_fullsized":"2024-09-10T20:28:57.954788+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":31,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T20:28:57.953467+0000","last_change":"2024-09-10T20:28:57.953378+0000","last_active":"2024-09-10T20:28:57.953467+0000","last_peered":"2024-09-10T20:28:57.953467+0000","last_clean":"2024-09-10T20:28:57.953467+0000","last_became_active":"2024-09-10T20:28:57.953005+0000","last_became_peered":"2024-09-10T20:28:57.953005+0000","last_unstale":"2024-09-10T20:28:57.953467+0000","last_undegraded":"2024-09-10T20:28:57.953467+0000","last_fullsized":"2024-09-10T20:28:57.953467+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:49:56.659658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"23'5","reported_seq":34,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:28:57.952364+0000","last_change":"2024-09-10T20:28:57.952624+0000","last_active":"2024-09-10T20:28:57.952364+0000","last_peered":"2024-09-10T20:28:57.952364+0000","last_clean":"2024-09-10T20:28:45.772428+0000","last_became_active":"2024-09-10T20:28:57.952163+0000","last_became_peered":"2024-09-10T20:28:57.952163+0000","last_unstale":"2024-09-10T20:28:57.952364+0000","last_undegraded":"2024-09-10T20:28:57.946122+0000","last_fullsized":"2024-09-10T20:28:57.952364+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"23'3","reported_seq":39,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:28:57.954471+0000","last_change":"2024-09-10T20:28:57.954639+0000","last_active":"2024-09-10T20:28:57.954471+0000","last_peered":"2024-09-10T20:28:57.954471+0000","last_clean":"2024-09-10T20:28:46.621675+0000","last_became_active":"2024-09-10T20:28:57.954371+0000","last_became_peered":"2024-09-10T20:28:57.954371+0000","last_unstale":"2024-09-10T20:28:57.954471+0000","last_undegraded":"2024-09-10T20:28:57.947548+0000","last_fullsized":"2024-09-10T20:28:57.954471+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"23'4","reported_seq":40,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:28:57.955618+0000","last_change":"2024-09-10T20:28:57.955806+0000","last_active":"2024-09-10T20:28:57.955618+0000","last_peered":"2024-09-10T20:28:57.955618+0000","last_clean":"2024-09-10T20:28:46.021581+0000","last_became_active":"2024-09-10T20:28:57.955527+0000","last_became_peered":"2024-09-10T20:28:57.955527+0000","last_unstale":"2024-09-10T20:28:57.955618+0000","last_undegraded":"2024-09-10T20:28:57.946696+0000","last_fullsized":"2024-09-10T20:28:57.955618+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"23'1","reported_seq":31,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:28:57.952389+0000","last_change":"2024-09-10T20:28:57.952681+0000","last_active":"2024-09-10T20:28:57.952389+0000","last_peered":"2024-09-10T20:28:57.952389+0000","last_clean":"2024-09-10T20:28:40.535680+0000","last_became_active":"2024-09-10T20:28:57.952299+0000","last_became_peered":"2024-09-10T20:28:57.952299+0000","last_unstale":"2024-09-10T20:28:57.952389+0000","last_undegraded":"2024-09-10T20:28:57.946168+0000","last_fullsized":"2024-09-10T20:28:57.952389+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":53,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T20:28:57.952947+0000","last_change":"2024-09-10T20:28:57.952857+0000","last_active":"2024-09-10T20:28:57.952947+0000","last_peered":"2024-09-10T20:28:57.952947+0000","last_clean":"2024-09-10T20:28:57.952947+0000","last_became_active":"2024-09-10T20:28:57.952525+0000","last_became_peered":"2024-09-10T20:28:57.952525+0000","last_unstale":"2024-09-10T20:28:57.952947+0000","last_undegraded":"2024-09-10T20:28:57.952947+0000","last_fullsized":"2024-09-10T20:28:57.952947+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:45.850968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"23'5","reported_seq":46,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:28:57.955846+0000","last_change":"2024-09-10T20:28:57.956434+0000","last_active":"2024-09-10T20:28:57.955846+0000","last_peered":"2024-09-10T20:28:57.955846+0000","last_clean":"2024-09-10T20:28:46.470645+0000","last_became_active":"2024-09-10T20:28:57.955757+0000","last_became_peered":"2024-09-10T20:28:57.955757+0000","last_unstale":"2024-09-10T20:28:57.955846+0000","last_undegraded":"2024-09-10T20:28:57.947921+0000","last_fullsized":"2024-09-10T20:28:57.955846+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":44,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T20:28:57.953768+0000","last_change":"2024-09-10T20:28:57.953029+0000","last_active":"2024-09-10T20:28:57.953768+0000","last_peered":"2024-09-10T20:28:57.953768+0000","last_clean":"2024-09-10T20:28:57.953768+0000","last_became_active":"2024-09-10T20:28:57.951812+0000","last_became_peered":"2024-09-10T20:28:57.951812+0000","last_unstale":"2024-09-10T20:28:57.953768+0000","last_undegraded":"2024-09-10T20:28:57.953768+0000","last_fullsized":"2024-09-10T20:28:57.953768+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:17:12.140395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'6","reported_seq":32,"reported_epoch":26,"state":"active+recovering+degraded","last_fresh":"2024-09-10T20:28:57.952423+0000","last_change":"2024-09-10T20:28:57.957077+0000","last_active":"2024-09-10T20:28:57.952423+0000","last_peered":"2024-09-10T20:28:57.952423+0000","last_clean":"2024-09-10T20:28:46.672065+0000","last_became_active":"2024-09-10T20:28:57.952071+0000","last_became_peered":"2024-09-10T20:28:57.952071+0000","last_unstale":"2024-09-10T20:28:57.952423+0000","last_undegraded":"2024-09-10T20:28:57.946645+0000","last_fullsized":"2024-09-10T20:28:57.952423+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":38,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T20:28:57.953188+0000","last_change":"2024-09-10T20:28:57.953086+0000","last_active":"2024-09-10T20:28:57.953188+0000","last_peered":"2024-09-10T20:28:57.953188+0000","last_clean":"2024-09-10T20:28:57.953188+0000","last_became_active":"2024-09-10T20:28:57.952342+0000","last_became_peered":"2024-09-10T20:28:57.952342+0000","last_unstale":"2024-09-10T20:28:57.953188+0000","last_undegraded":"2024-09-10T20:28:57.953188+0000","last_fullsized":"2024-09-10T20:28:57.953188+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:07.618480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'3","reported_seq":39,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:28:57.954761+0000","last_change":"2024-09-10T20:28:57.954935+0000","last_active":"2024-09-10T20:28:57.954761+0000","last_peered":"2024-09-10T20:28:57.954761+0000","last_clean":"2024-09-10T20:28:45.722105+0000","last_became_active":"2024-09-10T20:28:57.954659+0000","last_became_peered":"2024-09-10T20:28:57.954659+0000","last_unstale":"2024-09-10T20:28:57.954761+0000","last_undegraded":"2024-09-10T20:28:57.946686+0000","last_fullsized":"2024-09-10T20:28:57.954761+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":63,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T20:28:57.956697+0000","last_change":"2024-09-10T20:28:57.956607+0000","last_active":"2024-09-10T20:28:57.956697+0000","last_peered":"2024-09-10T20:28:57.956697+0000","last_clean":"2024-09-10T20:28:57.956697+0000","last_became_active":"2024-09-10T20:28:57.956196+0000","last_became_peered":"2024-09-10T20:28:57.956196+0000","last_unstale":"2024-09-10T20:28:57.956697+0000","last_undegraded":"2024-09-10T20:28:57.956697+0000","last_fullsized":"2024-09-10T20:28:57.956697+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:10.581574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"23'4","reported_seq":41,"reported_epoch":26,"state":"active+recovering+degraded","last_fresh":"2024-09-10T20:28:57.950136+0000","last_change":"2024-09-10T20:28:57.953517+0000","last_active":"2024-09-10T20:28:57.950136+0000","last_peered":"2024-09-10T20:28:57.950136+0000","last_clean":"2024-09-10T20:28:46.120829+0000","last_became_active":"2024-09-10T20:28:57.949847+0000","last_became_peered":"2024-09-10T20:28:57.949847+0000","last_unstale":"2024-09-10T20:28:57.950136+0000","last_undegraded":"2024-09-10T20:28:57.945691+0000","last_fullsized":"2024-09-10T20:28:57.950136+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"23'1","reported_seq":36,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:28:57.953355+0000","last_change":"2024-09-10T20:28:57.953527+0000","last_active":"2024-09-10T20:28:57.953355+0000","last_peered":"2024-09-10T20:28:57.953355+0000","last_clean":"2024-09-10T20:28:42.552296+0000","last_became_active":"2024-09-10T20:28:57.953265+0000","last_became_peered":"2024-09-10T20:28:57.953265+0000","last_unstale":"2024-09-10T20:28:57.953355+0000","last_undegraded":"2024-09-10T20:28:57.946626+0000","last_fullsized":"2024-09-10T20:28:57.953355+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":53,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T20:28:57.955651+0000","last_change":"2024-09-10T20:28:57.955552+0000","last_active":"2024-09-10T20:28:57.955651+0000","last_peered":"2024-09-10T20:28:57.955651+0000","last_clean":"2024-09-10T20:28:57.955651+0000","last_became_active":"2024-09-10T20:28:57.955272+0000","last_became_peered":"2024-09-10T20:28:57.955272+0000","last_unstale":"2024-09-10T20:28:57.955651+0000","last_undegraded":"2024-09-10T20:28:57.955651+0000","last_fullsized":"2024-09-10T20:28:57.955651+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:32:15.992591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'5","reported_seq":41,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:28:57.955822+0000","last_change":"2024-09-10T20:28:57.955989+0000","last_active":"2024-09-10T20:28:57.955822+0000","last_peered":"2024-09-10T20:28:57.955822+0000","last_clean":"2024-09-10T20:28:46.571322+0000","last_became_active":"2024-09-10T20:28:57.955712+0000","last_became_peered":"2024-09-10T20:28:57.955712+0000","last_unstale":"2024-09-10T20:28:57.955822+0000","last_undegraded":"2024-09-10T20:28:57.948623+0000","last_fullsized":"2024-09-10T20:28:57.955822+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":3}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":44,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T20:28:57.952401+0000","last_change":"2024-09-10T20:28:57.952296+0000","last_active":"2024-09-10T20:28:57.952401+0000","last_peered":"2024-09-10T20:28:57.952401+0000","last_clean":"2024-09-10T20:28:57.952401+0000","last_became_active":"2024-09-10T20:28:57.951870+0000","last_became_peered":"2024-09-10T20:28:57.951870+0000","last_unstale":"2024-09-10T20:28:57.952401+0000","last_undegraded":"2024-09-10T20:28:57.952401+0000","last_fullsized":"2024-09-10T20:28:57.952401+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:07:38.399386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"23'2","reported_seq":37,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:28:57.955424+0000","last_change":"2024-09-10T20:28:57.955656+0000","last_active":"2024-09-10T20:28:57.955424+0000","last_peered":"2024-09-10T20:28:57.955424+0000","last_clean":"2024-09-10T20:28:42.552437+0000","last_became_active":"2024-09-10T20:28:57.955338+0000","last_became_peered":"2024-09-10T20:28:57.955338+0000","last_unstale":"2024-09-10T20:28:57.955424+0000","last_undegraded":"2024-09-10T20:28:57.948036+0000","last_fullsized":"2024-09-10T20:28:57.955424+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":53,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T20:28:57.956344+0000","last_change":"2024-09-10T20:28:57.956248+0000","last_active":"2024-09-10T20:28:57.956344+0000","last_peered":"2024-09-10T20:28:57.956344+0000","last_clean":"2024-09-10T20:28:57.956344+0000","last_became_active":"2024-09-10T20:28:57.955944+0000","last_became_peered":"2024-09-10T20:28:57.955944+0000","last_unstale":"2024-09-10T20:28:57.956344+0000","last_undegraded":"2024-09-10T20:28:57.956344+0000","last_fullsized":"2024-09-10T20:28:57.956344+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:25:32.460559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":58,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T20:28:57.954141+0000","last_change":"2024-09-10T20:28:57.953824+0000","last_active":"2024-09-10T20:28:57.954141+0000","last_peered":"2024-09-10T20:28:57.954141+0000","last_clean":"2024-09-10T20:28:57.954141+0000","last_became_active":"2024-09-10T20:28:57.953449+0000","last_became_peered":"2024-09-10T20:28:57.953449+0000","last_unstale":"2024-09-10T20:28:57.954141+0000","last_undegraded":"2024-09-10T20:28:57.954141+0000","last_fullsized":"2024-09-10T20:28:57.954141+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:14.515384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"23'9","reported_seq":45,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:28:57.955348+0000","last_change":"2024-09-10T20:28:57.957124+0000","last_active":"2024-09-10T20:28:57.955348+0000","last_peered":"2024-09-10T20:28:57.955348+0000","last_clean":"2024-09-10T20:28:46.321317+0000","last_became_active":"2024-09-10T20:28:57.955259+0000","last_became_peered":"2024-09-10T20:28:57.955259+0000","last_unstale":"2024-09-10T20:28:57.955348+0000","last_undegraded":"2024-09-10T20:28:57.947776+0000","last_fullsized":"2024-09-10T20:28:57.955348+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1932,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"0,1","objects":1},{"shards":"1","objects":3}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":13524,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":27,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":18354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182403,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":260,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":266240,"data_stored":78031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":28110506},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738382,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":339968,"data_stored":86725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:29:04.294 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:29:04.646 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:29:04.646 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 27/60 objects degraded (45.000%), 14 pgs degraded","count":14},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":56,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":26,"num_osds":3,"num_up_osds":2,"osd_up_since":1726000136,"num_in_osds":2,"osd_in_since":1726000121,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+recovery_wait+degraded","count":12},{"state_name":"active+clean","count":10},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":13543,"bytes_used":56434688,"bytes_avail":193217093632,"bytes_total":193273528320,"degraded_objects":27,"degraded_total":60,"degraded_ratio":0.45000000000000001},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-09-10T20:28:08.372376+0000","services":{}},"progress_events":{}} 2024-09-10T20:29:04.659 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'fsid': '3053cb46-0512-45f5-9707-4eb48df5a665', 'health': {'status': 'HEALTH_WARN', 'checks': {'PG_DEGRADED': {'severity': 'HEALTH_WARN', 'summary': {'message': 'Degraded data redundancy: 27/60 objects degraded (45.000%), 14 pgs degraded', 'count': 14}, 'muted': False}}, 'mutes': []}, 'election_epoch': 6, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 56, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 26, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1726000136, 'num_in_osds': 2, 'osd_in_since': 1726000121, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+recovery_wait+degraded', 'count': 12}, {'state_name': 'active+clean', 'count': 10}, {'state_name': 'active+recovering+degraded', 'count': 2}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 13543, 'bytes_used': 56434688, 'bytes_avail': 193217093632, 'bytes_total': 193273528320, 'degraded_objects': 27, 'degraded_total': 60, 'degraded_ratio': 0.45}, 'fsmap': {'epoch': 1, 'btime': '2024-09-10T20:28:08:371114+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 1, 'modified': '2024-09-10T20:28:08.372376+0000', 'services': {}}, 'progress_events': {}} 2024-09-10T20:29:04.659 INFO:tasks.rep_lost_unfound_delete.ceph_manager:active! 2024-09-10T20:29:04.659 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-10T20:29:04.659 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T20:29:04.707 INFO:tasks.ceph.osd.1:Stopped 2024-09-10T20:29:04.707 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 1 2024-09-10T20:29:05.004 INFO:tasks.ceph.mon.a.smithi131.stderr:2024-09-10T20:29:04.999+0000 7f1e634ff640 -1 mon.a@0(leader).osd e27 definitely_dead 0 2024-09-10T20:29:05.004 INFO:teuthology.orchestra.run.smithi131.stderr:osd.1 is already down. 2024-09-10T20:29:05.017 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2024-09-10T20:29:05.309 INFO:tasks.ceph.mon.a.smithi131.stderr:2024-09-10T20:29:05.304+0000 7f1e634ff640 -1 mon.a@0(leader).osd e27 definitely_dead 0 2024-09-10T20:29:05.987 INFO:teuthology.orchestra.run.smithi131.stderr:marked out osd.1. 2024-09-10T20:29:06.002 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd lost 1 --yes-i-really-mean-it 2024-09-10T20:29:06.992 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:06.986+0000 7fb3185c9640 -1 log_channel(cluster) log [ERR] : 2.0 has 2 objects unfound and apparently lost 2024-09-10T20:29:06.992 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:06.986+0000 7fb313dc0640 -1 log_channel(cluster) log [ERR] : 2.1 has 2 objects unfound and apparently lost 2024-09-10T20:29:06.992 INFO:teuthology.orchestra.run.smithi131.stderr:marked osd lost in epoch 27 2024-09-10T20:29:06.993 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:06.986+0000 7fb317dc8640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-10T20:29:06.994 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:06.986+0000 7fb3175c7640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-10T20:29:06.994 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:06.986+0000 7fb3145c1640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-10T20:29:06.994 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:06.986+0000 7fb312dbe640 -1 log_channel(cluster) log [ERR] : 2.3 has 2 objects unfound and apparently lost 2024-09-10T20:29:06.994 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:06.986+0000 7fb3135bf640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-10T20:29:06.994 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:06.987+0000 7fb315dc4640 -1 log_channel(cluster) log [ERR] : 2.d has 1 objects unfound and apparently lost 2024-09-10T20:29:06.994 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:06.987+0000 7fb314dc2640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-10T20:29:06.994 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:06.987+0000 7fb312dbe640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-10T20:29:06.994 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:06.987+0000 7fb315dc4640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-10T20:29:06.994 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:06.987+0000 7fb3125bd640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-10T20:29:06.994 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:06.987+0000 7fb3155c3640 -1 log_channel(cluster) log [ERR] : 2.e has 2 objects unfound and apparently lost 2024-09-10T20:29:06.994 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:06.987+0000 7fb3115bb640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-10T20:29:07.008 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-10T20:29:07.008 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-10T20:29:07.011 INFO:tasks.ceph.osd.2:Started 2024-09-10T20:29:07.011 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-10T20:29:07.114 INFO:teuthology.orchestra.run.smithi131.stderr:no valid command found; 10 closest matches: 2024-09-10T20:29:07.115 INFO:teuthology.orchestra.run.smithi131.stderr:0 2024-09-10T20:29:07.115 INFO:teuthology.orchestra.run.smithi131.stderr:1 2024-09-10T20:29:07.115 INFO:teuthology.orchestra.run.smithi131.stderr:2 2024-09-10T20:29:07.115 INFO:teuthology.orchestra.run.smithi131.stderr:abort 2024-09-10T20:29:07.115 INFO:teuthology.orchestra.run.smithi131.stderr:assert 2024-09-10T20:29:07.115 INFO:teuthology.orchestra.run.smithi131.stderr:config diff 2024-09-10T20:29:07.115 INFO:teuthology.orchestra.run.smithi131.stderr:config diff get 2024-09-10T20:29:07.115 INFO:teuthology.orchestra.run.smithi131.stderr:config get 2024-09-10T20:29:07.115 INFO:teuthology.orchestra.run.smithi131.stderr:config help [] 2024-09-10T20:29:07.115 INFO:teuthology.orchestra.run.smithi131.stderr:config set ... 2024-09-10T20:29:07.115 INFO:teuthology.orchestra.run.smithi131.stderr:admin_socket: invalid command 2024-09-10T20:29:07.117 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-10T20:29:07.117 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-10T20:29:08.677 INFO:tasks.ceph.osd.2.smithi131.stderr:2024-09-10T20:29:08.672+0000 7f6c647bd640 -1 Falling back to public interface 2024-09-10T20:29:12.118 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-10T20:29:12.224 INFO:teuthology.orchestra.run.smithi131.stderr:no valid command found; 10 closest matches: 2024-09-10T20:29:12.224 INFO:teuthology.orchestra.run.smithi131.stderr:0 2024-09-10T20:29:12.225 INFO:teuthology.orchestra.run.smithi131.stderr:1 2024-09-10T20:29:12.225 INFO:teuthology.orchestra.run.smithi131.stderr:2 2024-09-10T20:29:12.225 INFO:teuthology.orchestra.run.smithi131.stderr:abort 2024-09-10T20:29:12.225 INFO:teuthology.orchestra.run.smithi131.stderr:assert 2024-09-10T20:29:12.225 INFO:teuthology.orchestra.run.smithi131.stderr:bluestore allocator dump block 2024-09-10T20:29:12.225 INFO:teuthology.orchestra.run.smithi131.stderr:bluestore allocator fragmentation block 2024-09-10T20:29:12.225 INFO:teuthology.orchestra.run.smithi131.stderr:bluestore allocator fragmentation histogram block [] [] 2024-09-10T20:29:12.225 INFO:teuthology.orchestra.run.smithi131.stderr:bluestore allocator score block 2024-09-10T20:29:12.225 INFO:teuthology.orchestra.run.smithi131.stderr:config diff 2024-09-10T20:29:12.226 INFO:teuthology.orchestra.run.smithi131.stderr:admin_socket: invalid command 2024-09-10T20:29:12.226 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-10T20:29:12.227 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-10T20:29:12.459 INFO:tasks.ceph.osd.2.smithi131.stderr:2024-09-10T20:29:12.454+0000 7f6c647bd640 -1 osd.2 18 log_to_monitors true 2024-09-10T20:29:12.989 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:12.983+0000 7fb3145c1640 -1 log_channel(cluster) log [ERR] : 2.0 has 2 objects unfound and apparently lost 2024-09-10T20:29:12.989 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:12.983+0000 7fb317dc8640 -1 log_channel(cluster) log [ERR] : 2.1 has 2 objects unfound and apparently lost 2024-09-10T20:29:12.989 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:12.983+0000 7fb313dc0640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-10T20:29:12.989 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:12.983+0000 7fb3145c1640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-10T20:29:12.989 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:12.983+0000 7fb316dc6640 -1 log_channel(cluster) log [ERR] : 2.3 has 2 objects unfound and apparently lost 2024-09-10T20:29:12.989 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:12.983+0000 7fb3135bf640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-10T20:29:12.989 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:12.983+0000 7fb314dc2640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-10T20:29:12.990 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:12.983+0000 7fb312dbe640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-10T20:29:12.990 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:12.983+0000 7fb3135bf640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-10T20:29:12.990 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:12.983+0000 7fb3155c3640 -1 log_channel(cluster) log [ERR] : 2.e has 2 objects unfound and apparently lost 2024-09-10T20:29:12.990 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:12.983+0000 7fb311dbc640 -1 log_channel(cluster) log [ERR] : 2.d has 1 objects unfound and apparently lost 2024-09-10T20:29:12.990 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:12.983+0000 7fb3155c3640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-10T20:29:12.990 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:12.983+0000 7fb3125bd640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-10T20:29:12.990 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:12.984+0000 7fb311dbc640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-10T20:29:13.002 INFO:tasks.ceph.osd.2.smithi131.stderr:2024-09-10T20:29:12.998+0000 7f6c5c573640 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T20:29:13.993 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:13.988+0000 7fb3145c1640 -1 log_channel(cluster) log [ERR] : 2.0 has 2 objects unfound and apparently lost 2024-09-10T20:29:13.993 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:13.988+0000 7fb313dc0640 -1 log_channel(cluster) log [ERR] : 2.1 has 2 objects unfound and apparently lost 2024-09-10T20:29:13.994 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:13.988+0000 7fb3145c1640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-10T20:29:13.996 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:13.989+0000 7fb311dbc640 -1 log_channel(cluster) log [ERR] : 2.d has 1 objects unfound and apparently lost 2024-09-10T20:29:13.996 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:13.989+0000 7fb311dbc640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-10T20:29:13.996 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:13.989+0000 7fb317dc8640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-10T20:29:13.996 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:13.989+0000 7fb3175c7640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-10T20:29:13.996 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:13.990+0000 7fb312dbe640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-10T20:29:13.996 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:13.990+0000 7fb3135bf640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-10T20:29:13.996 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:13.990+0000 7fb310dba640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-10T20:29:13.996 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:13.990+0000 7fb316dc6640 -1 log_channel(cluster) log [ERR] : 2.3 has 2 objects unfound and apparently lost 2024-09-10T20:29:13.996 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:13.990+0000 7fb3125bd640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-10T20:29:13.997 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:13.990+0000 7fb3155c3640 -1 log_channel(cluster) log [ERR] : 2.e has 2 objects unfound and apparently lost 2024-09-10T20:29:13.997 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:13.990+0000 7fb3115bb640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-10T20:29:14.997 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:14.993+0000 7fb3175c7640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-10T20:29:14.998 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:14.994+0000 7fb3165c5640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-10T20:29:14.998 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:14.994+0000 7fb310dba640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-10T20:29:14.998 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:14.994+0000 7fb313dc0640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-10T20:29:14.999 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:14.995+0000 7fb3135bf640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-10T20:29:14.999 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:14.995+0000 7fb3115bb640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-10T20:29:15.000 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:14.996+0000 7fb315dc4640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-10T20:29:15.999 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:15.995+0000 7fb3135bf640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-10T20:29:16.000 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:15.995+0000 7fb3135bf640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-10T20:29:16.000 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:15.996+0000 7fb313dc0640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-10T20:29:16.001 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:15.996+0000 7fb310dba640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-10T20:29:16.001 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:15.997+0000 7fb3165c5640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-10T20:29:16.001 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:15.997+0000 7fb315dc4640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-10T20:29:16.001 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:15.997+0000 7fb3115bb640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-10T20:29:17.228 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-10T20:29:17.344 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-10T20:29:17.344 INFO:teuthology.orchestra.run.smithi131.stdout: "ops": [], 2024-09-10T20:29:17.344 INFO:teuthology.orchestra.run.smithi131.stdout: "num_ops": 0 2024-09-10T20:29:17.344 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-10T20:29:17.353 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-09-10T20:29:17.645 INFO:tasks.ceph.mon.a.smithi131.stderr:2024-09-10T20:29:17.641+0000 7f1e634ff640 -1 mon.a@0(leader).osd e33 definitely_dead 0 2024-09-10T20:29:18.012 INFO:teuthology.orchestra.run.smithi131.stderr:marked in osd.2. 2024-09-10T20:29:18.034 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for osd.2 to be up 2024-09-10T20:29:18.034 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T20:29:18.320 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:29:18.320 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":34,"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","created":"2024-09-10T20:28:08.371455+0000","modified":"2024-09-10T20:29:18.008222+0000","last_up_change":"2024-09-10T20:29:13.985086+0000","last_in_change":"2024-09-10T20:29:18.008222+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-10T20:28:23.669739+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":1.4900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":1,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfounddel_pool","create_time":"2024-09-10T20:28:36.617409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":1.8700000047683716,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d2261461-51c1-40ff-9ad5-8f4b10800e7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":21,"up_from":25,"up_thru":32,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":567435338},{"type":"v1","addr":"172.21.15.131:6801","nonce":567435338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":567435338},{"type":"v1","addr":"172.21.15.131:6803","nonce":567435338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":567435338},{"type":"v1","addr":"172.21.15.131:6807","nonce":567435338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":567435338},{"type":"v1","addr":"172.21.15.131:6805","nonce":567435338}]},"public_addr":"172.21.15.131:6801/567435338","cluster_addr":"172.21.15.131:6803/567435338","heartbeat_back_addr":"172.21.15.131:6807/567435338","heartbeat_front_addr":"172.21.15.131:6805/567435338","state":["exists","up"]},{"osd":1,"uuid":"9cb327f8-9ea4-489b-9cdf-1007a983d385","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":25,"down_at":27,"lost_at":27,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":454060533},{"type":"v1","addr":"172.21.15.131:6816","nonce":454060533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":454060533},{"type":"v1","addr":"172.21.15.131:6819","nonce":454060533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":454060533},{"type":"v1","addr":"172.21.15.131:6823","nonce":454060533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":454060533},{"type":"v1","addr":"172.21.15.131:6821","nonce":454060533}]},"public_addr":"172.21.15.131:6816/454060533","cluster_addr":"172.21.15.131:6819/454060533","heartbeat_back_addr":"172.21.15.131:6823/454060533","heartbeat_front_addr":"172.21.15.131:6821/454060533","state":["exists"]},{"osd":2,"uuid":"2a1df90e-2933-4052-a62a-ffa454708bb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":18,"up_from":31,"up_thru":16,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":2043655954},{"type":"v1","addr":"172.21.15.131:6809","nonce":2043655954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2043655954},{"type":"v1","addr":"172.21.15.131:6811","nonce":2043655954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2043655954},{"type":"v1","addr":"172.21.15.131:6815","nonce":2043655954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2043655954},{"type":"v1","addr":"172.21.15.131:6813","nonce":2043655954}]},"public_addr":"172.21.15.131:6809/2043655954","cluster_addr":"172.21.15.131:6811/2043655954","heartbeat_back_addr":"172.21.15.131:6815/2043655954","heartbeat_front_addr":"172.21.15.131:6813/2043655954","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:28:16.289792+0000","dead_epoch":22},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":65536,"last_purged_snaps_scrub":"2024-09-10T20:28:15.574958+0000","dead_epoch":27},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:28:15.599924+0000","dead_epoch":19}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"1.1","osds":[0,2]},{"pgid":"1.3","osds":[0,2]},{"pgid":"1.5","osds":[0,2]},{"pgid":"1.7","osds":[0,2]},{"pgid":"2.0","osds":[0,2]},{"pgid":"2.1","osds":[0,2]},{"pgid":"2.3","osds":[0,2]},{"pgid":"2.c","osds":[0,2]},{"pgid":"2.d","osds":[0,2]},{"pgid":"2.e","osds":[0,2]}],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T20:29:18.333 INFO:tasks.rep_lost_unfound_delete.ceph_manager:osd.2 is up 2024-09-10T20:29:18.333 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-10T20:29:18.334 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-10T20:29:18.467 INFO:teuthology.orchestra.run.smithi131.stdout:107374182407 2024-09-10T20:29:18.468 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T20:29:18.469 INFO:teuthology.orchestra.run.smithi131.stdout:133143986179 2024-09-10T20:29:18.469 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T20:29:18.794 INFO:teuthology.orchestra.run.smithi131.stdout:107374182406 2024-09-10T20:29:18.795 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-09-10T20:29:18.808 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 107374182407 got 107374182406 for osd.0 2024-09-10T20:29:18.808 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 133143986179 got 1 for osd.2 2024-09-10T20:29:19.809 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T20:29:19.810 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T20:29:20.018 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:20.014+0000 7fb3125bd640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-10T20:29:20.018 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:20.014+0000 7fb315dc4640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-10T20:29:20.018 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:20.014+0000 7fb3135bf640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-10T20:29:20.019 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:20.014+0000 7fb316dc6640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-10T20:29:20.019 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:20.014+0000 7fb3135bf640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-10T20:29:20.019 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:20.015+0000 7fb310dba640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-10T20:29:20.019 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:20.015+0000 7fb3145c1640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-10T20:29:20.019 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:20.015+0000 7fb313dc0640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-10T20:29:20.019 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-10T20:29:20.015+0000 7fb3115bb640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-10T20:29:20.120 INFO:teuthology.orchestra.run.smithi131.stdout:107374182406 2024-09-10T20:29:20.123 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-09-10T20:29:20.134 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 107374182407 got 107374182406 for osd.0 2024-09-10T20:29:20.137 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 133143986179 got 1 for osd.2 2024-09-10T20:29:21.135 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T20:29:21.138 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T20:29:21.443 INFO:teuthology.orchestra.run.smithi131.stdout:107374182407 2024-09-10T20:29:21.445 INFO:teuthology.orchestra.run.smithi131.stdout:133143986179 2024-09-10T20:29:21.460 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 107374182407 got 107374182407 for osd.0 2024-09-10T20:29:21.460 DEBUG:teuthology.parallel:result is None 2024-09-10T20:29:21.462 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 133143986179 got 133143986179 for osd.2 2024-09-10T20:29:21.462 DEBUG:teuthology.parallel:result is None 2024-09-10T20:29:21.462 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting till active 2024-09-10T20:29:21.462 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:29:21.780 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:29:21.781 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:29:21.793 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-09-10T20:29:20.821356+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13543,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58,"ondisk_log_size":58,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55612,"kb_used_data":596,"kb_used_omap":8,"kb_used_meta":54967,"kb_avail":188688068,"statfs":{"total":193273528320,"available":193216581632,"internally_reserved":0,"allocated":610304,"data_stored":204120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8876,"internal_metadata":56286548},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-32,"num_objects_misplaced":0,"num_objects_unfound":-15,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.827136"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":40,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-10T20:29:15.997804+0000","last_change":"2024-09-10T20:29:18.019538+0000","last_active":"2024-09-10T20:29:18.019538+0000","last_peered":"2024-09-10T20:29:15.997804+0000","last_clean":"2024-09-10T20:28:57.955211+0000","last_became_active":"2024-09-10T20:29:05.983826+0000","last_became_peered":"2024-09-10T20:29:05.983826+0000","last_unstale":"2024-09-10T20:29:15.997804+0000","last_undegraded":"2024-09-10T20:29:15.997804+0000","last_fullsized":"2024-09-10T20:29:05.981204+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"23'4","reported_seq":50,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-10T20:29:16.005108+0000","last_change":"2024-09-10T20:29:18.019120+0000","last_active":"2024-09-10T20:29:18.019120+0000","last_peered":"2024-09-10T20:29:16.005108+0000","last_clean":"2024-09-10T20:28:45.872406+0000","last_became_active":"2024-09-10T20:29:16.004797+0000","last_became_peered":"2024-09-10T20:29:16.004797+0000","last_unstale":"2024-09-10T20:29:16.005108+0000","last_undegraded":"2024-09-10T20:29:15.998492+0000","last_fullsized":"2024-09-10T20:29:16.005108+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"23'3","reported_seq":62,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-10T20:29:16.004897+0000","last_change":"2024-09-10T20:29:18.017247+0000","last_active":"2024-09-10T20:29:18.017247+0000","last_peered":"2024-09-10T20:29:16.004897+0000","last_clean":"2024-09-10T20:28:46.420618+0000","last_became_active":"2024-09-10T20:29:16.004579+0000","last_became_peered":"2024-09-10T20:29:16.004579+0000","last_unstale":"2024-09-10T20:29:16.004897+0000","last_undegraded":"2024-09-10T20:29:15.998202+0000","last_fullsized":"2024-09-10T20:29:16.004897+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":55,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-10T20:29:16.006133+0000","last_change":"2024-09-10T20:29:18.016100+0000","last_active":"2024-09-10T20:29:18.016100+0000","last_peered":"2024-09-10T20:29:16.006133+0000","last_clean":"2024-09-10T20:29:16.006133+0000","last_became_active":"2024-09-10T20:29:16.004155+0000","last_became_peered":"2024-09-10T20:29:16.004155+0000","last_unstale":"2024-09-10T20:29:16.006133+0000","last_undegraded":"2024-09-10T20:29:16.006133+0000","last_fullsized":"2024-09-10T20:29:16.006133+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"23'5","reported_seq":55,"reported_epoch":34,"state":"peering","last_fresh":"2024-09-10T20:29:18.018785+0000","last_change":"2024-09-10T20:29:18.018785+0000","last_active":"2024-09-10T20:29:18.018785+0000","last_peered":"2024-09-10T20:29:16.004839+0000","last_clean":"2024-09-10T20:28:45.772428+0000","last_became_active":"2024-09-10T20:29:16.004537+0000","last_became_peered":"2024-09-10T20:29:16.004537+0000","last_unstale":"2024-09-10T20:29:18.018785+0000","last_undegraded":"2024-09-10T20:29:18.018785+0000","last_fullsized":"2024-09-10T20:29:18.018785+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"23'3","reported_seq":54,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-10T20:29:15.996409+0000","last_change":"2024-09-10T20:29:18.015716+0000","last_active":"2024-09-10T20:29:18.015716+0000","last_peered":"2024-09-10T20:29:15.996409+0000","last_clean":"2024-09-10T20:28:46.621675+0000","last_became_active":"2024-09-10T20:29:05.982935+0000","last_became_peered":"2024-09-10T20:29:05.982935+0000","last_unstale":"2024-09-10T20:29:15.996409+0000","last_undegraded":"2024-09-10T20:29:05.980651+0000","last_fullsized":"2024-09-10T20:29:05.980145+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"23'4","reported_seq":55,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-10T20:29:15.996952+0000","last_change":"2024-09-10T20:29:18.017214+0000","last_active":"2024-09-10T20:29:18.017214+0000","last_peered":"2024-09-10T20:29:15.996952+0000","last_clean":"2024-09-10T20:28:46.021581+0000","last_became_active":"2024-09-10T20:29:05.983206+0000","last_became_peered":"2024-09-10T20:29:05.983206+0000","last_unstale":"2024-09-10T20:29:15.996952+0000","last_undegraded":"2024-09-10T20:29:05.980534+0000","last_fullsized":"2024-09-10T20:29:05.980401+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"23'1","reported_seq":52,"reported_epoch":34,"state":"peering","last_fresh":"2024-09-10T20:29:18.018426+0000","last_change":"2024-09-10T20:29:18.018426+0000","last_active":"2024-09-10T20:29:18.018426+0000","last_peered":"2024-09-10T20:29:16.004375+0000","last_clean":"2024-09-10T20:28:40.535680+0000","last_became_active":"2024-09-10T20:29:16.004080+0000","last_became_peered":"2024-09-10T20:29:16.004080+0000","last_unstale":"2024-09-10T20:29:18.018426+0000","last_undegraded":"2024-09-10T20:29:18.018426+0000","last_fullsized":"2024-09-10T20:29:18.018426+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-10T20:29:15.996779+0000","last_change":"2024-09-10T20:29:18.014128+0000","last_active":"2024-09-10T20:29:18.014128+0000","last_peered":"2024-09-10T20:29:15.996779+0000","last_clean":"2024-09-10T20:28:42.552045+0000","last_became_active":"2024-09-10T20:29:05.982147+0000","last_became_peered":"2024-09-10T20:29:05.982147+0000","last_unstale":"2024-09-10T20:29:15.996779+0000","last_undegraded":"2024-09-10T20:29:15.996779+0000","last_fullsized":"2024-09-10T20:29:05.979897+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"23'5","reported_seq":64,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-10T20:29:16.005130+0000","last_change":"2024-09-10T20:29:18.018324+0000","last_active":"2024-09-10T20:29:18.018324+0000","last_peered":"2024-09-10T20:29:16.005130+0000","last_clean":"2024-09-10T20:28:46.470645+0000","last_became_active":"2024-09-10T20:29:16.004601+0000","last_became_peered":"2024-09-10T20:29:16.004601+0000","last_unstale":"2024-09-10T20:29:16.005130+0000","last_undegraded":"2024-09-10T20:29:15.998754+0000","last_fullsized":"2024-09-10T20:29:16.005130+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":68,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-10T20:29:16.003451+0000","last_change":"2024-09-10T20:29:18.017044+0000","last_active":"2024-09-10T20:29:18.017044+0000","last_peered":"2024-09-10T20:29:16.003451+0000","last_clean":"2024-09-10T20:29:16.003451+0000","last_became_active":"2024-09-10T20:29:16.001453+0000","last_became_peered":"2024-09-10T20:29:16.001453+0000","last_unstale":"2024-09-10T20:29:16.003451+0000","last_undegraded":"2024-09-10T20:29:16.003451+0000","last_fullsized":"2024-09-10T20:29:16.003451+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'6","reported_seq":49,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-10T20:29:15.996680+0000","last_change":"2024-09-10T20:29:18.016376+0000","last_active":"2024-09-10T20:29:18.016376+0000","last_peered":"2024-09-10T20:29:15.996680+0000","last_clean":"2024-09-10T20:28:46.672065+0000","last_became_active":"2024-09-10T20:29:05.983889+0000","last_became_peered":"2024-09-10T20:29:05.983889+0000","last_unstale":"2024-09-10T20:29:15.996680+0000","last_undegraded":"2024-09-10T20:29:05.981229+0000","last_fullsized":"2024-09-10T20:29:05.981105+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":56,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-10T20:29:15.996173+0000","last_change":"2024-09-10T20:29:18.015089+0000","last_active":"2024-09-10T20:29:18.015089+0000","last_peered":"2024-09-10T20:29:15.996173+0000","last_clean":"2024-09-10T20:28:57.955277+0000","last_became_active":"2024-09-10T20:29:05.981575+0000","last_became_peered":"2024-09-10T20:29:05.981575+0000","last_unstale":"2024-09-10T20:29:15.996173+0000","last_undegraded":"2024-09-10T20:29:05.980013+0000","last_fullsized":"2024-09-10T20:29:05.979929+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'3","reported_seq":57,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-10T20:29:16.004742+0000","last_change":"2024-09-10T20:29:18.014165+0000","last_active":"2024-09-10T20:29:18.014165+0000","last_peered":"2024-09-10T20:29:16.004742+0000","last_clean":"2024-09-10T20:28:45.722105+0000","last_became_active":"2024-09-10T20:29:16.004403+0000","last_became_peered":"2024-09-10T20:29:16.004403+0000","last_unstale":"2024-09-10T20:29:16.004742+0000","last_undegraded":"2024-09-10T20:29:15.997447+0000","last_fullsized":"2024-09-10T20:29:16.004742+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-10T20:29:16.003472+0000","last_change":"2024-09-10T20:29:18.017336+0000","last_active":"2024-09-10T20:29:18.017336+0000","last_peered":"2024-09-10T20:29:16.003472+0000","last_clean":"2024-09-10T20:29:16.003472+0000","last_became_active":"2024-09-10T20:29:16.001468+0000","last_became_peered":"2024-09-10T20:29:16.001468+0000","last_unstale":"2024-09-10T20:29:16.003472+0000","last_undegraded":"2024-09-10T20:29:16.003472+0000","last_fullsized":"2024-09-10T20:29:16.003472+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"23'4","reported_seq":59,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-10T20:29:16.004685+0000","last_change":"2024-09-10T20:29:18.014260+0000","last_active":"2024-09-10T20:29:18.014260+0000","last_peered":"2024-09-10T20:29:16.004685+0000","last_clean":"2024-09-10T20:28:46.120829+0000","last_became_active":"2024-09-10T20:29:16.004305+0000","last_became_peered":"2024-09-10T20:29:16.004305+0000","last_unstale":"2024-09-10T20:29:16.004685+0000","last_undegraded":"2024-09-10T20:29:15.997007+0000","last_fullsized":"2024-09-10T20:29:16.004685+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"23'1","reported_seq":51,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-10T20:29:15.997591+0000","last_change":"2024-09-10T20:29:18.018761+0000","last_active":"2024-09-10T20:29:18.018761+0000","last_peered":"2024-09-10T20:29:15.997591+0000","last_clean":"2024-09-10T20:28:42.552296+0000","last_became_active":"2024-09-10T20:29:05.983266+0000","last_became_peered":"2024-09-10T20:29:05.983266+0000","last_unstale":"2024-09-10T20:29:15.997591+0000","last_undegraded":"2024-09-10T20:29:05.981115+0000","last_fullsized":"2024-09-10T20:29:05.980975+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-10T20:29:15.998042+0000","last_change":"2024-09-10T20:29:18.015531+0000","last_active":"2024-09-10T20:29:18.015531+0000","last_peered":"2024-09-10T20:29:15.998042+0000","last_clean":"2024-09-10T20:28:42.552425+0000","last_became_active":"2024-09-10T20:29:05.984346+0000","last_became_peered":"2024-09-10T20:29:05.984346+0000","last_unstale":"2024-09-10T20:29:15.998042+0000","last_undegraded":"2024-09-10T20:29:15.998042+0000","last_fullsized":"2024-09-10T20:29:05.981330+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"23'5","reported_seq":56,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-10T20:29:15.998524+0000","last_change":"2024-09-10T20:29:18.019445+0000","last_active":"2024-09-10T20:29:18.019445+0000","last_peered":"2024-09-10T20:29:15.998524+0000","last_clean":"2024-09-10T20:28:46.571322+0000","last_became_active":"2024-09-10T20:29:05.983718+0000","last_became_peered":"2024-09-10T20:29:05.983718+0000","last_unstale":"2024-09-10T20:29:15.998524+0000","last_undegraded":"2024-09-10T20:29:05.981039+0000","last_fullsized":"2024-09-10T20:29:05.980906+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":68,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-10T20:29:16.006113+0000","last_change":"2024-09-10T20:29:18.017280+0000","last_active":"2024-09-10T20:29:18.017280+0000","last_peered":"2024-09-10T20:29:16.006113+0000","last_clean":"2024-09-10T20:29:16.006113+0000","last_became_active":"2024-09-10T20:29:16.004228+0000","last_became_peered":"2024-09-10T20:29:16.004228+0000","last_unstale":"2024-09-10T20:29:16.006113+0000","last_undegraded":"2024-09-10T20:29:16.006113+0000","last_fullsized":"2024-09-10T20:29:16.006113+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"23'2","reported_seq":52,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-10T20:29:15.998409+0000","last_change":"2024-09-10T20:29:18.019186+0000","last_active":"2024-09-10T20:29:18.019186+0000","last_peered":"2024-09-10T20:29:15.998409+0000","last_clean":"2024-09-10T20:28:42.552437+0000","last_became_active":"2024-09-10T20:29:05.983765+0000","last_became_peered":"2024-09-10T20:29:05.983765+0000","last_unstale":"2024-09-10T20:29:15.998409+0000","last_undegraded":"2024-09-10T20:29:05.981249+0000","last_fullsized":"2024-09-10T20:29:05.981122+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-10T20:29:15.998255+0000","last_change":"2024-09-10T20:29:18.016837+0000","last_active":"2024-09-10T20:29:18.016837+0000","last_peered":"2024-09-10T20:29:15.998255+0000","last_clean":"2024-09-10T20:28:42.552329+0000","last_became_active":"2024-09-10T20:29:05.982487+0000","last_became_peered":"2024-09-10T20:29:05.982487+0000","last_unstale":"2024-09-10T20:29:15.998255+0000","last_undegraded":"2024-09-10T20:29:15.998255+0000","last_fullsized":"2024-09-10T20:29:05.980406+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-10T20:29:16.005568+0000","last_change":"2024-09-10T20:29:18.017052+0000","last_active":"2024-09-10T20:29:18.017052+0000","last_peered":"2024-09-10T20:29:16.005568+0000","last_clean":"2024-09-10T20:29:16.005568+0000","last_became_active":"2024-09-10T20:29:16.003555+0000","last_became_peered":"2024-09-10T20:29:16.003555+0000","last_unstale":"2024-09-10T20:29:16.005568+0000","last_undegraded":"2024-09-10T20:29:16.005568+0000","last_fullsized":"2024-09-10T20:29:16.005568+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"23'9","reported_seq":61,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-10T20:29:15.997842+0000","last_change":"2024-09-10T20:29:18.015013+0000","last_active":"2024-09-10T20:29:18.015013+0000","last_peered":"2024-09-10T20:29:15.997842+0000","last_clean":"2024-09-10T20:28:46.321317+0000","last_became_active":"2024-09-10T20:29:05.982492+0000","last_became_peered":"2024-09-10T20:29:05.982492+0000","last_unstale":"2024-09-10T20:29:15.997842+0000","last_undegraded":"2024-09-10T20:29:05.980739+0000","last_fullsized":"2024-09-10T20:29:05.980608+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1932,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":13524,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":18354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182407,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":324,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":331776,"data_stored":106366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":28110506},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986179,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":272,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":278528,"data_stored":97754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":28176042},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:29:21.794 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:29:22.144 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:29:22.144 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":73,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":36,"num_osds":3,"num_up_osds":2,"osd_up_since":1726000153,"num_in_osds":2,"osd_in_since":1726000158,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"peering","count":14},{"state_name":"remapped+peering","count":10}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":13543,"bytes_used":56946688,"bytes_avail":193216581632,"bytes_total":193273528320,"inactive_pgs_ratio":1},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-10T20:29:14.819993+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T20:29:22.156 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'fsid': '3053cb46-0512-45f5-9707-4eb48df5a665', 'health': {'status': 'HEALTH_OK', 'checks': {}, 'mutes': []}, 'election_epoch': 6, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 73, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 36, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1726000153, 'num_in_osds': 2, 'osd_in_since': 1726000158, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'peering', 'count': 14}, {'state_name': 'remapped+peering', 'count': 10}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 13543, 'bytes_used': 56946688, 'bytes_avail': 193216581632, 'bytes_total': 193273528320, 'inactive_pgs_ratio': 1}, 'fsmap': {'epoch': 1, 'btime': '2024-09-10T20:28:08:371114+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 2, 'modified': '2024-09-10T20:29:14.819993+0000', 'services': {'osd': {'daemons': {'summary': '', '0': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, '1': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, '2': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}}}, 'progress_events': {}} 2024-09-10T20:29:25.029 INFO:tasks.ceph.mon.a.smithi131.stderr:2024-09-10T20:29:25.026+0000 7f1e65d04640 -1 log_channel(cluster) log [ERR] : Health check failed: Possible data damage: 5 pgs recovery_unfound (PG_DAMAGED) 2024-09-10T20:29:25.157 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:29:25.450 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:29:25.451 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:29:25.463 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2024-09-10T20:29:24.822325+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13543,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":54,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58,"ondisk_log_size":58,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55656,"kb_used_data":636,"kb_used_omap":8,"kb_used_meta":54967,"kb_avail":188688024,"statfs":{"total":193273528320,"available":193216536576,"internally_reserved":0,"allocated":651264,"data_stored":220819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8876,"internal_metadata":56286548},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":12,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.809907"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":51,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.015844+0000","last_change":"2024-09-10T20:29:19.019964+0000","last_active":"2024-09-10T20:29:20.015844+0000","last_peered":"2024-09-10T20:29:20.015844+0000","last_clean":"2024-09-10T20:29:20.015844+0000","last_became_active":"2024-09-10T20:29:19.019691+0000","last_became_peered":"2024-09-10T20:29:19.019691+0000","last_unstale":"2024-09-10T20:29:20.015844+0000","last_undegraded":"2024-09-10T20:29:20.015844+0000","last_fullsized":"2024-09-10T20:29:20.015844+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:46.881470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"23'4","reported_seq":56,"reported_epoch":36,"state":"active+recovery_unfound","last_fresh":"2024-09-10T20:29:20.023822+0000","last_change":"2024-09-10T20:29:20.024061+0000","last_active":"2024-09-10T20:29:20.023822+0000","last_peered":"2024-09-10T20:29:20.023822+0000","last_clean":"2024-09-10T20:28:45.872406+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:29:20.023822+0000","last_undegraded":"2024-09-10T20:29:20.017083+0000","last_fullsized":"2024-09-10T20:29:20.023822+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"23'3","reported_seq":69,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-10T20:29:20.023125+0000","last_change":"2024-09-10T20:29:22.991654+0000","last_active":"2024-09-10T20:29:20.023125+0000","last_peered":"2024-09-10T20:29:20.023125+0000","last_clean":"2024-09-10T20:28:46.420618+0000","last_became_active":"2024-09-10T20:29:20.020548+0000","last_became_peered":"2024-09-10T20:29:20.020548+0000","last_unstale":"2024-09-10T20:29:20.023125+0000","last_undegraded":"2024-09-10T20:29:20.023125+0000","last_fullsized":"2024-09-10T20:29:20.023125+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":61,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.018986+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:29:20.018986+0000","last_peered":"2024-09-10T20:29:20.018986+0000","last_clean":"2024-09-10T20:29:20.018986+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:29:20.018986+0000","last_undegraded":"2024-09-10T20:29:20.018986+0000","last_fullsized":"2024-09-10T20:29:20.018986+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"23'5","reported_seq":63,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:20.015478+0000","last_change":"2024-09-10T20:29:19.022136+0000","last_active":"2024-09-10T20:29:20.015478+0000","last_peered":"2024-09-10T20:29:20.015478+0000","last_clean":"2024-09-10T20:28:45.772428+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:29:20.015478+0000","last_undegraded":"2024-09-10T20:29:19.016520+0000","last_fullsized":"2024-09-10T20:29:20.015478+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"23'3","reported_seq":62,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:20.015307+0000","last_change":"2024-09-10T20:29:19.022275+0000","last_active":"2024-09-10T20:29:20.015307+0000","last_peered":"2024-09-10T20:29:20.015307+0000","last_clean":"2024-09-10T20:28:46.621675+0000","last_became_active":"2024-09-10T20:29:19.022041+0000","last_became_peered":"2024-09-10T20:29:19.022041+0000","last_unstale":"2024-09-10T20:29:20.015307+0000","last_undegraded":"2024-09-10T20:29:19.014144+0000","last_fullsized":"2024-09-10T20:29:20.015307+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"23'4","reported_seq":63,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:20.016112+0000","last_change":"2024-09-10T20:29:19.024530+0000","last_active":"2024-09-10T20:29:20.016112+0000","last_peered":"2024-09-10T20:29:20.016112+0000","last_clean":"2024-09-10T20:28:46.021581+0000","last_became_active":"2024-09-10T20:29:19.024227+0000","last_became_peered":"2024-09-10T20:29:19.024227+0000","last_unstale":"2024-09-10T20:29:20.016112+0000","last_undegraded":"2024-09-10T20:29:19.016822+0000","last_fullsized":"2024-09-10T20:29:20.016112+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"23'1","reported_seq":61,"reported_epoch":36,"state":"active+recovering","last_fresh":"2024-09-10T20:29:20.016123+0000","last_change":"2024-09-10T20:29:20.016123+0000","last_active":"2024-09-10T20:29:20.016123+0000","last_peered":"2024-09-10T20:29:20.016123+0000","last_clean":"2024-09-10T20:28:40.535680+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:29:20.016123+0000","last_undegraded":"2024-09-10T20:29:20.016123+0000","last_fullsized":"2024-09-10T20:29:20.016123+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.016038+0000","last_change":"2024-09-10T20:29:19.019807+0000","last_active":"2024-09-10T20:29:20.016038+0000","last_peered":"2024-09-10T20:29:20.016038+0000","last_clean":"2024-09-10T20:29:20.016038+0000","last_became_active":"2024-09-10T20:29:19.019500+0000","last_became_peered":"2024-09-10T20:29:19.019500+0000","last_unstale":"2024-09-10T20:29:20.016038+0000","last_undegraded":"2024-09-10T20:29:20.016038+0000","last_fullsized":"2024-09-10T20:29:20.016038+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:17:47.161423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"23'5","reported_seq":71,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-10T20:29:20.023530+0000","last_change":"2024-09-10T20:29:22.991611+0000","last_active":"2024-09-10T20:29:20.023530+0000","last_peered":"2024-09-10T20:29:20.023530+0000","last_clean":"2024-09-10T20:28:46.470645+0000","last_became_active":"2024-09-10T20:29:20.020530+0000","last_became_peered":"2024-09-10T20:29:20.020530+0000","last_unstale":"2024-09-10T20:29:20.023530+0000","last_undegraded":"2024-09-10T20:29:20.023530+0000","last_fullsized":"2024-09-10T20:29:20.023530+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.019440+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:29:20.019440+0000","last_peered":"2024-09-10T20:29:20.019440+0000","last_clean":"2024-09-10T20:29:20.019440+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:29:20.019440+0000","last_undegraded":"2024-09-10T20:29:20.019440+0000","last_fullsized":"2024-09-10T20:29:20.019440+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"23'6","reported_seq":57,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:20.015555+0000","last_change":"2024-09-10T20:29:19.024601+0000","last_active":"2024-09-10T20:29:20.015555+0000","last_peered":"2024-09-10T20:29:20.015555+0000","last_clean":"2024-09-10T20:28:46.672065+0000","last_became_active":"2024-09-10T20:29:19.024360+0000","last_became_peered":"2024-09-10T20:29:19.024360+0000","last_unstale":"2024-09-10T20:29:20.015555+0000","last_undegraded":"2024-09-10T20:29:19.014503+0000","last_fullsized":"2024-09-10T20:29:20.015555+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":64,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:20.015786+0000","last_change":"2024-09-10T20:29:19.021713+0000","last_active":"2024-09-10T20:29:20.015786+0000","last_peered":"2024-09-10T20:29:20.015786+0000","last_clean":"2024-09-10T20:28:57.955277+0000","last_became_active":"2024-09-10T20:29:19.021487+0000","last_became_peered":"2024-09-10T20:29:19.021487+0000","last_unstale":"2024-09-10T20:29:20.015786+0000","last_undegraded":"2024-09-10T20:29:19.013787+0000","last_fullsized":"2024-09-10T20:29:20.015786+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'3","reported_seq":63,"reported_epoch":36,"state":"active+recovery_unfound","last_fresh":"2024-09-10T20:29:20.022461+0000","last_change":"2024-09-10T20:29:20.022641+0000","last_active":"2024-09-10T20:29:20.022461+0000","last_peered":"2024-09-10T20:29:20.022461+0000","last_clean":"2024-09-10T20:28:45.722105+0000","last_became_active":"2024-09-10T20:29:20.018890+0000","last_became_peered":"2024-09-10T20:29:20.018890+0000","last_unstale":"2024-09-10T20:29:20.022461+0000","last_undegraded":"2024-09-10T20:29:20.015328+0000","last_fullsized":"2024-09-10T20:29:20.022461+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":92,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.022839+0000","last_change":"2024-09-10T20:29:20.021021+0000","last_active":"2024-09-10T20:29:20.022839+0000","last_peered":"2024-09-10T20:29:20.022839+0000","last_clean":"2024-09-10T20:29:20.022839+0000","last_became_active":"2024-09-10T20:29:20.020575+0000","last_became_peered":"2024-09-10T20:29:20.020575+0000","last_unstale":"2024-09-10T20:29:20.022839+0000","last_undegraded":"2024-09-10T20:29:20.022839+0000","last_fullsized":"2024-09-10T20:29:20.022839+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:23:19.560105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"23'4","reported_seq":65,"reported_epoch":36,"state":"active+recovery_unfound","last_fresh":"2024-09-10T20:29:20.021297+0000","last_change":"2024-09-10T20:29:20.021584+0000","last_active":"2024-09-10T20:29:20.021297+0000","last_peered":"2024-09-10T20:29:20.021297+0000","last_clean":"2024-09-10T20:28:46.120829+0000","last_became_active":"2024-09-10T20:29:20.018640+0000","last_became_peered":"2024-09-10T20:29:20.018640+0000","last_unstale":"2024-09-10T20:29:20.021297+0000","last_undegraded":"2024-09-10T20:29:20.016410+0000","last_fullsized":"2024-09-10T20:29:20.021297+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"23'1","reported_seq":59,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:20.015808+0000","last_change":"2024-09-10T20:29:19.022418+0000","last_active":"2024-09-10T20:29:20.015808+0000","last_peered":"2024-09-10T20:29:20.015808+0000","last_clean":"2024-09-10T20:28:42.552296+0000","last_became_active":"2024-09-10T20:29:19.022182+0000","last_became_peered":"2024-09-10T20:29:19.022182+0000","last_unstale":"2024-09-10T20:29:20.015808+0000","last_undegraded":"2024-09-10T20:29:19.016061+0000","last_fullsized":"2024-09-10T20:29:20.015808+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.015089+0000","last_change":"2024-09-10T20:29:19.022445+0000","last_active":"2024-09-10T20:29:20.015089+0000","last_peered":"2024-09-10T20:29:20.015089+0000","last_clean":"2024-09-10T20:29:20.015089+0000","last_became_active":"2024-09-10T20:29:19.022165+0000","last_became_peered":"2024-09-10T20:29:19.022165+0000","last_unstale":"2024-09-10T20:29:20.015089+0000","last_undegraded":"2024-09-10T20:29:20.015089+0000","last_fullsized":"2024-09-10T20:29:20.015089+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:57:59.626770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"23'5","reported_seq":64,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:20.016271+0000","last_change":"2024-09-10T20:29:19.024585+0000","last_active":"2024-09-10T20:29:20.016271+0000","last_peered":"2024-09-10T20:29:20.016271+0000","last_clean":"2024-09-10T20:28:46.571322+0000","last_became_active":"2024-09-10T20:29:19.024283+0000","last_became_peered":"2024-09-10T20:29:19.024283+0000","last_unstale":"2024-09-10T20:29:20.016271+0000","last_undegraded":"2024-09-10T20:29:19.017318+0000","last_fullsized":"2024-09-10T20:29:20.016271+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":3}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.020926+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:29:20.020926+0000","last_peered":"2024-09-10T20:29:20.020926+0000","last_clean":"2024-09-10T20:29:20.020926+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:29:20.020926+0000","last_undegraded":"2024-09-10T20:29:20.020926+0000","last_fullsized":"2024-09-10T20:29:20.020926+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"23'2","reported_seq":60,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:20.015284+0000","last_change":"2024-09-10T20:29:19.022172+0000","last_active":"2024-09-10T20:29:20.015284+0000","last_peered":"2024-09-10T20:29:20.015284+0000","last_clean":"2024-09-10T20:28:42.552437+0000","last_became_active":"2024-09-10T20:29:19.021920+0000","last_became_peered":"2024-09-10T20:29:19.021920+0000","last_unstale":"2024-09-10T20:29:20.015284+0000","last_undegraded":"2024-09-10T20:29:19.015669+0000","last_fullsized":"2024-09-10T20:29:20.015284+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.016002+0000","last_change":"2024-09-10T20:29:19.022093+0000","last_active":"2024-09-10T20:29:20.016002+0000","last_peered":"2024-09-10T20:29:20.016002+0000","last_clean":"2024-09-10T20:29:20.016002+0000","last_became_active":"2024-09-10T20:29:19.021778+0000","last_became_peered":"2024-09-10T20:29:19.021778+0000","last_unstale":"2024-09-10T20:29:20.016002+0000","last_undegraded":"2024-09-10T20:29:20.016002+0000","last_fullsized":"2024-09-10T20:29:20.016002+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:25:12.240664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.019846+0000","last_change":"2024-09-10T20:29:20.018969+0000","last_active":"2024-09-10T20:29:20.019846+0000","last_peered":"2024-09-10T20:29:20.019846+0000","last_clean":"2024-09-10T20:29:20.019846+0000","last_became_active":"2024-09-10T20:29:20.018675+0000","last_became_peered":"2024-09-10T20:29:20.018675+0000","last_unstale":"2024-09-10T20:29:20.019846+0000","last_undegraded":"2024-09-10T20:29:20.019846+0000","last_fullsized":"2024-09-10T20:29:20.019846+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:50.839971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"23'9","reported_seq":69,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:20.014830+0000","last_change":"2024-09-10T20:29:19.024494+0000","last_active":"2024-09-10T20:29:20.014830+0000","last_peered":"2024-09-10T20:29:20.014830+0000","last_clean":"2024-09-10T20:28:46.321317+0000","last_became_active":"2024-09-10T20:29:19.024217+0000","last_became_peered":"2024-09-10T20:29:19.024217+0000","last_unstale":"2024-09-10T20:29:20.014830+0000","last_undegraded":"2024-09-10T20:29:19.016668+0000","last_fullsized":"2024-09-10T20:29:20.014830+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1932,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":3}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":13524,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":53,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":18354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182408,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":340,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":348160,"data_stored":112834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":28110506},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986180,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":296,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":303104,"data_stored":107985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":28176042},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:29:25.465 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:29:25.814 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:29:25.814 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_ERR","checks":{"OBJECT_UNFOUND":{"severity":"HEALTH_WARN","summary":{"message":"27/30 objects unfound (90.000%)","count":27},"muted":false},"PG_DAMAGED":{"severity":"HEALTH_ERR","summary":{"message":"Possible data damage: 5 pgs recovery_unfound","count":5},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 54/60 objects degraded (90.000%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":77,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":36,"num_osds":3,"num_up_osds":2,"osd_up_since":1726000153,"num_in_osds":2,"osd_in_since":1726000158,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":9},{"state_name":"active+recovery_wait+degraded","count":9},{"state_name":"active+recovery_unfound","count":3},{"state_name":"active+recovery_unfound+degraded","count":2},{"state_name":"active+recovering","count":1}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":13543,"bytes_used":56991744,"bytes_avail":193216536576,"bytes_total":193273528320,"degraded_objects":54,"degraded_total":60,"degraded_ratio":0.90000000000000002,"unfound_objects":27,"unfound_total":30,"unfound_ratio":0.90000000000000002},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-10T20:29:14.819993+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T20:29:25.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'fsid': '3053cb46-0512-45f5-9707-4eb48df5a665', 'health': {'status': 'HEALTH_ERR', 'checks': {'OBJECT_UNFOUND': {'severity': 'HEALTH_WARN', 'summary': {'message': '27/30 objects unfound (90.000%)', 'count': 27}, 'muted': False}, 'PG_DAMAGED': {'severity': 'HEALTH_ERR', 'summary': {'message': 'Possible data damage: 5 pgs recovery_unfound', 'count': 5}, 'muted': False}, 'PG_DEGRADED': {'severity': 'HEALTH_WARN', 'summary': {'message': 'Degraded data redundancy: 54/60 objects degraded (90.000%), 11 pgs degraded', 'count': 11}, 'muted': False}}, 'mutes': []}, 'election_epoch': 6, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 77, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 36, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1726000153, 'num_in_osds': 2, 'osd_in_since': 1726000158, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+clean', 'count': 9}, {'state_name': 'active+recovery_wait+degraded', 'count': 9}, {'state_name': 'active+recovery_unfound', 'count': 3}, {'state_name': 'active+recovery_unfound+degraded', 'count': 2}, {'state_name': 'active+recovering', 'count': 1}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 13543, 'bytes_used': 56991744, 'bytes_avail': 193216536576, 'bytes_total': 193273528320, 'degraded_objects': 54, 'degraded_total': 60, 'degraded_ratio': 0.9, 'unfound_objects': 27, 'unfound_total': 30, 'unfound_ratio': 0.9}, 'fsmap': {'epoch': 1, 'btime': '2024-09-10T20:28:08:371114+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 2, 'modified': '2024-09-10T20:29:14.819993+0000', 'services': {'osd': {'daemons': {'summary': '', '0': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, '1': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, '2': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}}}, 'progress_events': {}} 2024-09-10T20:29:25.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:active! 2024-09-10T20:29:25.827 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-10T20:29:25.828 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-10T20:29:25.976 INFO:teuthology.orchestra.run.smithi131.stdout:107374182410 2024-09-10T20:29:25.977 INFO:teuthology.orchestra.run.smithi131.stdout:133143986181 2024-09-10T20:29:25.977 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T20:29:25.978 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T20:29:26.274 INFO:teuthology.orchestra.run.smithi131.stdout:133143986180 2024-09-10T20:29:26.274 INFO:teuthology.orchestra.run.smithi131.stdout:107374182408 2024-09-10T20:29:26.288 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 133143986181 got 133143986180 for osd.2 2024-09-10T20:29:26.289 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 107374182410 got 107374182408 for osd.0 2024-09-10T20:29:27.289 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T20:29:27.290 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T20:29:27.598 INFO:teuthology.orchestra.run.smithi131.stdout:133143986181 2024-09-10T20:29:27.598 INFO:teuthology.orchestra.run.smithi131.stdout:107374182410 2024-09-10T20:29:27.617 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 133143986181 got 133143986181 for osd.2 2024-09-10T20:29:27.617 DEBUG:teuthology.parallel:result is None 2024-09-10T20:29:27.617 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 107374182410 got 107374182410 for osd.0 2024-09-10T20:29:27.618 DEBUG:teuthology.parallel:result is None 2024-09-10T20:29:27.618 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:29:27.963 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:29:27.963 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_ERR","checks":{"OBJECT_UNFOUND":{"severity":"HEALTH_WARN","summary":{"message":"27/30 objects unfound (90.000%)","count":27},"muted":false},"PG_DAMAGED":{"severity":"HEALTH_ERR","summary":{"message":"Possible data damage: 5 pgs recovery_unfound","count":5},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 54/60 objects degraded (90.000%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":79,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":36,"num_osds":3,"num_up_osds":2,"osd_up_since":1726000153,"num_in_osds":2,"osd_in_since":1726000158,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":9},{"state_name":"active+recovery_wait+degraded","count":9},{"state_name":"active+recovery_unfound","count":3},{"state_name":"active+recovery_unfound+degraded","count":2},{"state_name":"active+recovering","count":1}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":13543,"bytes_used":56991744,"bytes_avail":193216536576,"bytes_total":193273528320,"degraded_objects":54,"degraded_total":60,"degraded_ratio":0.90000000000000002,"unfound_objects":27,"unfound_total":30,"unfound_ratio":0.90000000000000002},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-10T20:29:14.819993+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T20:29:27.976 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'fsid': '3053cb46-0512-45f5-9707-4eb48df5a665', 'health': {'status': 'HEALTH_ERR', 'checks': {'OBJECT_UNFOUND': {'severity': 'HEALTH_WARN', 'summary': {'message': '27/30 objects unfound (90.000%)', 'count': 27}, 'muted': False}, 'PG_DAMAGED': {'severity': 'HEALTH_ERR', 'summary': {'message': 'Possible data damage: 5 pgs recovery_unfound', 'count': 5}, 'muted': False}, 'PG_DEGRADED': {'severity': 'HEALTH_WARN', 'summary': {'message': 'Degraded data redundancy: 54/60 objects degraded (90.000%), 11 pgs degraded', 'count': 11}, 'muted': False}}, 'mutes': []}, 'election_epoch': 6, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 79, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 36, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1726000153, 'num_in_osds': 2, 'osd_in_since': 1726000158, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+clean', 'count': 9}, {'state_name': 'active+recovery_wait+degraded', 'count': 9}, {'state_name': 'active+recovery_unfound', 'count': 3}, {'state_name': 'active+recovery_unfound+degraded', 'count': 2}, {'state_name': 'active+recovering', 'count': 1}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 13543, 'bytes_used': 56991744, 'bytes_avail': 193216536576, 'bytes_total': 193273528320, 'degraded_objects': 54, 'degraded_total': 60, 'degraded_ratio': 0.9, 'unfound_objects': 27, 'unfound_total': 30, 'unfound_ratio': 0.9}, 'fsmap': {'epoch': 1, 'btime': '2024-09-10T20:28:08:371114+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 2, 'modified': '2024-09-10T20:29:14.819993+0000', 'services': {'osd': {'daemons': {'summary': '', '0': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, '1': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, '2': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}}}, 'progress_events': {}} 2024-09-10T20:29:27.977 INFO:tasks.rep_lost_unfound_delete:there are 27 unfound objects 2024-09-10T20:29:27.977 DEBUG:teuthology.orchestra.run.smithi131:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.admin -b 4096 -p unfounddel_pool -t 20 bench 240 write' 2024-09-10T20:29:28.030 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout:hints = 1 2024-09-10T20:29:28.030 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout:Maintaining 20 concurrent writes of 4096 bytes to objects of size 4096 for up to 240 seconds or 0 objects 2024-09-10T20:29:28.030 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout:Object prefix: benchmark_data_smithi131_42778 2024-09-10T20:29:28.030 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T20:29:28.030 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 0 0 0 0 0 0 - 0 2024-09-10T20:29:29.030 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 1 20 3217 3197 12.4876 12.4883 0.0165549 0.00618613 2024-09-10T20:29:30.030 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 2 20 6388 6368 12.4366 12.3867 0.00341829 0.00626884 2024-09-10T20:29:31.030 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 3 20 9253 9233 12.0213 11.1914 0.0067456 0.00648934 2024-09-10T20:29:32.030 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 4 20 11943 11923 11.6427 10.5078 0.00890303 0.0067017 2024-09-10T20:29:33.030 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 5 20 14428 14408 11.2554 9.70703 0.0105352 0.00693461 2024-09-10T20:29:34.030 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 6 20 16839 16819 10.949 9.41797 0.00811504 0.00712719 2024-09-10T20:29:35.030 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 7 20 19178 19158 10.69 9.13672 0.00562709 0.00729864 2024-09-10T20:29:36.030 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 8 20 21261 21241 10.3708 8.13672 0.00549896 0.00744234 2024-09-10T20:29:37.030 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 9 20 22752 22732 9.86556 5.82422 0.0104998 0.00791201 2024-09-10T20:29:37.981 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:29:38.031 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 10 20 24797 24777 9.67734 7.98828 0.0156294 0.00806621 2024-09-10T20:29:38.505 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:29:38.505 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:29:38.523 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2024-09-10T20:29:36.825520+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":76436711,"num_objects":18688,"num_object_clones":0,"num_object_copies":37376,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":54,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":18688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18715,"num_write_kb":74680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18716,"ondisk_log_size":18716,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":260536,"kb_used_data":134744,"kb_used_omap":8,"kb_used_meta":125751,"kb_avail":188483144,"statfs":{"total":193273528320,"available":193006739456,"internally_reserved":0,"allocated":137977856,"data_stored":137547411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8876,"internal_metadata":128769364},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[12,3,7,12],"upper_bound":16},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":76423168,"num_objects":18658,"num_object_clones":0,"num_object_copies":37316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18658,"num_write_kb":74632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003195"},"pg_stats":[{"pgid":"2.f","version":"36'1348","reported_seq":1399,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:35.937267+0000","last_change":"2024-09-10T20:29:19.019964+0000","last_active":"2024-09-10T20:29:35.937267+0000","last_peered":"2024-09-10T20:29:35.937267+0000","last_clean":"2024-09-10T20:29:35.937267+0000","last_became_active":"2024-09-10T20:29:19.019691+0000","last_became_peered":"2024-09-10T20:29:19.019691+0000","last_unstale":"2024-09-10T20:29:35.937267+0000","last_undegraded":"2024-09-10T20:29:35.937267+0000","last_fullsized":"2024-09-10T20:29:35.937267+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":1348,"log_dups_size":0,"ondisk_log_size":1348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:46.881470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5521408,"num_objects":1348,"num_object_clones":0,"num_object_copies":2696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1348,"num_write_kb":5392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'957","reported_seq":1009,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-10T20:29:32.990228+0000","last_change":"2024-09-10T20:29:28.036652+0000","last_active":"2024-09-10T20:29:32.990228+0000","last_peered":"2024-09-10T20:29:32.990228+0000","last_clean":"2024-09-10T20:28:45.872406+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:29:32.990228+0000","last_undegraded":"2024-09-10T20:29:20.024061+0000","last_fullsized":"2024-09-10T20:29:32.990228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3904454,"num_objects":955,"num_object_clones":0,"num_object_copies":1910,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":957,"num_write_kb":3816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":955}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'924","reported_seq":988,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-10T20:29:32.970369+0000","last_change":"2024-09-10T20:29:22.991654+0000","last_active":"2024-09-10T20:29:32.970369+0000","last_peered":"2024-09-10T20:29:32.970369+0000","last_clean":"2024-09-10T20:28:46.420618+0000","last_became_active":"2024-09-10T20:29:20.020548+0000","last_became_peered":"2024-09-10T20:29:20.020548+0000","last_unstale":"2024-09-10T20:29:32.970369+0000","last_undegraded":"2024-09-10T20:29:20.023125+0000","last_fullsized":"2024-09-10T20:29:32.970369+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":924,"log_dups_size":0,"ondisk_log_size":924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3772899,"num_objects":922,"num_object_clones":0,"num_object_copies":1844,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":922,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":924,"num_write_kb":3686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":922}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'916","reported_seq":977,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:32.992730+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:29:32.992730+0000","last_peered":"2024-09-10T20:29:32.992730+0000","last_clean":"2024-09-10T20:29:32.992730+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:29:32.992730+0000","last_undegraded":"2024-09-10T20:29:32.992730+0000","last_fullsized":"2024-09-10T20:29:32.992730+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":916,"log_dups_size":0,"ondisk_log_size":916,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3751936,"num_objects":916,"num_object_clones":0,"num_object_copies":1832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":916,"num_write_kb":3664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'1360","reported_seq":1418,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:35.931660+0000","last_change":"2024-09-10T20:29:19.022136+0000","last_active":"2024-09-10T20:29:35.931660+0000","last_peered":"2024-09-10T20:29:35.931660+0000","last_clean":"2024-09-10T20:28:45.772428+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:29:35.931660+0000","last_undegraded":"2024-09-10T20:29:19.016520+0000","last_fullsized":"2024-09-10T20:29:35.931660+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":1360,"log_dups_size":0,"ondisk_log_size":1360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5551046,"num_objects":1357,"num_object_clones":0,"num_object_copies":2714,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1360,"num_write_kb":5424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1357}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'1286","reported_seq":1345,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:35.935826+0000","last_change":"2024-09-10T20:29:19.022275+0000","last_active":"2024-09-10T20:29:35.935826+0000","last_peered":"2024-09-10T20:29:35.935826+0000","last_clean":"2024-09-10T20:28:46.621675+0000","last_became_active":"2024-09-10T20:29:19.022041+0000","last_became_peered":"2024-09-10T20:29:19.022041+0000","last_unstale":"2024-09-10T20:29:35.935826+0000","last_undegraded":"2024-09-10T20:29:19.014144+0000","last_fullsized":"2024-09-10T20:29:35.935826+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":1286,"log_dups_size":0,"ondisk_log_size":1286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5256134,"num_objects":1285,"num_object_clones":0,"num_object_copies":2570,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1286,"num_write_kb":5135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1285}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'1340","reported_seq":1399,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:35.937971+0000","last_change":"2024-09-10T20:29:19.024530+0000","last_active":"2024-09-10T20:29:35.937971+0000","last_peered":"2024-09-10T20:29:35.937971+0000","last_clean":"2024-09-10T20:28:46.021581+0000","last_became_active":"2024-09-10T20:29:19.024227+0000","last_became_peered":"2024-09-10T20:29:19.024227+0000","last_unstale":"2024-09-10T20:29:35.937971+0000","last_undegraded":"2024-09-10T20:29:19.016822+0000","last_fullsized":"2024-09-10T20:29:35.937971+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":1340,"log_dups_size":0,"ondisk_log_size":1340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5473222,"num_objects":1338,"num_object_clones":0,"num_object_copies":2676,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1340,"num_write_kb":5348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1338}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'1328","reported_seq":1388,"reported_epoch":36,"state":"active+recovering","last_fresh":"2024-09-10T20:29:35.925868+0000","last_change":"2024-09-10T20:29:20.016123+0000","last_active":"2024-09-10T20:29:35.925868+0000","last_peered":"2024-09-10T20:29:35.925868+0000","last_clean":"2024-09-10T20:28:40.535680+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:29:35.925868+0000","last_undegraded":"2024-09-10T20:29:35.925868+0000","last_fullsized":"2024-09-10T20:29:35.925868+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":1328,"log_dups_size":0,"ondisk_log_size":1328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5435875,"num_objects":1328,"num_object_clones":0,"num_object_copies":2656,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1328,"num_write_kb":5309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1328}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.016038+0000","last_change":"2024-09-10T20:29:19.019807+0000","last_active":"2024-09-10T20:29:20.016038+0000","last_peered":"2024-09-10T20:29:20.016038+0000","last_clean":"2024-09-10T20:29:20.016038+0000","last_became_active":"2024-09-10T20:29:19.019500+0000","last_became_peered":"2024-09-10T20:29:19.019500+0000","last_unstale":"2024-09-10T20:29:20.016038+0000","last_undegraded":"2024-09-10T20:29:20.016038+0000","last_fullsized":"2024-09-10T20:29:20.016038+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:17:47.161423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'907","reported_seq":973,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-10T20:29:32.985449+0000","last_change":"2024-09-10T20:29:22.991611+0000","last_active":"2024-09-10T20:29:32.985449+0000","last_peered":"2024-09-10T20:29:32.985449+0000","last_clean":"2024-09-10T20:28:46.470645+0000","last_became_active":"2024-09-10T20:29:20.020530+0000","last_became_peered":"2024-09-10T20:29:20.020530+0000","last_unstale":"2024-09-10T20:29:32.985449+0000","last_undegraded":"2024-09-10T20:29:20.023530+0000","last_fullsized":"2024-09-10T20:29:32.985449+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3695558,"num_objects":904,"num_object_clones":0,"num_object_copies":1808,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":907,"num_write_kb":3612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":904}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.019440+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:29:20.019440+0000","last_peered":"2024-09-10T20:29:20.019440+0000","last_clean":"2024-09-10T20:29:20.019440+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:29:20.019440+0000","last_undegraded":"2024-09-10T20:29:20.019440+0000","last_fullsized":"2024-09-10T20:29:20.019440+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'1346","reported_seq":1397,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:35.922109+0000","last_change":"2024-09-10T20:29:19.024601+0000","last_active":"2024-09-10T20:29:35.922109+0000","last_peered":"2024-09-10T20:29:35.922109+0000","last_clean":"2024-09-10T20:28:46.672065+0000","last_became_active":"2024-09-10T20:29:19.024360+0000","last_became_peered":"2024-09-10T20:29:19.024360+0000","last_unstale":"2024-09-10T20:29:35.922109+0000","last_undegraded":"2024-09-10T20:29:19.014503+0000","last_fullsized":"2024-09-10T20:29:35.922109+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":1346,"log_dups_size":0,"ondisk_log_size":1346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5489606,"num_objects":1342,"num_object_clones":0,"num_object_copies":2684,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1346,"num_write_kb":5364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1342}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":64,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:20.015786+0000","last_change":"2024-09-10T20:29:19.021713+0000","last_active":"2024-09-10T20:29:20.015786+0000","last_peered":"2024-09-10T20:29:20.015786+0000","last_clean":"2024-09-10T20:28:57.955277+0000","last_became_active":"2024-09-10T20:29:19.021487+0000","last_became_peered":"2024-09-10T20:29:19.021487+0000","last_unstale":"2024-09-10T20:29:20.015786+0000","last_undegraded":"2024-09-10T20:29:19.013787+0000","last_fullsized":"2024-09-10T20:29:20.015786+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'882","reported_seq":942,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-10T20:29:32.982201+0000","last_change":"2024-09-10T20:29:28.036372+0000","last_active":"2024-09-10T20:29:32.982201+0000","last_peered":"2024-09-10T20:29:32.982201+0000","last_clean":"2024-09-10T20:28:45.722105+0000","last_became_active":"2024-09-10T20:29:20.018890+0000","last_became_peered":"2024-09-10T20:29:20.018890+0000","last_unstale":"2024-09-10T20:29:32.982201+0000","last_undegraded":"2024-09-10T20:29:20.022641+0000","last_fullsized":"2024-09-10T20:29:32.982201+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3601350,"num_objects":881,"num_object_clones":0,"num_object_copies":1762,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":882,"num_write_kb":3519,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":881}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":92,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.022839+0000","last_change":"2024-09-10T20:29:20.021021+0000","last_active":"2024-09-10T20:29:20.022839+0000","last_peered":"2024-09-10T20:29:20.022839+0000","last_clean":"2024-09-10T20:29:20.022839+0000","last_became_active":"2024-09-10T20:29:20.020575+0000","last_became_peered":"2024-09-10T20:29:20.020575+0000","last_unstale":"2024-09-10T20:29:20.022839+0000","last_undegraded":"2024-09-10T20:29:20.022839+0000","last_fullsized":"2024-09-10T20:29:20.022839+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:23:19.560105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'888","reported_seq":949,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-10T20:29:32.982321+0000","last_change":"2024-09-10T20:29:28.036341+0000","last_active":"2024-09-10T20:29:32.982321+0000","last_peered":"2024-09-10T20:29:32.982321+0000","last_clean":"2024-09-10T20:28:46.120829+0000","last_became_active":"2024-09-10T20:29:20.018640+0000","last_became_peered":"2024-09-10T20:29:20.018640+0000","last_unstale":"2024-09-10T20:29:32.982321+0000","last_undegraded":"2024-09-10T20:29:20.021584+0000","last_fullsized":"2024-09-10T20:29:32.982321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3621830,"num_objects":886,"num_object_clones":0,"num_object_copies":1772,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":886,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":888,"num_write_kb":3539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":886}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'1326","reported_seq":1384,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:35.916484+0000","last_change":"2024-09-10T20:29:19.022418+0000","last_active":"2024-09-10T20:29:35.916484+0000","last_peered":"2024-09-10T20:29:35.916484+0000","last_clean":"2024-09-10T20:28:42.552296+0000","last_became_active":"2024-09-10T20:29:19.022182+0000","last_became_peered":"2024-09-10T20:29:19.022182+0000","last_unstale":"2024-09-10T20:29:35.916484+0000","last_undegraded":"2024-09-10T20:29:19.016061+0000","last_fullsized":"2024-09-10T20:29:35.916484+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":1326,"log_dups_size":0,"ondisk_log_size":1326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5427683,"num_objects":1326,"num_object_clones":0,"num_object_copies":2652,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1326,"num_write_kb":5301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1326}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.015089+0000","last_change":"2024-09-10T20:29:19.022445+0000","last_active":"2024-09-10T20:29:20.015089+0000","last_peered":"2024-09-10T20:29:20.015089+0000","last_clean":"2024-09-10T20:29:20.015089+0000","last_became_active":"2024-09-10T20:29:19.022165+0000","last_became_peered":"2024-09-10T20:29:19.022165+0000","last_unstale":"2024-09-10T20:29:20.015089+0000","last_undegraded":"2024-09-10T20:29:20.015089+0000","last_fullsized":"2024-09-10T20:29:20.015089+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:57:59.626770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'1327","reported_seq":1386,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:35.913169+0000","last_change":"2024-09-10T20:29:19.024585+0000","last_active":"2024-09-10T20:29:35.913169+0000","last_peered":"2024-09-10T20:29:35.913169+0000","last_clean":"2024-09-10T20:28:46.571322+0000","last_became_active":"2024-09-10T20:29:19.024283+0000","last_became_peered":"2024-09-10T20:29:19.024283+0000","last_unstale":"2024-09-10T20:29:35.913169+0000","last_undegraded":"2024-09-10T20:29:19.017318+0000","last_fullsized":"2024-09-10T20:29:35.913169+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":1327,"log_dups_size":0,"ondisk_log_size":1327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5416361,"num_objects":1325,"num_object_clones":0,"num_object_copies":2650,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":1325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1327,"num_write_kb":5292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1325}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.020926+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:29:20.020926+0000","last_peered":"2024-09-10T20:29:20.020926+0000","last_clean":"2024-09-10T20:29:20.020926+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:29:20.020926+0000","last_undegraded":"2024-09-10T20:29:20.020926+0000","last_fullsized":"2024-09-10T20:29:20.020926+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'1296","reported_seq":1354,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:35.928129+0000","last_change":"2024-09-10T20:29:19.022172+0000","last_active":"2024-09-10T20:29:35.928129+0000","last_peered":"2024-09-10T20:29:35.928129+0000","last_clean":"2024-09-10T20:28:42.552437+0000","last_became_active":"2024-09-10T20:29:19.021920+0000","last_became_peered":"2024-09-10T20:29:19.021920+0000","last_unstale":"2024-09-10T20:29:35.928129+0000","last_undegraded":"2024-09-10T20:29:19.015669+0000","last_fullsized":"2024-09-10T20:29:35.928129+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":1296,"log_dups_size":0,"ondisk_log_size":1296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5301190,"num_objects":1296,"num_object_clones":0,"num_object_copies":2592,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1296,"num_write_kb":5178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1296}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.016002+0000","last_change":"2024-09-10T20:29:19.022093+0000","last_active":"2024-09-10T20:29:20.016002+0000","last_peered":"2024-09-10T20:29:20.016002+0000","last_clean":"2024-09-10T20:29:20.016002+0000","last_became_active":"2024-09-10T20:29:19.021778+0000","last_became_peered":"2024-09-10T20:29:19.021778+0000","last_unstale":"2024-09-10T20:29:20.016002+0000","last_undegraded":"2024-09-10T20:29:20.016002+0000","last_fullsized":"2024-09-10T20:29:20.016002+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:25:12.240664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.019846+0000","last_change":"2024-09-10T20:29:20.018969+0000","last_active":"2024-09-10T20:29:20.019846+0000","last_peered":"2024-09-10T20:29:20.019846+0000","last_clean":"2024-09-10T20:29:20.019846+0000","last_became_active":"2024-09-10T20:29:20.018675+0000","last_became_peered":"2024-09-10T20:29:20.018675+0000","last_unstale":"2024-09-10T20:29:20.019846+0000","last_undegraded":"2024-09-10T20:29:20.019846+0000","last_fullsized":"2024-09-10T20:29:20.019846+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:50.839971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'1282","reported_seq":1342,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:35.936527+0000","last_change":"2024-09-10T20:29:19.024494+0000","last_active":"2024-09-10T20:29:35.936527+0000","last_peered":"2024-09-10T20:29:35.936527+0000","last_clean":"2024-09-10T20:28:46.321317+0000","last_became_active":"2024-09-10T20:29:19.024217+0000","last_became_peered":"2024-09-10T20:29:19.024217+0000","last_unstale":"2024-09-10T20:29:35.936527+0000","last_undegraded":"2024-09-10T20:29:19.016668+0000","last_fullsized":"2024-09-10T20:29:35.936527+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":1282,"log_dups_size":0,"ondisk_log_size":1282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5216140,"num_objects":1277,"num_object_clones":0,"num_object_copies":2554,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":1277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1282,"num_write_kb":5099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":1276}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":76436692,"num_objects":18686,"num_object_clones":0,"num_object_copies":37372,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":53,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":18686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18713,"num_write_kb":74678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":145956864,"data_stored":145819570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18713,"ondisk_log_size":18713,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182412,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":148516,"kb_used_data":80788,"kb_used_omap":4,"kb_used_meta":67707,"kb_avail":94223324,"statfs":{"total":96636764160,"available":96484683776,"internally_reserved":0,"allocated":82726912,"data_stored":82491586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":69332650},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,1,5,4],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986183,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":112020,"kb_used_data":53956,"kb_used_omap":4,"kb_used_meta":58043,"kb_avail":94259820,"statfs":{"total":96636764160,"available":96522055680,"internally_reserved":0,"allocated":55250944,"data_stored":55055825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":59436714},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,2,2,8],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":87093248,"data_stored":87057118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":58748928,"data_stored":58748928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:29:38.525 INFO:tasks.rep_lost_unfound_delete:no unfound in 2.f 2024-09-10T20:29:38.525 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.e state active+recovery_unfound+degraded 2024-09-10T20:29:38.525 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.e list_unfound '{}' 2024-09-10T20:29:38.724 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-10T20:29:38.724 INFO:teuthology.orchestra.run.smithi131.stdout: "offset": { 2024-09-10T20:29:38.724 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": "", 2024-09-10T20:29:38.724 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "", 2024-09-10T20:29:38.724 INFO:teuthology.orchestra.run.smithi131.stdout: "snapid": 0, 2024-09-10T20:29:38.725 INFO:teuthology.orchestra.run.smithi131.stdout: "hash": 0, 2024-09-10T20:29:38.725 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 0, 2024-09-10T20:29:38.725 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": -9223372036854775808, 2024-09-10T20:29:38.725 INFO:teuthology.orchestra.run.smithi131.stdout: "namespace": "" 2024-09-10T20:29:38.725 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:38.726 INFO:teuthology.orchestra.run.smithi131.stdout: "num_missing": 2, 2024-09-10T20:29:38.726 INFO:teuthology.orchestra.run.smithi131.stdout: "num_unfound": 2, 2024-09-10T20:29:38.726 INFO:teuthology.orchestra.run.smithi131.stdout: "objects": [ 2024-09-10T20:29:38.726 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-10T20:29:38.727 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": { 2024-09-10T20:29:38.727 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": "existing_4", 2024-09-10T20:29:38.727 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "", 2024-09-10T20:29:38.727 INFO:teuthology.orchestra.run.smithi131.stdout: "snapid": -2, 2024-09-10T20:29:38.727 INFO:teuthology.orchestra.run.smithi131.stdout: "hash": 1224592654, 2024-09-10T20:29:38.728 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 0, 2024-09-10T20:29:38.728 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": 2, 2024-09-10T20:29:38.728 INFO:teuthology.orchestra.run.smithi131.stdout: "namespace": "" 2024-09-10T20:29:38.728 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:38.728 INFO:teuthology.orchestra.run.smithi131.stdout: "need": "23'3", 2024-09-10T20:29:38.729 INFO:teuthology.orchestra.run.smithi131.stdout: "have": "21'1", 2024-09-10T20:29:38.729 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": "none", 2024-09-10T20:29:38.729 INFO:teuthology.orchestra.run.smithi131.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-10T20:29:38.729 INFO:teuthology.orchestra.run.smithi131.stdout: "locations": [] 2024-09-10T20:29:38.729 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:38.730 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-10T20:29:38.730 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": { 2024-09-10T20:29:38.730 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": "existing_6", 2024-09-10T20:29:38.730 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "", 2024-09-10T20:29:38.730 INFO:teuthology.orchestra.run.smithi131.stdout: "snapid": -2, 2024-09-10T20:29:38.731 INFO:teuthology.orchestra.run.smithi131.stdout: "hash": 3101443614, 2024-09-10T20:29:38.731 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 0, 2024-09-10T20:29:38.731 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": 2, 2024-09-10T20:29:38.731 INFO:teuthology.orchestra.run.smithi131.stdout: "namespace": "" 2024-09-10T20:29:38.731 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:38.732 INFO:teuthology.orchestra.run.smithi131.stdout: "need": "23'4", 2024-09-10T20:29:38.732 INFO:teuthology.orchestra.run.smithi131.stdout: "have": "21'2", 2024-09-10T20:29:38.732 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": "none", 2024-09-10T20:29:38.732 INFO:teuthology.orchestra.run.smithi131.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-10T20:29:38.732 INFO:teuthology.orchestra.run.smithi131.stdout: "locations": [] 2024-09-10T20:29:38.733 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-10T20:29:38.733 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-09-10T20:29:38.733 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "NotRecovering", 2024-09-10T20:29:38.733 INFO:teuthology.orchestra.run.smithi131.stdout: "available_might_have_unfound": true, 2024-09-10T20:29:38.733 INFO:teuthology.orchestra.run.smithi131.stdout: "might_have_unfound": [ 2024-09-10T20:29:38.734 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-10T20:29:38.734 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "1", 2024-09-10T20:29:38.734 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "osd is down" 2024-09-10T20:29:38.734 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-10T20:29:38.734 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-09-10T20:29:38.734 INFO:teuthology.orchestra.run.smithi131.stdout: "more": false 2024-09-10T20:29:38.735 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-10T20:29:38.748 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.e on osd.2 2024-09-10T20:29:38.748 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.e mark_unfound_lost delete 2024-09-10T20:29:38.967 INFO:teuthology.orchestra.run.smithi131.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-10T20:29:38.988 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.d state active+recovery_unfound+degraded 2024-09-10T20:29:38.988 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.d list_unfound '{}' 2024-09-10T20:29:39.031 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 11 20 26559 26539 9.42325 6.88281 0.0137695 0.0082833 2024-09-10T20:29:39.204 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-10T20:29:39.204 INFO:teuthology.orchestra.run.smithi131.stdout: "offset": { 2024-09-10T20:29:39.205 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": "", 2024-09-10T20:29:39.205 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "", 2024-09-10T20:29:39.205 INFO:teuthology.orchestra.run.smithi131.stdout: "snapid": 0, 2024-09-10T20:29:39.205 INFO:teuthology.orchestra.run.smithi131.stdout: "hash": 0, 2024-09-10T20:29:39.205 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 0, 2024-09-10T20:29:39.205 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": -9223372036854775808, 2024-09-10T20:29:39.206 INFO:teuthology.orchestra.run.smithi131.stdout: "namespace": "" 2024-09-10T20:29:39.206 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:39.206 INFO:teuthology.orchestra.run.smithi131.stdout: "num_missing": 1, 2024-09-10T20:29:39.206 INFO:teuthology.orchestra.run.smithi131.stdout: "num_unfound": 1, 2024-09-10T20:29:39.206 INFO:teuthology.orchestra.run.smithi131.stdout: "objects": [ 2024-09-10T20:29:39.207 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-10T20:29:39.207 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": { 2024-09-10T20:29:39.207 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": "existed_7", 2024-09-10T20:29:39.207 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "", 2024-09-10T20:29:39.207 INFO:teuthology.orchestra.run.smithi131.stdout: "snapid": -2, 2024-09-10T20:29:39.207 INFO:teuthology.orchestra.run.smithi131.stdout: "hash": 2358579165, 2024-09-10T20:29:39.208 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 0, 2024-09-10T20:29:39.208 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": 2, 2024-09-10T20:29:39.208 INFO:teuthology.orchestra.run.smithi131.stdout: "namespace": "" 2024-09-10T20:29:39.208 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:39.208 INFO:teuthology.orchestra.run.smithi131.stdout: "need": "23'3", 2024-09-10T20:29:39.209 INFO:teuthology.orchestra.run.smithi131.stdout: "have": "0'0", 2024-09-10T20:29:39.209 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": "none", 2024-09-10T20:29:39.209 INFO:teuthology.orchestra.run.smithi131.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T20:29:39.209 INFO:teuthology.orchestra.run.smithi131.stdout: "locations": [] 2024-09-10T20:29:39.209 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-10T20:29:39.209 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-09-10T20:29:39.210 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "NotRecovering", 2024-09-10T20:29:39.210 INFO:teuthology.orchestra.run.smithi131.stdout: "available_might_have_unfound": true, 2024-09-10T20:29:39.210 INFO:teuthology.orchestra.run.smithi131.stdout: "might_have_unfound": [ 2024-09-10T20:29:39.210 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-10T20:29:39.210 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "1", 2024-09-10T20:29:39.211 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "osd is down" 2024-09-10T20:29:39.211 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-10T20:29:39.211 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-09-10T20:29:39.211 INFO:teuthology.orchestra.run.smithi131.stdout: "more": false 2024-09-10T20:29:39.211 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-10T20:29:39.221 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.d on osd.2 2024-09-10T20:29:39.222 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.d mark_unfound_lost delete 2024-09-10T20:29:39.436 INFO:teuthology.orchestra.run.smithi131.stderr:pg has 1 objects unfound and apparently lost marking 2024-09-10T20:29:39.454 INFO:tasks.rep_lost_unfound_delete:no unfound in 2.c 2024-09-10T20:29:39.454 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.b state active+recovery_wait+degraded 2024-09-10T20:29:39.454 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.b list_unfound '{}' 2024-09-10T20:29:39.660 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-10T20:29:39.660 INFO:teuthology.orchestra.run.smithi131.stdout: "offset": { 2024-09-10T20:29:39.660 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": "", 2024-09-10T20:29:39.660 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "", 2024-09-10T20:29:39.660 INFO:teuthology.orchestra.run.smithi131.stdout: "snapid": 0, 2024-09-10T20:29:39.660 INFO:teuthology.orchestra.run.smithi131.stdout: "hash": 0, 2024-09-10T20:29:39.660 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 0, 2024-09-10T20:29:39.660 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": -9223372036854775808, 2024-09-10T20:29:39.660 INFO:teuthology.orchestra.run.smithi131.stdout: "namespace": "" 2024-09-10T20:29:39.660 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:39.660 INFO:teuthology.orchestra.run.smithi131.stdout: "num_missing": 2, 2024-09-10T20:29:39.661 INFO:teuthology.orchestra.run.smithi131.stdout: "num_unfound": 2, 2024-09-10T20:29:39.661 INFO:teuthology.orchestra.run.smithi131.stdout: "objects": [ 2024-09-10T20:29:39.661 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-10T20:29:39.661 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": { 2024-09-10T20:29:39.661 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": "existed_3", 2024-09-10T20:29:39.661 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "", 2024-09-10T20:29:39.661 INFO:teuthology.orchestra.run.smithi131.stdout: "snapid": -2, 2024-09-10T20:29:39.661 INFO:teuthology.orchestra.run.smithi131.stdout: "hash": 943960171, 2024-09-10T20:29:39.661 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 0, 2024-09-10T20:29:39.661 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": 2, 2024-09-10T20:29:39.661 INFO:teuthology.orchestra.run.smithi131.stdout: "namespace": "" 2024-09-10T20:29:39.661 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:39.661 INFO:teuthology.orchestra.run.smithi131.stdout: "need": "23'5", 2024-09-10T20:29:39.661 INFO:teuthology.orchestra.run.smithi131.stdout: "have": "0'0", 2024-09-10T20:29:39.662 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": "none", 2024-09-10T20:29:39.662 INFO:teuthology.orchestra.run.smithi131.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T20:29:39.662 INFO:teuthology.orchestra.run.smithi131.stdout: "locations": [] 2024-09-10T20:29:39.662 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:39.662 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-10T20:29:39.662 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": { 2024-09-10T20:29:39.662 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": "existing_1", 2024-09-10T20:29:39.662 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "", 2024-09-10T20:29:39.662 INFO:teuthology.orchestra.run.smithi131.stdout: "snapid": -2, 2024-09-10T20:29:39.662 INFO:teuthology.orchestra.run.smithi131.stdout: "hash": 2885884443, 2024-09-10T20:29:39.662 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 0, 2024-09-10T20:29:39.662 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": 2, 2024-09-10T20:29:39.662 INFO:teuthology.orchestra.run.smithi131.stdout: "namespace": "" 2024-09-10T20:29:39.663 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:39.663 INFO:teuthology.orchestra.run.smithi131.stdout: "need": "23'4", 2024-09-10T20:29:39.663 INFO:teuthology.orchestra.run.smithi131.stdout: "have": "21'1", 2024-09-10T20:29:39.663 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": "none", 2024-09-10T20:29:39.663 INFO:teuthology.orchestra.run.smithi131.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-10T20:29:39.663 INFO:teuthology.orchestra.run.smithi131.stdout: "locations": [] 2024-09-10T20:29:39.663 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-10T20:29:39.663 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-09-10T20:29:39.663 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "Active", 2024-09-10T20:29:39.663 INFO:teuthology.orchestra.run.smithi131.stdout: "available_might_have_unfound": true, 2024-09-10T20:29:39.663 INFO:teuthology.orchestra.run.smithi131.stdout: "might_have_unfound": [ 2024-09-10T20:29:39.663 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-10T20:29:39.663 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "1", 2024-09-10T20:29:39.663 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "osd is down" 2024-09-10T20:29:39.664 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-10T20:29:39.664 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-09-10T20:29:39.664 INFO:teuthology.orchestra.run.smithi131.stdout: "more": false 2024-09-10T20:29:39.664 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-10T20:29:39.683 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.b on osd.0 2024-09-10T20:29:39.684 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.b mark_unfound_lost delete 2024-09-10T20:29:39.913 INFO:teuthology.orchestra.run.smithi131.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-10T20:29:39.933 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.a state active+recovery_wait+degraded 2024-09-10T20:29:39.933 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.a list_unfound '{}' 2024-09-10T20:29:40.031 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 12 20 28089 28069 9.136 5.97656 0.0132461 0.00854242 2024-09-10T20:29:40.133 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-10T20:29:40.133 INFO:teuthology.orchestra.run.smithi131.stdout: "offset": { 2024-09-10T20:29:40.133 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": "", 2024-09-10T20:29:40.133 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "", 2024-09-10T20:29:40.133 INFO:teuthology.orchestra.run.smithi131.stdout: "snapid": 0, 2024-09-10T20:29:40.133 INFO:teuthology.orchestra.run.smithi131.stdout: "hash": 0, 2024-09-10T20:29:40.133 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 0, 2024-09-10T20:29:40.133 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": -9223372036854775808, 2024-09-10T20:29:40.133 INFO:teuthology.orchestra.run.smithi131.stdout: "namespace": "" 2024-09-10T20:29:40.133 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:40.133 INFO:teuthology.orchestra.run.smithi131.stdout: "num_missing": 2, 2024-09-10T20:29:40.134 INFO:teuthology.orchestra.run.smithi131.stdout: "num_unfound": 2, 2024-09-10T20:29:40.134 INFO:teuthology.orchestra.run.smithi131.stdout: "objects": [ 2024-09-10T20:29:40.134 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-10T20:29:40.134 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": { 2024-09-10T20:29:40.134 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": "existing_9", 2024-09-10T20:29:40.134 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "", 2024-09-10T20:29:40.134 INFO:teuthology.orchestra.run.smithi131.stdout: "snapid": -2, 2024-09-10T20:29:40.134 INFO:teuthology.orchestra.run.smithi131.stdout: "hash": 1217469130, 2024-09-10T20:29:40.134 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 0, 2024-09-10T20:29:40.134 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": 2, 2024-09-10T20:29:40.134 INFO:teuthology.orchestra.run.smithi131.stdout: "namespace": "" 2024-09-10T20:29:40.134 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:40.134 INFO:teuthology.orchestra.run.smithi131.stdout: "need": "23'3", 2024-09-10T20:29:40.135 INFO:teuthology.orchestra.run.smithi131.stdout: "have": "21'1", 2024-09-10T20:29:40.135 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": "none", 2024-09-10T20:29:40.135 INFO:teuthology.orchestra.run.smithi131.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-10T20:29:40.135 INFO:teuthology.orchestra.run.smithi131.stdout: "locations": [] 2024-09-10T20:29:40.135 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:40.135 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-10T20:29:40.135 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": { 2024-09-10T20:29:40.135 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": "new_9", 2024-09-10T20:29:40.135 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "", 2024-09-10T20:29:40.135 INFO:teuthology.orchestra.run.smithi131.stdout: "snapid": -2, 2024-09-10T20:29:40.135 INFO:teuthology.orchestra.run.smithi131.stdout: "hash": 170713514, 2024-09-10T20:29:40.135 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 0, 2024-09-10T20:29:40.135 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": 2, 2024-09-10T20:29:40.135 INFO:teuthology.orchestra.run.smithi131.stdout: "namespace": "" 2024-09-10T20:29:40.136 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:40.136 INFO:teuthology.orchestra.run.smithi131.stdout: "need": "23'2", 2024-09-10T20:29:40.136 INFO:teuthology.orchestra.run.smithi131.stdout: "have": "0'0", 2024-09-10T20:29:40.136 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": "none", 2024-09-10T20:29:40.136 INFO:teuthology.orchestra.run.smithi131.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T20:29:40.136 INFO:teuthology.orchestra.run.smithi131.stdout: "locations": [] 2024-09-10T20:29:40.136 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-10T20:29:40.136 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-09-10T20:29:40.136 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "Active", 2024-09-10T20:29:40.136 INFO:teuthology.orchestra.run.smithi131.stdout: "available_might_have_unfound": true, 2024-09-10T20:29:40.136 INFO:teuthology.orchestra.run.smithi131.stdout: "might_have_unfound": [ 2024-09-10T20:29:40.136 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-10T20:29:40.136 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "1", 2024-09-10T20:29:40.137 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "osd is down" 2024-09-10T20:29:40.137 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-10T20:29:40.137 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-09-10T20:29:40.137 INFO:teuthology.orchestra.run.smithi131.stdout: "more": false 2024-09-10T20:29:40.137 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-10T20:29:40.153 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.a on osd.0 2024-09-10T20:29:40.153 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.a mark_unfound_lost delete 2024-09-10T20:29:40.383 INFO:teuthology.orchestra.run.smithi131.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-10T20:29:40.405 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.9 state active+recovery_wait+degraded 2024-09-10T20:29:40.406 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.9 list_unfound '{}' 2024-09-10T20:29:40.605 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-10T20:29:40.605 INFO:teuthology.orchestra.run.smithi131.stdout: "offset": { 2024-09-10T20:29:40.605 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": "", 2024-09-10T20:29:40.605 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "", 2024-09-10T20:29:40.605 INFO:teuthology.orchestra.run.smithi131.stdout: "snapid": 0, 2024-09-10T20:29:40.606 INFO:teuthology.orchestra.run.smithi131.stdout: "hash": 0, 2024-09-10T20:29:40.606 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 0, 2024-09-10T20:29:40.606 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": -9223372036854775808, 2024-09-10T20:29:40.606 INFO:teuthology.orchestra.run.smithi131.stdout: "namespace": "" 2024-09-10T20:29:40.606 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:40.606 INFO:teuthology.orchestra.run.smithi131.stdout: "num_missing": 2, 2024-09-10T20:29:40.606 INFO:teuthology.orchestra.run.smithi131.stdout: "num_unfound": 2, 2024-09-10T20:29:40.606 INFO:teuthology.orchestra.run.smithi131.stdout: "objects": [ 2024-09-10T20:29:40.606 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-10T20:29:40.606 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": { 2024-09-10T20:29:40.606 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": "existing_5", 2024-09-10T20:29:40.606 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "", 2024-09-10T20:29:40.606 INFO:teuthology.orchestra.run.smithi131.stdout: "snapid": -2, 2024-09-10T20:29:40.606 INFO:teuthology.orchestra.run.smithi131.stdout: "hash": 2998549577, 2024-09-10T20:29:40.607 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 0, 2024-09-10T20:29:40.607 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": 2, 2024-09-10T20:29:40.607 INFO:teuthology.orchestra.run.smithi131.stdout: "namespace": "" 2024-09-10T20:29:40.607 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:40.607 INFO:teuthology.orchestra.run.smithi131.stdout: "need": "23'4", 2024-09-10T20:29:40.607 INFO:teuthology.orchestra.run.smithi131.stdout: "have": "21'2", 2024-09-10T20:29:40.607 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": "none", 2024-09-10T20:29:40.607 INFO:teuthology.orchestra.run.smithi131.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-10T20:29:40.607 INFO:teuthology.orchestra.run.smithi131.stdout: "locations": [] 2024-09-10T20:29:40.607 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:40.607 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-10T20:29:40.607 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": { 2024-09-10T20:29:40.607 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": "existing_2", 2024-09-10T20:29:40.607 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "", 2024-09-10T20:29:40.608 INFO:teuthology.orchestra.run.smithi131.stdout: "snapid": -2, 2024-09-10T20:29:40.608 INFO:teuthology.orchestra.run.smithi131.stdout: "hash": 4032093161, 2024-09-10T20:29:40.608 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 0, 2024-09-10T20:29:40.608 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": 2, 2024-09-10T20:29:40.608 INFO:teuthology.orchestra.run.smithi131.stdout: "namespace": "" 2024-09-10T20:29:40.608 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:40.608 INFO:teuthology.orchestra.run.smithi131.stdout: "need": "23'3", 2024-09-10T20:29:40.608 INFO:teuthology.orchestra.run.smithi131.stdout: "have": "21'1", 2024-09-10T20:29:40.608 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": "none", 2024-09-10T20:29:40.608 INFO:teuthology.orchestra.run.smithi131.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-10T20:29:40.608 INFO:teuthology.orchestra.run.smithi131.stdout: "locations": [] 2024-09-10T20:29:40.608 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-10T20:29:40.608 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-09-10T20:29:40.609 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "Active", 2024-09-10T20:29:40.609 INFO:teuthology.orchestra.run.smithi131.stdout: "available_might_have_unfound": true, 2024-09-10T20:29:40.609 INFO:teuthology.orchestra.run.smithi131.stdout: "might_have_unfound": [ 2024-09-10T20:29:40.609 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-10T20:29:40.609 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "1", 2024-09-10T20:29:40.609 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "osd is down" 2024-09-10T20:29:40.609 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-10T20:29:40.609 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-09-10T20:29:40.609 INFO:teuthology.orchestra.run.smithi131.stdout: "more": false 2024-09-10T20:29:40.609 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-10T20:29:40.629 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.9 on osd.0 2024-09-10T20:29:40.629 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.9 mark_unfound_lost delete 2024-09-10T20:29:40.853 INFO:teuthology.orchestra.run.smithi131.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-10T20:29:40.871 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.8 state active+recovering 2024-09-10T20:29:40.871 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.8 list_unfound '{}' 2024-09-10T20:29:41.031 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 13 20 29706 29686 8.91908 6.31641 0.00918901 0.00875063 2024-09-10T20:29:41.070 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-10T20:29:41.070 INFO:teuthology.orchestra.run.smithi131.stdout: "offset": { 2024-09-10T20:29:41.070 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": "", 2024-09-10T20:29:41.070 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "", 2024-09-10T20:29:41.070 INFO:teuthology.orchestra.run.smithi131.stdout: "snapid": 0, 2024-09-10T20:29:41.070 INFO:teuthology.orchestra.run.smithi131.stdout: "hash": 0, 2024-09-10T20:29:41.070 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 0, 2024-09-10T20:29:41.070 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": -9223372036854775808, 2024-09-10T20:29:41.070 INFO:teuthology.orchestra.run.smithi131.stdout: "namespace": "" 2024-09-10T20:29:41.070 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:41.070 INFO:teuthology.orchestra.run.smithi131.stdout: "num_missing": 1, 2024-09-10T20:29:41.070 INFO:teuthology.orchestra.run.smithi131.stdout: "num_unfound": 1, 2024-09-10T20:29:41.071 INFO:teuthology.orchestra.run.smithi131.stdout: "objects": [ 2024-09-10T20:29:41.071 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-10T20:29:41.071 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": { 2024-09-10T20:29:41.071 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": "new_8", 2024-09-10T20:29:41.071 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "", 2024-09-10T20:29:41.071 INFO:teuthology.orchestra.run.smithi131.stdout: "snapid": -2, 2024-09-10T20:29:41.071 INFO:teuthology.orchestra.run.smithi131.stdout: "hash": 2730327960, 2024-09-10T20:29:41.071 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 0, 2024-09-10T20:29:41.071 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": 2, 2024-09-10T20:29:41.071 INFO:teuthology.orchestra.run.smithi131.stdout: "namespace": "" 2024-09-10T20:29:41.071 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:41.071 INFO:teuthology.orchestra.run.smithi131.stdout: "need": "23'1", 2024-09-10T20:29:41.071 INFO:teuthology.orchestra.run.smithi131.stdout: "have": "0'0", 2024-09-10T20:29:41.072 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": "none", 2024-09-10T20:29:41.072 INFO:teuthology.orchestra.run.smithi131.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T20:29:41.072 INFO:teuthology.orchestra.run.smithi131.stdout: "locations": [] 2024-09-10T20:29:41.072 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-10T20:29:41.072 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-09-10T20:29:41.072 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "Active", 2024-09-10T20:29:41.072 INFO:teuthology.orchestra.run.smithi131.stdout: "available_might_have_unfound": true, 2024-09-10T20:29:41.072 INFO:teuthology.orchestra.run.smithi131.stdout: "might_have_unfound": [ 2024-09-10T20:29:41.072 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-10T20:29:41.072 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "1", 2024-09-10T20:29:41.072 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "osd is down" 2024-09-10T20:29:41.072 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-10T20:29:41.072 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-09-10T20:29:41.072 INFO:teuthology.orchestra.run.smithi131.stdout: "more": false 2024-09-10T20:29:41.073 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-10T20:29:41.090 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.8 on osd.0 2024-09-10T20:29:41.091 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.8 mark_unfound_lost delete 2024-09-10T20:29:41.314 INFO:teuthology.orchestra.run.smithi131.stderr:pg has 1 objects unfound and apparently lost marking 2024-09-10T20:29:41.335 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.0 2024-09-10T20:29:41.336 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.3 state active+recovery_unfound+degraded 2024-09-10T20:29:41.336 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.3 list_unfound '{}' 2024-09-10T20:29:41.553 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-10T20:29:41.553 INFO:teuthology.orchestra.run.smithi131.stdout: "offset": { 2024-09-10T20:29:41.553 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": "", 2024-09-10T20:29:41.553 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "", 2024-09-10T20:29:41.553 INFO:teuthology.orchestra.run.smithi131.stdout: "snapid": 0, 2024-09-10T20:29:41.553 INFO:teuthology.orchestra.run.smithi131.stdout: "hash": 0, 2024-09-10T20:29:41.553 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 0, 2024-09-10T20:29:41.553 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": -9223372036854775808, 2024-09-10T20:29:41.553 INFO:teuthology.orchestra.run.smithi131.stdout: "namespace": "" 2024-09-10T20:29:41.553 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:41.554 INFO:teuthology.orchestra.run.smithi131.stdout: "num_missing": 2, 2024-09-10T20:29:41.554 INFO:teuthology.orchestra.run.smithi131.stdout: "num_unfound": 2, 2024-09-10T20:29:41.554 INFO:teuthology.orchestra.run.smithi131.stdout: "objects": [ 2024-09-10T20:29:41.554 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-10T20:29:41.554 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": { 2024-09-10T20:29:41.554 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": "existing_8", 2024-09-10T20:29:41.554 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "", 2024-09-10T20:29:41.554 INFO:teuthology.orchestra.run.smithi131.stdout: "snapid": -2, 2024-09-10T20:29:41.554 INFO:teuthology.orchestra.run.smithi131.stdout: "hash": 1901787523, 2024-09-10T20:29:41.554 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 0, 2024-09-10T20:29:41.554 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": 2, 2024-09-10T20:29:41.554 INFO:teuthology.orchestra.run.smithi131.stdout: "namespace": "" 2024-09-10T20:29:41.554 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:41.554 INFO:teuthology.orchestra.run.smithi131.stdout: "need": "23'5", 2024-09-10T20:29:41.555 INFO:teuthology.orchestra.run.smithi131.stdout: "have": "21'3", 2024-09-10T20:29:41.555 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": "none", 2024-09-10T20:29:41.555 INFO:teuthology.orchestra.run.smithi131.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-10T20:29:41.555 INFO:teuthology.orchestra.run.smithi131.stdout: "locations": [] 2024-09-10T20:29:41.555 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:41.555 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-10T20:29:41.555 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": { 2024-09-10T20:29:41.555 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": "existed_1", 2024-09-10T20:29:41.555 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "", 2024-09-10T20:29:41.555 INFO:teuthology.orchestra.run.smithi131.stdout: "snapid": -2, 2024-09-10T20:29:41.555 INFO:teuthology.orchestra.run.smithi131.stdout: "hash": 3122379363, 2024-09-10T20:29:41.555 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 0, 2024-09-10T20:29:41.555 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": 2, 2024-09-10T20:29:41.556 INFO:teuthology.orchestra.run.smithi131.stdout: "namespace": "" 2024-09-10T20:29:41.556 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:41.556 INFO:teuthology.orchestra.run.smithi131.stdout: "need": "23'4", 2024-09-10T20:29:41.556 INFO:teuthology.orchestra.run.smithi131.stdout: "have": "0'0", 2024-09-10T20:29:41.556 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": "none", 2024-09-10T20:29:41.556 INFO:teuthology.orchestra.run.smithi131.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T20:29:41.556 INFO:teuthology.orchestra.run.smithi131.stdout: "locations": [] 2024-09-10T20:29:41.556 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-10T20:29:41.556 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-09-10T20:29:41.556 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "NotRecovering", 2024-09-10T20:29:41.556 INFO:teuthology.orchestra.run.smithi131.stdout: "available_might_have_unfound": true, 2024-09-10T20:29:41.556 INFO:teuthology.orchestra.run.smithi131.stdout: "might_have_unfound": [ 2024-09-10T20:29:41.556 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-10T20:29:41.556 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "1", 2024-09-10T20:29:41.557 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "osd is down" 2024-09-10T20:29:41.557 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-10T20:29:41.557 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-09-10T20:29:41.557 INFO:teuthology.orchestra.run.smithi131.stdout: "more": false 2024-09-10T20:29:41.557 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-10T20:29:41.570 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.3 on osd.2 2024-09-10T20:29:41.570 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.3 mark_unfound_lost delete 2024-09-10T20:29:41.788 INFO:teuthology.orchestra.run.smithi131.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-10T20:29:41.808 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.1 2024-09-10T20:29:41.808 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.2 state active+recovery_wait+degraded 2024-09-10T20:29:41.809 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.2 list_unfound '{}' 2024-09-10T20:29:42.023 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-10T20:29:42.023 INFO:teuthology.orchestra.run.smithi131.stdout: "offset": { 2024-09-10T20:29:42.023 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": "", 2024-09-10T20:29:42.023 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "", 2024-09-10T20:29:42.023 INFO:teuthology.orchestra.run.smithi131.stdout: "snapid": 0, 2024-09-10T20:29:42.023 INFO:teuthology.orchestra.run.smithi131.stdout: "hash": 0, 2024-09-10T20:29:42.023 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 0, 2024-09-10T20:29:42.023 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": -9223372036854775808, 2024-09-10T20:29:42.023 INFO:teuthology.orchestra.run.smithi131.stdout: "namespace": "" 2024-09-10T20:29:42.024 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:42.024 INFO:teuthology.orchestra.run.smithi131.stdout: "num_missing": 2, 2024-09-10T20:29:42.024 INFO:teuthology.orchestra.run.smithi131.stdout: "num_unfound": 2, 2024-09-10T20:29:42.024 INFO:teuthology.orchestra.run.smithi131.stdout: "objects": [ 2024-09-10T20:29:42.024 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-10T20:29:42.024 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": { 2024-09-10T20:29:42.024 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": "existed_6", 2024-09-10T20:29:42.024 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "", 2024-09-10T20:29:42.024 INFO:teuthology.orchestra.run.smithi131.stdout: "snapid": -2, 2024-09-10T20:29:42.024 INFO:teuthology.orchestra.run.smithi131.stdout: "hash": 1708673474, 2024-09-10T20:29:42.024 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 0, 2024-09-10T20:29:42.024 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": 2, 2024-09-10T20:29:42.025 INFO:teuthology.orchestra.run.smithi131.stdout: "namespace": "" 2024-09-10T20:29:42.025 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:42.025 INFO:teuthology.orchestra.run.smithi131.stdout: "need": "23'5", 2024-09-10T20:29:42.025 INFO:teuthology.orchestra.run.smithi131.stdout: "have": "0'0", 2024-09-10T20:29:42.025 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": "none", 2024-09-10T20:29:42.025 INFO:teuthology.orchestra.run.smithi131.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T20:29:42.025 INFO:teuthology.orchestra.run.smithi131.stdout: "locations": [] 2024-09-10T20:29:42.025 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:42.025 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-10T20:29:42.025 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": { 2024-09-10T20:29:42.025 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": "existed_9", 2024-09-10T20:29:42.025 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "", 2024-09-10T20:29:42.025 INFO:teuthology.orchestra.run.smithi131.stdout: "snapid": -2, 2024-09-10T20:29:42.025 INFO:teuthology.orchestra.run.smithi131.stdout: "hash": 831402034, 2024-09-10T20:29:42.025 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 0, 2024-09-10T20:29:42.026 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": 2, 2024-09-10T20:29:42.026 INFO:teuthology.orchestra.run.smithi131.stdout: "namespace": "" 2024-09-10T20:29:42.026 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:42.026 INFO:teuthology.orchestra.run.smithi131.stdout: "need": "23'6", 2024-09-10T20:29:42.026 INFO:teuthology.orchestra.run.smithi131.stdout: "have": "0'0", 2024-09-10T20:29:42.026 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": "none", 2024-09-10T20:29:42.026 INFO:teuthology.orchestra.run.smithi131.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T20:29:42.026 INFO:teuthology.orchestra.run.smithi131.stdout: "locations": [] 2024-09-10T20:29:42.026 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-10T20:29:42.026 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-09-10T20:29:42.026 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "Active", 2024-09-10T20:29:42.026 INFO:teuthology.orchestra.run.smithi131.stdout: "available_might_have_unfound": true, 2024-09-10T20:29:42.026 INFO:teuthology.orchestra.run.smithi131.stdout: "might_have_unfound": [ 2024-09-10T20:29:42.026 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-10T20:29:42.027 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "1", 2024-09-10T20:29:42.027 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "osd is down" 2024-09-10T20:29:42.027 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-10T20:29:42.027 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-09-10T20:29:42.027 INFO:teuthology.orchestra.run.smithi131.stdout: "more": false 2024-09-10T20:29:42.027 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-10T20:29:42.031 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 14 20 31330 31310 8.73509 6.34375 0.0131274 0.00893601 2024-09-10T20:29:42.040 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.2 on osd.0 2024-09-10T20:29:42.040 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.2 mark_unfound_lost delete 2024-09-10T20:29:42.265 INFO:teuthology.orchestra.run.smithi131.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-10T20:29:42.288 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.2 2024-09-10T20:29:42.288 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.1 state active+recovery_unfound+degraded 2024-09-10T20:29:42.288 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.1 list_unfound '{}' 2024-09-10T20:29:42.500 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-10T20:29:42.500 INFO:teuthology.orchestra.run.smithi131.stdout: "offset": { 2024-09-10T20:29:42.500 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": "", 2024-09-10T20:29:42.500 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "", 2024-09-10T20:29:42.500 INFO:teuthology.orchestra.run.smithi131.stdout: "snapid": 0, 2024-09-10T20:29:42.500 INFO:teuthology.orchestra.run.smithi131.stdout: "hash": 0, 2024-09-10T20:29:42.500 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 0, 2024-09-10T20:29:42.500 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": -9223372036854775808, 2024-09-10T20:29:42.501 INFO:teuthology.orchestra.run.smithi131.stdout: "namespace": "" 2024-09-10T20:29:42.501 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:42.501 INFO:teuthology.orchestra.run.smithi131.stdout: "num_missing": 2, 2024-09-10T20:29:42.501 INFO:teuthology.orchestra.run.smithi131.stdout: "num_unfound": 2, 2024-09-10T20:29:42.501 INFO:teuthology.orchestra.run.smithi131.stdout: "objects": [ 2024-09-10T20:29:42.501 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-10T20:29:42.501 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": { 2024-09-10T20:29:42.501 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": "existing_3", 2024-09-10T20:29:42.501 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "", 2024-09-10T20:29:42.501 INFO:teuthology.orchestra.run.smithi131.stdout: "snapid": -2, 2024-09-10T20:29:42.501 INFO:teuthology.orchestra.run.smithi131.stdout: "hash": 459768081, 2024-09-10T20:29:42.501 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 0, 2024-09-10T20:29:42.502 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": 2, 2024-09-10T20:29:42.502 INFO:teuthology.orchestra.run.smithi131.stdout: "namespace": "" 2024-09-10T20:29:42.502 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:42.502 INFO:teuthology.orchestra.run.smithi131.stdout: "need": "23'3", 2024-09-10T20:29:42.502 INFO:teuthology.orchestra.run.smithi131.stdout: "have": "21'1", 2024-09-10T20:29:42.502 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": "none", 2024-09-10T20:29:42.502 INFO:teuthology.orchestra.run.smithi131.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-10T20:29:42.502 INFO:teuthology.orchestra.run.smithi131.stdout: "locations": [] 2024-09-10T20:29:42.502 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:42.502 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-10T20:29:42.502 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": { 2024-09-10T20:29:42.503 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": "new_3", 2024-09-10T20:29:42.503 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "", 2024-09-10T20:29:42.503 INFO:teuthology.orchestra.run.smithi131.stdout: "snapid": -2, 2024-09-10T20:29:42.503 INFO:teuthology.orchestra.run.smithi131.stdout: "hash": 480221105, 2024-09-10T20:29:42.503 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 0, 2024-09-10T20:29:42.503 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": 2, 2024-09-10T20:29:42.503 INFO:teuthology.orchestra.run.smithi131.stdout: "namespace": "" 2024-09-10T20:29:42.503 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:42.503 INFO:teuthology.orchestra.run.smithi131.stdout: "need": "23'2", 2024-09-10T20:29:42.503 INFO:teuthology.orchestra.run.smithi131.stdout: "have": "0'0", 2024-09-10T20:29:42.503 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": "none", 2024-09-10T20:29:42.504 INFO:teuthology.orchestra.run.smithi131.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T20:29:42.504 INFO:teuthology.orchestra.run.smithi131.stdout: "locations": [] 2024-09-10T20:29:42.504 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-10T20:29:42.504 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-09-10T20:29:42.504 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "NotRecovering", 2024-09-10T20:29:42.504 INFO:teuthology.orchestra.run.smithi131.stdout: "available_might_have_unfound": true, 2024-09-10T20:29:42.504 INFO:teuthology.orchestra.run.smithi131.stdout: "might_have_unfound": [ 2024-09-10T20:29:42.504 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-10T20:29:42.504 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "1", 2024-09-10T20:29:42.504 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "osd is down" 2024-09-10T20:29:42.504 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-10T20:29:42.504 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-09-10T20:29:42.505 INFO:teuthology.orchestra.run.smithi131.stdout: "more": false 2024-09-10T20:29:42.505 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-10T20:29:42.522 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.1 on osd.2 2024-09-10T20:29:42.522 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.1 mark_unfound_lost delete 2024-09-10T20:29:42.752 INFO:teuthology.orchestra.run.smithi131.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-10T20:29:42.773 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.3 2024-09-10T20:29:42.773 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.0 state active+recovery_unfound+degraded 2024-09-10T20:29:42.773 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.0 list_unfound '{}' 2024-09-10T20:29:42.980 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-10T20:29:42.980 INFO:teuthology.orchestra.run.smithi131.stdout: "offset": { 2024-09-10T20:29:42.980 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": "", 2024-09-10T20:29:42.980 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "", 2024-09-10T20:29:42.980 INFO:teuthology.orchestra.run.smithi131.stdout: "snapid": 0, 2024-09-10T20:29:42.981 INFO:teuthology.orchestra.run.smithi131.stdout: "hash": 0, 2024-09-10T20:29:42.981 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 0, 2024-09-10T20:29:42.981 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": -9223372036854775808, 2024-09-10T20:29:42.981 INFO:teuthology.orchestra.run.smithi131.stdout: "namespace": "" 2024-09-10T20:29:42.981 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:42.981 INFO:teuthology.orchestra.run.smithi131.stdout: "num_missing": 2, 2024-09-10T20:29:42.981 INFO:teuthology.orchestra.run.smithi131.stdout: "num_unfound": 2, 2024-09-10T20:29:42.981 INFO:teuthology.orchestra.run.smithi131.stdout: "objects": [ 2024-09-10T20:29:42.981 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-10T20:29:42.981 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": { 2024-09-10T20:29:42.981 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": "new_5", 2024-09-10T20:29:42.981 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "", 2024-09-10T20:29:42.981 INFO:teuthology.orchestra.run.smithi131.stdout: "snapid": -2, 2024-09-10T20:29:42.982 INFO:teuthology.orchestra.run.smithi131.stdout: "hash": 4248945696, 2024-09-10T20:29:42.982 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 0, 2024-09-10T20:29:42.982 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": 2, 2024-09-10T20:29:42.982 INFO:teuthology.orchestra.run.smithi131.stdout: "namespace": "" 2024-09-10T20:29:42.982 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:42.982 INFO:teuthology.orchestra.run.smithi131.stdout: "need": "23'3", 2024-09-10T20:29:42.982 INFO:teuthology.orchestra.run.smithi131.stdout: "have": "0'0", 2024-09-10T20:29:42.982 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": "none", 2024-09-10T20:29:42.982 INFO:teuthology.orchestra.run.smithi131.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T20:29:42.982 INFO:teuthology.orchestra.run.smithi131.stdout: "locations": [] 2024-09-10T20:29:42.982 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:42.982 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-10T20:29:42.982 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": { 2024-09-10T20:29:42.982 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": "existed_5", 2024-09-10T20:29:42.983 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "", 2024-09-10T20:29:42.983 INFO:teuthology.orchestra.run.smithi131.stdout: "snapid": -2, 2024-09-10T20:29:42.983 INFO:teuthology.orchestra.run.smithi131.stdout: "hash": 4212789968, 2024-09-10T20:29:42.983 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 0, 2024-09-10T20:29:42.983 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": 2, 2024-09-10T20:29:42.983 INFO:teuthology.orchestra.run.smithi131.stdout: "namespace": "" 2024-09-10T20:29:42.983 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:42.983 INFO:teuthology.orchestra.run.smithi131.stdout: "need": "23'4", 2024-09-10T20:29:42.983 INFO:teuthology.orchestra.run.smithi131.stdout: "have": "0'0", 2024-09-10T20:29:42.983 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": "none", 2024-09-10T20:29:42.983 INFO:teuthology.orchestra.run.smithi131.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T20:29:42.983 INFO:teuthology.orchestra.run.smithi131.stdout: "locations": [] 2024-09-10T20:29:42.983 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-10T20:29:42.983 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-09-10T20:29:42.984 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "NotRecovering", 2024-09-10T20:29:42.984 INFO:teuthology.orchestra.run.smithi131.stdout: "available_might_have_unfound": true, 2024-09-10T20:29:42.984 INFO:teuthology.orchestra.run.smithi131.stdout: "might_have_unfound": [ 2024-09-10T20:29:42.984 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-10T20:29:42.984 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "1", 2024-09-10T20:29:42.984 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "osd is down" 2024-09-10T20:29:42.984 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-10T20:29:42.984 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-09-10T20:29:42.984 INFO:teuthology.orchestra.run.smithi131.stdout: "more": false 2024-09-10T20:29:42.984 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-10T20:29:43.002 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.0 on osd.2 2024-09-10T20:29:43.002 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.0 mark_unfound_lost delete 2024-09-10T20:29:43.031 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 15 20 32907 32887 8.56334 6.16016 0.0282185 0.00911558 2024-09-10T20:29:43.227 INFO:teuthology.orchestra.run.smithi131.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-10T20:29:43.249 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.7 state active+recovery_wait+degraded 2024-09-10T20:29:43.249 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.7 list_unfound '{}' 2024-09-10T20:29:43.493 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-10T20:29:43.494 INFO:teuthology.orchestra.run.smithi131.stdout: "offset": { 2024-09-10T20:29:43.494 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": "", 2024-09-10T20:29:43.494 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "", 2024-09-10T20:29:43.494 INFO:teuthology.orchestra.run.smithi131.stdout: "snapid": 0, 2024-09-10T20:29:43.494 INFO:teuthology.orchestra.run.smithi131.stdout: "hash": 0, 2024-09-10T20:29:43.494 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 0, 2024-09-10T20:29:43.494 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": -9223372036854775808, 2024-09-10T20:29:43.494 INFO:teuthology.orchestra.run.smithi131.stdout: "namespace": "" 2024-09-10T20:29:43.494 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:43.494 INFO:teuthology.orchestra.run.smithi131.stdout: "num_missing": 1, 2024-09-10T20:29:43.494 INFO:teuthology.orchestra.run.smithi131.stdout: "num_unfound": 1, 2024-09-10T20:29:43.494 INFO:teuthology.orchestra.run.smithi131.stdout: "objects": [ 2024-09-10T20:29:43.495 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-10T20:29:43.495 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": { 2024-09-10T20:29:43.495 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": "new_7", 2024-09-10T20:29:43.495 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "", 2024-09-10T20:29:43.495 INFO:teuthology.orchestra.run.smithi131.stdout: "snapid": -2, 2024-09-10T20:29:43.495 INFO:teuthology.orchestra.run.smithi131.stdout: "hash": 3761017527, 2024-09-10T20:29:43.495 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 0, 2024-09-10T20:29:43.495 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": 2, 2024-09-10T20:29:43.495 INFO:teuthology.orchestra.run.smithi131.stdout: "namespace": "" 2024-09-10T20:29:43.495 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:43.495 INFO:teuthology.orchestra.run.smithi131.stdout: "need": "23'1", 2024-09-10T20:29:43.495 INFO:teuthology.orchestra.run.smithi131.stdout: "have": "0'0", 2024-09-10T20:29:43.495 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": "none", 2024-09-10T20:29:43.495 INFO:teuthology.orchestra.run.smithi131.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T20:29:43.496 INFO:teuthology.orchestra.run.smithi131.stdout: "locations": [] 2024-09-10T20:29:43.496 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-10T20:29:43.496 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-09-10T20:29:43.496 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "Active", 2024-09-10T20:29:43.496 INFO:teuthology.orchestra.run.smithi131.stdout: "available_might_have_unfound": true, 2024-09-10T20:29:43.496 INFO:teuthology.orchestra.run.smithi131.stdout: "might_have_unfound": [ 2024-09-10T20:29:43.496 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-10T20:29:43.496 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "1", 2024-09-10T20:29:43.496 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "osd is down" 2024-09-10T20:29:43.496 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-10T20:29:43.496 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-09-10T20:29:43.496 INFO:teuthology.orchestra.run.smithi131.stdout: "more": false 2024-09-10T20:29:43.496 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-10T20:29:43.510 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.7 on osd.0 2024-09-10T20:29:43.511 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.7 mark_unfound_lost delete 2024-09-10T20:29:43.749 INFO:teuthology.orchestra.run.smithi131.stderr:pg has 1 objects unfound and apparently lost marking 2024-09-10T20:29:43.770 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.4 2024-09-10T20:29:43.770 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.6 state active+recovery_wait+degraded 2024-09-10T20:29:43.770 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.6 list_unfound '{}' 2024-09-10T20:29:43.971 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-10T20:29:43.971 INFO:teuthology.orchestra.run.smithi131.stdout: "offset": { 2024-09-10T20:29:43.971 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": "", 2024-09-10T20:29:43.971 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "", 2024-09-10T20:29:43.971 INFO:teuthology.orchestra.run.smithi131.stdout: "snapid": 0, 2024-09-10T20:29:43.971 INFO:teuthology.orchestra.run.smithi131.stdout: "hash": 0, 2024-09-10T20:29:43.971 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 0, 2024-09-10T20:29:43.971 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": -9223372036854775808, 2024-09-10T20:29:43.971 INFO:teuthology.orchestra.run.smithi131.stdout: "namespace": "" 2024-09-10T20:29:43.971 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:43.971 INFO:teuthology.orchestra.run.smithi131.stdout: "num_missing": 3, 2024-09-10T20:29:43.971 INFO:teuthology.orchestra.run.smithi131.stdout: "num_unfound": 3, 2024-09-10T20:29:43.972 INFO:teuthology.orchestra.run.smithi131.stdout: "objects": [ 2024-09-10T20:29:43.972 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-10T20:29:43.972 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": { 2024-09-10T20:29:43.972 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": "existed_8", 2024-09-10T20:29:43.972 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "", 2024-09-10T20:29:43.972 INFO:teuthology.orchestra.run.smithi131.stdout: "snapid": -2, 2024-09-10T20:29:43.972 INFO:teuthology.orchestra.run.smithi131.stdout: "hash": 1861424966, 2024-09-10T20:29:43.972 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 0, 2024-09-10T20:29:43.972 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": 2, 2024-09-10T20:29:43.972 INFO:teuthology.orchestra.run.smithi131.stdout: "namespace": "" 2024-09-10T20:29:43.972 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:43.972 INFO:teuthology.orchestra.run.smithi131.stdout: "need": "23'5", 2024-09-10T20:29:43.972 INFO:teuthology.orchestra.run.smithi131.stdout: "have": "0'0", 2024-09-10T20:29:43.973 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": "none", 2024-09-10T20:29:43.973 INFO:teuthology.orchestra.run.smithi131.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T20:29:43.973 INFO:teuthology.orchestra.run.smithi131.stdout: "locations": [] 2024-09-10T20:29:43.973 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:43.973 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-10T20:29:43.973 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": { 2024-09-10T20:29:43.973 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": "new_6", 2024-09-10T20:29:43.973 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "", 2024-09-10T20:29:43.973 INFO:teuthology.orchestra.run.smithi131.stdout: "snapid": -2, 2024-09-10T20:29:43.973 INFO:teuthology.orchestra.run.smithi131.stdout: "hash": 3910131350, 2024-09-10T20:29:43.973 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 0, 2024-09-10T20:29:43.973 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": 2, 2024-09-10T20:29:43.973 INFO:teuthology.orchestra.run.smithi131.stdout: "namespace": "" 2024-09-10T20:29:43.973 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:43.974 INFO:teuthology.orchestra.run.smithi131.stdout: "need": "23'4", 2024-09-10T20:29:43.974 INFO:teuthology.orchestra.run.smithi131.stdout: "have": "0'0", 2024-09-10T20:29:43.974 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": "none", 2024-09-10T20:29:43.974 INFO:teuthology.orchestra.run.smithi131.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T20:29:43.974 INFO:teuthology.orchestra.run.smithi131.stdout: "locations": [] 2024-09-10T20:29:43.974 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:43.974 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-10T20:29:43.974 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": { 2024-09-10T20:29:43.974 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": "new_4", 2024-09-10T20:29:43.974 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "", 2024-09-10T20:29:43.974 INFO:teuthology.orchestra.run.smithi131.stdout: "snapid": -2, 2024-09-10T20:29:43.974 INFO:teuthology.orchestra.run.smithi131.stdout: "hash": 2787839990, 2024-09-10T20:29:43.974 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 0, 2024-09-10T20:29:43.974 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": 2, 2024-09-10T20:29:43.975 INFO:teuthology.orchestra.run.smithi131.stdout: "namespace": "" 2024-09-10T20:29:43.975 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:43.975 INFO:teuthology.orchestra.run.smithi131.stdout: "need": "23'3", 2024-09-10T20:29:43.975 INFO:teuthology.orchestra.run.smithi131.stdout: "have": "0'0", 2024-09-10T20:29:43.975 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": "none", 2024-09-10T20:29:43.975 INFO:teuthology.orchestra.run.smithi131.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T20:29:43.975 INFO:teuthology.orchestra.run.smithi131.stdout: "locations": [] 2024-09-10T20:29:43.975 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-10T20:29:43.975 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-09-10T20:29:43.975 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "Active", 2024-09-10T20:29:43.975 INFO:teuthology.orchestra.run.smithi131.stdout: "available_might_have_unfound": true, 2024-09-10T20:29:43.976 INFO:teuthology.orchestra.run.smithi131.stdout: "might_have_unfound": [ 2024-09-10T20:29:43.976 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-10T20:29:43.976 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "1", 2024-09-10T20:29:43.976 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "osd is down" 2024-09-10T20:29:43.976 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-10T20:29:43.976 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-09-10T20:29:43.976 INFO:teuthology.orchestra.run.smithi131.stdout: "more": false 2024-09-10T20:29:43.976 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-10T20:29:43.993 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.6 on osd.0 2024-09-10T20:29:43.993 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.6 mark_unfound_lost delete 2024-09-10T20:29:44.032 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 16 20 34438 34418 8.40169 5.98047 0.00883397 0.00929104 2024-09-10T20:29:44.228 INFO:teuthology.orchestra.run.smithi131.stderr:pg has 3 objects unfound and apparently lost marking 2024-09-10T20:29:44.250 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.5 2024-09-10T20:29:44.250 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.5 state active+recovery_wait+degraded 2024-09-10T20:29:44.251 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.5 list_unfound '{}' 2024-09-10T20:29:44.499 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-10T20:29:44.499 INFO:teuthology.orchestra.run.smithi131.stdout: "offset": { 2024-09-10T20:29:44.499 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": "", 2024-09-10T20:29:44.499 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "", 2024-09-10T20:29:44.500 INFO:teuthology.orchestra.run.smithi131.stdout: "snapid": 0, 2024-09-10T20:29:44.500 INFO:teuthology.orchestra.run.smithi131.stdout: "hash": 0, 2024-09-10T20:29:44.500 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 0, 2024-09-10T20:29:44.500 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": -9223372036854775808, 2024-09-10T20:29:44.500 INFO:teuthology.orchestra.run.smithi131.stdout: "namespace": "" 2024-09-10T20:29:44.500 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:44.500 INFO:teuthology.orchestra.run.smithi131.stdout: "num_missing": 2, 2024-09-10T20:29:44.500 INFO:teuthology.orchestra.run.smithi131.stdout: "num_unfound": 2, 2024-09-10T20:29:44.500 INFO:teuthology.orchestra.run.smithi131.stdout: "objects": [ 2024-09-10T20:29:44.500 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-10T20:29:44.500 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": { 2024-09-10T20:29:44.500 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": "new_1", 2024-09-10T20:29:44.500 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "", 2024-09-10T20:29:44.501 INFO:teuthology.orchestra.run.smithi131.stdout: "snapid": -2, 2024-09-10T20:29:44.501 INFO:teuthology.orchestra.run.smithi131.stdout: "hash": 26281365, 2024-09-10T20:29:44.501 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 0, 2024-09-10T20:29:44.501 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": 2, 2024-09-10T20:29:44.501 INFO:teuthology.orchestra.run.smithi131.stdout: "namespace": "" 2024-09-10T20:29:44.501 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:44.501 INFO:teuthology.orchestra.run.smithi131.stdout: "need": "23'1", 2024-09-10T20:29:44.501 INFO:teuthology.orchestra.run.smithi131.stdout: "have": "0'0", 2024-09-10T20:29:44.501 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": "none", 2024-09-10T20:29:44.501 INFO:teuthology.orchestra.run.smithi131.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T20:29:44.501 INFO:teuthology.orchestra.run.smithi131.stdout: "locations": [] 2024-09-10T20:29:44.501 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:44.501 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-10T20:29:44.501 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": { 2024-09-10T20:29:44.501 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": "new_2", 2024-09-10T20:29:44.502 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "", 2024-09-10T20:29:44.502 INFO:teuthology.orchestra.run.smithi131.stdout: "snapid": -2, 2024-09-10T20:29:44.502 INFO:teuthology.orchestra.run.smithi131.stdout: "hash": 3196340981, 2024-09-10T20:29:44.502 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 0, 2024-09-10T20:29:44.502 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": 2, 2024-09-10T20:29:44.502 INFO:teuthology.orchestra.run.smithi131.stdout: "namespace": "" 2024-09-10T20:29:44.502 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:44.502 INFO:teuthology.orchestra.run.smithi131.stdout: "need": "23'2", 2024-09-10T20:29:44.502 INFO:teuthology.orchestra.run.smithi131.stdout: "have": "0'0", 2024-09-10T20:29:44.502 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": "none", 2024-09-10T20:29:44.502 INFO:teuthology.orchestra.run.smithi131.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T20:29:44.502 INFO:teuthology.orchestra.run.smithi131.stdout: "locations": [] 2024-09-10T20:29:44.502 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-10T20:29:44.502 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-09-10T20:29:44.503 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "Active", 2024-09-10T20:29:44.503 INFO:teuthology.orchestra.run.smithi131.stdout: "available_might_have_unfound": true, 2024-09-10T20:29:44.503 INFO:teuthology.orchestra.run.smithi131.stdout: "might_have_unfound": [ 2024-09-10T20:29:44.503 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-10T20:29:44.503 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "1", 2024-09-10T20:29:44.503 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "osd is down" 2024-09-10T20:29:44.503 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-10T20:29:44.503 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-09-10T20:29:44.503 INFO:teuthology.orchestra.run.smithi131.stdout: "more": false 2024-09-10T20:29:44.503 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-10T20:29:44.518 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.5 on osd.0 2024-09-10T20:29:44.518 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.5 mark_unfound_lost delete 2024-09-10T20:29:44.671 INFO:teuthology.orchestra.run.smithi131.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-10T20:29:44.682 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.6 2024-09-10T20:29:44.682 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.7 2024-09-10T20:29:44.682 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.4 state active+recovery_wait+degraded 2024-09-10T20:29:44.683 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.4 list_unfound '{}' 2024-09-10T20:29:44.852 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-10T20:29:44.852 INFO:teuthology.orchestra.run.smithi131.stdout: "offset": { 2024-09-10T20:29:44.852 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": "", 2024-09-10T20:29:44.852 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "", 2024-09-10T20:29:44.852 INFO:teuthology.orchestra.run.smithi131.stdout: "snapid": 0, 2024-09-10T20:29:44.852 INFO:teuthology.orchestra.run.smithi131.stdout: "hash": 0, 2024-09-10T20:29:44.852 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 0, 2024-09-10T20:29:44.852 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": -9223372036854775808, 2024-09-10T20:29:44.852 INFO:teuthology.orchestra.run.smithi131.stdout: "namespace": "" 2024-09-10T20:29:44.852 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:44.852 INFO:teuthology.orchestra.run.smithi131.stdout: "num_missing": 4, 2024-09-10T20:29:44.853 INFO:teuthology.orchestra.run.smithi131.stdout: "num_unfound": 3, 2024-09-10T20:29:44.853 INFO:teuthology.orchestra.run.smithi131.stdout: "objects": [ 2024-09-10T20:29:44.853 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-10T20:29:44.853 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": { 2024-09-10T20:29:44.853 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": "existed_4", 2024-09-10T20:29:44.853 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "", 2024-09-10T20:29:44.853 INFO:teuthology.orchestra.run.smithi131.stdout: "snapid": -2, 2024-09-10T20:29:44.853 INFO:teuthology.orchestra.run.smithi131.stdout: "hash": 873891716, 2024-09-10T20:29:44.853 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 0, 2024-09-10T20:29:44.853 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": 2, 2024-09-10T20:29:44.853 INFO:teuthology.orchestra.run.smithi131.stdout: "namespace": "" 2024-09-10T20:29:44.854 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:44.854 INFO:teuthology.orchestra.run.smithi131.stdout: "need": "23'8", 2024-09-10T20:29:44.854 INFO:teuthology.orchestra.run.smithi131.stdout: "have": "0'0", 2024-09-10T20:29:44.854 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": "none", 2024-09-10T20:29:44.854 INFO:teuthology.orchestra.run.smithi131.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T20:29:44.854 INFO:teuthology.orchestra.run.smithi131.stdout: "locations": [] 2024-09-10T20:29:44.854 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:44.854 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-10T20:29:44.854 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": { 2024-09-10T20:29:44.854 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": "existing_7", 2024-09-10T20:29:44.854 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "", 2024-09-10T20:29:44.854 INFO:teuthology.orchestra.run.smithi131.stdout: "snapid": -2, 2024-09-10T20:29:44.855 INFO:teuthology.orchestra.run.smithi131.stdout: "hash": 3996155460, 2024-09-10T20:29:44.855 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 0, 2024-09-10T20:29:44.855 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": 2, 2024-09-10T20:29:44.855 INFO:teuthology.orchestra.run.smithi131.stdout: "namespace": "" 2024-09-10T20:29:44.855 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:44.855 INFO:teuthology.orchestra.run.smithi131.stdout: "need": "23'9", 2024-09-10T20:29:44.855 INFO:teuthology.orchestra.run.smithi131.stdout: "have": "21'6", 2024-09-10T20:29:44.855 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": "none", 2024-09-10T20:29:44.855 INFO:teuthology.orchestra.run.smithi131.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-10T20:29:44.855 INFO:teuthology.orchestra.run.smithi131.stdout: "locations": [] 2024-09-10T20:29:44.855 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:44.855 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-10T20:29:44.855 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": { 2024-09-10T20:29:44.855 INFO:teuthology.orchestra.run.smithi131.stdout: "oid": "existed_2", 2024-09-10T20:29:44.856 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "", 2024-09-10T20:29:44.856 INFO:teuthology.orchestra.run.smithi131.stdout: "snapid": -2, 2024-09-10T20:29:44.856 INFO:teuthology.orchestra.run.smithi131.stdout: "hash": 1767667508, 2024-09-10T20:29:44.856 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 0, 2024-09-10T20:29:44.856 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": 2, 2024-09-10T20:29:44.856 INFO:teuthology.orchestra.run.smithi131.stdout: "namespace": "" 2024-09-10T20:29:44.856 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-10T20:29:44.856 INFO:teuthology.orchestra.run.smithi131.stdout: "need": "23'7", 2024-09-10T20:29:44.856 INFO:teuthology.orchestra.run.smithi131.stdout: "have": "0'0", 2024-09-10T20:29:44.856 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": "none", 2024-09-10T20:29:44.856 INFO:teuthology.orchestra.run.smithi131.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T20:29:44.856 INFO:teuthology.orchestra.run.smithi131.stdout: "locations": [] 2024-09-10T20:29:44.856 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-10T20:29:44.856 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-09-10T20:29:44.857 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "Active", 2024-09-10T20:29:44.857 INFO:teuthology.orchestra.run.smithi131.stdout: "available_might_have_unfound": true, 2024-09-10T20:29:44.857 INFO:teuthology.orchestra.run.smithi131.stdout: "might_have_unfound": [ 2024-09-10T20:29:44.857 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-10T20:29:44.857 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "1", 2024-09-10T20:29:44.857 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "osd is down" 2024-09-10T20:29:44.857 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-10T20:29:44.857 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-09-10T20:29:44.857 INFO:teuthology.orchestra.run.smithi131.stdout: "more": false 2024-09-10T20:29:44.857 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-10T20:29:44.871 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.4 on osd.0 2024-09-10T20:29:44.871 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.4 mark_unfound_lost delete 2024-09-10T20:29:45.032 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 17 20 35629 35609 8.18113 4.65234 0.00882972 0.00954351 2024-09-10T20:29:45.095 INFO:teuthology.orchestra.run.smithi131.stderr:pg has 3 objects unfound and apparently lost marking 2024-09-10T20:29:45.118 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 debug kick_recovery_wq 5 2024-09-10T20:29:45.147 INFO:tasks.ceph.mon.a.smithi131.stderr:2024-09-10T20:29:45.143+0000 7f1e65d04640 -1 log_channel(cluster) log [ERR] : Health check update: Possible data damage: 1 pg recovery_unfound (PG_DAMAGED) 2024-09-10T20:29:45.338 INFO:teuthology.orchestra.run.smithi131.stderr:kicking recovery queue. set osd_recovery_delay_start to 5 2024-09-10T20:29:45.361 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 debug kick_recovery_wq 5 2024-09-10T20:29:45.614 INFO:teuthology.orchestra.run.smithi131.stderr:kicking recovery queue. set osd_recovery_delay_start to 5 2024-09-10T20:29:45.636 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-10T20:29:45.636 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-10T20:29:45.873 INFO:teuthology.orchestra.run.smithi131.stdout:107374182414 2024-09-10T20:29:45.873 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T20:29:45.879 INFO:teuthology.orchestra.run.smithi131.stdout:133143986186 2024-09-10T20:29:45.880 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T20:29:46.032 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 18 20 37122 37102 8.05062 5.83203 0.0154295 0.00969744 2024-09-10T20:29:46.412 INFO:teuthology.orchestra.run.smithi131.stdout:107374182413 2024-09-10T20:29:46.435 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 107374182414 got 107374182413 for osd.0 2024-09-10T20:29:46.478 INFO:teuthology.orchestra.run.smithi131.stdout:133143986185 2024-09-10T20:29:46.501 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 133143986186 got 133143986185 for osd.2 2024-09-10T20:29:47.032 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 19 20 38672 38652 7.94542 6.05469 0.0109107 0.00982573 2024-09-10T20:29:47.436 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T20:29:47.502 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T20:29:48.014 INFO:teuthology.orchestra.run.smithi131.stdout:107374182415 2024-09-10T20:29:48.032 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout:2024-09-10T20:29:48.029908+0000 min lat: 0.00191414 max lat: 0.233393 avg lat: 0.00995851 2024-09-10T20:29:48.032 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T20:29:48.032 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 20 20 40168 40148 7.84032 5.84375 0.0145923 0.00995851 2024-09-10T20:29:48.036 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 107374182414 got 107374182415 for osd.0 2024-09-10T20:29:48.036 DEBUG:teuthology.parallel:result is None 2024-09-10T20:29:48.097 INFO:teuthology.orchestra.run.smithi131.stdout:133143986186 2024-09-10T20:29:48.118 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 133143986186 got 133143986186 for osd.2 2024-09-10T20:29:48.118 DEBUG:teuthology.parallel:result is None 2024-09-10T20:29:48.118 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for recovery to complete 2024-09-10T20:29:48.118 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:29:48.668 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:29:48.668 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:29:48.689 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2024-09-10T20:29:46.828112+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":151450604,"num_objects":36994,"num_object_clones":0,"num_object_copies":73988,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26960,"num_write_kb":107720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37058,"ondisk_log_size":37058,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":503496,"kb_used_data":291624,"kb_used_omap":8,"kb_used_meta":211831,"kb_avail":188240184,"statfs":{"total":193273528320,"available":192757948416,"internally_reserved":0,"allocated":298622976,"data_stored":298206983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8876,"internal_metadata":216915284},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,5,3,9,10,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":106360581,"num_objects":25959,"num_object_clones":0,"num_object_copies":51918,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-27,"num_objects_dirty":25959,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15898,"num_write_kb":63652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003119"},"pg_stats":[{"pgid":"2.f","version":"36'2285","reported_seq":2336,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:45.934488+0000","last_change":"2024-09-10T20:29:19.019964+0000","last_active":"2024-09-10T20:29:45.934488+0000","last_peered":"2024-09-10T20:29:45.934488+0000","last_clean":"2024-09-10T20:29:45.934488+0000","last_became_active":"2024-09-10T20:29:19.019691+0000","last_became_peered":"2024-09-10T20:29:19.019691+0000","last_unstale":"2024-09-10T20:29:45.934488+0000","last_undegraded":"2024-09-10T20:29:45.934488+0000","last_fullsized":"2024-09-10T20:29:45.934488+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2285,"log_dups_size":0,"ondisk_log_size":2285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:46.881470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9359360,"num_objects":2285,"num_object_clones":0,"num_object_copies":4570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2285,"num_write_kb":9140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'2411","reported_seq":2502,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:45.839171+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:29:45.839171+0000","last_peered":"2024-09-10T20:29:45.839171+0000","last_clean":"2024-09-10T20:29:45.839171+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:29:45.839171+0000","last_undegraded":"2024-09-10T20:29:45.839171+0000","last_fullsized":"2024-09-10T20:29:45.839171+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":2411,"log_dups_size":0,"ondisk_log_size":2411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9850880,"num_objects":2405,"num_object_clones":0,"num_object_copies":4810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":597,"num_write_kb":2388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'2302","reported_seq":2405,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:45.817239+0000","last_change":"2024-09-10T20:29:41.318456+0000","last_active":"2024-09-10T20:29:45.817239+0000","last_peered":"2024-09-10T20:29:45.817239+0000","last_clean":"2024-09-10T20:29:45.817239+0000","last_became_active":"2024-09-10T20:29:20.020548+0000","last_became_peered":"2024-09-10T20:29:20.020548+0000","last_unstale":"2024-09-10T20:29:45.817239+0000","last_undegraded":"2024-09-10T20:29:45.817239+0000","last_fullsized":"2024-09-10T20:29:45.817239+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":2302,"log_dups_size":0,"ondisk_log_size":2302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:41:34.883083+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9412608,"num_objects":2298,"num_object_clones":0,"num_object_copies":4596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":430,"num_write_kb":1720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'2309","reported_seq":2369,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:45.848915+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:29:45.848915+0000","last_peered":"2024-09-10T20:29:45.848915+0000","last_clean":"2024-09-10T20:29:45.848915+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:29:45.848915+0000","last_undegraded":"2024-09-10T20:29:45.848915+0000","last_fullsized":"2024-09-10T20:29:45.848915+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2309,"log_dups_size":0,"ondisk_log_size":2309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9457664,"num_objects":2309,"num_object_clones":0,"num_object_copies":4618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2309,"num_write_kb":9236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'2322","reported_seq":2379,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:45.930007+0000","last_change":"2024-09-10T20:29:19.022136+0000","last_active":"2024-09-10T20:29:45.930007+0000","last_peered":"2024-09-10T20:29:45.930007+0000","last_clean":"2024-09-10T20:28:45.772428+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:29:45.930007+0000","last_undegraded":"2024-09-10T20:29:19.016520+0000","last_fullsized":"2024-09-10T20:29:45.930007+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2322,"log_dups_size":0,"ondisk_log_size":2322,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9483206,"num_objects":2317,"num_object_clones":0,"num_object_copies":4634,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2317,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2320,"num_write_kb":9264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2317}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'2311","reported_seq":2369,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:45.960730+0000","last_change":"2024-09-10T20:29:19.022275+0000","last_active":"2024-09-10T20:29:45.960730+0000","last_peered":"2024-09-10T20:29:45.960730+0000","last_clean":"2024-09-10T20:28:46.621675+0000","last_became_active":"2024-09-10T20:29:19.022041+0000","last_became_peered":"2024-09-10T20:29:19.022041+0000","last_unstale":"2024-09-10T20:29:45.960730+0000","last_undegraded":"2024-09-10T20:29:19.014144+0000","last_fullsized":"2024-09-10T20:29:45.960730+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2311,"log_dups_size":0,"ondisk_log_size":2311,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9446342,"num_objects":2308,"num_object_clones":0,"num_object_copies":4616,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2309,"num_write_kb":9227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2308}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'2269","reported_seq":2326,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:45.957863+0000","last_change":"2024-09-10T20:29:19.024530+0000","last_active":"2024-09-10T20:29:45.957863+0000","last_peered":"2024-09-10T20:29:45.957863+0000","last_clean":"2024-09-10T20:28:46.021581+0000","last_became_active":"2024-09-10T20:29:19.024227+0000","last_became_peered":"2024-09-10T20:29:19.024227+0000","last_unstale":"2024-09-10T20:29:45.957863+0000","last_undegraded":"2024-09-10T20:29:19.016822+0000","last_fullsized":"2024-09-10T20:29:45.957863+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2269,"log_dups_size":0,"ondisk_log_size":2269,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9270214,"num_objects":2265,"num_object_clones":0,"num_object_copies":4530,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2267,"num_write_kb":9056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2265}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'2338","reported_seq":2398,"reported_epoch":36,"state":"active+recovering","last_fresh":"2024-09-10T20:29:45.959603+0000","last_change":"2024-09-10T20:29:20.016123+0000","last_active":"2024-09-10T20:29:45.959603+0000","last_peered":"2024-09-10T20:29:45.959603+0000","last_clean":"2024-09-10T20:28:40.535680+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:29:45.959603+0000","last_undegraded":"2024-09-10T20:29:45.959603+0000","last_fullsized":"2024-09-10T20:29:45.959603+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2338,"log_dups_size":0,"ondisk_log_size":2338,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9568739,"num_objects":2337,"num_object_clones":0,"num_object_copies":4674,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2337,"num_write_kb":9345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2337}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.016038+0000","last_change":"2024-09-10T20:29:19.019807+0000","last_active":"2024-09-10T20:29:20.016038+0000","last_peered":"2024-09-10T20:29:20.016038+0000","last_clean":"2024-09-10T20:29:20.016038+0000","last_became_active":"2024-09-10T20:29:19.019500+0000","last_became_peered":"2024-09-10T20:29:19.019500+0000","last_unstale":"2024-09-10T20:29:20.016038+0000","last_undegraded":"2024-09-10T20:29:20.016038+0000","last_fullsized":"2024-09-10T20:29:20.016038+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:17:47.161423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'2255","reported_seq":2362,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:45.831397+0000","last_change":"2024-09-10T20:29:43.227345+0000","last_active":"2024-09-10T20:29:45.831397+0000","last_peered":"2024-09-10T20:29:45.831397+0000","last_clean":"2024-09-10T20:29:45.831397+0000","last_became_active":"2024-09-10T20:29:20.020530+0000","last_became_peered":"2024-09-10T20:29:20.020530+0000","last_unstale":"2024-09-10T20:29:45.831397+0000","last_undegraded":"2024-09-10T20:29:45.831397+0000","last_fullsized":"2024-09-10T20:29:45.831397+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":2255,"log_dups_size":0,"ondisk_log_size":2255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:18.980351+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9207808,"num_objects":2248,"num_object_clones":0,"num_object_copies":4496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":248,"num_write_kb":992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.019440+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:29:20.019440+0000","last_peered":"2024-09-10T20:29:20.019440+0000","last_clean":"2024-09-10T20:29:20.019440+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:29:20.019440+0000","last_undegraded":"2024-09-10T20:29:20.019440+0000","last_fullsized":"2024-09-10T20:29:20.019440+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'2356","reported_seq":2405,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:45.973679+0000","last_change":"2024-09-10T20:29:19.024601+0000","last_active":"2024-09-10T20:29:45.973679+0000","last_peered":"2024-09-10T20:29:45.973679+0000","last_clean":"2024-09-10T20:28:46.672065+0000","last_became_active":"2024-09-10T20:29:19.024360+0000","last_became_peered":"2024-09-10T20:29:19.024360+0000","last_unstale":"2024-09-10T20:29:45.973679+0000","last_undegraded":"2024-09-10T20:29:19.014503+0000","last_fullsized":"2024-09-10T20:29:45.973679+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2356,"log_dups_size":0,"ondisk_log_size":2356,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9618374,"num_objects":2350,"num_object_clones":0,"num_object_copies":4700,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2354,"num_write_kb":9396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2350}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":64,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:20.015786+0000","last_change":"2024-09-10T20:29:19.021713+0000","last_active":"2024-09-10T20:29:20.015786+0000","last_peered":"2024-09-10T20:29:20.015786+0000","last_clean":"2024-09-10T20:28:57.955277+0000","last_became_active":"2024-09-10T20:29:19.021487+0000","last_became_peered":"2024-09-10T20:29:19.021487+0000","last_unstale":"2024-09-10T20:29:20.015786+0000","last_undegraded":"2024-09-10T20:29:19.013787+0000","last_fullsized":"2024-09-10T20:29:20.015786+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'2317","reported_seq":2419,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:45.834727+0000","last_change":"2024-09-10T20:29:44.304315+0000","last_active":"2024-09-10T20:29:45.834727+0000","last_peered":"2024-09-10T20:29:45.834727+0000","last_clean":"2024-09-10T20:29:45.834727+0000","last_became_active":"2024-09-10T20:29:20.018890+0000","last_became_peered":"2024-09-10T20:29:20.018890+0000","last_unstale":"2024-09-10T20:29:45.834727+0000","last_undegraded":"2024-09-10T20:29:45.834727+0000","last_fullsized":"2024-09-10T20:29:45.834727+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":2317,"log_dups_size":0,"ondisk_log_size":2317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:57:54.859061+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9469952,"num_objects":2312,"num_object_clones":0,"num_object_copies":4624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":145,"num_write_kb":580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":92,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.022839+0000","last_change":"2024-09-10T20:29:20.021021+0000","last_active":"2024-09-10T20:29:20.022839+0000","last_peered":"2024-09-10T20:29:20.022839+0000","last_clean":"2024-09-10T20:29:20.022839+0000","last_became_active":"2024-09-10T20:29:20.020575+0000","last_became_peered":"2024-09-10T20:29:20.020575+0000","last_unstale":"2024-09-10T20:29:20.022839+0000","last_undegraded":"2024-09-10T20:29:20.022839+0000","last_fullsized":"2024-09-10T20:29:20.022839+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:23:19.560105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'2317","reported_seq":2419,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:45.835197+0000","last_change":"2024-09-10T20:29:45.018166+0000","last_active":"2024-09-10T20:29:45.835197+0000","last_peered":"2024-09-10T20:29:45.835197+0000","last_clean":"2024-09-10T20:29:45.835197+0000","last_became_active":"2024-09-10T20:29:20.018640+0000","last_became_peered":"2024-09-10T20:29:20.018640+0000","last_unstale":"2024-09-10T20:29:45.835197+0000","last_undegraded":"2024-09-10T20:29:45.835197+0000","last_fullsized":"2024-09-10T20:29:45.835197+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":2317,"log_dups_size":0,"ondisk_log_size":2317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:08:13.349234+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9465856,"num_objects":2311,"num_object_clones":0,"num_object_copies":4622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103,"num_write_kb":412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'2363","reported_seq":2421,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:45.913949+0000","last_change":"2024-09-10T20:29:19.022418+0000","last_active":"2024-09-10T20:29:45.913949+0000","last_peered":"2024-09-10T20:29:45.913949+0000","last_clean":"2024-09-10T20:28:42.552296+0000","last_became_active":"2024-09-10T20:29:19.022182+0000","last_became_peered":"2024-09-10T20:29:19.022182+0000","last_unstale":"2024-09-10T20:29:45.913949+0000","last_undegraded":"2024-09-10T20:29:19.016061+0000","last_fullsized":"2024-09-10T20:29:45.913949+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2363,"log_dups_size":0,"ondisk_log_size":2363,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9671139,"num_objects":2362,"num_object_clones":0,"num_object_copies":4724,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2362,"num_write_kb":9445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2362}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.015089+0000","last_change":"2024-09-10T20:29:19.022445+0000","last_active":"2024-09-10T20:29:20.015089+0000","last_peered":"2024-09-10T20:29:20.015089+0000","last_clean":"2024-09-10T20:29:20.015089+0000","last_became_active":"2024-09-10T20:29:19.022165+0000","last_became_peered":"2024-09-10T20:29:19.022165+0000","last_unstale":"2024-09-10T20:29:20.015089+0000","last_undegraded":"2024-09-10T20:29:20.015089+0000","last_fullsized":"2024-09-10T20:29:20.015089+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:57:59.626770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'2254","reported_seq":2310,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:45.957679+0000","last_change":"2024-09-10T20:29:19.024585+0000","last_active":"2024-09-10T20:29:45.957679+0000","last_peered":"2024-09-10T20:29:45.957679+0000","last_clean":"2024-09-10T20:28:46.571322+0000","last_became_active":"2024-09-10T20:29:19.024283+0000","last_became_peered":"2024-09-10T20:29:19.024283+0000","last_unstale":"2024-09-10T20:29:45.957679+0000","last_undegraded":"2024-09-10T20:29:19.017318+0000","last_fullsized":"2024-09-10T20:29:45.957679+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2254,"log_dups_size":0,"ondisk_log_size":2254,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9201065,"num_objects":2249,"num_object_clones":0,"num_object_copies":4498,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2251,"num_write_kb":8988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2249}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.020926+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:29:20.020926+0000","last_peered":"2024-09-10T20:29:20.020926+0000","last_clean":"2024-09-10T20:29:20.020926+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:29:20.020926+0000","last_undegraded":"2024-09-10T20:29:20.020926+0000","last_fullsized":"2024-09-10T20:29:20.020926+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'2375","reported_seq":2432,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:45.950713+0000","last_change":"2024-09-10T20:29:19.022172+0000","last_active":"2024-09-10T20:29:45.950713+0000","last_peered":"2024-09-10T20:29:45.950713+0000","last_clean":"2024-09-10T20:28:42.552437+0000","last_became_active":"2024-09-10T20:29:19.021920+0000","last_became_peered":"2024-09-10T20:29:19.021920+0000","last_unstale":"2024-09-10T20:29:45.950713+0000","last_undegraded":"2024-09-10T20:29:19.015669+0000","last_fullsized":"2024-09-10T20:29:45.950713+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2375,"log_dups_size":0,"ondisk_log_size":2375,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9712582,"num_objects":2373,"num_object_clones":0,"num_object_copies":4746,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2373,"num_write_kb":9486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2373}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.016002+0000","last_change":"2024-09-10T20:29:19.022093+0000","last_active":"2024-09-10T20:29:20.016002+0000","last_peered":"2024-09-10T20:29:20.016002+0000","last_clean":"2024-09-10T20:29:20.016002+0000","last_became_active":"2024-09-10T20:29:19.021778+0000","last_became_peered":"2024-09-10T20:29:19.021778+0000","last_unstale":"2024-09-10T20:29:20.016002+0000","last_undegraded":"2024-09-10T20:29:20.016002+0000","last_fullsized":"2024-09-10T20:29:20.016002+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:25:12.240664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.019846+0000","last_change":"2024-09-10T20:29:20.018969+0000","last_active":"2024-09-10T20:29:20.019846+0000","last_peered":"2024-09-10T20:29:20.019846+0000","last_clean":"2024-09-10T20:29:20.019846+0000","last_became_active":"2024-09-10T20:29:20.018675+0000","last_became_peered":"2024-09-10T20:29:20.018675+0000","last_unstale":"2024-09-10T20:29:20.019846+0000","last_undegraded":"2024-09-10T20:29:20.019846+0000","last_fullsized":"2024-09-10T20:29:20.019846+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:50.839971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'2271","reported_seq":2328,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:45.973743+0000","last_change":"2024-09-10T20:29:19.024494+0000","last_active":"2024-09-10T20:29:45.973743+0000","last_peered":"2024-09-10T20:29:45.973743+0000","last_clean":"2024-09-10T20:28:46.321317+0000","last_became_active":"2024-09-10T20:29:19.024217+0000","last_became_peered":"2024-09-10T20:29:19.024217+0000","last_unstale":"2024-09-10T20:29:45.973743+0000","last_undegraded":"2024-09-10T20:29:19.016668+0000","last_fullsized":"2024-09-10T20:29:45.973743+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2271,"log_dups_size":0,"ondisk_log_size":2271,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9254796,"num_objects":2263,"num_object_clones":0,"num_object_copies":4526,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2268,"num_write_kb":9043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":2262}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":151450585,"num_objects":36992,"num_object_clones":0,"num_object_copies":73984,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36992,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26958,"num_write_kb":107718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":302919680,"data_stored":302796838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37055,"ondisk_log_size":37055,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182415,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":248836,"kb_used_data":145012,"kb_used_omap":4,"kb_used_meta":103803,"kb_avail":94123004,"statfs":{"total":96636764160,"available":96381956096,"internally_reserved":0,"allocated":148492288,"data_stored":148271414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":106294954},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,5,8],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986186,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":254660,"kb_used_data":146612,"kb_used_omap":4,"kb_used_meta":108027,"kb_avail":94117180,"statfs":{"total":96636764160,"available":96375992320,"internally_reserved":0,"allocated":150130688,"data_stored":149935569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":110620330},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,2,4,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":151777280,"data_stored":151755602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":151027712,"data_stored":151027712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:29:48.691 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:29:49.032 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 21 20 41667 41647 7.74579 5.85547 0.00877187 0.0100804 2024-09-10T20:29:49.210 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:29:49.211 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:29:49.238 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2024-09-10T20:29:48.828560+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":156222444,"num_objects":38159,"num_object_clones":0,"num_object_copies":76318,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28125,"num_write_kb":112380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38223,"ondisk_log_size":38223,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":519856,"kb_used_data":303824,"kb_used_omap":8,"kb_used_meta":215991,"kb_avail":188223824,"statfs":{"total":193273528320,"available":192741195776,"internally_reserved":0,"allocated":311115776,"data_stored":310699783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8876,"internal_metadata":221175124},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,5,13,10],"upper_bound":32},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":79785733,"num_objects":19471,"num_object_clones":0,"num_object_copies":38942,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-27,"num_objects_dirty":19471,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9410,"num_write_kb":37700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003039"},"pg_stats":[{"pgid":"2.f","version":"36'2285","reported_seq":2336,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:45.934488+0000","last_change":"2024-09-10T20:29:19.019964+0000","last_active":"2024-09-10T20:29:45.934488+0000","last_peered":"2024-09-10T20:29:45.934488+0000","last_clean":"2024-09-10T20:29:45.934488+0000","last_became_active":"2024-09-10T20:29:19.019691+0000","last_became_peered":"2024-09-10T20:29:19.019691+0000","last_unstale":"2024-09-10T20:29:45.934488+0000","last_undegraded":"2024-09-10T20:29:45.934488+0000","last_fullsized":"2024-09-10T20:29:45.934488+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2285,"log_dups_size":0,"ondisk_log_size":2285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:46.881470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9359360,"num_objects":2285,"num_object_clones":0,"num_object_copies":4570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2285,"num_write_kb":9140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'2612","reported_seq":2703,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:47.999691+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:29:47.999691+0000","last_peered":"2024-09-10T20:29:47.999691+0000","last_clean":"2024-09-10T20:29:47.999691+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:29:47.999691+0000","last_undegraded":"2024-09-10T20:29:47.999691+0000","last_fullsized":"2024-09-10T20:29:47.999691+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":2612,"log_dups_size":0,"ondisk_log_size":2612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10674176,"num_objects":2606,"num_object_clones":0,"num_object_copies":5212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":798,"num_write_kb":3192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'2499","reported_seq":2602,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:48.005538+0000","last_change":"2024-09-10T20:29:41.318456+0000","last_active":"2024-09-10T20:29:48.005538+0000","last_peered":"2024-09-10T20:29:48.005538+0000","last_clean":"2024-09-10T20:29:48.005538+0000","last_became_active":"2024-09-10T20:29:20.020548+0000","last_became_peered":"2024-09-10T20:29:20.020548+0000","last_unstale":"2024-09-10T20:29:48.005538+0000","last_undegraded":"2024-09-10T20:29:48.005538+0000","last_fullsized":"2024-09-10T20:29:48.005538+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":2499,"log_dups_size":0,"ondisk_log_size":2499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:41:34.883083+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10219520,"num_objects":2495,"num_object_clones":0,"num_object_copies":4990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":627,"num_write_kb":2508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'2498","reported_seq":2559,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:47.968331+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:29:47.968331+0000","last_peered":"2024-09-10T20:29:47.968331+0000","last_clean":"2024-09-10T20:29:47.968331+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:29:47.968331+0000","last_undegraded":"2024-09-10T20:29:47.968331+0000","last_fullsized":"2024-09-10T20:29:47.968331+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2498,"log_dups_size":0,"ondisk_log_size":2498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10231808,"num_objects":2498,"num_object_clones":0,"num_object_copies":4996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2498,"num_write_kb":9992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'2322","reported_seq":2379,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:45.930007+0000","last_change":"2024-09-10T20:29:19.022136+0000","last_active":"2024-09-10T20:29:45.930007+0000","last_peered":"2024-09-10T20:29:45.930007+0000","last_clean":"2024-09-10T20:28:45.772428+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:29:45.930007+0000","last_undegraded":"2024-09-10T20:29:19.016520+0000","last_fullsized":"2024-09-10T20:29:45.930007+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2322,"log_dups_size":0,"ondisk_log_size":2322,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9483206,"num_objects":2317,"num_object_clones":0,"num_object_copies":4634,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2317,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2320,"num_write_kb":9264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2317}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'2311","reported_seq":2369,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:45.960730+0000","last_change":"2024-09-10T20:29:19.022275+0000","last_active":"2024-09-10T20:29:45.960730+0000","last_peered":"2024-09-10T20:29:45.960730+0000","last_clean":"2024-09-10T20:28:46.621675+0000","last_became_active":"2024-09-10T20:29:19.022041+0000","last_became_peered":"2024-09-10T20:29:19.022041+0000","last_unstale":"2024-09-10T20:29:45.960730+0000","last_undegraded":"2024-09-10T20:29:19.014144+0000","last_fullsized":"2024-09-10T20:29:45.960730+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2311,"log_dups_size":0,"ondisk_log_size":2311,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9446342,"num_objects":2308,"num_object_clones":0,"num_object_copies":4616,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2309,"num_write_kb":9227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2308}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'2269","reported_seq":2326,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:45.957863+0000","last_change":"2024-09-10T20:29:19.024530+0000","last_active":"2024-09-10T20:29:45.957863+0000","last_peered":"2024-09-10T20:29:45.957863+0000","last_clean":"2024-09-10T20:28:46.021581+0000","last_became_active":"2024-09-10T20:29:19.024227+0000","last_became_peered":"2024-09-10T20:29:19.024227+0000","last_unstale":"2024-09-10T20:29:45.957863+0000","last_undegraded":"2024-09-10T20:29:19.016822+0000","last_fullsized":"2024-09-10T20:29:45.957863+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2269,"log_dups_size":0,"ondisk_log_size":2269,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9270214,"num_objects":2265,"num_object_clones":0,"num_object_copies":4530,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2267,"num_write_kb":9056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2265}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'2338","reported_seq":2398,"reported_epoch":36,"state":"active+recovering","last_fresh":"2024-09-10T20:29:45.959603+0000","last_change":"2024-09-10T20:29:20.016123+0000","last_active":"2024-09-10T20:29:45.959603+0000","last_peered":"2024-09-10T20:29:45.959603+0000","last_clean":"2024-09-10T20:28:40.535680+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:29:45.959603+0000","last_undegraded":"2024-09-10T20:29:45.959603+0000","last_fullsized":"2024-09-10T20:29:45.959603+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2338,"log_dups_size":0,"ondisk_log_size":2338,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9568739,"num_objects":2337,"num_object_clones":0,"num_object_copies":4674,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2337,"num_write_kb":9345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2337}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.016038+0000","last_change":"2024-09-10T20:29:19.019807+0000","last_active":"2024-09-10T20:29:20.016038+0000","last_peered":"2024-09-10T20:29:20.016038+0000","last_clean":"2024-09-10T20:29:20.016038+0000","last_became_active":"2024-09-10T20:29:19.019500+0000","last_became_peered":"2024-09-10T20:29:19.019500+0000","last_unstale":"2024-09-10T20:29:20.016038+0000","last_undegraded":"2024-09-10T20:29:20.016038+0000","last_fullsized":"2024-09-10T20:29:20.016038+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:17:47.161423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'2433","reported_seq":2540,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:48.009709+0000","last_change":"2024-09-10T20:29:43.227345+0000","last_active":"2024-09-10T20:29:48.009709+0000","last_peered":"2024-09-10T20:29:48.009709+0000","last_clean":"2024-09-10T20:29:48.009709+0000","last_became_active":"2024-09-10T20:29:20.020530+0000","last_became_peered":"2024-09-10T20:29:20.020530+0000","last_unstale":"2024-09-10T20:29:48.009709+0000","last_undegraded":"2024-09-10T20:29:48.009709+0000","last_fullsized":"2024-09-10T20:29:48.009709+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":2433,"log_dups_size":0,"ondisk_log_size":2433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:18.980351+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9936896,"num_objects":2426,"num_object_clones":0,"num_object_copies":4852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":426,"num_write_kb":1704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.019440+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:29:20.019440+0000","last_peered":"2024-09-10T20:29:20.019440+0000","last_clean":"2024-09-10T20:29:20.019440+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:29:20.019440+0000","last_undegraded":"2024-09-10T20:29:20.019440+0000","last_fullsized":"2024-09-10T20:29:20.019440+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'2356","reported_seq":2405,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:45.973679+0000","last_change":"2024-09-10T20:29:19.024601+0000","last_active":"2024-09-10T20:29:45.973679+0000","last_peered":"2024-09-10T20:29:45.973679+0000","last_clean":"2024-09-10T20:28:46.672065+0000","last_became_active":"2024-09-10T20:29:19.024360+0000","last_became_peered":"2024-09-10T20:29:19.024360+0000","last_unstale":"2024-09-10T20:29:45.973679+0000","last_undegraded":"2024-09-10T20:29:19.014503+0000","last_fullsized":"2024-09-10T20:29:45.973679+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2356,"log_dups_size":0,"ondisk_log_size":2356,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9618374,"num_objects":2350,"num_object_clones":0,"num_object_copies":4700,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2354,"num_write_kb":9396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2350}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":64,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:20.015786+0000","last_change":"2024-09-10T20:29:19.021713+0000","last_active":"2024-09-10T20:29:20.015786+0000","last_peered":"2024-09-10T20:29:20.015786+0000","last_clean":"2024-09-10T20:28:57.955277+0000","last_became_active":"2024-09-10T20:29:19.021487+0000","last_became_peered":"2024-09-10T20:29:19.021487+0000","last_unstale":"2024-09-10T20:29:20.015786+0000","last_undegraded":"2024-09-10T20:29:19.013787+0000","last_fullsized":"2024-09-10T20:29:20.015786+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'2521","reported_seq":2623,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:47.984557+0000","last_change":"2024-09-10T20:29:44.304315+0000","last_active":"2024-09-10T20:29:47.984557+0000","last_peered":"2024-09-10T20:29:47.984557+0000","last_clean":"2024-09-10T20:29:47.984557+0000","last_became_active":"2024-09-10T20:29:20.018890+0000","last_became_peered":"2024-09-10T20:29:20.018890+0000","last_unstale":"2024-09-10T20:29:47.984557+0000","last_undegraded":"2024-09-10T20:29:47.984557+0000","last_fullsized":"2024-09-10T20:29:47.984557+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":2521,"log_dups_size":0,"ondisk_log_size":2521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:57:54.859061+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10305536,"num_objects":2516,"num_object_clones":0,"num_object_copies":5032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":349,"num_write_kb":1396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":92,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.022839+0000","last_change":"2024-09-10T20:29:20.021021+0000","last_active":"2024-09-10T20:29:20.022839+0000","last_peered":"2024-09-10T20:29:20.022839+0000","last_clean":"2024-09-10T20:29:20.022839+0000","last_became_active":"2024-09-10T20:29:20.020575+0000","last_became_peered":"2024-09-10T20:29:20.020575+0000","last_unstale":"2024-09-10T20:29:20.022839+0000","last_undegraded":"2024-09-10T20:29:20.022839+0000","last_fullsized":"2024-09-10T20:29:20.022839+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:23:19.560105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'2513","reported_seq":2615,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:48.014314+0000","last_change":"2024-09-10T20:29:45.018166+0000","last_active":"2024-09-10T20:29:48.014314+0000","last_peered":"2024-09-10T20:29:48.014314+0000","last_clean":"2024-09-10T20:29:48.014314+0000","last_became_active":"2024-09-10T20:29:20.018640+0000","last_became_peered":"2024-09-10T20:29:20.018640+0000","last_unstale":"2024-09-10T20:29:48.014314+0000","last_undegraded":"2024-09-10T20:29:48.014314+0000","last_fullsized":"2024-09-10T20:29:48.014314+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":2513,"log_dups_size":0,"ondisk_log_size":2513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:08:13.349234+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10268672,"num_objects":2507,"num_object_clones":0,"num_object_copies":5014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":299,"num_write_kb":1196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'2363","reported_seq":2421,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:45.913949+0000","last_change":"2024-09-10T20:29:19.022418+0000","last_active":"2024-09-10T20:29:45.913949+0000","last_peered":"2024-09-10T20:29:45.913949+0000","last_clean":"2024-09-10T20:28:42.552296+0000","last_became_active":"2024-09-10T20:29:19.022182+0000","last_became_peered":"2024-09-10T20:29:19.022182+0000","last_unstale":"2024-09-10T20:29:45.913949+0000","last_undegraded":"2024-09-10T20:29:19.016061+0000","last_fullsized":"2024-09-10T20:29:45.913949+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2363,"log_dups_size":0,"ondisk_log_size":2363,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9671139,"num_objects":2362,"num_object_clones":0,"num_object_copies":4724,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2362,"num_write_kb":9445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2362}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.015089+0000","last_change":"2024-09-10T20:29:19.022445+0000","last_active":"2024-09-10T20:29:20.015089+0000","last_peered":"2024-09-10T20:29:20.015089+0000","last_clean":"2024-09-10T20:29:20.015089+0000","last_became_active":"2024-09-10T20:29:19.022165+0000","last_became_peered":"2024-09-10T20:29:19.022165+0000","last_unstale":"2024-09-10T20:29:20.015089+0000","last_undegraded":"2024-09-10T20:29:20.015089+0000","last_fullsized":"2024-09-10T20:29:20.015089+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:57:59.626770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'2254","reported_seq":2310,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:45.957679+0000","last_change":"2024-09-10T20:29:19.024585+0000","last_active":"2024-09-10T20:29:45.957679+0000","last_peered":"2024-09-10T20:29:45.957679+0000","last_clean":"2024-09-10T20:28:46.571322+0000","last_became_active":"2024-09-10T20:29:19.024283+0000","last_became_peered":"2024-09-10T20:29:19.024283+0000","last_unstale":"2024-09-10T20:29:45.957679+0000","last_undegraded":"2024-09-10T20:29:19.017318+0000","last_fullsized":"2024-09-10T20:29:45.957679+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2254,"log_dups_size":0,"ondisk_log_size":2254,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9201065,"num_objects":2249,"num_object_clones":0,"num_object_copies":4498,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2251,"num_write_kb":8988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2249}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.020926+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:29:20.020926+0000","last_peered":"2024-09-10T20:29:20.020926+0000","last_clean":"2024-09-10T20:29:20.020926+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:29:20.020926+0000","last_undegraded":"2024-09-10T20:29:20.020926+0000","last_fullsized":"2024-09-10T20:29:20.020926+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'2375","reported_seq":2432,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:45.950713+0000","last_change":"2024-09-10T20:29:19.022172+0000","last_active":"2024-09-10T20:29:45.950713+0000","last_peered":"2024-09-10T20:29:45.950713+0000","last_clean":"2024-09-10T20:28:42.552437+0000","last_became_active":"2024-09-10T20:29:19.021920+0000","last_became_peered":"2024-09-10T20:29:19.021920+0000","last_unstale":"2024-09-10T20:29:45.950713+0000","last_undegraded":"2024-09-10T20:29:19.015669+0000","last_fullsized":"2024-09-10T20:29:45.950713+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2375,"log_dups_size":0,"ondisk_log_size":2375,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9712582,"num_objects":2373,"num_object_clones":0,"num_object_copies":4746,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2373,"num_write_kb":9486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2373}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.016002+0000","last_change":"2024-09-10T20:29:19.022093+0000","last_active":"2024-09-10T20:29:20.016002+0000","last_peered":"2024-09-10T20:29:20.016002+0000","last_clean":"2024-09-10T20:29:20.016002+0000","last_became_active":"2024-09-10T20:29:19.021778+0000","last_became_peered":"2024-09-10T20:29:19.021778+0000","last_unstale":"2024-09-10T20:29:20.016002+0000","last_undegraded":"2024-09-10T20:29:20.016002+0000","last_fullsized":"2024-09-10T20:29:20.016002+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:25:12.240664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.019846+0000","last_change":"2024-09-10T20:29:20.018969+0000","last_active":"2024-09-10T20:29:20.019846+0000","last_peered":"2024-09-10T20:29:20.019846+0000","last_clean":"2024-09-10T20:29:20.019846+0000","last_became_active":"2024-09-10T20:29:20.018675+0000","last_became_peered":"2024-09-10T20:29:20.018675+0000","last_unstale":"2024-09-10T20:29:20.019846+0000","last_undegraded":"2024-09-10T20:29:20.019846+0000","last_fullsized":"2024-09-10T20:29:20.019846+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:50.839971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'2271","reported_seq":2328,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:45.973743+0000","last_change":"2024-09-10T20:29:19.024494+0000","last_active":"2024-09-10T20:29:45.973743+0000","last_peered":"2024-09-10T20:29:45.973743+0000","last_clean":"2024-09-10T20:28:46.321317+0000","last_became_active":"2024-09-10T20:29:19.024217+0000","last_became_peered":"2024-09-10T20:29:19.024217+0000","last_unstale":"2024-09-10T20:29:45.973743+0000","last_undegraded":"2024-09-10T20:29:19.016668+0000","last_fullsized":"2024-09-10T20:29:45.973743+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2271,"log_dups_size":0,"ondisk_log_size":2271,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9254796,"num_objects":2263,"num_object_clones":0,"num_object_copies":4526,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2268,"num_write_kb":9043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":2262}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":156222425,"num_objects":38157,"num_object_clones":0,"num_object_copies":76314,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28123,"num_write_kb":112378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":316354560,"data_stored":316231718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38220,"ondisk_log_size":38220,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182415,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":248836,"kb_used_data":145012,"kb_used_omap":4,"kb_used_meta":103803,"kb_avail":94123004,"statfs":{"total":96636764160,"available":96381956096,"internally_reserved":0,"allocated":148492288,"data_stored":148271414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":106294954},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,5,8],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986187,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":271020,"kb_used_data":158812,"kb_used_omap":4,"kb_used_meta":112187,"kb_avail":94100820,"statfs":{"total":96636764160,"available":96359239680,"internally_reserved":0,"allocated":162623488,"data_stored":162428369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":114880170},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,8,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":151777280,"data_stored":151755602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":164462592,"data_stored":164462592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:29:49.239 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:29:49.752 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:29:49.752 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:29:49.772 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2024-09-10T20:29:48.828560+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":156222444,"num_objects":38159,"num_object_clones":0,"num_object_copies":76318,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28125,"num_write_kb":112380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38223,"ondisk_log_size":38223,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":519856,"kb_used_data":303824,"kb_used_omap":8,"kb_used_meta":215991,"kb_avail":188223824,"statfs":{"total":193273528320,"available":192741195776,"internally_reserved":0,"allocated":311115776,"data_stored":310699783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8876,"internal_metadata":221175124},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,5,13,10],"upper_bound":32},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":79785733,"num_objects":19471,"num_object_clones":0,"num_object_copies":38942,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-27,"num_objects_dirty":19471,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9410,"num_write_kb":37700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003039"},"pg_stats":[{"pgid":"2.f","version":"36'2285","reported_seq":2336,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:45.934488+0000","last_change":"2024-09-10T20:29:19.019964+0000","last_active":"2024-09-10T20:29:45.934488+0000","last_peered":"2024-09-10T20:29:45.934488+0000","last_clean":"2024-09-10T20:29:45.934488+0000","last_became_active":"2024-09-10T20:29:19.019691+0000","last_became_peered":"2024-09-10T20:29:19.019691+0000","last_unstale":"2024-09-10T20:29:45.934488+0000","last_undegraded":"2024-09-10T20:29:45.934488+0000","last_fullsized":"2024-09-10T20:29:45.934488+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2285,"log_dups_size":0,"ondisk_log_size":2285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:46.881470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9359360,"num_objects":2285,"num_object_clones":0,"num_object_copies":4570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2285,"num_write_kb":9140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'2612","reported_seq":2703,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:47.999691+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:29:47.999691+0000","last_peered":"2024-09-10T20:29:47.999691+0000","last_clean":"2024-09-10T20:29:47.999691+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:29:47.999691+0000","last_undegraded":"2024-09-10T20:29:47.999691+0000","last_fullsized":"2024-09-10T20:29:47.999691+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":2612,"log_dups_size":0,"ondisk_log_size":2612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10674176,"num_objects":2606,"num_object_clones":0,"num_object_copies":5212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":798,"num_write_kb":3192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'2499","reported_seq":2602,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:48.005538+0000","last_change":"2024-09-10T20:29:41.318456+0000","last_active":"2024-09-10T20:29:48.005538+0000","last_peered":"2024-09-10T20:29:48.005538+0000","last_clean":"2024-09-10T20:29:48.005538+0000","last_became_active":"2024-09-10T20:29:20.020548+0000","last_became_peered":"2024-09-10T20:29:20.020548+0000","last_unstale":"2024-09-10T20:29:48.005538+0000","last_undegraded":"2024-09-10T20:29:48.005538+0000","last_fullsized":"2024-09-10T20:29:48.005538+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":2499,"log_dups_size":0,"ondisk_log_size":2499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:41:34.883083+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10219520,"num_objects":2495,"num_object_clones":0,"num_object_copies":4990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":627,"num_write_kb":2508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'2498","reported_seq":2559,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:47.968331+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:29:47.968331+0000","last_peered":"2024-09-10T20:29:47.968331+0000","last_clean":"2024-09-10T20:29:47.968331+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:29:47.968331+0000","last_undegraded":"2024-09-10T20:29:47.968331+0000","last_fullsized":"2024-09-10T20:29:47.968331+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2498,"log_dups_size":0,"ondisk_log_size":2498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10231808,"num_objects":2498,"num_object_clones":0,"num_object_copies":4996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2498,"num_write_kb":9992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'2322","reported_seq":2379,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:45.930007+0000","last_change":"2024-09-10T20:29:19.022136+0000","last_active":"2024-09-10T20:29:45.930007+0000","last_peered":"2024-09-10T20:29:45.930007+0000","last_clean":"2024-09-10T20:28:45.772428+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:29:45.930007+0000","last_undegraded":"2024-09-10T20:29:19.016520+0000","last_fullsized":"2024-09-10T20:29:45.930007+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2322,"log_dups_size":0,"ondisk_log_size":2322,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9483206,"num_objects":2317,"num_object_clones":0,"num_object_copies":4634,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2317,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2320,"num_write_kb":9264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2317}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'2311","reported_seq":2369,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:45.960730+0000","last_change":"2024-09-10T20:29:19.022275+0000","last_active":"2024-09-10T20:29:45.960730+0000","last_peered":"2024-09-10T20:29:45.960730+0000","last_clean":"2024-09-10T20:28:46.621675+0000","last_became_active":"2024-09-10T20:29:19.022041+0000","last_became_peered":"2024-09-10T20:29:19.022041+0000","last_unstale":"2024-09-10T20:29:45.960730+0000","last_undegraded":"2024-09-10T20:29:19.014144+0000","last_fullsized":"2024-09-10T20:29:45.960730+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2311,"log_dups_size":0,"ondisk_log_size":2311,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9446342,"num_objects":2308,"num_object_clones":0,"num_object_copies":4616,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2309,"num_write_kb":9227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2308}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'2269","reported_seq":2326,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:45.957863+0000","last_change":"2024-09-10T20:29:19.024530+0000","last_active":"2024-09-10T20:29:45.957863+0000","last_peered":"2024-09-10T20:29:45.957863+0000","last_clean":"2024-09-10T20:28:46.021581+0000","last_became_active":"2024-09-10T20:29:19.024227+0000","last_became_peered":"2024-09-10T20:29:19.024227+0000","last_unstale":"2024-09-10T20:29:45.957863+0000","last_undegraded":"2024-09-10T20:29:19.016822+0000","last_fullsized":"2024-09-10T20:29:45.957863+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2269,"log_dups_size":0,"ondisk_log_size":2269,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9270214,"num_objects":2265,"num_object_clones":0,"num_object_copies":4530,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2267,"num_write_kb":9056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2265}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'2338","reported_seq":2398,"reported_epoch":36,"state":"active+recovering","last_fresh":"2024-09-10T20:29:45.959603+0000","last_change":"2024-09-10T20:29:20.016123+0000","last_active":"2024-09-10T20:29:45.959603+0000","last_peered":"2024-09-10T20:29:45.959603+0000","last_clean":"2024-09-10T20:28:40.535680+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:29:45.959603+0000","last_undegraded":"2024-09-10T20:29:45.959603+0000","last_fullsized":"2024-09-10T20:29:45.959603+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2338,"log_dups_size":0,"ondisk_log_size":2338,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9568739,"num_objects":2337,"num_object_clones":0,"num_object_copies":4674,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2337,"num_write_kb":9345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2337}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.016038+0000","last_change":"2024-09-10T20:29:19.019807+0000","last_active":"2024-09-10T20:29:20.016038+0000","last_peered":"2024-09-10T20:29:20.016038+0000","last_clean":"2024-09-10T20:29:20.016038+0000","last_became_active":"2024-09-10T20:29:19.019500+0000","last_became_peered":"2024-09-10T20:29:19.019500+0000","last_unstale":"2024-09-10T20:29:20.016038+0000","last_undegraded":"2024-09-10T20:29:20.016038+0000","last_fullsized":"2024-09-10T20:29:20.016038+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:17:47.161423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'2433","reported_seq":2540,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:48.009709+0000","last_change":"2024-09-10T20:29:43.227345+0000","last_active":"2024-09-10T20:29:48.009709+0000","last_peered":"2024-09-10T20:29:48.009709+0000","last_clean":"2024-09-10T20:29:48.009709+0000","last_became_active":"2024-09-10T20:29:20.020530+0000","last_became_peered":"2024-09-10T20:29:20.020530+0000","last_unstale":"2024-09-10T20:29:48.009709+0000","last_undegraded":"2024-09-10T20:29:48.009709+0000","last_fullsized":"2024-09-10T20:29:48.009709+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":2433,"log_dups_size":0,"ondisk_log_size":2433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:18.980351+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9936896,"num_objects":2426,"num_object_clones":0,"num_object_copies":4852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":426,"num_write_kb":1704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.019440+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:29:20.019440+0000","last_peered":"2024-09-10T20:29:20.019440+0000","last_clean":"2024-09-10T20:29:20.019440+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:29:20.019440+0000","last_undegraded":"2024-09-10T20:29:20.019440+0000","last_fullsized":"2024-09-10T20:29:20.019440+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'2356","reported_seq":2405,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:45.973679+0000","last_change":"2024-09-10T20:29:19.024601+0000","last_active":"2024-09-10T20:29:45.973679+0000","last_peered":"2024-09-10T20:29:45.973679+0000","last_clean":"2024-09-10T20:28:46.672065+0000","last_became_active":"2024-09-10T20:29:19.024360+0000","last_became_peered":"2024-09-10T20:29:19.024360+0000","last_unstale":"2024-09-10T20:29:45.973679+0000","last_undegraded":"2024-09-10T20:29:19.014503+0000","last_fullsized":"2024-09-10T20:29:45.973679+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2356,"log_dups_size":0,"ondisk_log_size":2356,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9618374,"num_objects":2350,"num_object_clones":0,"num_object_copies":4700,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2354,"num_write_kb":9396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2350}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":64,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:20.015786+0000","last_change":"2024-09-10T20:29:19.021713+0000","last_active":"2024-09-10T20:29:20.015786+0000","last_peered":"2024-09-10T20:29:20.015786+0000","last_clean":"2024-09-10T20:28:57.955277+0000","last_became_active":"2024-09-10T20:29:19.021487+0000","last_became_peered":"2024-09-10T20:29:19.021487+0000","last_unstale":"2024-09-10T20:29:20.015786+0000","last_undegraded":"2024-09-10T20:29:19.013787+0000","last_fullsized":"2024-09-10T20:29:20.015786+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'2521","reported_seq":2623,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:47.984557+0000","last_change":"2024-09-10T20:29:44.304315+0000","last_active":"2024-09-10T20:29:47.984557+0000","last_peered":"2024-09-10T20:29:47.984557+0000","last_clean":"2024-09-10T20:29:47.984557+0000","last_became_active":"2024-09-10T20:29:20.018890+0000","last_became_peered":"2024-09-10T20:29:20.018890+0000","last_unstale":"2024-09-10T20:29:47.984557+0000","last_undegraded":"2024-09-10T20:29:47.984557+0000","last_fullsized":"2024-09-10T20:29:47.984557+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":2521,"log_dups_size":0,"ondisk_log_size":2521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:57:54.859061+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10305536,"num_objects":2516,"num_object_clones":0,"num_object_copies":5032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":349,"num_write_kb":1396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":92,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.022839+0000","last_change":"2024-09-10T20:29:20.021021+0000","last_active":"2024-09-10T20:29:20.022839+0000","last_peered":"2024-09-10T20:29:20.022839+0000","last_clean":"2024-09-10T20:29:20.022839+0000","last_became_active":"2024-09-10T20:29:20.020575+0000","last_became_peered":"2024-09-10T20:29:20.020575+0000","last_unstale":"2024-09-10T20:29:20.022839+0000","last_undegraded":"2024-09-10T20:29:20.022839+0000","last_fullsized":"2024-09-10T20:29:20.022839+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:23:19.560105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'2513","reported_seq":2615,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:48.014314+0000","last_change":"2024-09-10T20:29:45.018166+0000","last_active":"2024-09-10T20:29:48.014314+0000","last_peered":"2024-09-10T20:29:48.014314+0000","last_clean":"2024-09-10T20:29:48.014314+0000","last_became_active":"2024-09-10T20:29:20.018640+0000","last_became_peered":"2024-09-10T20:29:20.018640+0000","last_unstale":"2024-09-10T20:29:48.014314+0000","last_undegraded":"2024-09-10T20:29:48.014314+0000","last_fullsized":"2024-09-10T20:29:48.014314+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":2513,"log_dups_size":0,"ondisk_log_size":2513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:08:13.349234+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10268672,"num_objects":2507,"num_object_clones":0,"num_object_copies":5014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":299,"num_write_kb":1196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'2363","reported_seq":2421,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:45.913949+0000","last_change":"2024-09-10T20:29:19.022418+0000","last_active":"2024-09-10T20:29:45.913949+0000","last_peered":"2024-09-10T20:29:45.913949+0000","last_clean":"2024-09-10T20:28:42.552296+0000","last_became_active":"2024-09-10T20:29:19.022182+0000","last_became_peered":"2024-09-10T20:29:19.022182+0000","last_unstale":"2024-09-10T20:29:45.913949+0000","last_undegraded":"2024-09-10T20:29:19.016061+0000","last_fullsized":"2024-09-10T20:29:45.913949+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2363,"log_dups_size":0,"ondisk_log_size":2363,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9671139,"num_objects":2362,"num_object_clones":0,"num_object_copies":4724,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2362,"num_write_kb":9445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2362}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.015089+0000","last_change":"2024-09-10T20:29:19.022445+0000","last_active":"2024-09-10T20:29:20.015089+0000","last_peered":"2024-09-10T20:29:20.015089+0000","last_clean":"2024-09-10T20:29:20.015089+0000","last_became_active":"2024-09-10T20:29:19.022165+0000","last_became_peered":"2024-09-10T20:29:19.022165+0000","last_unstale":"2024-09-10T20:29:20.015089+0000","last_undegraded":"2024-09-10T20:29:20.015089+0000","last_fullsized":"2024-09-10T20:29:20.015089+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:57:59.626770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'2254","reported_seq":2310,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:45.957679+0000","last_change":"2024-09-10T20:29:19.024585+0000","last_active":"2024-09-10T20:29:45.957679+0000","last_peered":"2024-09-10T20:29:45.957679+0000","last_clean":"2024-09-10T20:28:46.571322+0000","last_became_active":"2024-09-10T20:29:19.024283+0000","last_became_peered":"2024-09-10T20:29:19.024283+0000","last_unstale":"2024-09-10T20:29:45.957679+0000","last_undegraded":"2024-09-10T20:29:19.017318+0000","last_fullsized":"2024-09-10T20:29:45.957679+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2254,"log_dups_size":0,"ondisk_log_size":2254,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9201065,"num_objects":2249,"num_object_clones":0,"num_object_copies":4498,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2251,"num_write_kb":8988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2249}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.020926+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:29:20.020926+0000","last_peered":"2024-09-10T20:29:20.020926+0000","last_clean":"2024-09-10T20:29:20.020926+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:29:20.020926+0000","last_undegraded":"2024-09-10T20:29:20.020926+0000","last_fullsized":"2024-09-10T20:29:20.020926+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'2375","reported_seq":2432,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:45.950713+0000","last_change":"2024-09-10T20:29:19.022172+0000","last_active":"2024-09-10T20:29:45.950713+0000","last_peered":"2024-09-10T20:29:45.950713+0000","last_clean":"2024-09-10T20:28:42.552437+0000","last_became_active":"2024-09-10T20:29:19.021920+0000","last_became_peered":"2024-09-10T20:29:19.021920+0000","last_unstale":"2024-09-10T20:29:45.950713+0000","last_undegraded":"2024-09-10T20:29:19.015669+0000","last_fullsized":"2024-09-10T20:29:45.950713+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2375,"log_dups_size":0,"ondisk_log_size":2375,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9712582,"num_objects":2373,"num_object_clones":0,"num_object_copies":4746,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2373,"num_write_kb":9486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2373}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.016002+0000","last_change":"2024-09-10T20:29:19.022093+0000","last_active":"2024-09-10T20:29:20.016002+0000","last_peered":"2024-09-10T20:29:20.016002+0000","last_clean":"2024-09-10T20:29:20.016002+0000","last_became_active":"2024-09-10T20:29:19.021778+0000","last_became_peered":"2024-09-10T20:29:19.021778+0000","last_unstale":"2024-09-10T20:29:20.016002+0000","last_undegraded":"2024-09-10T20:29:20.016002+0000","last_fullsized":"2024-09-10T20:29:20.016002+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:25:12.240664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.019846+0000","last_change":"2024-09-10T20:29:20.018969+0000","last_active":"2024-09-10T20:29:20.019846+0000","last_peered":"2024-09-10T20:29:20.019846+0000","last_clean":"2024-09-10T20:29:20.019846+0000","last_became_active":"2024-09-10T20:29:20.018675+0000","last_became_peered":"2024-09-10T20:29:20.018675+0000","last_unstale":"2024-09-10T20:29:20.019846+0000","last_undegraded":"2024-09-10T20:29:20.019846+0000","last_fullsized":"2024-09-10T20:29:20.019846+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:50.839971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'2271","reported_seq":2328,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T20:29:45.973743+0000","last_change":"2024-09-10T20:29:19.024494+0000","last_active":"2024-09-10T20:29:45.973743+0000","last_peered":"2024-09-10T20:29:45.973743+0000","last_clean":"2024-09-10T20:28:46.321317+0000","last_became_active":"2024-09-10T20:29:19.024217+0000","last_became_peered":"2024-09-10T20:29:19.024217+0000","last_unstale":"2024-09-10T20:29:45.973743+0000","last_undegraded":"2024-09-10T20:29:19.016668+0000","last_fullsized":"2024-09-10T20:29:45.973743+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2271,"log_dups_size":0,"ondisk_log_size":2271,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9254796,"num_objects":2263,"num_object_clones":0,"num_object_copies":4526,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2268,"num_write_kb":9043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":2262}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":156222425,"num_objects":38157,"num_object_clones":0,"num_object_copies":76314,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28123,"num_write_kb":112378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":316354560,"data_stored":316231718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38220,"ondisk_log_size":38220,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182415,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":248836,"kb_used_data":145012,"kb_used_omap":4,"kb_used_meta":103803,"kb_avail":94123004,"statfs":{"total":96636764160,"available":96381956096,"internally_reserved":0,"allocated":148492288,"data_stored":148271414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":106294954},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,5,8],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986187,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":271020,"kb_used_data":158812,"kb_used_omap":4,"kb_used_meta":112187,"kb_avail":94100820,"statfs":{"total":96636764160,"available":96359239680,"internally_reserved":0,"allocated":162623488,"data_stored":162428369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":114880170},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,8,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":151777280,"data_stored":151755602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":164462592,"data_stored":164462592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:29:50.034 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 22 20 43176 43156 7.66121 5.89453 0.0100521 0.0101909 2024-09-10T20:29:51.034 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 23 20 44802 44782 7.60426 6.35156 0.0151426 0.0102675 2024-09-10T20:29:52.034 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 24 20 46340 46320 7.53772 6.00781 0.00885813 0.0103594 2024-09-10T20:29:52.775 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:29:53.034 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 25 20 47688 47668 7.44682 5.26562 0.0178544 0.0104839 2024-09-10T20:29:53.303 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:29:53.304 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:29:53.324 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2024-09-10T20:29:52.829356+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":176219116,"num_objects":43041,"num_object_clones":0,"num_object_copies":86082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33007,"num_write_kb":131908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":502,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43105,"ondisk_log_size":43105,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":563044,"kb_used_data":334532,"kb_used_omap":8,"kb_used_meta":228471,"kb_avail":188180636,"statfs":{"total":193273528320,"available":192696971264,"internally_reserved":0,"allocated":342560768,"data_stored":342144775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8876,"internal_metadata":233954644},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[9,1,5,13,9,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":83848965,"num_objects":20463,"num_object_clones":0,"num_object_copies":40926,"num_objects_missing_on_primary":-27,"num_objects_missing":-27,"num_objects_degraded":-54,"num_objects_misplaced":0,"num_objects_unfound":-27,"num_objects_dirty":20463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10402,"num_write_kb":41668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":502,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002742"},"pg_stats":[{"pgid":"2.f","version":"36'2802","reported_seq":2852,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:50.977292+0000","last_change":"2024-09-10T20:29:19.019964+0000","last_active":"2024-09-10T20:29:50.977292+0000","last_peered":"2024-09-10T20:29:50.977292+0000","last_clean":"2024-09-10T20:29:50.977292+0000","last_became_active":"2024-09-10T20:29:19.019691+0000","last_became_peered":"2024-09-10T20:29:19.019691+0000","last_unstale":"2024-09-10T20:29:50.977292+0000","last_undegraded":"2024-09-10T20:29:50.977292+0000","last_fullsized":"2024-09-10T20:29:50.977292+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2802,"log_dups_size":0,"ondisk_log_size":2802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:46.881470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11476992,"num_objects":2802,"num_object_clones":0,"num_object_copies":5604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2802,"num_write_kb":11208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'2612","reported_seq":2703,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:47.999691+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:29:47.999691+0000","last_peered":"2024-09-10T20:29:47.999691+0000","last_clean":"2024-09-10T20:29:47.999691+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:29:47.999691+0000","last_undegraded":"2024-09-10T20:29:47.999691+0000","last_fullsized":"2024-09-10T20:29:47.999691+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":2612,"log_dups_size":0,"ondisk_log_size":2612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10674176,"num_objects":2606,"num_object_clones":0,"num_object_copies":5212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":798,"num_write_kb":3192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'2499","reported_seq":2602,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:48.005538+0000","last_change":"2024-09-10T20:29:41.318456+0000","last_active":"2024-09-10T20:29:48.005538+0000","last_peered":"2024-09-10T20:29:48.005538+0000","last_clean":"2024-09-10T20:29:48.005538+0000","last_became_active":"2024-09-10T20:29:20.020548+0000","last_became_peered":"2024-09-10T20:29:20.020548+0000","last_unstale":"2024-09-10T20:29:48.005538+0000","last_undegraded":"2024-09-10T20:29:48.005538+0000","last_fullsized":"2024-09-10T20:29:48.005538+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":2499,"log_dups_size":0,"ondisk_log_size":2499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:41:34.883083+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10219520,"num_objects":2495,"num_object_clones":0,"num_object_copies":4990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":627,"num_write_kb":2508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'2498","reported_seq":2559,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:47.968331+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:29:47.968331+0000","last_peered":"2024-09-10T20:29:47.968331+0000","last_clean":"2024-09-10T20:29:47.968331+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:29:47.968331+0000","last_undegraded":"2024-09-10T20:29:47.968331+0000","last_fullsized":"2024-09-10T20:29:47.968331+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2498,"log_dups_size":0,"ondisk_log_size":2498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10231808,"num_objects":2498,"num_object_clones":0,"num_object_copies":4996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2498,"num_write_kb":9992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'2804","reported_seq":2872,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:50.984541+0000","last_change":"2024-09-10T20:29:50.526711+0000","last_active":"2024-09-10T20:29:50.984541+0000","last_peered":"2024-09-10T20:29:50.984541+0000","last_clean":"2024-09-10T20:29:50.984541+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:29:50.984541+0000","last_undegraded":"2024-09-10T20:29:50.984541+0000","last_fullsized":"2024-09-10T20:29:50.984541+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2804,"log_dups_size":0,"ondisk_log_size":2804,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11457478,"num_objects":2799,"num_object_clones":0,"num_object_copies":5598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2799,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2802,"num_write_kb":11192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'2806","reported_seq":2874,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:50.976089+0000","last_change":"2024-09-10T20:29:50.637910+0000","last_active":"2024-09-10T20:29:50.976089+0000","last_peered":"2024-09-10T20:29:50.976089+0000","last_clean":"2024-09-10T20:29:50.976089+0000","last_became_active":"2024-09-10T20:29:19.022041+0000","last_became_peered":"2024-09-10T20:29:19.022041+0000","last_unstale":"2024-09-10T20:29:50.976089+0000","last_undegraded":"2024-09-10T20:29:50.976089+0000","last_fullsized":"2024-09-10T20:29:50.976089+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2806,"log_dups_size":0,"ondisk_log_size":2806,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11473862,"num_objects":2803,"num_object_clones":0,"num_object_copies":5606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2804,"num_write_kb":11207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'2752","reported_seq":2821,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:50.977135+0000","last_change":"2024-09-10T20:29:50.797207+0000","last_active":"2024-09-10T20:29:50.977135+0000","last_peered":"2024-09-10T20:29:50.977135+0000","last_clean":"2024-09-10T20:29:50.977135+0000","last_became_active":"2024-09-10T20:29:19.024227+0000","last_became_peered":"2024-09-10T20:29:19.024227+0000","last_unstale":"2024-09-10T20:29:50.977135+0000","last_undegraded":"2024-09-10T20:29:50.977135+0000","last_fullsized":"2024-09-10T20:29:50.977135+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2752,"log_dups_size":0,"ondisk_log_size":2752,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11248582,"num_objects":2748,"num_object_clones":0,"num_object_copies":5496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2750,"num_write_kb":10988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'2835","reported_seq":2902,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:50.953647+0000","last_change":"2024-09-10T20:29:50.472066+0000","last_active":"2024-09-10T20:29:50.953647+0000","last_peered":"2024-09-10T20:29:50.953647+0000","last_clean":"2024-09-10T20:29:50.953647+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:29:50.953647+0000","last_undegraded":"2024-09-10T20:29:50.953647+0000","last_fullsized":"2024-09-10T20:29:50.953647+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2835,"log_dups_size":0,"ondisk_log_size":2835,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11604451,"num_objects":2834,"num_object_clones":0,"num_object_copies":5668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2834,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2834,"num_write_kb":11333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.016038+0000","last_change":"2024-09-10T20:29:19.019807+0000","last_active":"2024-09-10T20:29:20.016038+0000","last_peered":"2024-09-10T20:29:20.016038+0000","last_clean":"2024-09-10T20:29:20.016038+0000","last_became_active":"2024-09-10T20:29:19.019500+0000","last_became_peered":"2024-09-10T20:29:19.019500+0000","last_unstale":"2024-09-10T20:29:20.016038+0000","last_undegraded":"2024-09-10T20:29:20.016038+0000","last_fullsized":"2024-09-10T20:29:20.016038+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:17:47.161423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'2433","reported_seq":2540,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:48.009709+0000","last_change":"2024-09-10T20:29:43.227345+0000","last_active":"2024-09-10T20:29:48.009709+0000","last_peered":"2024-09-10T20:29:48.009709+0000","last_clean":"2024-09-10T20:29:48.009709+0000","last_became_active":"2024-09-10T20:29:20.020530+0000","last_became_peered":"2024-09-10T20:29:20.020530+0000","last_unstale":"2024-09-10T20:29:48.009709+0000","last_undegraded":"2024-09-10T20:29:48.009709+0000","last_fullsized":"2024-09-10T20:29:48.009709+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":2433,"log_dups_size":0,"ondisk_log_size":2433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:18.980351+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9936896,"num_objects":2426,"num_object_clones":0,"num_object_copies":4852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":426,"num_write_kb":1704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.019440+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:29:20.019440+0000","last_peered":"2024-09-10T20:29:20.019440+0000","last_clean":"2024-09-10T20:29:20.019440+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:29:20.019440+0000","last_undegraded":"2024-09-10T20:29:20.019440+0000","last_fullsized":"2024-09-10T20:29:20.019440+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'2814","reported_seq":2875,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:50.968228+0000","last_change":"2024-09-10T20:29:50.934449+0000","last_active":"2024-09-10T20:29:50.968228+0000","last_peered":"2024-09-10T20:29:50.968228+0000","last_clean":"2024-09-10T20:29:50.968228+0000","last_became_active":"2024-09-10T20:29:19.024360+0000","last_became_peered":"2024-09-10T20:29:19.024360+0000","last_unstale":"2024-09-10T20:29:50.968228+0000","last_undegraded":"2024-09-10T20:29:50.968228+0000","last_fullsized":"2024-09-10T20:29:50.968228+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2814,"log_dups_size":0,"ondisk_log_size":2814,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11494342,"num_objects":2808,"num_object_clones":0,"num_object_copies":5616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2808,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2812,"num_write_kb":11228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":71,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:50.950293+0000","last_change":"2024-09-10T20:29:50.950210+0000","last_active":"2024-09-10T20:29:50.950293+0000","last_peered":"2024-09-10T20:29:50.950293+0000","last_clean":"2024-09-10T20:29:50.950293+0000","last_became_active":"2024-09-10T20:29:19.021487+0000","last_became_peered":"2024-09-10T20:29:19.021487+0000","last_unstale":"2024-09-10T20:29:50.950293+0000","last_undegraded":"2024-09-10T20:29:50.950293+0000","last_fullsized":"2024-09-10T20:29:50.950293+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:29:49.877011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'2521","reported_seq":2623,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:47.984557+0000","last_change":"2024-09-10T20:29:44.304315+0000","last_active":"2024-09-10T20:29:47.984557+0000","last_peered":"2024-09-10T20:29:47.984557+0000","last_clean":"2024-09-10T20:29:47.984557+0000","last_became_active":"2024-09-10T20:29:20.018890+0000","last_became_peered":"2024-09-10T20:29:20.018890+0000","last_unstale":"2024-09-10T20:29:47.984557+0000","last_undegraded":"2024-09-10T20:29:47.984557+0000","last_fullsized":"2024-09-10T20:29:47.984557+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":2521,"log_dups_size":0,"ondisk_log_size":2521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:57:54.859061+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10305536,"num_objects":2516,"num_object_clones":0,"num_object_copies":5032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":349,"num_write_kb":1396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":92,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.022839+0000","last_change":"2024-09-10T20:29:20.021021+0000","last_active":"2024-09-10T20:29:20.022839+0000","last_peered":"2024-09-10T20:29:20.022839+0000","last_clean":"2024-09-10T20:29:20.022839+0000","last_became_active":"2024-09-10T20:29:20.020575+0000","last_became_peered":"2024-09-10T20:29:20.020575+0000","last_unstale":"2024-09-10T20:29:20.022839+0000","last_undegraded":"2024-09-10T20:29:20.022839+0000","last_fullsized":"2024-09-10T20:29:20.022839+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:23:19.560105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'2513","reported_seq":2615,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:48.014314+0000","last_change":"2024-09-10T20:29:45.018166+0000","last_active":"2024-09-10T20:29:48.014314+0000","last_peered":"2024-09-10T20:29:48.014314+0000","last_clean":"2024-09-10T20:29:48.014314+0000","last_became_active":"2024-09-10T20:29:20.018640+0000","last_became_peered":"2024-09-10T20:29:20.018640+0000","last_unstale":"2024-09-10T20:29:48.014314+0000","last_undegraded":"2024-09-10T20:29:48.014314+0000","last_fullsized":"2024-09-10T20:29:48.014314+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":2513,"log_dups_size":0,"ondisk_log_size":2513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:08:13.349234+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10268672,"num_objects":2507,"num_object_clones":0,"num_object_copies":5014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":299,"num_write_kb":1196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'2847","reported_seq":2912,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:50.990507+0000","last_change":"2024-09-10T20:29:50.680602+0000","last_active":"2024-09-10T20:29:50.990507+0000","last_peered":"2024-09-10T20:29:50.990507+0000","last_clean":"2024-09-10T20:29:50.990507+0000","last_became_active":"2024-09-10T20:29:19.022182+0000","last_became_peered":"2024-09-10T20:29:19.022182+0000","last_unstale":"2024-09-10T20:29:50.990507+0000","last_undegraded":"2024-09-10T20:29:50.990507+0000","last_fullsized":"2024-09-10T20:29:50.990507+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2847,"log_dups_size":0,"ondisk_log_size":2847,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11653603,"num_objects":2846,"num_object_clones":0,"num_object_copies":5692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2846,"num_write_kb":11381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.015089+0000","last_change":"2024-09-10T20:29:19.022445+0000","last_active":"2024-09-10T20:29:20.015089+0000","last_peered":"2024-09-10T20:29:20.015089+0000","last_clean":"2024-09-10T20:29:20.015089+0000","last_became_active":"2024-09-10T20:29:19.022165+0000","last_became_peered":"2024-09-10T20:29:19.022165+0000","last_unstale":"2024-09-10T20:29:20.015089+0000","last_undegraded":"2024-09-10T20:29:20.015089+0000","last_fullsized":"2024-09-10T20:29:20.015089+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:57:59.626770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'2748","reported_seq":2819,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:50.970749+0000","last_change":"2024-09-10T20:29:50.878965+0000","last_active":"2024-09-10T20:29:50.970749+0000","last_peered":"2024-09-10T20:29:50.970749+0000","last_clean":"2024-09-10T20:29:50.970749+0000","last_became_active":"2024-09-10T20:29:19.024283+0000","last_became_peered":"2024-09-10T20:29:19.024283+0000","last_unstale":"2024-09-10T20:29:50.970749+0000","last_undegraded":"2024-09-10T20:29:50.970749+0000","last_fullsized":"2024-09-10T20:29:50.970749+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2748,"log_dups_size":0,"ondisk_log_size":2748,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11224489,"num_objects":2743,"num_object_clones":0,"num_object_copies":5486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2745,"num_write_kb":10964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.020926+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:29:20.020926+0000","last_peered":"2024-09-10T20:29:20.020926+0000","last_clean":"2024-09-10T20:29:20.020926+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:29:20.020926+0000","last_undegraded":"2024-09-10T20:29:20.020926+0000","last_fullsized":"2024-09-10T20:29:20.020926+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'2891","reported_seq":2959,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:50.977154+0000","last_change":"2024-09-10T20:29:50.571043+0000","last_active":"2024-09-10T20:29:50.977154+0000","last_peered":"2024-09-10T20:29:50.977154+0000","last_clean":"2024-09-10T20:29:50.977154+0000","last_became_active":"2024-09-10T20:29:19.021920+0000","last_became_peered":"2024-09-10T20:29:19.021920+0000","last_unstale":"2024-09-10T20:29:50.977154+0000","last_undegraded":"2024-09-10T20:29:50.977154+0000","last_fullsized":"2024-09-10T20:29:50.977154+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2891,"log_dups_size":0,"ondisk_log_size":2891,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11826118,"num_objects":2889,"num_object_clones":0,"num_object_copies":5778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2889,"num_write_kb":11550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.016002+0000","last_change":"2024-09-10T20:29:19.022093+0000","last_active":"2024-09-10T20:29:20.016002+0000","last_peered":"2024-09-10T20:29:20.016002+0000","last_clean":"2024-09-10T20:29:20.016002+0000","last_became_active":"2024-09-10T20:29:19.021778+0000","last_became_peered":"2024-09-10T20:29:19.021778+0000","last_unstale":"2024-09-10T20:29:20.016002+0000","last_undegraded":"2024-09-10T20:29:20.016002+0000","last_fullsized":"2024-09-10T20:29:20.016002+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:25:12.240664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:20.019846+0000","last_change":"2024-09-10T20:29:20.018969+0000","last_active":"2024-09-10T20:29:20.019846+0000","last_peered":"2024-09-10T20:29:20.019846+0000","last_clean":"2024-09-10T20:29:20.019846+0000","last_became_active":"2024-09-10T20:29:20.018675+0000","last_became_peered":"2024-09-10T20:29:20.018675+0000","last_unstale":"2024-09-10T20:29:20.019846+0000","last_undegraded":"2024-09-10T20:29:20.019846+0000","last_fullsized":"2024-09-10T20:29:20.019846+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:50.839971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'2727","reported_seq":2801,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T20:29:50.962837+0000","last_change":"2024-09-10T20:29:50.749998+0000","last_active":"2024-09-10T20:29:50.962837+0000","last_peered":"2024-09-10T20:29:50.962837+0000","last_clean":"2024-09-10T20:29:50.962837+0000","last_became_active":"2024-09-10T20:29:19.024217+0000","last_became_peered":"2024-09-10T20:29:19.024217+0000","last_unstale":"2024-09-10T20:29:50.962837+0000","last_undegraded":"2024-09-10T20:29:50.962837+0000","last_fullsized":"2024-09-10T20:29:50.962837+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":2727,"log_dups_size":0,"ondisk_log_size":2727,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11122572,"num_objects":2719,"num_object_clones":0,"num_object_copies":5438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2724,"num_write_kb":10867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":176219097,"num_objects":43039,"num_object_clones":0,"num_object_copies":86078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33005,"num_write_kb":131906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":347811840,"data_stored":347707063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43102,"ondisk_log_size":43102,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182416,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292024,"kb_used_data":175720,"kb_used_omap":4,"kb_used_meta":116283,"kb_avail":94079816,"statfs":{"total":96636764160,"available":96337731584,"internally_reserved":0,"allocated":179937280,"data_stored":179716406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":119074474},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,1,1,5,7,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986187,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":271020,"kb_used_data":158812,"kb_used_omap":4,"kb_used_meta":112187,"kb_avail":94100820,"statfs":{"total":96636764160,"available":96359239680,"internally_reserved":0,"allocated":162623488,"data_stored":162428369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":114880170},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,8,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":183234560,"data_stored":183230947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":164462592,"data_stored":164462592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:29:53.326 INFO:tasks.rep_lost_unfound_delete.ceph_manager:recovered! 2024-09-10T20:29:53.326 INFO:tasks.util.rados:rados -p unfounddel_pool get new_1 - 2024-09-10T20:29:53.326 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_1 - 2024-09-10T20:29:53.419 INFO:teuthology.orchestra.run.smithi131.stderr:error getting unfounddel_pool/new_1: (2) No such file or directory 2024-09-10T20:29:53.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:29:53.425 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_1 - 2024-09-10T20:29:53.425 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_1 - 2024-09-10T20:29:53.625 INFO:teuthology.orchestra.run.smithi131.stderr:error getting unfounddel_pool/existed_1: (2) No such file or directory 2024-09-10T20:29:53.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:29:53.635 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_1 - 2024-09-10T20:29:53.635 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_1 - 2024-09-10T20:29:53.720 INFO:teuthology.orchestra.run.smithi131.stderr:error getting unfounddel_pool/existing_1: (2) No such file or directory 2024-09-10T20:29:53.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:29:53.727 INFO:tasks.util.rados:rados -p unfounddel_pool get new_2 - 2024-09-10T20:29:53.727 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_2 - 2024-09-10T20:29:53.833 INFO:teuthology.orchestra.run.smithi131.stderr:error getting unfounddel_pool/new_2: (2) No such file or directory 2024-09-10T20:29:53.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:29:53.837 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_2 - 2024-09-10T20:29:53.838 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_2 - 2024-09-10T20:29:53.936 INFO:teuthology.orchestra.run.smithi131.stderr:error getting unfounddel_pool/existed_2: (2) No such file or directory 2024-09-10T20:29:53.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:29:53.945 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_2 - 2024-09-10T20:29:53.945 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_2 - 2024-09-10T20:29:54.034 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 26 20 48764 48744 7.32205 4.20312 0.0150101 0.0106621 2024-09-10T20:29:54.047 INFO:teuthology.orchestra.run.smithi131.stderr:error getting unfounddel_pool/existing_2: (2) No such file or directory 2024-09-10T20:29:54.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:29:54.052 INFO:tasks.util.rados:rados -p unfounddel_pool get new_3 - 2024-09-10T20:29:54.052 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_3 - 2024-09-10T20:29:54.142 INFO:teuthology.orchestra.run.smithi131.stderr:error getting unfounddel_pool/new_3: (2) No such file or directory 2024-09-10T20:29:54.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:29:54.150 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_3 - 2024-09-10T20:29:54.151 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_3 - 2024-09-10T20:29:54.241 INFO:teuthology.orchestra.run.smithi131.stderr:error getting unfounddel_pool/existed_3: (2) No such file or directory 2024-09-10T20:29:54.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:29:54.250 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_3 - 2024-09-10T20:29:54.250 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_3 - 2024-09-10T20:29:54.349 INFO:teuthology.orchestra.run.smithi131.stderr:error getting unfounddel_pool/existing_3: (2) No such file or directory 2024-09-10T20:29:54.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:29:54.361 INFO:tasks.util.rados:rados -p unfounddel_pool get new_4 - 2024-09-10T20:29:54.361 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_4 - 2024-09-10T20:29:54.464 INFO:teuthology.orchestra.run.smithi131.stderr:error getting unfounddel_pool/new_4: (2) No such file or directory 2024-09-10T20:29:54.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:29:54.470 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_4 - 2024-09-10T20:29:54.470 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_4 - 2024-09-10T20:29:54.569 INFO:teuthology.orchestra.run.smithi131.stderr:error getting unfounddel_pool/existed_4: (2) No such file or directory 2024-09-10T20:29:54.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:29:54.579 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_4 - 2024-09-10T20:29:54.579 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_4 - 2024-09-10T20:29:54.681 INFO:teuthology.orchestra.run.smithi131.stderr:error getting unfounddel_pool/existing_4: (2) No such file or directory 2024-09-10T20:29:54.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:29:54.690 INFO:tasks.util.rados:rados -p unfounddel_pool get new_5 - 2024-09-10T20:29:54.690 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_5 - 2024-09-10T20:29:54.779 INFO:teuthology.orchestra.run.smithi131.stderr:error getting unfounddel_pool/new_5: (2) No such file or directory 2024-09-10T20:29:54.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:29:54.786 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_5 - 2024-09-10T20:29:54.786 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_5 - 2024-09-10T20:29:54.881 INFO:teuthology.orchestra.run.smithi131.stderr:error getting unfounddel_pool/existed_5: (2) No such file or directory 2024-09-10T20:29:54.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:29:54.893 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_5 - 2024-09-10T20:29:54.893 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_5 - 2024-09-10T20:29:54.987 INFO:teuthology.orchestra.run.smithi131.stderr:error getting unfounddel_pool/existing_5: (2) No such file or directory 2024-09-10T20:29:54.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:29:54.997 INFO:tasks.util.rados:rados -p unfounddel_pool get new_6 - 2024-09-10T20:29:54.998 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_6 - 2024-09-10T20:29:55.034 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 27 20 50001 49981 7.22982 4.83203 0.0177155 0.0107995 2024-09-10T20:29:55.096 INFO:teuthology.orchestra.run.smithi131.stderr:error getting unfounddel_pool/new_6: (2) No such file or directory 2024-09-10T20:29:55.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:29:55.105 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_6 - 2024-09-10T20:29:55.105 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_6 - 2024-09-10T20:29:55.243 INFO:teuthology.orchestra.run.smithi131.stderr:error getting unfounddel_pool/existed_6: (2) No such file or directory 2024-09-10T20:29:55.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:29:55.251 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_6 - 2024-09-10T20:29:55.251 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_6 - 2024-09-10T20:29:55.349 INFO:teuthology.orchestra.run.smithi131.stderr:error getting unfounddel_pool/existing_6: (2) No such file or directory 2024-09-10T20:29:55.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:29:55.358 INFO:tasks.util.rados:rados -p unfounddel_pool get new_7 - 2024-09-10T20:29:55.358 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_7 - 2024-09-10T20:29:55.451 INFO:teuthology.orchestra.run.smithi131.stderr:error getting unfounddel_pool/new_7: (2) No such file or directory 2024-09-10T20:29:55.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:29:55.458 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_7 - 2024-09-10T20:29:55.459 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_7 - 2024-09-10T20:29:55.548 INFO:teuthology.orchestra.run.smithi131.stderr:error getting unfounddel_pool/existed_7: (2) No such file or directory 2024-09-10T20:29:55.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:29:55.555 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_7 - 2024-09-10T20:29:55.555 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_7 - 2024-09-10T20:29:55.653 INFO:teuthology.orchestra.run.smithi131.stderr:error getting unfounddel_pool/existing_7: (2) No such file or directory 2024-09-10T20:29:55.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:29:55.661 INFO:tasks.util.rados:rados -p unfounddel_pool get new_8 - 2024-09-10T20:29:55.661 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_8 - 2024-09-10T20:29:55.753 INFO:teuthology.orchestra.run.smithi131.stderr:error getting unfounddel_pool/new_8: (2) No such file or directory 2024-09-10T20:29:55.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:29:55.761 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_8 - 2024-09-10T20:29:55.761 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_8 - 2024-09-10T20:29:55.846 INFO:teuthology.orchestra.run.smithi131.stderr:error getting unfounddel_pool/existed_8: (2) No such file or directory 2024-09-10T20:29:55.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:29:55.858 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_8 - 2024-09-10T20:29:55.858 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_8 - 2024-09-10T20:29:55.959 INFO:teuthology.orchestra.run.smithi131.stderr:error getting unfounddel_pool/existing_8: (2) No such file or directory 2024-09-10T20:29:55.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:29:55.971 INFO:tasks.util.rados:rados -p unfounddel_pool get new_9 - 2024-09-10T20:29:55.972 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_9 - 2024-09-10T20:29:56.034 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 28 20 51212 51192 7.14056 4.73047 0.0113045 0.0109334 2024-09-10T20:29:56.062 INFO:teuthology.orchestra.run.smithi131.stderr:error getting unfounddel_pool/new_9: (2) No such file or directory 2024-09-10T20:29:56.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:29:56.072 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_9 - 2024-09-10T20:29:56.072 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_9 - 2024-09-10T20:29:56.165 INFO:teuthology.orchestra.run.smithi131.stderr:error getting unfounddel_pool/existed_9: (2) No such file or directory 2024-09-10T20:29:56.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:29:56.174 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_9 - 2024-09-10T20:29:56.174 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_9 - 2024-09-10T20:29:56.345 INFO:teuthology.orchestra.run.smithi131.stderr:error getting unfounddel_pool/existing_9: (2) No such file or directory 2024-09-10T20:29:56.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:29:56.353 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2024-09-10T20:29:56.906 INFO:tasks.ceph.mon.a.smithi131.stderr:2024-09-10T20:29:56.901+0000 7f1e634ff640 -1 mon.a@0(leader).osd e36 definitely_dead 0 2024-09-10T20:29:57.034 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 29 20 52340 52320 7.04627 4.40625 0.0171573 0.0110803 2024-09-10T20:29:57.245 INFO:teuthology.orchestra.run.smithi131.stderr:marked in osd.1. 2024-09-10T20:29:57.268 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-10T20:29:57.269 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-10T20:29:57.271 INFO:tasks.ceph.osd.1:Started 2024-09-10T20:29:57.271 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-10T20:29:57.369 INFO:teuthology.orchestra.run.smithi131.stderr:no valid command found; 10 closest matches: 2024-09-10T20:29:57.369 INFO:teuthology.orchestra.run.smithi131.stderr:0 2024-09-10T20:29:57.369 INFO:teuthology.orchestra.run.smithi131.stderr:1 2024-09-10T20:29:57.369 INFO:teuthology.orchestra.run.smithi131.stderr:2 2024-09-10T20:29:57.369 INFO:teuthology.orchestra.run.smithi131.stderr:abort 2024-09-10T20:29:57.369 INFO:teuthology.orchestra.run.smithi131.stderr:assert 2024-09-10T20:29:57.369 INFO:teuthology.orchestra.run.smithi131.stderr:config diff 2024-09-10T20:29:57.369 INFO:teuthology.orchestra.run.smithi131.stderr:config diff get 2024-09-10T20:29:57.369 INFO:teuthology.orchestra.run.smithi131.stderr:config get 2024-09-10T20:29:57.369 INFO:teuthology.orchestra.run.smithi131.stderr:config help [] 2024-09-10T20:29:57.370 INFO:teuthology.orchestra.run.smithi131.stderr:config set ... 2024-09-10T20:29:57.370 INFO:teuthology.orchestra.run.smithi131.stderr:admin_socket: invalid command 2024-09-10T20:29:57.371 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-10T20:29:57.371 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-10T20:29:58.034 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 30 20 52639 52619 6.85033 1.16797 0.00251576 0.0111024 2024-09-10T20:29:58.186 INFO:tasks.ceph.osd.1.smithi131.stderr:2024-09-10T20:29:58.181+0000 7fe821a11640 -1 Falling back to public interface 2024-09-10T20:29:59.035 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 31 20 52639 52619 6.62937 0 - 0.0111024 2024-09-10T20:30:00.035 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 32 19 53683 53664 6.54976 2.04102 0.0104639 0.0119219 2024-09-10T20:30:01.035 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 33 20 54987 54967 6.50551 5.08984 0.015162 0.0120033 2024-09-10T20:30:02.035 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 34 20 56161 56141 6.44905 4.58594 0.00941751 0.0121067 2024-09-10T20:30:02.066 INFO:tasks.ceph.osd.1.smithi131.stderr:2024-09-10T20:30:02.062+0000 7fe821a11640 -1 osd.1 26 log_to_monitors true 2024-09-10T20:30:02.319 INFO:tasks.ceph.osd.1.smithi131.stderr:2024-09-10T20:30:02.314+0000 7fe819145640 -1 osd.1 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T20:30:02.372 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-10T20:30:02.568 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-10T20:30:02.568 INFO:teuthology.orchestra.run.smithi131.stdout: "ops": [], 2024-09-10T20:30:02.568 INFO:teuthology.orchestra.run.smithi131.stdout: "num_ops": 0 2024-09-10T20:30:02.568 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-10T20:30:02.582 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for osd.1 to be up 2024-09-10T20:30:02.583 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T20:30:03.035 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 35 20 57295 57275 6.39134 4.42969 0.0190194 0.0122169 2024-09-10T20:30:03.122 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:03.122 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":40,"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","created":"2024-09-10T20:28:08.371455+0000","modified":"2024-09-10T20:30:02.278625+0000","last_up_change":"2024-09-10T20:29:13.985086+0000","last_in_change":"2024-09-10T20:29:57.228193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-10T20:28:23.669739+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.4900000095367432,"score_stable":1.4900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":1,"raw_score_stable":1,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfounddel_pool","create_time":"2024-09-10T20:28:36.617409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8700000047683716,"score_stable":1.8700000047683716,"optimal_score":0.67000001668930054,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d2261461-51c1-40ff-9ad5-8f4b10800e7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":21,"up_from":25,"up_thru":38,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":567435338},{"type":"v1","addr":"172.21.15.131:6801","nonce":567435338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":567435338},{"type":"v1","addr":"172.21.15.131:6803","nonce":567435338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":567435338},{"type":"v1","addr":"172.21.15.131:6807","nonce":567435338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":567435338},{"type":"v1","addr":"172.21.15.131:6805","nonce":567435338}]},"public_addr":"172.21.15.131:6801/567435338","cluster_addr":"172.21.15.131:6803/567435338","heartbeat_back_addr":"172.21.15.131:6807/567435338","heartbeat_front_addr":"172.21.15.131:6805/567435338","state":["exists","up"]},{"osd":1,"uuid":"9cb327f8-9ea4-489b-9cdf-1007a983d385","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":25,"down_at":27,"lost_at":27,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":454060533},{"type":"v1","addr":"172.21.15.131:6816","nonce":454060533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":454060533},{"type":"v1","addr":"172.21.15.131:6819","nonce":454060533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":454060533},{"type":"v1","addr":"172.21.15.131:6823","nonce":454060533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":454060533},{"type":"v1","addr":"172.21.15.131:6821","nonce":454060533}]},"public_addr":"172.21.15.131:6816/454060533","cluster_addr":"172.21.15.131:6819/454060533","heartbeat_back_addr":"172.21.15.131:6823/454060533","heartbeat_front_addr":"172.21.15.131:6821/454060533","state":["exists"]},{"osd":2,"uuid":"2a1df90e-2933-4052-a62a-ffa454708bb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":18,"up_from":31,"up_thru":38,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":2043655954},{"type":"v1","addr":"172.21.15.131:6809","nonce":2043655954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2043655954},{"type":"v1","addr":"172.21.15.131:6811","nonce":2043655954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2043655954},{"type":"v1","addr":"172.21.15.131:6815","nonce":2043655954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2043655954},{"type":"v1","addr":"172.21.15.131:6813","nonce":2043655954}]},"public_addr":"172.21.15.131:6809/2043655954","cluster_addr":"172.21.15.131:6811/2043655954","heartbeat_back_addr":"172.21.15.131:6815/2043655954","heartbeat_front_addr":"172.21.15.131:6813/2043655954","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:28:16.289792+0000","dead_epoch":22},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:28:15.574958+0000","dead_epoch":27},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:28:15.599924+0000","dead_epoch":19}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"1.0","osds":[0,2]},{"pgid":"1.2","osds":[0,2]},{"pgid":"1.3","osds":[2,0]},{"pgid":"1.4","osds":[0,2]},{"pgid":"1.6","osds":[0,2]},{"pgid":"1.7","osds":[2,0]},{"pgid":"2.0","osds":[2,0]},{"pgid":"2.1","osds":[2,0]},{"pgid":"2.2","osds":[0,2]},{"pgid":"2.3","osds":[2,0]},{"pgid":"2.4","osds":[0,2]},{"pgid":"2.5","osds":[0,2]},{"pgid":"2.6","osds":[0,2]},{"pgid":"2.7","osds":[0,2]},{"pgid":"2.9","osds":[0,2]},{"pgid":"2.a","osds":[0,2]},{"pgid":"2.d","osds":[2,0]},{"pgid":"2.f","osds":[0,2]}],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T20:30:04.035 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 36 20 57631 57611 6.25026 1.3125 0.00260162 0.0122445 2024-09-10T20:30:05.035 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 37 20 57631 57611 6.08134 0 - 0.0122445 2024-09-10T20:30:06.035 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 38 20 57840 57820 5.9428 0.408203 0.0435106 0.0131372 2024-09-10T20:30:06.151 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T20:30:06.702 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:06.702 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":43,"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","created":"2024-09-10T20:28:08.371455+0000","modified":"2024-09-10T20:30:05.300290+0000","last_up_change":"2024-09-10T20:30:03.291576+0000","last_in_change":"2024-09-10T20:29:57.228193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-10T20:28:23.669739+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfounddel_pool","create_time":"2024-09-10T20:28:36.617409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d2261461-51c1-40ff-9ad5-8f4b10800e7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":21,"up_from":25,"up_thru":42,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":567435338},{"type":"v1","addr":"172.21.15.131:6801","nonce":567435338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":567435338},{"type":"v1","addr":"172.21.15.131:6803","nonce":567435338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":567435338},{"type":"v1","addr":"172.21.15.131:6807","nonce":567435338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":567435338},{"type":"v1","addr":"172.21.15.131:6805","nonce":567435338}]},"public_addr":"172.21.15.131:6801/567435338","cluster_addr":"172.21.15.131:6803/567435338","heartbeat_back_addr":"172.21.15.131:6807/567435338","heartbeat_front_addr":"172.21.15.131:6805/567435338","state":["exists","up"]},{"osd":1,"uuid":"9cb327f8-9ea4-489b-9cdf-1007a983d385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":26,"up_from":41,"up_thru":42,"down_at":27,"lost_at":27,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1855213743},{"type":"v1","addr":"172.21.15.131:6817","nonce":1855213743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1855213743},{"type":"v1","addr":"172.21.15.131:6819","nonce":1855213743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1855213743},{"type":"v1","addr":"172.21.15.131:6823","nonce":1855213743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1855213743},{"type":"v1","addr":"172.21.15.131:6821","nonce":1855213743}]},"public_addr":"172.21.15.131:6817/1855213743","cluster_addr":"172.21.15.131:6819/1855213743","heartbeat_back_addr":"172.21.15.131:6823/1855213743","heartbeat_front_addr":"172.21.15.131:6821/1855213743","state":["exists","up"]},{"osd":2,"uuid":"2a1df90e-2933-4052-a62a-ffa454708bb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":18,"up_from":31,"up_thru":42,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":2043655954},{"type":"v1","addr":"172.21.15.131:6809","nonce":2043655954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2043655954},{"type":"v1","addr":"172.21.15.131:6811","nonce":2043655954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2043655954},{"type":"v1","addr":"172.21.15.131:6815","nonce":2043655954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2043655954},{"type":"v1","addr":"172.21.15.131:6813","nonce":2043655954}]},"public_addr":"172.21.15.131:6809/2043655954","cluster_addr":"172.21.15.131:6811/2043655954","heartbeat_back_addr":"172.21.15.131:6815/2043655954","heartbeat_front_addr":"172.21.15.131:6813/2043655954","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:28:16.289792+0000","dead_epoch":22},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:28:15.574958+0000","dead_epoch":27},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:28:15.599924+0000","dead_epoch":19}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"2.0","osds":[2]},{"pgid":"2.1","osds":[2]},{"pgid":"2.2","osds":[0]},{"pgid":"2.3","osds":[2]},{"pgid":"2.4","osds":[0]},{"pgid":"2.5","osds":[0]},{"pgid":"2.6","osds":[0]},{"pgid":"2.7","osds":[0]},{"pgid":"2.9","osds":[0]},{"pgid":"2.a","osds":[0]},{"pgid":"2.d","osds":[2]},{"pgid":"2.f","osds":[0]}],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T20:30:06.732 INFO:tasks.rep_lost_unfound_delete.ceph_manager:osd.1 is up 2024-09-10T20:30:06.732 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for clean 2024-09-10T20:30:06.733 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:30:07.035 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 39 20 58503 58483 5.85681 2.58984 0.0262404 0.0133306 2024-09-10T20:30:07.298 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:07.299 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:30:07.328 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2024-09-10T20:30:06.831829+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":236217788,"num_objects":57675,"num_object_clones":0,"num_object_copies":115350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28767,"num_objects_misplaced":14300,"num_objects_unfound":0,"num_objects_dirty":57675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22050,"num_write_kb":88174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":4115,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57755,"ondisk_log_size":57755,"up":39,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":72,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":748832,"kb_used_data":459432,"kb_used_omap":12,"kb_used_meta":289331,"kb_avail":282366688,"statfs":{"total":289910292480,"available":289143488512,"internally_reserved":0,"allocated":470458368,"data_stored":469752106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":296274946},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,25],"upper_bound":4096},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":14603921,"num_objects":3561,"num_object_clones":0,"num_object_copies":7122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28767,"num_objects_misplaced":-13150,"num_objects_unfound":0,"num_objects_dirty":3561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-3227,"num_write_kb":-12867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-6,"num_bytes_recovered":-483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.545794"},"pg_stats":[{"pgid":"2.f","version":"43'3624","reported_seq":3698,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:06.055566+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:30:06.055566+0000","last_peered":"2024-09-10T20:30:06.055566+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:30:06.055566+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":3624,"log_dups_size":0,"ondisk_log_size":3624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14843904,"num_objects":3624,"num_object_clones":0,"num_object_copies":7248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3624,"num_write_kb":14496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":17},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"40'3635","reported_seq":3734,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:30:03.024080+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:30:03.024080+0000","last_peered":"2024-09-10T20:30:03.024080+0000","last_clean":"2024-09-10T20:30:03.024080+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:30:03.024080+0000","last_undegraded":"2024-09-10T20:30:03.024080+0000","last_fullsized":"2024-09-10T20:30:03.024080+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":3635,"log_dups_size":0,"ondisk_log_size":3635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14864384,"num_objects":3629,"num_object_clones":0,"num_object_copies":7258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1821,"num_write_kb":7284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"40'3589","reported_seq":3705,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T20:30:03.034166+0000","last_change":"2024-09-10T20:29:59.267508+0000","last_active":"2024-09-10T20:30:03.034166+0000","last_peered":"2024-09-10T20:30:03.034166+0000","last_clean":"2024-09-10T20:30:03.034166+0000","last_became_active":"2024-09-10T20:29:59.264128+0000","last_became_peered":"2024-09-10T20:29:59.264128+0000","last_unstale":"2024-09-10T20:30:03.034166+0000","last_undegraded":"2024-09-10T20:30:03.034166+0000","last_fullsized":"2024-09-10T20:30:03.034166+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":3589,"log_dups_size":0,"ondisk_log_size":3589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:30.100436+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14684160,"num_objects":3585,"num_object_clones":0,"num_object_copies":7170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3585,"num_objects_unfound":0,"num_objects_dirty":3585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1717,"num_write_kb":6868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3585}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"40'3543","reported_seq":3612,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:30:03.035556+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:30:03.035556+0000","last_peered":"2024-09-10T20:30:03.035556+0000","last_clean":"2024-09-10T20:30:03.035556+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:30:03.035556+0000","last_undegraded":"2024-09-10T20:30:03.035556+0000","last_fullsized":"2024-09-10T20:30:03.035556+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":3543,"log_dups_size":0,"ondisk_log_size":3543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14512128,"num_objects":3543,"num_object_clones":0,"num_object_copies":7086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3543,"num_write_kb":14172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'3681","reported_seq":3802,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:06.055205+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:30:06.055205+0000","last_peered":"2024-09-10T20:30:06.055205+0000","last_clean":"2024-09-10T20:30:06.055205+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:30:06.055205+0000","last_undegraded":"2024-09-10T20:30:06.055205+0000","last_fullsized":"2024-09-10T20:30:06.055205+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":3681,"log_dups_size":0,"ondisk_log_size":3681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15048704,"num_objects":3674,"num_object_clones":0,"num_object_copies":7348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":677,"num_write_kb":2708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'3628","reported_seq":3759,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:06.037889+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:30:06.037889+0000","last_peered":"2024-09-10T20:30:06.037889+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:30:06.037889+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":3628,"log_dups_size":0,"ondisk_log_size":3628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14839808,"num_objects":3623,"num_object_clones":0,"num_object_copies":7246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":492,"num_write_kb":1968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":9},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'3582","reported_seq":3715,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:06.060861+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:30:06.060861+0000","last_peered":"2024-09-10T20:30:06.060861+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:30:06.060861+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":3582,"log_dups_size":0,"ondisk_log_size":3582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14647296,"num_objects":3576,"num_object_clones":0,"num_object_copies":7152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":177,"num_write_kb":708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":21},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'3654","reported_seq":3773,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:06.065305+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:30:06.065305+0000","last_peered":"2024-09-10T20:30:06.065305+0000","last_clean":"2024-09-10T20:30:06.065305+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:30:06.065305+0000","last_undegraded":"2024-09-10T20:30:06.065305+0000","last_fullsized":"2024-09-10T20:30:06.065305+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":3654,"log_dups_size":0,"ondisk_log_size":3654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14958592,"num_objects":3652,"num_object_clones":0,"num_object_copies":7304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":725,"num_write_kb":2900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":91,"reported_epoch":39,"state":"active+clean+remapped","last_fresh":"2024-09-10T20:29:59.254405+0000","last_change":"2024-09-10T20:29:59.254253+0000","last_active":"2024-09-10T20:29:59.254405+0000","last_peered":"2024-09-10T20:29:59.254405+0000","last_clean":"2024-09-10T20:29:59.254405+0000","last_became_active":"2024-09-10T20:29:59.253047+0000","last_became_peered":"2024-09-10T20:29:59.253047+0000","last_unstale":"2024-09-10T20:29:59.254405+0000","last_undegraded":"2024-09-10T20:29:59.254405+0000","last_fullsized":"2024-09-10T20:29:59.254405+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:44.668186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"40'3516","reported_seq":3635,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T20:30:03.020216+0000","last_change":"2024-09-10T20:29:59.265690+0000","last_active":"2024-09-10T20:30:03.020216+0000","last_peered":"2024-09-10T20:30:03.020216+0000","last_clean":"2024-09-10T20:30:03.020216+0000","last_became_active":"2024-09-10T20:29:59.261575+0000","last_became_peered":"2024-09-10T20:29:59.261575+0000","last_unstale":"2024-09-10T20:30:03.020216+0000","last_undegraded":"2024-09-10T20:30:03.020216+0000","last_fullsized":"2024-09-10T20:30:03.020216+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":3516,"log_dups_size":0,"ondisk_log_size":3516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:38:59.574765+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14372864,"num_objects":3509,"num_object_clones":0,"num_object_copies":7018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3509,"num_objects_unfound":0,"num_objects_dirty":3509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1509,"num_write_kb":6036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3509}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:30:02.301409+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:02.301409+0000","last_peered":"2024-09-10T20:30:02.301409+0000","last_clean":"2024-09-10T20:30:02.301409+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:02.301409+0000","last_undegraded":"2024-09-10T20:30:02.301409+0000","last_fullsized":"2024-09-10T20:30:02.301409+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'3638","reported_seq":3739,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:06.084363+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:30:06.084363+0000","last_peered":"2024-09-10T20:30:06.084363+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:30:06.084363+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":3638,"log_dups_size":0,"ondisk_log_size":3638,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14869446,"num_objects":3632,"num_object_clones":0,"num_object_copies":7264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3636,"num_write_kb":14524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":17},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.413176+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.413176+0000","last_peered":"2024-09-10T20:30:05.413176+0000","last_clean":"2024-09-10T20:30:05.413176+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.413176+0000","last_undegraded":"2024-09-10T20:30:05.413176+0000","last_fullsized":"2024-09-10T20:30:05.413176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"40'3622","reported_seq":3734,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T20:30:03.008352+0000","last_change":"2024-09-10T20:29:59.258677+0000","last_active":"2024-09-10T20:30:03.008352+0000","last_peered":"2024-09-10T20:30:03.008352+0000","last_clean":"2024-09-10T20:30:03.008352+0000","last_became_active":"2024-09-10T20:29:59.253659+0000","last_became_peered":"2024-09-10T20:29:59.253659+0000","last_unstale":"2024-09-10T20:30:03.008352+0000","last_undegraded":"2024-09-10T20:30:03.008352+0000","last_fullsized":"2024-09-10T20:30:03.008352+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":3622,"log_dups_size":0,"ondisk_log_size":3622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:33.522383+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14815232,"num_objects":3617,"num_object_clones":0,"num_object_copies":7234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3617,"num_objects_unfound":0,"num_objects_dirty":3617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1450,"num_write_kb":5800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3617}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":105,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T20:30:02.299141+0000","last_change":"2024-09-10T20:29:59.262015+0000","last_active":"2024-09-10T20:30:02.299141+0000","last_peered":"2024-09-10T20:30:02.299141+0000","last_clean":"2024-09-10T20:30:02.299141+0000","last_became_active":"2024-09-10T20:29:59.261632+0000","last_became_peered":"2024-09-10T20:29:59.261632+0000","last_unstale":"2024-09-10T20:30:02.299141+0000","last_undegraded":"2024-09-10T20:30:02.299141+0000","last_fullsized":"2024-09-10T20:30:02.299141+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:15.907091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"40'3594","reported_seq":3708,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T20:30:03.020270+0000","last_change":"2024-09-10T20:29:59.259062+0000","last_active":"2024-09-10T20:30:03.020270+0000","last_peered":"2024-09-10T20:30:03.020270+0000","last_clean":"2024-09-10T20:30:03.020270+0000","last_became_active":"2024-09-10T20:29:59.253560+0000","last_became_peered":"2024-09-10T20:29:59.253560+0000","last_unstale":"2024-09-10T20:30:03.020270+0000","last_undegraded":"2024-09-10T20:30:03.020270+0000","last_fullsized":"2024-09-10T20:30:03.020270+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":3594,"log_dups_size":0,"ondisk_log_size":3594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:28.056718+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14696448,"num_objects":3588,"num_object_clones":0,"num_object_copies":7176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3588,"num_objects_unfound":0,"num_objects_dirty":3588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1380,"num_write_kb":5520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3588}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'3660","reported_seq":3790,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:06.004424+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:30:06.004424+0000","last_peered":"2024-09-10T20:30:06.004424+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:30:06.004424+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":3660,"log_dups_size":0,"ondisk_log_size":3660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14983168,"num_objects":3658,"num_object_clones":0,"num_object_copies":7316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":383,"num_write_kb":1532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":10},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":91,"reported_epoch":39,"state":"active+clean+remapped","last_fresh":"2024-09-10T20:29:59.262308+0000","last_change":"2024-09-10T20:29:59.262201+0000","last_active":"2024-09-10T20:29:59.262308+0000","last_peered":"2024-09-10T20:29:59.262308+0000","last_clean":"2024-09-10T20:29:59.262308+0000","last_became_active":"2024-09-10T20:29:59.261897+0000","last_became_peered":"2024-09-10T20:29:59.261897+0000","last_unstale":"2024-09-10T20:29:59.262308+0000","last_undegraded":"2024-09-10T20:29:59.262308+0000","last_fullsized":"2024-09-10T20:29:59.262308+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:46:21.332712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"43'3557","reported_seq":3690,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:06.050870+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:30:06.050870+0000","last_peered":"2024-09-10T20:30:06.050870+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:30:06.050870+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":3557,"log_dups_size":0,"ondisk_log_size":3557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14536704,"num_objects":3549,"num_object_clones":0,"num_object_copies":7098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":251,"num_write_kb":1004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":17},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:30:02.297984+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:02.297984+0000","last_peered":"2024-09-10T20:30:02.297984+0000","last_clean":"2024-09-10T20:30:02.297984+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:02.297984+0000","last_undegraded":"2024-09-10T20:30:02.297984+0000","last_fullsized":"2024-09-10T20:30:02.297984+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'3736","reported_seq":3861,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:06.067845+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:30:06.067845+0000","last_peered":"2024-09-10T20:30:06.067845+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:30:06.067845+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":3736,"log_dups_size":0,"ondisk_log_size":3736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15286272,"num_objects":3732,"num_object_clones":0,"num_object_copies":7464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":563,"num_write_kb":2252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":19},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":91,"reported_epoch":39,"state":"active+clean+remapped","last_fresh":"2024-09-10T20:29:59.255443+0000","last_change":"2024-09-10T20:29:59.255326+0000","last_active":"2024-09-10T20:29:59.255443+0000","last_peered":"2024-09-10T20:29:59.255443+0000","last_clean":"2024-09-10T20:29:59.255443+0000","last_became_active":"2024-09-10T20:29:59.254983+0000","last_became_peered":"2024-09-10T20:29:59.254983+0000","last_unstale":"2024-09-10T20:29:59.255443+0000","last_undegraded":"2024-09-10T20:29:59.255443+0000","last_fullsized":"2024-09-10T20:29:59.255443+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:47.725162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":100,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T20:30:02.298723+0000","last_change":"2024-09-10T20:29:59.278181+0000","last_active":"2024-09-10T20:30:02.298723+0000","last_peered":"2024-09-10T20:30:02.298723+0000","last_clean":"2024-09-10T20:30:02.298723+0000","last_became_active":"2024-09-10T20:29:59.277822+0000","last_became_peered":"2024-09-10T20:29:59.277822+0000","last_unstale":"2024-09-10T20:30:02.298723+0000","last_undegraded":"2024-09-10T20:30:02.298723+0000","last_fullsized":"2024-09-10T20:30:02.298723+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:18:55.527871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"43'3493","reported_seq":3653,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:05.987311+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:30:05.987311+0000","last_peered":"2024-09-10T20:30:05.987311+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:30:05.987311+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":3493,"log_dups_size":0,"ondisk_log_size":3493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14258659,"num_objects":3482,"num_object_clones":0,"num_object_copies":6964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100,"num_write_kb":400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":14},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":236217769,"num_objects":57673,"num_object_clones":0,"num_object_copies":115346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28767,"num_objects_misplaced":14299,"num_objects_unfound":0,"num_objects_dirty":57673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22048,"num_write_kb":88172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":471683072,"data_stored":471574682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57752,"ondisk_log_size":57752,"up":28,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":11,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182419,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":370768,"kb_used_data":230844,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":94001072,"statfs":{"total":96636764160,"available":96257097728,"internally_reserved":0,"allocated":236384256,"data_stored":236151123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":143257258},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,17],"upper_bound":4096},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986190,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":350196,"kb_used_data":228256,"kb_used_omap":4,"kb_used_meta":121915,"kb_avail":94021644,"statfs":{"total":96636764160,"available":96278163456,"internally_reserved":0,"allocated":233734144,"data_stored":233514258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":124841642},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,8],"upper_bound":4096},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":0,"seq":1,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":332,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":339968,"data_stored":86725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":28176046},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":236806144,"data_stored":236802531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":234762240,"data_stored":234758627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:30:07.330 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:30:07.886 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:07.888 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:30:07.914 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2024-09-10T20:30:06.831829+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":236217788,"num_objects":57675,"num_object_clones":0,"num_object_copies":115350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28767,"num_objects_misplaced":14300,"num_objects_unfound":0,"num_objects_dirty":57675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22050,"num_write_kb":88174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":4115,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57755,"ondisk_log_size":57755,"up":39,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":72,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":748832,"kb_used_data":459432,"kb_used_omap":12,"kb_used_meta":289331,"kb_avail":282366688,"statfs":{"total":289910292480,"available":289143488512,"internally_reserved":0,"allocated":470458368,"data_stored":469752106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":296274946},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,25],"upper_bound":4096},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":14603921,"num_objects":3561,"num_object_clones":0,"num_object_copies":7122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28767,"num_objects_misplaced":-13150,"num_objects_unfound":0,"num_objects_dirty":3561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-3227,"num_write_kb":-12867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-6,"num_bytes_recovered":-483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.545794"},"pg_stats":[{"pgid":"2.f","version":"43'3624","reported_seq":3698,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:06.055566+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:30:06.055566+0000","last_peered":"2024-09-10T20:30:06.055566+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:30:06.055566+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":3624,"log_dups_size":0,"ondisk_log_size":3624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14843904,"num_objects":3624,"num_object_clones":0,"num_object_copies":7248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3624,"num_write_kb":14496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":17},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"40'3635","reported_seq":3734,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:30:03.024080+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:30:03.024080+0000","last_peered":"2024-09-10T20:30:03.024080+0000","last_clean":"2024-09-10T20:30:03.024080+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:30:03.024080+0000","last_undegraded":"2024-09-10T20:30:03.024080+0000","last_fullsized":"2024-09-10T20:30:03.024080+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":3635,"log_dups_size":0,"ondisk_log_size":3635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14864384,"num_objects":3629,"num_object_clones":0,"num_object_copies":7258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1821,"num_write_kb":7284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"40'3589","reported_seq":3705,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T20:30:03.034166+0000","last_change":"2024-09-10T20:29:59.267508+0000","last_active":"2024-09-10T20:30:03.034166+0000","last_peered":"2024-09-10T20:30:03.034166+0000","last_clean":"2024-09-10T20:30:03.034166+0000","last_became_active":"2024-09-10T20:29:59.264128+0000","last_became_peered":"2024-09-10T20:29:59.264128+0000","last_unstale":"2024-09-10T20:30:03.034166+0000","last_undegraded":"2024-09-10T20:30:03.034166+0000","last_fullsized":"2024-09-10T20:30:03.034166+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":3589,"log_dups_size":0,"ondisk_log_size":3589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:30.100436+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14684160,"num_objects":3585,"num_object_clones":0,"num_object_copies":7170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3585,"num_objects_unfound":0,"num_objects_dirty":3585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1717,"num_write_kb":6868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3585}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"40'3543","reported_seq":3612,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:30:03.035556+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:30:03.035556+0000","last_peered":"2024-09-10T20:30:03.035556+0000","last_clean":"2024-09-10T20:30:03.035556+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:30:03.035556+0000","last_undegraded":"2024-09-10T20:30:03.035556+0000","last_fullsized":"2024-09-10T20:30:03.035556+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":3543,"log_dups_size":0,"ondisk_log_size":3543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14512128,"num_objects":3543,"num_object_clones":0,"num_object_copies":7086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3543,"num_write_kb":14172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'3681","reported_seq":3802,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:06.055205+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:30:06.055205+0000","last_peered":"2024-09-10T20:30:06.055205+0000","last_clean":"2024-09-10T20:30:06.055205+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:30:06.055205+0000","last_undegraded":"2024-09-10T20:30:06.055205+0000","last_fullsized":"2024-09-10T20:30:06.055205+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":3681,"log_dups_size":0,"ondisk_log_size":3681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15048704,"num_objects":3674,"num_object_clones":0,"num_object_copies":7348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":677,"num_write_kb":2708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'3628","reported_seq":3759,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:06.037889+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:30:06.037889+0000","last_peered":"2024-09-10T20:30:06.037889+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:30:06.037889+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":3628,"log_dups_size":0,"ondisk_log_size":3628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14839808,"num_objects":3623,"num_object_clones":0,"num_object_copies":7246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":492,"num_write_kb":1968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":9},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'3582","reported_seq":3715,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:06.060861+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:30:06.060861+0000","last_peered":"2024-09-10T20:30:06.060861+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:30:06.060861+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":3582,"log_dups_size":0,"ondisk_log_size":3582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14647296,"num_objects":3576,"num_object_clones":0,"num_object_copies":7152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":177,"num_write_kb":708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":21},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'3654","reported_seq":3773,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:06.065305+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:30:06.065305+0000","last_peered":"2024-09-10T20:30:06.065305+0000","last_clean":"2024-09-10T20:30:06.065305+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:30:06.065305+0000","last_undegraded":"2024-09-10T20:30:06.065305+0000","last_fullsized":"2024-09-10T20:30:06.065305+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":3654,"log_dups_size":0,"ondisk_log_size":3654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14958592,"num_objects":3652,"num_object_clones":0,"num_object_copies":7304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":725,"num_write_kb":2900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":91,"reported_epoch":39,"state":"active+clean+remapped","last_fresh":"2024-09-10T20:29:59.254405+0000","last_change":"2024-09-10T20:29:59.254253+0000","last_active":"2024-09-10T20:29:59.254405+0000","last_peered":"2024-09-10T20:29:59.254405+0000","last_clean":"2024-09-10T20:29:59.254405+0000","last_became_active":"2024-09-10T20:29:59.253047+0000","last_became_peered":"2024-09-10T20:29:59.253047+0000","last_unstale":"2024-09-10T20:29:59.254405+0000","last_undegraded":"2024-09-10T20:29:59.254405+0000","last_fullsized":"2024-09-10T20:29:59.254405+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:44.668186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"40'3516","reported_seq":3635,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T20:30:03.020216+0000","last_change":"2024-09-10T20:29:59.265690+0000","last_active":"2024-09-10T20:30:03.020216+0000","last_peered":"2024-09-10T20:30:03.020216+0000","last_clean":"2024-09-10T20:30:03.020216+0000","last_became_active":"2024-09-10T20:29:59.261575+0000","last_became_peered":"2024-09-10T20:29:59.261575+0000","last_unstale":"2024-09-10T20:30:03.020216+0000","last_undegraded":"2024-09-10T20:30:03.020216+0000","last_fullsized":"2024-09-10T20:30:03.020216+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":3516,"log_dups_size":0,"ondisk_log_size":3516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:38:59.574765+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14372864,"num_objects":3509,"num_object_clones":0,"num_object_copies":7018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3509,"num_objects_unfound":0,"num_objects_dirty":3509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1509,"num_write_kb":6036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3509}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:30:02.301409+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:02.301409+0000","last_peered":"2024-09-10T20:30:02.301409+0000","last_clean":"2024-09-10T20:30:02.301409+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:02.301409+0000","last_undegraded":"2024-09-10T20:30:02.301409+0000","last_fullsized":"2024-09-10T20:30:02.301409+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'3638","reported_seq":3739,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:06.084363+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:30:06.084363+0000","last_peered":"2024-09-10T20:30:06.084363+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:30:06.084363+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":3638,"log_dups_size":0,"ondisk_log_size":3638,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14869446,"num_objects":3632,"num_object_clones":0,"num_object_copies":7264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3636,"num_write_kb":14524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":17},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.413176+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.413176+0000","last_peered":"2024-09-10T20:30:05.413176+0000","last_clean":"2024-09-10T20:30:05.413176+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.413176+0000","last_undegraded":"2024-09-10T20:30:05.413176+0000","last_fullsized":"2024-09-10T20:30:05.413176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"40'3622","reported_seq":3734,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T20:30:03.008352+0000","last_change":"2024-09-10T20:29:59.258677+0000","last_active":"2024-09-10T20:30:03.008352+0000","last_peered":"2024-09-10T20:30:03.008352+0000","last_clean":"2024-09-10T20:30:03.008352+0000","last_became_active":"2024-09-10T20:29:59.253659+0000","last_became_peered":"2024-09-10T20:29:59.253659+0000","last_unstale":"2024-09-10T20:30:03.008352+0000","last_undegraded":"2024-09-10T20:30:03.008352+0000","last_fullsized":"2024-09-10T20:30:03.008352+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":3622,"log_dups_size":0,"ondisk_log_size":3622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:33.522383+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14815232,"num_objects":3617,"num_object_clones":0,"num_object_copies":7234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3617,"num_objects_unfound":0,"num_objects_dirty":3617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1450,"num_write_kb":5800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3617}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":105,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T20:30:02.299141+0000","last_change":"2024-09-10T20:29:59.262015+0000","last_active":"2024-09-10T20:30:02.299141+0000","last_peered":"2024-09-10T20:30:02.299141+0000","last_clean":"2024-09-10T20:30:02.299141+0000","last_became_active":"2024-09-10T20:29:59.261632+0000","last_became_peered":"2024-09-10T20:29:59.261632+0000","last_unstale":"2024-09-10T20:30:02.299141+0000","last_undegraded":"2024-09-10T20:30:02.299141+0000","last_fullsized":"2024-09-10T20:30:02.299141+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:15.907091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"40'3594","reported_seq":3708,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T20:30:03.020270+0000","last_change":"2024-09-10T20:29:59.259062+0000","last_active":"2024-09-10T20:30:03.020270+0000","last_peered":"2024-09-10T20:30:03.020270+0000","last_clean":"2024-09-10T20:30:03.020270+0000","last_became_active":"2024-09-10T20:29:59.253560+0000","last_became_peered":"2024-09-10T20:29:59.253560+0000","last_unstale":"2024-09-10T20:30:03.020270+0000","last_undegraded":"2024-09-10T20:30:03.020270+0000","last_fullsized":"2024-09-10T20:30:03.020270+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":3594,"log_dups_size":0,"ondisk_log_size":3594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:28.056718+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14696448,"num_objects":3588,"num_object_clones":0,"num_object_copies":7176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3588,"num_objects_unfound":0,"num_objects_dirty":3588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1380,"num_write_kb":5520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3588}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'3660","reported_seq":3790,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:06.004424+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:30:06.004424+0000","last_peered":"2024-09-10T20:30:06.004424+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:30:06.004424+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":3660,"log_dups_size":0,"ondisk_log_size":3660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14983168,"num_objects":3658,"num_object_clones":0,"num_object_copies":7316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":383,"num_write_kb":1532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":10},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":91,"reported_epoch":39,"state":"active+clean+remapped","last_fresh":"2024-09-10T20:29:59.262308+0000","last_change":"2024-09-10T20:29:59.262201+0000","last_active":"2024-09-10T20:29:59.262308+0000","last_peered":"2024-09-10T20:29:59.262308+0000","last_clean":"2024-09-10T20:29:59.262308+0000","last_became_active":"2024-09-10T20:29:59.261897+0000","last_became_peered":"2024-09-10T20:29:59.261897+0000","last_unstale":"2024-09-10T20:29:59.262308+0000","last_undegraded":"2024-09-10T20:29:59.262308+0000","last_fullsized":"2024-09-10T20:29:59.262308+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:46:21.332712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"43'3557","reported_seq":3690,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:06.050870+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:30:06.050870+0000","last_peered":"2024-09-10T20:30:06.050870+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:30:06.050870+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":3557,"log_dups_size":0,"ondisk_log_size":3557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14536704,"num_objects":3549,"num_object_clones":0,"num_object_copies":7098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":251,"num_write_kb":1004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":17},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:30:02.297984+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:02.297984+0000","last_peered":"2024-09-10T20:30:02.297984+0000","last_clean":"2024-09-10T20:30:02.297984+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:02.297984+0000","last_undegraded":"2024-09-10T20:30:02.297984+0000","last_fullsized":"2024-09-10T20:30:02.297984+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'3736","reported_seq":3861,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:06.067845+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:30:06.067845+0000","last_peered":"2024-09-10T20:30:06.067845+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:30:06.067845+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":3736,"log_dups_size":0,"ondisk_log_size":3736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15286272,"num_objects":3732,"num_object_clones":0,"num_object_copies":7464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":563,"num_write_kb":2252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":19},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":91,"reported_epoch":39,"state":"active+clean+remapped","last_fresh":"2024-09-10T20:29:59.255443+0000","last_change":"2024-09-10T20:29:59.255326+0000","last_active":"2024-09-10T20:29:59.255443+0000","last_peered":"2024-09-10T20:29:59.255443+0000","last_clean":"2024-09-10T20:29:59.255443+0000","last_became_active":"2024-09-10T20:29:59.254983+0000","last_became_peered":"2024-09-10T20:29:59.254983+0000","last_unstale":"2024-09-10T20:29:59.255443+0000","last_undegraded":"2024-09-10T20:29:59.255443+0000","last_fullsized":"2024-09-10T20:29:59.255443+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:47.725162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":100,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T20:30:02.298723+0000","last_change":"2024-09-10T20:29:59.278181+0000","last_active":"2024-09-10T20:30:02.298723+0000","last_peered":"2024-09-10T20:30:02.298723+0000","last_clean":"2024-09-10T20:30:02.298723+0000","last_became_active":"2024-09-10T20:29:59.277822+0000","last_became_peered":"2024-09-10T20:29:59.277822+0000","last_unstale":"2024-09-10T20:30:02.298723+0000","last_undegraded":"2024-09-10T20:30:02.298723+0000","last_fullsized":"2024-09-10T20:30:02.298723+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:18:55.527871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"43'3493","reported_seq":3653,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:05.987311+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:30:05.987311+0000","last_peered":"2024-09-10T20:30:05.987311+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:30:05.987311+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":3493,"log_dups_size":0,"ondisk_log_size":3493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14258659,"num_objects":3482,"num_object_clones":0,"num_object_copies":6964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100,"num_write_kb":400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":14},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":236217769,"num_objects":57673,"num_object_clones":0,"num_object_copies":115346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28767,"num_objects_misplaced":14299,"num_objects_unfound":0,"num_objects_dirty":57673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22048,"num_write_kb":88172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":471683072,"data_stored":471574682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57752,"ondisk_log_size":57752,"up":28,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":11,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182419,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":370768,"kb_used_data":230844,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":94001072,"statfs":{"total":96636764160,"available":96257097728,"internally_reserved":0,"allocated":236384256,"data_stored":236151123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":143257258},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,17],"upper_bound":4096},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986190,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":350196,"kb_used_data":228256,"kb_used_omap":4,"kb_used_meta":121915,"kb_avail":94021644,"statfs":{"total":96636764160,"available":96278163456,"internally_reserved":0,"allocated":233734144,"data_stored":233514258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":124841642},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,8],"upper_bound":4096},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":0,"seq":1,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":332,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":339968,"data_stored":86725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":28176046},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":236806144,"data_stored":236802531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":234762240,"data_stored":234758627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:30:07.915 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:30:08.036 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout:2024-09-10T20:30:08.032979+0000 min lat: 0.00191414 max lat: 2.51074 avg lat: 0.013525 2024-09-10T20:30:08.036 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T20:30:08.036 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 40 20 59138 59118 5.7724 2.48047 0.0409902 0.013525 2024-09-10T20:30:08.487 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:08.487 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:30:08.512 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2024-09-10T20:30:06.831829+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":236217788,"num_objects":57675,"num_object_clones":0,"num_object_copies":115350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28767,"num_objects_misplaced":14300,"num_objects_unfound":0,"num_objects_dirty":57675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22050,"num_write_kb":88174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":4115,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57755,"ondisk_log_size":57755,"up":39,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":72,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":748832,"kb_used_data":459432,"kb_used_omap":12,"kb_used_meta":289331,"kb_avail":282366688,"statfs":{"total":289910292480,"available":289143488512,"internally_reserved":0,"allocated":470458368,"data_stored":469752106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":296274946},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,25],"upper_bound":4096},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":14603921,"num_objects":3561,"num_object_clones":0,"num_object_copies":7122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28767,"num_objects_misplaced":-13150,"num_objects_unfound":0,"num_objects_dirty":3561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-3227,"num_write_kb":-12867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-6,"num_bytes_recovered":-483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.545794"},"pg_stats":[{"pgid":"2.f","version":"43'3624","reported_seq":3698,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:06.055566+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:30:06.055566+0000","last_peered":"2024-09-10T20:30:06.055566+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:30:06.055566+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":3624,"log_dups_size":0,"ondisk_log_size":3624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14843904,"num_objects":3624,"num_object_clones":0,"num_object_copies":7248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3624,"num_write_kb":14496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":17},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"40'3635","reported_seq":3734,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:30:03.024080+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:30:03.024080+0000","last_peered":"2024-09-10T20:30:03.024080+0000","last_clean":"2024-09-10T20:30:03.024080+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:30:03.024080+0000","last_undegraded":"2024-09-10T20:30:03.024080+0000","last_fullsized":"2024-09-10T20:30:03.024080+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":3635,"log_dups_size":0,"ondisk_log_size":3635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14864384,"num_objects":3629,"num_object_clones":0,"num_object_copies":7258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1821,"num_write_kb":7284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"40'3589","reported_seq":3705,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T20:30:03.034166+0000","last_change":"2024-09-10T20:29:59.267508+0000","last_active":"2024-09-10T20:30:03.034166+0000","last_peered":"2024-09-10T20:30:03.034166+0000","last_clean":"2024-09-10T20:30:03.034166+0000","last_became_active":"2024-09-10T20:29:59.264128+0000","last_became_peered":"2024-09-10T20:29:59.264128+0000","last_unstale":"2024-09-10T20:30:03.034166+0000","last_undegraded":"2024-09-10T20:30:03.034166+0000","last_fullsized":"2024-09-10T20:30:03.034166+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":3589,"log_dups_size":0,"ondisk_log_size":3589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:30.100436+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14684160,"num_objects":3585,"num_object_clones":0,"num_object_copies":7170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3585,"num_objects_unfound":0,"num_objects_dirty":3585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1717,"num_write_kb":6868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3585}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"40'3543","reported_seq":3612,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:30:03.035556+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:30:03.035556+0000","last_peered":"2024-09-10T20:30:03.035556+0000","last_clean":"2024-09-10T20:30:03.035556+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:30:03.035556+0000","last_undegraded":"2024-09-10T20:30:03.035556+0000","last_fullsized":"2024-09-10T20:30:03.035556+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":3543,"log_dups_size":0,"ondisk_log_size":3543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14512128,"num_objects":3543,"num_object_clones":0,"num_object_copies":7086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3543,"num_write_kb":14172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'3681","reported_seq":3802,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:06.055205+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:30:06.055205+0000","last_peered":"2024-09-10T20:30:06.055205+0000","last_clean":"2024-09-10T20:30:06.055205+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:30:06.055205+0000","last_undegraded":"2024-09-10T20:30:06.055205+0000","last_fullsized":"2024-09-10T20:30:06.055205+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":3681,"log_dups_size":0,"ondisk_log_size":3681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15048704,"num_objects":3674,"num_object_clones":0,"num_object_copies":7348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":677,"num_write_kb":2708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'3628","reported_seq":3759,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:06.037889+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:30:06.037889+0000","last_peered":"2024-09-10T20:30:06.037889+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:30:06.037889+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":3628,"log_dups_size":0,"ondisk_log_size":3628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14839808,"num_objects":3623,"num_object_clones":0,"num_object_copies":7246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":492,"num_write_kb":1968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":9},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'3582","reported_seq":3715,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:06.060861+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:30:06.060861+0000","last_peered":"2024-09-10T20:30:06.060861+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:30:06.060861+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":3582,"log_dups_size":0,"ondisk_log_size":3582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14647296,"num_objects":3576,"num_object_clones":0,"num_object_copies":7152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":177,"num_write_kb":708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":21},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'3654","reported_seq":3773,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:06.065305+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:30:06.065305+0000","last_peered":"2024-09-10T20:30:06.065305+0000","last_clean":"2024-09-10T20:30:06.065305+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:30:06.065305+0000","last_undegraded":"2024-09-10T20:30:06.065305+0000","last_fullsized":"2024-09-10T20:30:06.065305+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":3654,"log_dups_size":0,"ondisk_log_size":3654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14958592,"num_objects":3652,"num_object_clones":0,"num_object_copies":7304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":725,"num_write_kb":2900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":91,"reported_epoch":39,"state":"active+clean+remapped","last_fresh":"2024-09-10T20:29:59.254405+0000","last_change":"2024-09-10T20:29:59.254253+0000","last_active":"2024-09-10T20:29:59.254405+0000","last_peered":"2024-09-10T20:29:59.254405+0000","last_clean":"2024-09-10T20:29:59.254405+0000","last_became_active":"2024-09-10T20:29:59.253047+0000","last_became_peered":"2024-09-10T20:29:59.253047+0000","last_unstale":"2024-09-10T20:29:59.254405+0000","last_undegraded":"2024-09-10T20:29:59.254405+0000","last_fullsized":"2024-09-10T20:29:59.254405+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:44.668186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"40'3516","reported_seq":3635,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T20:30:03.020216+0000","last_change":"2024-09-10T20:29:59.265690+0000","last_active":"2024-09-10T20:30:03.020216+0000","last_peered":"2024-09-10T20:30:03.020216+0000","last_clean":"2024-09-10T20:30:03.020216+0000","last_became_active":"2024-09-10T20:29:59.261575+0000","last_became_peered":"2024-09-10T20:29:59.261575+0000","last_unstale":"2024-09-10T20:30:03.020216+0000","last_undegraded":"2024-09-10T20:30:03.020216+0000","last_fullsized":"2024-09-10T20:30:03.020216+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":3516,"log_dups_size":0,"ondisk_log_size":3516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:38:59.574765+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14372864,"num_objects":3509,"num_object_clones":0,"num_object_copies":7018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3509,"num_objects_unfound":0,"num_objects_dirty":3509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1509,"num_write_kb":6036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3509}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:30:02.301409+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:02.301409+0000","last_peered":"2024-09-10T20:30:02.301409+0000","last_clean":"2024-09-10T20:30:02.301409+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:02.301409+0000","last_undegraded":"2024-09-10T20:30:02.301409+0000","last_fullsized":"2024-09-10T20:30:02.301409+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'3638","reported_seq":3739,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:06.084363+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:30:06.084363+0000","last_peered":"2024-09-10T20:30:06.084363+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:30:06.084363+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":3638,"log_dups_size":0,"ondisk_log_size":3638,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14869446,"num_objects":3632,"num_object_clones":0,"num_object_copies":7264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3636,"num_write_kb":14524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":17},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.413176+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.413176+0000","last_peered":"2024-09-10T20:30:05.413176+0000","last_clean":"2024-09-10T20:30:05.413176+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.413176+0000","last_undegraded":"2024-09-10T20:30:05.413176+0000","last_fullsized":"2024-09-10T20:30:05.413176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"40'3622","reported_seq":3734,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T20:30:03.008352+0000","last_change":"2024-09-10T20:29:59.258677+0000","last_active":"2024-09-10T20:30:03.008352+0000","last_peered":"2024-09-10T20:30:03.008352+0000","last_clean":"2024-09-10T20:30:03.008352+0000","last_became_active":"2024-09-10T20:29:59.253659+0000","last_became_peered":"2024-09-10T20:29:59.253659+0000","last_unstale":"2024-09-10T20:30:03.008352+0000","last_undegraded":"2024-09-10T20:30:03.008352+0000","last_fullsized":"2024-09-10T20:30:03.008352+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":3622,"log_dups_size":0,"ondisk_log_size":3622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:33.522383+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14815232,"num_objects":3617,"num_object_clones":0,"num_object_copies":7234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3617,"num_objects_unfound":0,"num_objects_dirty":3617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1450,"num_write_kb":5800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3617}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":105,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T20:30:02.299141+0000","last_change":"2024-09-10T20:29:59.262015+0000","last_active":"2024-09-10T20:30:02.299141+0000","last_peered":"2024-09-10T20:30:02.299141+0000","last_clean":"2024-09-10T20:30:02.299141+0000","last_became_active":"2024-09-10T20:29:59.261632+0000","last_became_peered":"2024-09-10T20:29:59.261632+0000","last_unstale":"2024-09-10T20:30:02.299141+0000","last_undegraded":"2024-09-10T20:30:02.299141+0000","last_fullsized":"2024-09-10T20:30:02.299141+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:15.907091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"40'3594","reported_seq":3708,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T20:30:03.020270+0000","last_change":"2024-09-10T20:29:59.259062+0000","last_active":"2024-09-10T20:30:03.020270+0000","last_peered":"2024-09-10T20:30:03.020270+0000","last_clean":"2024-09-10T20:30:03.020270+0000","last_became_active":"2024-09-10T20:29:59.253560+0000","last_became_peered":"2024-09-10T20:29:59.253560+0000","last_unstale":"2024-09-10T20:30:03.020270+0000","last_undegraded":"2024-09-10T20:30:03.020270+0000","last_fullsized":"2024-09-10T20:30:03.020270+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":3594,"log_dups_size":0,"ondisk_log_size":3594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:28.056718+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14696448,"num_objects":3588,"num_object_clones":0,"num_object_copies":7176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3588,"num_objects_unfound":0,"num_objects_dirty":3588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1380,"num_write_kb":5520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3588}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'3660","reported_seq":3790,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:06.004424+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:30:06.004424+0000","last_peered":"2024-09-10T20:30:06.004424+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:30:06.004424+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":3660,"log_dups_size":0,"ondisk_log_size":3660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14983168,"num_objects":3658,"num_object_clones":0,"num_object_copies":7316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":383,"num_write_kb":1532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":10},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":91,"reported_epoch":39,"state":"active+clean+remapped","last_fresh":"2024-09-10T20:29:59.262308+0000","last_change":"2024-09-10T20:29:59.262201+0000","last_active":"2024-09-10T20:29:59.262308+0000","last_peered":"2024-09-10T20:29:59.262308+0000","last_clean":"2024-09-10T20:29:59.262308+0000","last_became_active":"2024-09-10T20:29:59.261897+0000","last_became_peered":"2024-09-10T20:29:59.261897+0000","last_unstale":"2024-09-10T20:29:59.262308+0000","last_undegraded":"2024-09-10T20:29:59.262308+0000","last_fullsized":"2024-09-10T20:29:59.262308+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:46:21.332712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"43'3557","reported_seq":3690,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:06.050870+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:30:06.050870+0000","last_peered":"2024-09-10T20:30:06.050870+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:30:06.050870+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":3557,"log_dups_size":0,"ondisk_log_size":3557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14536704,"num_objects":3549,"num_object_clones":0,"num_object_copies":7098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":251,"num_write_kb":1004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":17},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:30:02.297984+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:02.297984+0000","last_peered":"2024-09-10T20:30:02.297984+0000","last_clean":"2024-09-10T20:30:02.297984+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:02.297984+0000","last_undegraded":"2024-09-10T20:30:02.297984+0000","last_fullsized":"2024-09-10T20:30:02.297984+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'3736","reported_seq":3861,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:06.067845+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:30:06.067845+0000","last_peered":"2024-09-10T20:30:06.067845+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:30:06.067845+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":3736,"log_dups_size":0,"ondisk_log_size":3736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15286272,"num_objects":3732,"num_object_clones":0,"num_object_copies":7464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":563,"num_write_kb":2252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":19},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":91,"reported_epoch":39,"state":"active+clean+remapped","last_fresh":"2024-09-10T20:29:59.255443+0000","last_change":"2024-09-10T20:29:59.255326+0000","last_active":"2024-09-10T20:29:59.255443+0000","last_peered":"2024-09-10T20:29:59.255443+0000","last_clean":"2024-09-10T20:29:59.255443+0000","last_became_active":"2024-09-10T20:29:59.254983+0000","last_became_peered":"2024-09-10T20:29:59.254983+0000","last_unstale":"2024-09-10T20:29:59.255443+0000","last_undegraded":"2024-09-10T20:29:59.255443+0000","last_fullsized":"2024-09-10T20:29:59.255443+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:47.725162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":100,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T20:30:02.298723+0000","last_change":"2024-09-10T20:29:59.278181+0000","last_active":"2024-09-10T20:30:02.298723+0000","last_peered":"2024-09-10T20:30:02.298723+0000","last_clean":"2024-09-10T20:30:02.298723+0000","last_became_active":"2024-09-10T20:29:59.277822+0000","last_became_peered":"2024-09-10T20:29:59.277822+0000","last_unstale":"2024-09-10T20:30:02.298723+0000","last_undegraded":"2024-09-10T20:30:02.298723+0000","last_fullsized":"2024-09-10T20:30:02.298723+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:18:55.527871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"43'3493","reported_seq":3653,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:05.987311+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:30:05.987311+0000","last_peered":"2024-09-10T20:30:05.987311+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:30:05.987311+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":3493,"log_dups_size":0,"ondisk_log_size":3493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14258659,"num_objects":3482,"num_object_clones":0,"num_object_copies":6964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100,"num_write_kb":400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":14},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":236217769,"num_objects":57673,"num_object_clones":0,"num_object_copies":115346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28767,"num_objects_misplaced":14299,"num_objects_unfound":0,"num_objects_dirty":57673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22048,"num_write_kb":88172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":471683072,"data_stored":471574682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57752,"ondisk_log_size":57752,"up":28,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":11,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182419,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":370768,"kb_used_data":230844,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":94001072,"statfs":{"total":96636764160,"available":96257097728,"internally_reserved":0,"allocated":236384256,"data_stored":236151123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":143257258},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,17],"upper_bound":4096},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986190,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":350196,"kb_used_data":228256,"kb_used_omap":4,"kb_used_meta":121915,"kb_avail":94021644,"statfs":{"total":96636764160,"available":96278163456,"internally_reserved":0,"allocated":233734144,"data_stored":233514258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":124841642},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,8],"upper_bound":4096},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":0,"seq":1,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":332,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":339968,"data_stored":86725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":28176046},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":236806144,"data_stored":236802531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":234762240,"data_stored":234758627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:30:08.513 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:30:08.513 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'3624", 'reported_seq': 3698, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:06.055566+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:30:06.055566+0000', 'last_peered': '2024-09-10T20:30:06.055566+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:30:06.055566+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 3624, 'log_dups_size': 0, 'ondisk_log_size': 3624, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14843904, 'num_objects': 3624, 'num_object_clones': 0, 'num_object_copies': 7248, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3607, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3624, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3624, 'num_write_kb': 14496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 17}, {'shards': '0,2', 'objects': 3607}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:08.513 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T20:30:08.513 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "40'3589", 'reported_seq': 3705, 'reported_epoch': 40, 'state': 'active+clean+remapped', 'last_fresh': '2024-09-10T20:30:03.034166+0000', 'last_change': '2024-09-10T20:29:59.267508+0000', 'last_active': '2024-09-10T20:30:03.034166+0000', 'last_peered': '2024-09-10T20:30:03.034166+0000', 'last_clean': '2024-09-10T20:30:03.034166+0000', 'last_became_active': '2024-09-10T20:29:59.264128+0000', 'last_became_peered': '2024-09-10T20:29:59.264128+0000', 'last_unstale': '2024-09-10T20:30:03.034166+0000', 'last_undegraded': '2024-09-10T20:30:03.034166+0000', 'last_fullsized': '2024-09-10T20:30:03.034166+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1910", 'last_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'objects_scrubbed': 1868, 'log_size': 3589, 'log_dups_size': 0, 'ondisk_log_size': 3589, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-12T00:32:30.100436+0000', 'scrub_duration': 673, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14684160, 'num_objects': 3585, 'num_object_clones': 0, 'num_object_copies': 7170, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3585, 'num_objects_unfound': 0, 'num_objects_dirty': 3585, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1717, 'num_write_kb': 6868, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2], 'acting': [2, 0], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3585}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:08.513 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:30:08.513 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'3628", 'reported_seq': 3759, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:06.037889+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:30:06.037889+0000', 'last_peered': '2024-09-10T20:30:06.037889+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:30:06.037889+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 3628, 'log_dups_size': 0, 'ondisk_log_size': 3628, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14839808, 'num_objects': 3623, 'num_object_clones': 0, 'num_object_copies': 7246, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3616, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3623, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 492, 'num_write_kb': 1968, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 9}, {'shards': '0,2', 'objects': 3614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:08.514 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:30:08.514 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'3582", 'reported_seq': 3715, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:06.060861+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:30:06.060861+0000', 'last_peered': '2024-09-10T20:30:06.060861+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:30:06.060861+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 3582, 'log_dups_size': 0, 'ondisk_log_size': 3582, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14647296, 'num_objects': 3576, 'num_object_clones': 0, 'num_object_copies': 7152, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3557, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3576, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 177, 'num_write_kb': 708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 21}, {'shards': '0,2', 'objects': 3555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:08.514 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 1.0 is not active+clean 2024-09-10T20:30:08.514 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 91, 'reported_epoch': 39, 'state': 'active+clean+remapped', 'last_fresh': '2024-09-10T20:29:59.254405+0000', 'last_change': '2024-09-10T20:29:59.254253+0000', 'last_active': '2024-09-10T20:29:59.254405+0000', 'last_peered': '2024-09-10T20:29:59.254405+0000', 'last_clean': '2024-09-10T20:29:59.254405+0000', 'last_became_active': '2024-09-10T20:29:59.253047+0000', 'last_became_peered': '2024-09-10T20:29:59.253047+0000', 'last_unstale': '2024-09-10T20:29:59.254405+0000', 'last_undegraded': '2024-09-10T20:29:59.254405+0000', 'last_fullsized': '2024-09-10T20:29:59.254405+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 11, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:24.477689+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:24.477689+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:24.477689+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-12T05:15:44.668186+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000479713, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0], 'acting': [0, 2], 'avail_no_missing': ['0', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:08.514 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:30:08.514 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "40'3516", 'reported_seq': 3635, 'reported_epoch': 40, 'state': 'active+clean+remapped', 'last_fresh': '2024-09-10T20:30:03.020216+0000', 'last_change': '2024-09-10T20:29:59.265690+0000', 'last_active': '2024-09-10T20:30:03.020216+0000', 'last_peered': '2024-09-10T20:30:03.020216+0000', 'last_clean': '2024-09-10T20:30:03.020216+0000', 'last_became_active': '2024-09-10T20:29:59.261575+0000', 'last_became_peered': '2024-09-10T20:29:59.261575+0000', 'last_unstale': '2024-09-10T20:30:03.020216+0000', 'last_undegraded': '2024-09-10T20:30:03.020216+0000', 'last_fullsized': '2024-09-10T20:30:03.020216+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 3516, 'log_dups_size': 0, 'ondisk_log_size': 3516, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-12T01:38:59.574765+0000', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14372864, 'num_objects': 3509, 'num_object_clones': 0, 'num_object_copies': 7018, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3509, 'num_objects_unfound': 0, 'num_objects_dirty': 3509, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1509, 'num_write_kb': 6036, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2], 'acting': [2, 0], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3509}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:08.514 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:30:08.514 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'3638", 'reported_seq': 3739, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:06.084363+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:30:06.084363+0000', 'last_peered': '2024-09-10T20:30:06.084363+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:30:06.084363+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 3638, 'log_dups_size': 0, 'ondisk_log_size': 3638, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14869446, 'num_objects': 3632, 'num_object_clones': 0, 'num_object_copies': 7264, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3617, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3632, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3636, 'num_write_kb': 14524, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 17}, {'shards': '0,2', 'objects': 3615}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:08.514 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:30:08.514 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "40'3622", 'reported_seq': 3734, 'reported_epoch': 40, 'state': 'active+clean+remapped', 'last_fresh': '2024-09-10T20:30:03.008352+0000', 'last_change': '2024-09-10T20:29:59.258677+0000', 'last_active': '2024-09-10T20:30:03.008352+0000', 'last_peered': '2024-09-10T20:30:03.008352+0000', 'last_clean': '2024-09-10T20:30:03.008352+0000', 'last_became_active': '2024-09-10T20:29:59.253659+0000', 'last_became_peered': '2024-09-10T20:29:59.253659+0000', 'last_unstale': '2024-09-10T20:30:03.008352+0000', 'last_undegraded': '2024-09-10T20:30:03.008352+0000', 'last_fullsized': '2024-09-10T20:30:03.008352+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 3622, 'log_dups_size': 0, 'ondisk_log_size': 3622, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-11T21:50:33.522383+0000', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14815232, 'num_objects': 3617, 'num_object_clones': 0, 'num_object_copies': 7234, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3617, 'num_objects_unfound': 0, 'num_objects_dirty': 3617, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1450, 'num_write_kb': 5800, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2], 'acting': [2, 0], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3617}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:08.515 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 1.3 is not active+clean 2024-09-10T20:30:08.515 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '1.3', 'version': "12'1", 'reported_seq': 105, 'reported_epoch': 40, 'state': 'active+clean+remapped', 'last_fresh': '2024-09-10T20:30:02.299141+0000', 'last_change': '2024-09-10T20:29:59.262015+0000', 'last_active': '2024-09-10T20:30:02.299141+0000', 'last_peered': '2024-09-10T20:30:02.299141+0000', 'last_clean': '2024-09-10T20:30:02.299141+0000', 'last_became_active': '2024-09-10T20:29:59.261632+0000', 'last_became_peered': '2024-09-10T20:29:59.261632+0000', 'last_unstale': '2024-09-10T20:30:02.299141+0000', 'last_undegraded': '2024-09-10T20:30:02.299141+0000', 'last_fullsized': '2024-09-10T20:30:02.299141+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 11, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:24.477689+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:24.477689+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:24.477689+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-12T03:26:15.907091+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000500755, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2], 'acting': [2, 0], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:08.515 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:30:08.515 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "40'3594", 'reported_seq': 3708, 'reported_epoch': 40, 'state': 'active+clean+remapped', 'last_fresh': '2024-09-10T20:30:03.020270+0000', 'last_change': '2024-09-10T20:29:59.259062+0000', 'last_active': '2024-09-10T20:30:03.020270+0000', 'last_peered': '2024-09-10T20:30:03.020270+0000', 'last_clean': '2024-09-10T20:30:03.020270+0000', 'last_became_active': '2024-09-10T20:29:59.253560+0000', 'last_became_peered': '2024-09-10T20:29:59.253560+0000', 'last_unstale': '2024-09-10T20:30:03.020270+0000', 'last_undegraded': '2024-09-10T20:30:03.020270+0000', 'last_fullsized': '2024-09-10T20:30:03.020270+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 3594, 'log_dups_size': 0, 'ondisk_log_size': 3594, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-12T06:27:28.056718+0000', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14696448, 'num_objects': 3588, 'num_object_clones': 0, 'num_object_copies': 7176, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3588, 'num_objects_unfound': 0, 'num_objects_dirty': 3588, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1380, 'num_write_kb': 5520, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2], 'acting': [2, 0], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3588}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:08.515 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:30:08.515 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'3660", 'reported_seq': 3790, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:06.004424+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:30:06.004424+0000', 'last_peered': '2024-09-10T20:30:06.004424+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:30:06.004424+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 3660, 'log_dups_size': 0, 'ondisk_log_size': 3660, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14983168, 'num_objects': 3658, 'num_object_clones': 0, 'num_object_copies': 7316, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3649, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3658, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 383, 'num_write_kb': 1532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 10}, {'shards': '0,2', 'objects': 3648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:08.515 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 1.4 is not active+clean 2024-09-10T20:30:08.515 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '1.4', 'version': "0'0", 'reported_seq': 91, 'reported_epoch': 39, 'state': 'active+clean+remapped', 'last_fresh': '2024-09-10T20:29:59.262308+0000', 'last_change': '2024-09-10T20:29:59.262201+0000', 'last_active': '2024-09-10T20:29:59.262308+0000', 'last_peered': '2024-09-10T20:29:59.262308+0000', 'last_clean': '2024-09-10T20:29:59.262308+0000', 'last_became_active': '2024-09-10T20:29:59.261897+0000', 'last_became_peered': '2024-09-10T20:29:59.261897+0000', 'last_unstale': '2024-09-10T20:29:59.262308+0000', 'last_undegraded': '2024-09-10T20:29:59.262308+0000', 'last_fullsized': '2024-09-10T20:29:59.262308+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 11, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:24.477689+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:24.477689+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:24.477689+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-12T02:46:21.332712+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.00071858, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0], 'acting': [0, 2], 'avail_no_missing': ['0', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:08.515 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:30:08.515 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'3557", 'reported_seq': 3690, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:06.050870+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:30:06.050870+0000', 'last_peered': '2024-09-10T20:30:06.050870+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:30:06.050870+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 3557, 'log_dups_size': 0, 'ondisk_log_size': 3557, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14536704, 'num_objects': 3549, 'num_object_clones': 0, 'num_object_copies': 7098, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3535, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3549, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 251, 'num_write_kb': 1004, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 17}, {'shards': '0,2', 'objects': 3532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:08.515 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:30:08.515 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'3736", 'reported_seq': 3861, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:06.067845+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:30:06.067845+0000', 'last_peered': '2024-09-10T20:30:06.067845+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:30:06.067845+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 3736, 'log_dups_size': 0, 'ondisk_log_size': 3736, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15286272, 'num_objects': 3732, 'num_object_clones': 0, 'num_object_copies': 7464, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3715, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3732, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 563, 'num_write_kb': 2252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 19}, {'shards': '0,2', 'objects': 3713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:08.516 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 1.6 is not active+clean 2024-09-10T20:30:08.516 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '1.6', 'version': "0'0", 'reported_seq': 91, 'reported_epoch': 39, 'state': 'active+clean+remapped', 'last_fresh': '2024-09-10T20:29:59.255443+0000', 'last_change': '2024-09-10T20:29:59.255326+0000', 'last_active': '2024-09-10T20:29:59.255443+0000', 'last_peered': '2024-09-10T20:29:59.255443+0000', 'last_clean': '2024-09-10T20:29:59.255443+0000', 'last_became_active': '2024-09-10T20:29:59.254983+0000', 'last_became_peered': '2024-09-10T20:29:59.254983+0000', 'last_unstale': '2024-09-10T20:29:59.255443+0000', 'last_undegraded': '2024-09-10T20:29:59.255443+0000', 'last_fullsized': '2024-09-10T20:29:59.255443+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 11, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:24.477689+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:24.477689+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:24.477689+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-12T03:38:47.725162+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000668863, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0], 'acting': [0, 2], 'avail_no_missing': ['0', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:08.516 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 1.7 is not active+clean 2024-09-10T20:30:08.516 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '1.7', 'version': "0'0", 'reported_seq': 100, 'reported_epoch': 40, 'state': 'active+clean+remapped', 'last_fresh': '2024-09-10T20:30:02.298723+0000', 'last_change': '2024-09-10T20:29:59.278181+0000', 'last_active': '2024-09-10T20:30:02.298723+0000', 'last_peered': '2024-09-10T20:30:02.298723+0000', 'last_clean': '2024-09-10T20:30:02.298723+0000', 'last_became_active': '2024-09-10T20:29:59.277822+0000', 'last_became_peered': '2024-09-10T20:29:59.277822+0000', 'last_unstale': '2024-09-10T20:30:02.298723+0000', 'last_undegraded': '2024-09-10T20:30:02.298723+0000', 'last_fullsized': '2024-09-10T20:30:02.298723+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 11, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:24.477689+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:24.477689+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:24.477689+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-12T06:18:55.527871+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000810878, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2], 'acting': [2, 0], 'avail_no_missing': ['2', '0'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:08.516 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:30:08.516 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'3493", 'reported_seq': 3653, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:05.987311+0000', 'last_change': '2024-09-10T20:30:05.690055+0000', 'last_active': '2024-09-10T20:30:05.987311+0000', 'last_peered': '2024-09-10T20:30:05.987311+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:30:05.987311+0000', 'last_undegraded': '2024-09-10T20:30:05.378941+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 3493, 'log_dups_size': 0, 'ondisk_log_size': 3493, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14258659, 'num_objects': 3482, 'num_object_clones': 0, 'num_object_copies': 6964, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3471, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3482, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 100, 'num_write_kb': 400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 14}, {'shards': '0,2', 'objects': 3468}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:08.516 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:30:09.036 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 41 20 59781 59761 5.69287 2.51172 0.036049 0.013713 2024-09-10T20:30:09.207 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:09.207 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 28767/115350 objects degraded (24.939%), 8 pgs degraded","count":8},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":120,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean+remapped","count":9},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+clean","count":7}],"num_pgs":24,"num_pools":2,"num_objects":57675,"data_bytes":236217788,"bytes_used":766803968,"bytes_avail":289143488512,"bytes_total":289910292480,"degraded_objects":28767,"degraded_total":115350,"degraded_ratio":0.24938881664499349,"misplaced_objects":14300,"misplaced_total":115350,"misplaced_ratio":0.12397052449068054},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-10T20:29:56.830168+0000","services":{"osd":{"daemons":{"summary":"","2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T20:30:09.236 INFO:tasks.rep_lost_unfound_delete.ceph_manager:no progress seen, keeping timeout for now 2024-09-10T20:30:09.236 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:30:09.762 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:09.763 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:30:09.796 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2024-09-10T20:30:08.832286+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":238982588,"num_objects":58350,"num_object_clones":0,"num_object_copies":116700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":3548,"num_objects_unfound":0,"num_objects_dirty":58350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22725,"num_write_kb":90874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":229395,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58430,"ondisk_log_size":58430,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":64,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":801428,"kb_used_data":467612,"kb_used_omap":12,"kb_used_meta":333747,"kb_avail":282314092,"statfs":{"total":289910292480,"available":289089630208,"internally_reserved":0,"allocated":478834688,"data_stored":478059880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":341756930},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,2,6,3,5,0,0,0,0,17],"upper_bound":4096},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17368721,"num_objects":4236,"num_object_clones":0,"num_object_copies":8472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":-23902,"num_objects_unfound":0,"num_objects_dirty":4236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-2552,"num_write_kb":-10167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":224797,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001224"},"pg_stats":[{"pgid":"2.f","version":"43'3624","reported_seq":3698,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:06.055566+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:30:06.055566+0000","last_peered":"2024-09-10T20:30:06.055566+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:30:06.055566+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":3624,"log_dups_size":0,"ondisk_log_size":3624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14843904,"num_objects":3624,"num_object_clones":0,"num_object_copies":7248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3624,"num_write_kb":14496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":17},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'3733","reported_seq":3838,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:08.012924+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:30:08.012924+0000","last_peered":"2024-09-10T20:30:08.012924+0000","last_clean":"2024-09-10T20:30:08.012924+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:30:08.012924+0000","last_undegraded":"2024-09-10T20:30:08.012924+0000","last_fullsized":"2024-09-10T20:30:08.012924+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":3733,"log_dups_size":0,"ondisk_log_size":3733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15265792,"num_objects":3727,"num_object_clones":0,"num_object_copies":7454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1919,"num_write_kb":7676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'3697","reported_seq":3932,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:30:08.020393+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:30:08.020393+0000","last_peered":"2024-09-10T20:30:08.020393+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:30:08.020393+0000","last_undegraded":"2024-09-10T20:30:08.020393+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":3697,"log_dups_size":0,"ondisk_log_size":3697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15126528,"num_objects":3693,"num_object_clones":0,"num_object_copies":7386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3548,"num_objects_unfound":0,"num_objects_dirty":3693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1825,"num_write_kb":7300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":225280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3548},{"shards":"2","objects":145}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'3658","reported_seq":3733,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:08.017847+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:30:08.017847+0000","last_peered":"2024-09-10T20:30:08.017847+0000","last_clean":"2024-09-10T20:30:08.017847+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:30:08.017847+0000","last_undegraded":"2024-09-10T20:30:08.017847+0000","last_fullsized":"2024-09-10T20:30:08.017847+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":3658,"log_dups_size":0,"ondisk_log_size":3658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14983168,"num_objects":3658,"num_object_clones":0,"num_object_copies":7316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3658,"num_write_kb":14632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'3681","reported_seq":3802,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:06.055205+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:30:06.055205+0000","last_peered":"2024-09-10T20:30:06.055205+0000","last_clean":"2024-09-10T20:30:06.055205+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:30:06.055205+0000","last_undegraded":"2024-09-10T20:30:06.055205+0000","last_fullsized":"2024-09-10T20:30:06.055205+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":3681,"log_dups_size":0,"ondisk_log_size":3681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15048704,"num_objects":3674,"num_object_clones":0,"num_object_copies":7348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":677,"num_write_kb":2708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'3628","reported_seq":3759,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:06.037889+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:30:06.037889+0000","last_peered":"2024-09-10T20:30:06.037889+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:30:06.037889+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":3628,"log_dups_size":0,"ondisk_log_size":3628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14839808,"num_objects":3623,"num_object_clones":0,"num_object_copies":7246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":492,"num_write_kb":1968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":9},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'3582","reported_seq":3715,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:06.060861+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:30:06.060861+0000","last_peered":"2024-09-10T20:30:06.060861+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:30:06.060861+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":3582,"log_dups_size":0,"ondisk_log_size":3582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14647296,"num_objects":3576,"num_object_clones":0,"num_object_copies":7152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":177,"num_write_kb":708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":21},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'3654","reported_seq":3773,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:06.065305+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:30:06.065305+0000","last_peered":"2024-09-10T20:30:06.065305+0000","last_clean":"2024-09-10T20:30:06.065305+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:30:06.065305+0000","last_undegraded":"2024-09-10T20:30:06.065305+0000","last_fullsized":"2024-09-10T20:30:06.065305+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":3654,"log_dups_size":0,"ondisk_log_size":3654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14958592,"num_objects":3652,"num_object_clones":0,"num_object_copies":7304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":725,"num_write_kb":2900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'3644","reported_seq":3771,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:08.033022+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:30:08.033022+0000","last_peered":"2024-09-10T20:30:08.033022+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:30:08.033022+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":3644,"log_dups_size":0,"ondisk_log_size":3644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14897152,"num_objects":3637,"num_object_clones":0,"num_object_copies":7274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1637,"num_write_kb":6548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":111}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355438+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355438+0000","last_peered":"2024-09-10T20:30:05.355438+0000","last_clean":"2024-09-10T20:30:05.355438+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355438+0000","last_undegraded":"2024-09-10T20:30:05.355438+0000","last_fullsized":"2024-09-10T20:30:05.355438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'3638","reported_seq":3739,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:06.084363+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:30:06.084363+0000","last_peered":"2024-09-10T20:30:06.084363+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:30:06.084363+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":3638,"log_dups_size":0,"ondisk_log_size":3638,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14869446,"num_objects":3632,"num_object_clones":0,"num_object_copies":7264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3636,"num_write_kb":14524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":17},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.413176+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.413176+0000","last_peered":"2024-09-10T20:30:05.413176+0000","last_clean":"2024-09-10T20:30:05.413176+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.413176+0000","last_undegraded":"2024-09-10T20:30:05.413176+0000","last_fullsized":"2024-09-10T20:30:05.413176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'3738","reported_seq":3861,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:07.945539+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:30:07.945539+0000","last_peered":"2024-09-10T20:30:07.945539+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:30:07.945539+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":3738,"log_dups_size":0,"ondisk_log_size":3738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15290368,"num_objects":3733,"num_object_clones":0,"num_object_copies":7466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3733,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1566,"num_write_kb":6264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":100}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'3704","reported_seq":3827,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:07.981951+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:30:07.981951+0000","last_peered":"2024-09-10T20:30:07.981951+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:30:07.981951+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":3704,"log_dups_size":0,"ondisk_log_size":3704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15147008,"num_objects":3698,"num_object_clones":0,"num_object_copies":7396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3698,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1490,"num_write_kb":5960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":88}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'3660","reported_seq":3790,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:06.004424+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:30:06.004424+0000","last_peered":"2024-09-10T20:30:06.004424+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:30:06.004424+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":3660,"log_dups_size":0,"ondisk_log_size":3660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14983168,"num_objects":3658,"num_object_clones":0,"num_object_copies":7316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":383,"num_write_kb":1532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":10},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'3557","reported_seq":3690,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:06.050870+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:30:06.050870+0000","last_peered":"2024-09-10T20:30:06.050870+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:30:06.050870+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":3557,"log_dups_size":0,"ondisk_log_size":3557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14536704,"num_objects":3549,"num_object_clones":0,"num_object_copies":7098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":251,"num_write_kb":1004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":17},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345705+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345705+0000","last_peered":"2024-09-10T20:30:05.345705+0000","last_clean":"2024-09-10T20:30:05.345705+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345705+0000","last_undegraded":"2024-09-10T20:30:05.345705+0000","last_fullsized":"2024-09-10T20:30:05.345705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'3736","reported_seq":3861,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:06.067845+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:30:06.067845+0000","last_peered":"2024-09-10T20:30:06.067845+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:30:06.067845+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":3736,"log_dups_size":0,"ondisk_log_size":3736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15286272,"num_objects":3732,"num_object_clones":0,"num_object_copies":7464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":563,"num_write_kb":2252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":19},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'3493","reported_seq":3653,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:05.987311+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:30:05.987311+0000","last_peered":"2024-09-10T20:30:05.987311+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:30:05.987311+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":3493,"log_dups_size":0,"ondisk_log_size":3493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14258659,"num_objects":3482,"num_object_clones":0,"num_object_copies":6964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100,"num_write_kb":400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":14},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":238982569,"num_objects":58348,"num_object_clones":0,"num_object_copies":116696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":3548,"num_objects_unfound":0,"num_objects_dirty":58348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22723,"num_write_kb":90872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":229376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":479440896,"data_stored":479350571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58427,"ondisk_log_size":58427,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182419,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":370768,"kb_used_data":230844,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":94001072,"statfs":{"total":96636764160,"available":96257097728,"internally_reserved":0,"allocated":236384256,"data_stored":236151123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":143257258},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,17],"upper_bound":4096},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986191,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":377632,"kb_used_data":233488,"kb_used_omap":4,"kb_used_meta":144123,"kb_avail":93994208,"statfs":{"total":96636764160,"available":96250068992,"internally_reserved":0,"allocated":239091712,"data_stored":238862656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":147582634},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,4,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659138,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53028,"kb_used_data":3280,"kb_used_omap":4,"kb_used_meta":49723,"kb_avail":94318812,"statfs":{"total":96636764160,"available":96582463488,"internally_reserved":0,"allocated":3358720,"data_stored":3046101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":50917038},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,2,1,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":236806144,"data_stored":236802531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3461120,"data_stored":3378021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":239173632,"data_stored":239170019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:30:10.036 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 42 20 60433 60413 5.61796 2.54688 0.0579989 0.0139004 2024-09-10T20:30:11.036 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 43 20 61158 61138 5.55317 2.83203 0.0342189 0.0140607 2024-09-10T20:30:12.037 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 44 20 61823 61803 5.486 2.59766 0.0559114 0.0142308 2024-09-10T20:30:12.798 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:30:13.036 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 45 20 62488 62468 5.42182 2.59766 0.0121276 0.0144017 2024-09-10T20:30:13.338 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:13.341 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:30:13.364 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":99,"stamp":"2024-09-10T20:30:12.834240+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":247473596,"num_objects":60423,"num_object_clones":0,"num_object_copies":120846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":3548,"num_objects_unfound":0,"num_objects_dirty":60423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24798,"num_write_kb":99166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":229395,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":60503,"ondisk_log_size":60503,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":834252,"kb_used_data":487964,"kb_used_omap":12,"kb_used_meta":346227,"kb_avail":282281268,"statfs":{"total":289910292480,"available":289056018432,"internally_reserved":0,"allocated":499675136,"data_stored":498900328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":354536450},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,4,16,7,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18818705,"num_objects":4590,"num_object_clones":0,"num_object_copies":9180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":-38202,"num_objects_unfound":0,"num_objects_dirty":4590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-2198,"num_write_kb":-8751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":224797,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.532858"},"pg_stats":[{"pgid":"2.f","version":"43'3837","reported_seq":3912,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:11.041956+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:30:11.041956+0000","last_peered":"2024-09-10T20:30:11.041956+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:30:11.041956+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":3837,"log_dups_size":0,"ondisk_log_size":3837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15716352,"num_objects":3837,"num_object_clones":0,"num_object_copies":7674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3837,"num_write_kb":15348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":230},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'3733","reported_seq":3838,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:08.012924+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:30:08.012924+0000","last_peered":"2024-09-10T20:30:08.012924+0000","last_clean":"2024-09-10T20:30:08.012924+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:30:08.012924+0000","last_undegraded":"2024-09-10T20:30:08.012924+0000","last_fullsized":"2024-09-10T20:30:08.012924+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":3733,"log_dups_size":0,"ondisk_log_size":3733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15265792,"num_objects":3727,"num_object_clones":0,"num_object_copies":7454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1919,"num_write_kb":7676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'3697","reported_seq":3932,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:30:08.020393+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:30:08.020393+0000","last_peered":"2024-09-10T20:30:08.020393+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:30:08.020393+0000","last_undegraded":"2024-09-10T20:30:08.020393+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":3697,"log_dups_size":0,"ondisk_log_size":3697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15126528,"num_objects":3693,"num_object_clones":0,"num_object_copies":7386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3548,"num_objects_unfound":0,"num_objects_dirty":3693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1825,"num_write_kb":7300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":225280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3548},{"shards":"2","objects":145}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'3658","reported_seq":3733,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:08.017847+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:30:08.017847+0000","last_peered":"2024-09-10T20:30:08.017847+0000","last_clean":"2024-09-10T20:30:08.017847+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:30:08.017847+0000","last_undegraded":"2024-09-10T20:30:08.017847+0000","last_fullsized":"2024-09-10T20:30:08.017847+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":3658,"log_dups_size":0,"ondisk_log_size":3658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14983168,"num_objects":3658,"num_object_clones":0,"num_object_copies":7316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3658,"num_write_kb":14632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'3901","reported_seq":4022,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:11.080692+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:30:11.080692+0000","last_peered":"2024-09-10T20:30:11.080692+0000","last_clean":"2024-09-10T20:30:11.080692+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:30:11.080692+0000","last_undegraded":"2024-09-10T20:30:11.080692+0000","last_fullsized":"2024-09-10T20:30:11.080692+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":3901,"log_dups_size":0,"ondisk_log_size":3901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15949824,"num_objects":3894,"num_object_clones":0,"num_object_copies":7788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":897,"num_write_kb":3588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'3841","reported_seq":3972,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:11.042799+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:30:11.042799+0000","last_peered":"2024-09-10T20:30:11.042799+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:30:11.042799+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":3841,"log_dups_size":0,"ondisk_log_size":3841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15712256,"num_objects":3836,"num_object_clones":0,"num_object_copies":7672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3836,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":705,"num_write_kb":2820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":222},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'3783","reported_seq":3916,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:11.039478+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:30:11.039478+0000","last_peered":"2024-09-10T20:30:11.039478+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:30:11.039478+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":3783,"log_dups_size":0,"ondisk_log_size":3783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15470592,"num_objects":3777,"num_object_clones":0,"num_object_copies":7554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":378,"num_write_kb":1512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":222},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'3858","reported_seq":3977,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:11.076129+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:30:11.076129+0000","last_peered":"2024-09-10T20:30:11.076129+0000","last_clean":"2024-09-10T20:30:11.076129+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:30:11.076129+0000","last_undegraded":"2024-09-10T20:30:11.076129+0000","last_fullsized":"2024-09-10T20:30:11.076129+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":3858,"log_dups_size":0,"ondisk_log_size":3858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15794176,"num_objects":3856,"num_object_clones":0,"num_object_copies":7712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":929,"num_write_kb":3716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'3644","reported_seq":3771,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:08.033022+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:30:08.033022+0000","last_peered":"2024-09-10T20:30:08.033022+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:30:08.033022+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":3644,"log_dups_size":0,"ondisk_log_size":3644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14897152,"num_objects":3637,"num_object_clones":0,"num_object_copies":7274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1637,"num_write_kb":6548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":111}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355438+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355438+0000","last_peered":"2024-09-10T20:30:05.355438+0000","last_clean":"2024-09-10T20:30:05.355438+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355438+0000","last_undegraded":"2024-09-10T20:30:05.355438+0000","last_fullsized":"2024-09-10T20:30:05.355438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'3855","reported_seq":3958,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:11.069348+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:30:11.069348+0000","last_peered":"2024-09-10T20:30:11.069348+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:30:11.069348+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":3855,"log_dups_size":0,"ondisk_log_size":3855,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15758278,"num_objects":3849,"num_object_clones":0,"num_object_copies":7698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3853,"num_write_kb":15392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":234},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.413176+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.413176+0000","last_peered":"2024-09-10T20:30:05.413176+0000","last_clean":"2024-09-10T20:30:05.413176+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.413176+0000","last_undegraded":"2024-09-10T20:30:05.413176+0000","last_fullsized":"2024-09-10T20:30:05.413176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'3738","reported_seq":3861,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:07.945539+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:30:07.945539+0000","last_peered":"2024-09-10T20:30:07.945539+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:30:07.945539+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":3738,"log_dups_size":0,"ondisk_log_size":3738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15290368,"num_objects":3733,"num_object_clones":0,"num_object_copies":7466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3733,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1566,"num_write_kb":6264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":100}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'3704","reported_seq":3827,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:07.981951+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:30:07.981951+0000","last_peered":"2024-09-10T20:30:07.981951+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:30:07.981951+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":3704,"log_dups_size":0,"ondisk_log_size":3704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15147008,"num_objects":3698,"num_object_clones":0,"num_object_copies":7396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3698,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1490,"num_write_kb":5960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":88}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'3844","reported_seq":3974,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:11.086183+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:30:11.086183+0000","last_peered":"2024-09-10T20:30:11.086183+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:30:11.086183+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":3844,"log_dups_size":0,"ondisk_log_size":3844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15736832,"num_objects":3842,"num_object_clones":0,"num_object_copies":7684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":567,"num_write_kb":2268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":194},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'3771","reported_seq":3904,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:11.065098+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:30:11.065098+0000","last_peered":"2024-09-10T20:30:11.065098+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:30:11.065098+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":3771,"log_dups_size":0,"ondisk_log_size":3771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15413248,"num_objects":3763,"num_object_clones":0,"num_object_copies":7526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":465,"num_write_kb":1860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":231},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345705+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345705+0000","last_peered":"2024-09-10T20:30:05.345705+0000","last_clean":"2024-09-10T20:30:05.345705+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345705+0000","last_undegraded":"2024-09-10T20:30:05.345705+0000","last_fullsized":"2024-09-10T20:30:05.345705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'3942","reported_seq":4068,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:11.069497+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:30:11.069497+0000","last_peered":"2024-09-10T20:30:11.069497+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:30:11.069497+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":3942,"log_dups_size":0,"ondisk_log_size":3942,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16130048,"num_objects":3938,"num_object_clones":0,"num_object_copies":7876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":769,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":225},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'3694","reported_seq":3847,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:11.085529+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:30:11.085529+0000","last_peered":"2024-09-10T20:30:11.085529+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:30:11.085529+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":3694,"log_dups_size":0,"ondisk_log_size":3694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15081955,"num_objects":3683,"num_object_clones":0,"num_object_copies":7366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":301,"num_write_kb":1204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":215},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":247473577,"num_objects":60421,"num_object_clones":0,"num_object_copies":120842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":3548,"num_objects_unfound":0,"num_objects_dirty":60421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24796,"num_write_kb":99164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":229376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":500445184,"data_stored":500354859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":60500,"ondisk_log_size":60500,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182420,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":384736,"kb_used_data":240656,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93987104,"statfs":{"total":96636764160,"available":96242794496,"internally_reserved":0,"allocated":246431744,"data_stored":246198611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":147517098},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,3,5,2,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986191,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":377632,"kb_used_data":233488,"kb_used_omap":4,"kb_used_meta":144123,"kb_avail":93994208,"statfs":{"total":96636764160,"available":96250068992,"internally_reserved":0,"allocated":239091712,"data_stored":238862656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":147582634},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,4,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659139,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71884,"kb_used_data":13820,"kb_used_omap":4,"kb_used_meta":58043,"kb_avail":94299956,"statfs":{"total":96636764160,"available":96563154944,"internally_reserved":0,"allocated":14151680,"data_stored":13839061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":59436718},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,7,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":247083008,"data_stored":247079395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":14188544,"data_stored":14105445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":239173632,"data_stored":239170019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:30:13.365 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:30:13.919 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:13.920 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:30:13.949 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":99,"stamp":"2024-09-10T20:30:12.834240+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":247473596,"num_objects":60423,"num_object_clones":0,"num_object_copies":120846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":3548,"num_objects_unfound":0,"num_objects_dirty":60423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24798,"num_write_kb":99166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":229395,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":60503,"ondisk_log_size":60503,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":834252,"kb_used_data":487964,"kb_used_omap":12,"kb_used_meta":346227,"kb_avail":282281268,"statfs":{"total":289910292480,"available":289056018432,"internally_reserved":0,"allocated":499675136,"data_stored":498900328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":354536450},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,4,16,7,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18818705,"num_objects":4590,"num_object_clones":0,"num_object_copies":9180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":-38202,"num_objects_unfound":0,"num_objects_dirty":4590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-2198,"num_write_kb":-8751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":224797,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.532858"},"pg_stats":[{"pgid":"2.f","version":"43'3837","reported_seq":3912,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:11.041956+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:30:11.041956+0000","last_peered":"2024-09-10T20:30:11.041956+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:30:11.041956+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":3837,"log_dups_size":0,"ondisk_log_size":3837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15716352,"num_objects":3837,"num_object_clones":0,"num_object_copies":7674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3837,"num_write_kb":15348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":230},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'3733","reported_seq":3838,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:08.012924+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:30:08.012924+0000","last_peered":"2024-09-10T20:30:08.012924+0000","last_clean":"2024-09-10T20:30:08.012924+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:30:08.012924+0000","last_undegraded":"2024-09-10T20:30:08.012924+0000","last_fullsized":"2024-09-10T20:30:08.012924+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":3733,"log_dups_size":0,"ondisk_log_size":3733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15265792,"num_objects":3727,"num_object_clones":0,"num_object_copies":7454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1919,"num_write_kb":7676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'3697","reported_seq":3932,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:30:08.020393+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:30:08.020393+0000","last_peered":"2024-09-10T20:30:08.020393+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:30:08.020393+0000","last_undegraded":"2024-09-10T20:30:08.020393+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":3697,"log_dups_size":0,"ondisk_log_size":3697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15126528,"num_objects":3693,"num_object_clones":0,"num_object_copies":7386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3548,"num_objects_unfound":0,"num_objects_dirty":3693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1825,"num_write_kb":7300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":225280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3548},{"shards":"2","objects":145}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'3658","reported_seq":3733,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:08.017847+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:30:08.017847+0000","last_peered":"2024-09-10T20:30:08.017847+0000","last_clean":"2024-09-10T20:30:08.017847+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:30:08.017847+0000","last_undegraded":"2024-09-10T20:30:08.017847+0000","last_fullsized":"2024-09-10T20:30:08.017847+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":3658,"log_dups_size":0,"ondisk_log_size":3658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14983168,"num_objects":3658,"num_object_clones":0,"num_object_copies":7316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3658,"num_write_kb":14632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'3901","reported_seq":4022,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:11.080692+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:30:11.080692+0000","last_peered":"2024-09-10T20:30:11.080692+0000","last_clean":"2024-09-10T20:30:11.080692+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:30:11.080692+0000","last_undegraded":"2024-09-10T20:30:11.080692+0000","last_fullsized":"2024-09-10T20:30:11.080692+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":3901,"log_dups_size":0,"ondisk_log_size":3901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15949824,"num_objects":3894,"num_object_clones":0,"num_object_copies":7788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":897,"num_write_kb":3588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'3841","reported_seq":3972,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:11.042799+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:30:11.042799+0000","last_peered":"2024-09-10T20:30:11.042799+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:30:11.042799+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":3841,"log_dups_size":0,"ondisk_log_size":3841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15712256,"num_objects":3836,"num_object_clones":0,"num_object_copies":7672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3836,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":705,"num_write_kb":2820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":222},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'3783","reported_seq":3916,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:11.039478+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:30:11.039478+0000","last_peered":"2024-09-10T20:30:11.039478+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:30:11.039478+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":3783,"log_dups_size":0,"ondisk_log_size":3783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15470592,"num_objects":3777,"num_object_clones":0,"num_object_copies":7554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":378,"num_write_kb":1512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":222},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'3858","reported_seq":3977,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:11.076129+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:30:11.076129+0000","last_peered":"2024-09-10T20:30:11.076129+0000","last_clean":"2024-09-10T20:30:11.076129+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:30:11.076129+0000","last_undegraded":"2024-09-10T20:30:11.076129+0000","last_fullsized":"2024-09-10T20:30:11.076129+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":3858,"log_dups_size":0,"ondisk_log_size":3858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15794176,"num_objects":3856,"num_object_clones":0,"num_object_copies":7712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":929,"num_write_kb":3716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'3644","reported_seq":3771,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:08.033022+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:30:08.033022+0000","last_peered":"2024-09-10T20:30:08.033022+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:30:08.033022+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":3644,"log_dups_size":0,"ondisk_log_size":3644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14897152,"num_objects":3637,"num_object_clones":0,"num_object_copies":7274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1637,"num_write_kb":6548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":111}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355438+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355438+0000","last_peered":"2024-09-10T20:30:05.355438+0000","last_clean":"2024-09-10T20:30:05.355438+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355438+0000","last_undegraded":"2024-09-10T20:30:05.355438+0000","last_fullsized":"2024-09-10T20:30:05.355438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'3855","reported_seq":3958,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:11.069348+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:30:11.069348+0000","last_peered":"2024-09-10T20:30:11.069348+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:30:11.069348+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":3855,"log_dups_size":0,"ondisk_log_size":3855,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15758278,"num_objects":3849,"num_object_clones":0,"num_object_copies":7698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3853,"num_write_kb":15392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":234},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.413176+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.413176+0000","last_peered":"2024-09-10T20:30:05.413176+0000","last_clean":"2024-09-10T20:30:05.413176+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.413176+0000","last_undegraded":"2024-09-10T20:30:05.413176+0000","last_fullsized":"2024-09-10T20:30:05.413176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'3738","reported_seq":3861,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:07.945539+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:30:07.945539+0000","last_peered":"2024-09-10T20:30:07.945539+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:30:07.945539+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":3738,"log_dups_size":0,"ondisk_log_size":3738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15290368,"num_objects":3733,"num_object_clones":0,"num_object_copies":7466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3733,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1566,"num_write_kb":6264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":100}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'3704","reported_seq":3827,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:07.981951+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:30:07.981951+0000","last_peered":"2024-09-10T20:30:07.981951+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:30:07.981951+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":3704,"log_dups_size":0,"ondisk_log_size":3704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15147008,"num_objects":3698,"num_object_clones":0,"num_object_copies":7396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3698,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1490,"num_write_kb":5960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":88}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'3844","reported_seq":3974,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:11.086183+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:30:11.086183+0000","last_peered":"2024-09-10T20:30:11.086183+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:30:11.086183+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":3844,"log_dups_size":0,"ondisk_log_size":3844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15736832,"num_objects":3842,"num_object_clones":0,"num_object_copies":7684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":567,"num_write_kb":2268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":194},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'3771","reported_seq":3904,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:11.065098+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:30:11.065098+0000","last_peered":"2024-09-10T20:30:11.065098+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:30:11.065098+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":3771,"log_dups_size":0,"ondisk_log_size":3771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15413248,"num_objects":3763,"num_object_clones":0,"num_object_copies":7526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":465,"num_write_kb":1860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":231},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345705+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345705+0000","last_peered":"2024-09-10T20:30:05.345705+0000","last_clean":"2024-09-10T20:30:05.345705+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345705+0000","last_undegraded":"2024-09-10T20:30:05.345705+0000","last_fullsized":"2024-09-10T20:30:05.345705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'3942","reported_seq":4068,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:11.069497+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:30:11.069497+0000","last_peered":"2024-09-10T20:30:11.069497+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:30:11.069497+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":3942,"log_dups_size":0,"ondisk_log_size":3942,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16130048,"num_objects":3938,"num_object_clones":0,"num_object_copies":7876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":769,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":225},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'3694","reported_seq":3847,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:11.085529+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:30:11.085529+0000","last_peered":"2024-09-10T20:30:11.085529+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:30:11.085529+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":3694,"log_dups_size":0,"ondisk_log_size":3694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15081955,"num_objects":3683,"num_object_clones":0,"num_object_copies":7366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":301,"num_write_kb":1204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":215},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":247473577,"num_objects":60421,"num_object_clones":0,"num_object_copies":120842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":3548,"num_objects_unfound":0,"num_objects_dirty":60421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24796,"num_write_kb":99164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":229376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":500445184,"data_stored":500354859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":60500,"ondisk_log_size":60500,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182420,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":384736,"kb_used_data":240656,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93987104,"statfs":{"total":96636764160,"available":96242794496,"internally_reserved":0,"allocated":246431744,"data_stored":246198611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":147517098},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,3,5,2,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986191,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":377632,"kb_used_data":233488,"kb_used_omap":4,"kb_used_meta":144123,"kb_avail":93994208,"statfs":{"total":96636764160,"available":96250068992,"internally_reserved":0,"allocated":239091712,"data_stored":238862656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":147582634},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,4,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659139,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71884,"kb_used_data":13820,"kb_used_omap":4,"kb_used_meta":58043,"kb_avail":94299956,"statfs":{"total":96636764160,"available":96563154944,"internally_reserved":0,"allocated":14151680,"data_stored":13839061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":59436718},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,7,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":247083008,"data_stored":247079395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":14188544,"data_stored":14105445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":239173632,"data_stored":239170019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:30:13.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:30:13.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'3837", 'reported_seq': 3912, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:11.041956+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:30:11.041956+0000', 'last_peered': '2024-09-10T20:30:11.041956+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:30:11.041956+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 3837, 'log_dups_size': 0, 'ondisk_log_size': 3837, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15716352, 'num_objects': 3837, 'num_object_clones': 0, 'num_object_copies': 7674, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3607, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3837, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3837, 'num_write_kb': 15348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 230}, {'shards': '0,2', 'objects': 3607}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:13.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T20:30:13.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'3697", 'reported_seq': 3932, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:30:08.020393+0000', 'last_change': '2024-09-10T20:30:05.653560+0000', 'last_active': '2024-09-10T20:30:08.020393+0000', 'last_peered': '2024-09-10T20:30:08.020393+0000', 'last_clean': '2024-09-10T20:30:03.279259+0000', 'last_became_active': '2024-09-10T20:30:05.629664+0000', 'last_became_peered': '2024-09-10T20:30:05.629664+0000', 'last_unstale': '2024-09-10T20:30:08.020393+0000', 'last_undegraded': '2024-09-10T20:30:08.020393+0000', 'last_fullsized': '2024-09-10T20:30:05.312061+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1910", 'last_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'objects_scrubbed': 1868, 'log_size': 3697, 'log_dups_size': 0, 'ondisk_log_size': 3697, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 673, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15126528, 'num_objects': 3693, 'num_object_clones': 0, 'num_object_copies': 7386, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3548, 'num_objects_unfound': 0, 'num_objects_dirty': 3693, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1825, 'num_write_kb': 7300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 55, 'num_bytes_recovered': 225280, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3548}, {'shards': '2', 'objects': 145}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:13.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:30:13.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'3841", 'reported_seq': 3972, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:11.042799+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:30:11.042799+0000', 'last_peered': '2024-09-10T20:30:11.042799+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:30:11.042799+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 3841, 'log_dups_size': 0, 'ondisk_log_size': 3841, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15712256, 'num_objects': 3836, 'num_object_clones': 0, 'num_object_copies': 7672, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3616, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3836, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 705, 'num_write_kb': 2820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 222}, {'shards': '0,2', 'objects': 3614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:13.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:30:13.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'3783", 'reported_seq': 3916, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:11.039478+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:30:11.039478+0000', 'last_peered': '2024-09-10T20:30:11.039478+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:30:11.039478+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 3783, 'log_dups_size': 0, 'ondisk_log_size': 3783, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15470592, 'num_objects': 3777, 'num_object_clones': 0, 'num_object_copies': 7554, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3557, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3777, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 378, 'num_write_kb': 1512, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 222}, {'shards': '0,2', 'objects': 3555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:13.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:30:13.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'3644", 'reported_seq': 3771, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:08.033022+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:30:08.033022+0000', 'last_peered': '2024-09-10T20:30:08.033022+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:30:08.033022+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 3644, 'log_dups_size': 0, 'ondisk_log_size': 3644, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14897152, 'num_objects': 3637, 'num_object_clones': 0, 'num_object_copies': 7274, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3528, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3637, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1637, 'num_write_kb': 6548, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 111}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:13.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:30:13.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'3855", 'reported_seq': 3958, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:11.069348+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:30:11.069348+0000', 'last_peered': '2024-09-10T20:30:11.069348+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:30:11.069348+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 3855, 'log_dups_size': 0, 'ondisk_log_size': 3855, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15758278, 'num_objects': 3849, 'num_object_clones': 0, 'num_object_copies': 7698, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3617, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3849, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3853, 'num_write_kb': 15392, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 234}, {'shards': '0,2', 'objects': 3615}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:13.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:30:13.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'3738", 'reported_seq': 3861, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:07.945539+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:30:07.945539+0000', 'last_peered': '2024-09-10T20:30:07.945539+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:30:07.945539+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 3738, 'log_dups_size': 0, 'ondisk_log_size': 3738, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15290368, 'num_objects': 3733, 'num_object_clones': 0, 'num_object_copies': 7466, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3635, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3733, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1566, 'num_write_kb': 6264, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3633}, {'shards': '2', 'objects': 100}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:13.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:30:13.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'3704", 'reported_seq': 3827, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:07.981951+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:30:07.981951+0000', 'last_peered': '2024-09-10T20:30:07.981951+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:30:07.981951+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 3704, 'log_dups_size': 0, 'ondisk_log_size': 3704, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15147008, 'num_objects': 3698, 'num_object_clones': 0, 'num_object_copies': 7396, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3698, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1490, 'num_write_kb': 5960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3610}, {'shards': '2', 'objects': 88}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:13.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:30:13.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'3844", 'reported_seq': 3974, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:11.086183+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:30:11.086183+0000', 'last_peered': '2024-09-10T20:30:11.086183+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:30:11.086183+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 3844, 'log_dups_size': 0, 'ondisk_log_size': 3844, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15736832, 'num_objects': 3842, 'num_object_clones': 0, 'num_object_copies': 7684, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3649, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3842, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 567, 'num_write_kb': 2268, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 194}, {'shards': '0,2', 'objects': 3648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:13.952 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:30:13.952 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'3771", 'reported_seq': 3904, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:11.065098+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:30:11.065098+0000', 'last_peered': '2024-09-10T20:30:11.065098+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:30:11.065098+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 3771, 'log_dups_size': 0, 'ondisk_log_size': 3771, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15413248, 'num_objects': 3763, 'num_object_clones': 0, 'num_object_copies': 7526, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3535, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3763, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 465, 'num_write_kb': 1860, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 231}, {'shards': '0,2', 'objects': 3532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:13.952 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:30:13.952 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'3942", 'reported_seq': 4068, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:11.069497+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:30:11.069497+0000', 'last_peered': '2024-09-10T20:30:11.069497+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:30:11.069497+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 3942, 'log_dups_size': 0, 'ondisk_log_size': 3942, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16130048, 'num_objects': 3938, 'num_object_clones': 0, 'num_object_copies': 7876, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3715, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3938, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 769, 'num_write_kb': 3076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 225}, {'shards': '0,2', 'objects': 3713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:13.952 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:30:13.952 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'3694", 'reported_seq': 3847, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:11.085529+0000', 'last_change': '2024-09-10T20:30:05.690055+0000', 'last_active': '2024-09-10T20:30:11.085529+0000', 'last_peered': '2024-09-10T20:30:11.085529+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:30:11.085529+0000', 'last_undegraded': '2024-09-10T20:30:05.378941+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 3694, 'log_dups_size': 0, 'ondisk_log_size': 3694, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15081955, 'num_objects': 3683, 'num_object_clones': 0, 'num_object_copies': 7366, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3471, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3683, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 301, 'num_write_kb': 1204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 215}, {'shards': '0,2', 'objects': 3468}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:13.952 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:30:14.036 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 46 20 63099 63079 5.35584 2.38672 0.0224056 0.0145771 2024-09-10T20:30:14.634 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:14.634 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39542/120846 objects degraded (32.721%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":126,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":60423,"data_bytes":247473596,"bytes_used":854274048,"bytes_avail":289056018432,"bytes_total":289910292480,"degraded_objects":39542,"degraded_total":120846,"degraded_ratio":0.32720983731360576,"misplaced_objects":3548,"misplaced_total":120846,"misplaced_ratio":0.029359680916207401,"recovering_objects_per_sec":5,"recovering_bytes_per_sec":26344,"recovering_keys_per_sec":0,"num_objects_recovered":49,"num_bytes_recovered":224797,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-10T20:29:56.830168+0000","services":{"osd":{"daemons":{"summary":"","2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T20:30:14.663 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:30:14.663 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:30:15.036 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 47 20 63732 63712 5.29449 2.47266 0.0312401 0.0147473 2024-09-10T20:30:15.227 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:15.228 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:30:15.255 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2024-09-10T20:30:14.834713+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":252560828,"num_objects":61665,"num_object_clones":0,"num_object_copies":123330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":3431,"num_objects_unfound":0,"num_objects_dirty":61665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26040,"num_write_kb":104134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":708627,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":61745,"ondisk_log_size":61745,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":845036,"kb_used_data":494588,"kb_used_omap":12,"kb_used_meta":350387,"kb_avail":282270484,"statfs":{"total":289910292480,"available":289044975616,"internally_reserved":0,"allocated":506458112,"data_stored":505683304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":358796290},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,2,0,6,13,10,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23905937,"num_objects":5832,"num_object_clones":0,"num_object_copies":11664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":-38319,"num_objects_unfound":0,"num_objects_dirty":5832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-956,"num_write_kb":-3783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":704029,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003244"},"pg_stats":[{"pgid":"2.f","version":"43'3837","reported_seq":3912,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:11.041956+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:30:11.041956+0000","last_peered":"2024-09-10T20:30:11.041956+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:30:11.041956+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":3837,"log_dups_size":0,"ondisk_log_size":3837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15716352,"num_objects":3837,"num_object_clones":0,"num_object_copies":7674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3837,"num_write_kb":15348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":230},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'3966","reported_seq":4071,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:13.016050+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:30:13.016050+0000","last_peered":"2024-09-10T20:30:13.016050+0000","last_clean":"2024-09-10T20:30:13.016050+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:30:13.016050+0000","last_undegraded":"2024-09-10T20:30:13.016050+0000","last_fullsized":"2024-09-10T20:30:13.016050+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":3966,"log_dups_size":0,"ondisk_log_size":3966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16220160,"num_objects":3960,"num_object_clones":0,"num_object_copies":7920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2152,"num_write_kb":8608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'3899","reported_seq":4368,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:30:13.056036+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:30:13.056036+0000","last_peered":"2024-09-10T20:30:13.056036+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:30:13.056036+0000","last_undegraded":"2024-09-10T20:30:13.056036+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":3899,"log_dups_size":0,"ondisk_log_size":3899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15953920,"num_objects":3895,"num_object_clones":0,"num_object_copies":7790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3431,"num_objects_unfound":0,"num_objects_dirty":3895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2027,"num_write_kb":8108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":704512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3431},{"shards":"2","objects":464}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'3870","reported_seq":3945,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:12.968917+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:30:12.968917+0000","last_peered":"2024-09-10T20:30:12.968917+0000","last_clean":"2024-09-10T20:30:12.968917+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:30:12.968917+0000","last_undegraded":"2024-09-10T20:30:12.968917+0000","last_fullsized":"2024-09-10T20:30:12.968917+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":3870,"log_dups_size":0,"ondisk_log_size":3870,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15851520,"num_objects":3870,"num_object_clones":0,"num_object_copies":7740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3870,"num_write_kb":15480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'3901","reported_seq":4022,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:11.080692+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:30:11.080692+0000","last_peered":"2024-09-10T20:30:11.080692+0000","last_clean":"2024-09-10T20:30:11.080692+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:30:11.080692+0000","last_undegraded":"2024-09-10T20:30:11.080692+0000","last_fullsized":"2024-09-10T20:30:11.080692+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":3901,"log_dups_size":0,"ondisk_log_size":3901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15949824,"num_objects":3894,"num_object_clones":0,"num_object_copies":7788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":897,"num_write_kb":3588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'3841","reported_seq":3972,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:11.042799+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:30:11.042799+0000","last_peered":"2024-09-10T20:30:11.042799+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:30:11.042799+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":3841,"log_dups_size":0,"ondisk_log_size":3841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15712256,"num_objects":3836,"num_object_clones":0,"num_object_copies":7672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3836,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":705,"num_write_kb":2820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":222},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'3783","reported_seq":3916,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:11.039478+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:30:11.039478+0000","last_peered":"2024-09-10T20:30:11.039478+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:30:11.039478+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":3783,"log_dups_size":0,"ondisk_log_size":3783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15470592,"num_objects":3777,"num_object_clones":0,"num_object_copies":7554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":378,"num_write_kb":1512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":222},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'3858","reported_seq":3977,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:11.076129+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:30:11.076129+0000","last_peered":"2024-09-10T20:30:11.076129+0000","last_clean":"2024-09-10T20:30:11.076129+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:30:11.076129+0000","last_undegraded":"2024-09-10T20:30:11.076129+0000","last_fullsized":"2024-09-10T20:30:11.076129+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":3858,"log_dups_size":0,"ondisk_log_size":3858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15794176,"num_objects":3856,"num_object_clones":0,"num_object_copies":7712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":929,"num_write_kb":3716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'3838","reported_seq":3965,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:13.049888+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:30:13.049888+0000","last_peered":"2024-09-10T20:30:13.049888+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:30:13.049888+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":3838,"log_dups_size":0,"ondisk_log_size":3838,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15691776,"num_objects":3831,"num_object_clones":0,"num_object_copies":7662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1831,"num_write_kb":7324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":305}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355438+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355438+0000","last_peered":"2024-09-10T20:30:05.355438+0000","last_clean":"2024-09-10T20:30:05.355438+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355438+0000","last_undegraded":"2024-09-10T20:30:05.355438+0000","last_fullsized":"2024-09-10T20:30:05.355438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'3855","reported_seq":3958,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:11.069348+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:30:11.069348+0000","last_peered":"2024-09-10T20:30:11.069348+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:30:11.069348+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":3855,"log_dups_size":0,"ondisk_log_size":3855,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15758278,"num_objects":3849,"num_object_clones":0,"num_object_copies":7698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3853,"num_write_kb":15392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":234},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.413176+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.413176+0000","last_peered":"2024-09-10T20:30:05.413176+0000","last_clean":"2024-09-10T20:30:05.413176+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.413176+0000","last_undegraded":"2024-09-10T20:30:05.413176+0000","last_fullsized":"2024-09-10T20:30:05.413176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'3928","reported_seq":4051,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:13.000037+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:30:13.000037+0000","last_peered":"2024-09-10T20:30:13.000037+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:30:13.000037+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":3928,"log_dups_size":0,"ondisk_log_size":3928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16068608,"num_objects":3923,"num_object_clones":0,"num_object_copies":7846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1756,"num_write_kb":7024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":290}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'3915","reported_seq":4037,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:13.070664+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:30:13.070664+0000","last_peered":"2024-09-10T20:30:13.070664+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:30:13.070664+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":3915,"log_dups_size":0,"ondisk_log_size":3915,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16011264,"num_objects":3909,"num_object_clones":0,"num_object_copies":7818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1701,"num_write_kb":6804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":299}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'3844","reported_seq":3974,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:11.086183+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:30:11.086183+0000","last_peered":"2024-09-10T20:30:11.086183+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:30:11.086183+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":3844,"log_dups_size":0,"ondisk_log_size":3844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15736832,"num_objects":3842,"num_object_clones":0,"num_object_copies":7684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":567,"num_write_kb":2268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":194},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'3771","reported_seq":3904,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:11.065098+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:30:11.065098+0000","last_peered":"2024-09-10T20:30:11.065098+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:30:11.065098+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":3771,"log_dups_size":0,"ondisk_log_size":3771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15413248,"num_objects":3763,"num_object_clones":0,"num_object_copies":7526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":465,"num_write_kb":1860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":231},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345705+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345705+0000","last_peered":"2024-09-10T20:30:05.345705+0000","last_clean":"2024-09-10T20:30:05.345705+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345705+0000","last_undegraded":"2024-09-10T20:30:05.345705+0000","last_fullsized":"2024-09-10T20:30:05.345705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'3942","reported_seq":4068,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:11.069497+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:30:11.069497+0000","last_peered":"2024-09-10T20:30:11.069497+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:30:11.069497+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":3942,"log_dups_size":0,"ondisk_log_size":3942,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16130048,"num_objects":3938,"num_object_clones":0,"num_object_copies":7876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":769,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":225},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'3694","reported_seq":3847,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:11.085529+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:30:11.085529+0000","last_peered":"2024-09-10T20:30:11.085529+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:30:11.085529+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":3694,"log_dups_size":0,"ondisk_log_size":3694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15081955,"num_objects":3683,"num_object_clones":0,"num_object_copies":7366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":301,"num_write_kb":1204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":215},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":252560809,"num_objects":61663,"num_object_clones":0,"num_object_copies":123326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":3431,"num_objects_unfound":0,"num_objects_dirty":61663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26038,"num_write_kb":104132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":708608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":507269120,"data_stored":507178795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":61742,"ondisk_log_size":61742,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182420,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":384736,"kb_used_data":240656,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93987104,"statfs":{"total":96636764160,"available":96242794496,"internally_reserved":0,"allocated":246431744,"data_stored":246198611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":147517098},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,3,5,2,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986192,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":388416,"kb_used_data":240112,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":93983424,"statfs":{"total":96636764160,"available":96239026176,"internally_reserved":0,"allocated":245874688,"data_stored":245645632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":151842474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,3,1,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659139,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71884,"kb_used_data":13820,"kb_used_omap":4,"kb_used_meta":58043,"kb_avail":94299956,"statfs":{"total":96636764160,"available":96563154944,"internally_reserved":0,"allocated":14151680,"data_stored":13839061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":59436718},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,7,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":247083008,"data_stored":247079395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":14188544,"data_stored":14105445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":245997568,"data_stored":245993955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:30:16.036 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 48 20 64433 64413 5.24123 2.73828 0.0300244 0.014898 2024-09-10T20:30:17.038 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 49 20 65141 65121 5.19071 2.76562 0.00872199 0.0150431 2024-09-10T20:30:18.036 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 50 20 65854 65834 5.1426 2.78516 0.0105107 0.015183 2024-09-10T20:30:18.257 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:30:18.802 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:18.805 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:30:18.828 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2024-09-10T20:30:16.835151+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":261154236,"num_objects":63763,"num_object_clones":0,"num_object_copies":127526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":3431,"num_objects_unfound":0,"num_objects_dirty":63763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28138,"num_write_kb":112526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":708627,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63843,"ondisk_log_size":63843,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":846716,"kb_used_data":504588,"kb_used_omap":12,"kb_used_meta":342067,"kb_avail":282268804,"statfs":{"total":289910292480,"available":289043255296,"internally_reserved":0,"allocated":516698112,"data_stored":515923304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":350276610},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,0,6,14,9,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32499345,"num_objects":7930,"num_object_clones":0,"num_object_copies":15860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":-38319,"num_objects_unfound":0,"num_objects_dirty":7930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1142,"num_write_kb":4609,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":704029,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.530220"},"pg_stats":[{"pgid":"2.f","version":"43'4035","reported_seq":4108,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:16.084511+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:30:16.084511+0000","last_peered":"2024-09-10T20:30:16.084511+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:30:16.084511+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":4035,"log_dups_size":0,"ondisk_log_size":4035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16527360,"num_objects":4035,"num_object_clones":0,"num_object_copies":8070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4035,"num_write_kb":16140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":428},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'3966","reported_seq":4071,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:13.016050+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:30:13.016050+0000","last_peered":"2024-09-10T20:30:13.016050+0000","last_clean":"2024-09-10T20:30:13.016050+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:30:13.016050+0000","last_undegraded":"2024-09-10T20:30:13.016050+0000","last_fullsized":"2024-09-10T20:30:13.016050+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":3966,"log_dups_size":0,"ondisk_log_size":3966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16220160,"num_objects":3960,"num_object_clones":0,"num_object_copies":7920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2152,"num_write_kb":8608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'3899","reported_seq":4368,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:30:13.056036+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:30:13.056036+0000","last_peered":"2024-09-10T20:30:13.056036+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:30:13.056036+0000","last_undegraded":"2024-09-10T20:30:13.056036+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":3899,"log_dups_size":0,"ondisk_log_size":3899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15953920,"num_objects":3895,"num_object_clones":0,"num_object_copies":7790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3431,"num_objects_unfound":0,"num_objects_dirty":3895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2027,"num_write_kb":8108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":704512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3431},{"shards":"2","objects":464}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'3870","reported_seq":3945,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:12.968917+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:30:12.968917+0000","last_peered":"2024-09-10T20:30:12.968917+0000","last_clean":"2024-09-10T20:30:12.968917+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:30:12.968917+0000","last_undegraded":"2024-09-10T20:30:12.968917+0000","last_fullsized":"2024-09-10T20:30:12.968917+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":3870,"log_dups_size":0,"ondisk_log_size":3870,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15851520,"num_objects":3870,"num_object_clones":0,"num_object_copies":7740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3870,"num_write_kb":15480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4126","reported_seq":4247,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:16.073893+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:30:16.073893+0000","last_peered":"2024-09-10T20:30:16.073893+0000","last_clean":"2024-09-10T20:30:16.073893+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:30:16.073893+0000","last_undegraded":"2024-09-10T20:30:16.073893+0000","last_fullsized":"2024-09-10T20:30:16.073893+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":4126,"log_dups_size":0,"ondisk_log_size":4126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16871424,"num_objects":4119,"num_object_clones":0,"num_object_copies":8238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1122,"num_write_kb":4488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4057","reported_seq":4186,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:16.102830+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:30:16.102830+0000","last_peered":"2024-09-10T20:30:16.102830+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:30:16.102830+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":4057,"log_dups_size":0,"ondisk_log_size":4057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16596992,"num_objects":4052,"num_object_clones":0,"num_object_copies":8104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":921,"num_write_kb":3684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":438},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4000","reported_seq":4133,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:16.081927+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:30:16.081927+0000","last_peered":"2024-09-10T20:30:16.081927+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:30:16.081927+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":4000,"log_dups_size":0,"ondisk_log_size":4000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16359424,"num_objects":3994,"num_object_clones":0,"num_object_copies":7988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":595,"num_write_kb":2380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":439},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4064","reported_seq":4183,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:16.107474+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:30:16.107474+0000","last_peered":"2024-09-10T20:30:16.107474+0000","last_clean":"2024-09-10T20:30:16.107474+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:30:16.107474+0000","last_undegraded":"2024-09-10T20:30:16.107474+0000","last_fullsized":"2024-09-10T20:30:16.107474+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":4064,"log_dups_size":0,"ondisk_log_size":4064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16637952,"num_objects":4062,"num_object_clones":0,"num_object_copies":8124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1135,"num_write_kb":4540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'3838","reported_seq":3965,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:13.049888+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:30:13.049888+0000","last_peered":"2024-09-10T20:30:13.049888+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:30:13.049888+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":3838,"log_dups_size":0,"ondisk_log_size":3838,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15691776,"num_objects":3831,"num_object_clones":0,"num_object_copies":7662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1831,"num_write_kb":7324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":305}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355438+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355438+0000","last_peered":"2024-09-10T20:30:05.355438+0000","last_clean":"2024-09-10T20:30:05.355438+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355438+0000","last_undegraded":"2024-09-10T20:30:05.355438+0000","last_fullsized":"2024-09-10T20:30:05.355438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4066","reported_seq":4168,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:16.100281+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:30:16.100281+0000","last_peered":"2024-09-10T20:30:16.100281+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:30:16.100281+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":4066,"log_dups_size":0,"ondisk_log_size":4066,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16622534,"num_objects":4060,"num_object_clones":0,"num_object_copies":8120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4060,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4064,"num_write_kb":16236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":445},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.413176+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.413176+0000","last_peered":"2024-09-10T20:30:05.413176+0000","last_clean":"2024-09-10T20:30:05.413176+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.413176+0000","last_undegraded":"2024-09-10T20:30:05.413176+0000","last_fullsized":"2024-09-10T20:30:05.413176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'3928","reported_seq":4051,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:13.000037+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:30:13.000037+0000","last_peered":"2024-09-10T20:30:13.000037+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:30:13.000037+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":3928,"log_dups_size":0,"ondisk_log_size":3928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16068608,"num_objects":3923,"num_object_clones":0,"num_object_copies":7846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1756,"num_write_kb":7024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":290}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'3915","reported_seq":4037,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:13.070664+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:30:13.070664+0000","last_peered":"2024-09-10T20:30:13.070664+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:30:13.070664+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":3915,"log_dups_size":0,"ondisk_log_size":3915,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16011264,"num_objects":3909,"num_object_clones":0,"num_object_copies":7818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1701,"num_write_kb":6804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":299}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4020","reported_seq":4150,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:16.084551+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:30:16.084551+0000","last_peered":"2024-09-10T20:30:16.084551+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:30:16.084551+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":4020,"log_dups_size":0,"ondisk_log_size":4020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16457728,"num_objects":4018,"num_object_clones":0,"num_object_copies":8036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":743,"num_write_kb":2972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":370},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'3993","reported_seq":4126,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:16.073328+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:30:16.073328+0000","last_peered":"2024-09-10T20:30:16.073328+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:30:16.073328+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":3993,"log_dups_size":0,"ondisk_log_size":3993,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16322560,"num_objects":3985,"num_object_clones":0,"num_object_copies":7970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":687,"num_write_kb":2748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":453},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345705+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345705+0000","last_peered":"2024-09-10T20:30:05.345705+0000","last_clean":"2024-09-10T20:30:05.345705+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345705+0000","last_undegraded":"2024-09-10T20:30:05.345705+0000","last_fullsized":"2024-09-10T20:30:05.345705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4149","reported_seq":4275,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:16.048893+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:30:16.048893+0000","last_peered":"2024-09-10T20:30:16.048893+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:30:16.048893+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":4149,"log_dups_size":0,"ondisk_log_size":4149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16977920,"num_objects":4145,"num_object_clones":0,"num_object_copies":8290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":976,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":432},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'3914","reported_seq":4074,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:16.037728+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:30:16.037728+0000","last_peered":"2024-09-10T20:30:16.037728+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:30:16.037728+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":3914,"log_dups_size":0,"ondisk_log_size":3914,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15983075,"num_objects":3903,"num_object_clones":0,"num_object_copies":7806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":521,"num_write_kb":2084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":435},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":261154217,"num_objects":63761,"num_object_clones":0,"num_object_copies":127522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":3431,"num_objects_unfound":0,"num_objects_dirty":63761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28136,"num_write_kb":112524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":708608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":517476352,"data_stored":517386027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63840,"ondisk_log_size":63840,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182421,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":386416,"kb_used_data":250656,"kb_used_omap":4,"kb_used_meta":135739,"kb_avail":93985424,"statfs":{"total":96636764160,"available":96241074176,"internally_reserved":0,"allocated":256671744,"data_stored":256438611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":138997418},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,6,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986192,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":388416,"kb_used_data":240112,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":93983424,"statfs":{"total":96636764160,"available":96239026176,"internally_reserved":0,"allocated":245874688,"data_stored":245645632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":151842474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,3,1,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659139,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71884,"kb_used_data":13820,"kb_used_omap":4,"kb_used_meta":58043,"kb_avail":94299956,"statfs":{"total":96636764160,"available":96563154944,"internally_reserved":0,"allocated":14151680,"data_stored":13839061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":59436718},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,7,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":257290240,"data_stored":257286627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":14188544,"data_stored":14105445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":245997568,"data_stored":245993955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:30:18.831 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:30:19.037 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 51 20 66484 66464 5.08996 2.46094 0.0133226 0.01534 2024-09-10T20:30:19.378 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:19.380 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:30:19.405 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":102,"stamp":"2024-09-10T20:30:18.835602+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":266274236,"num_objects":65013,"num_object_clones":0,"num_object_copies":130026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":3314,"num_objects_unfound":0,"num_objects_dirty":65013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29388,"num_write_kb":117526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":294,"num_bytes_recovered":1187859,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65093,"ondisk_log_size":65093,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":838664,"kb_used_data":521496,"kb_used_omap":12,"kb_used_meta":317107,"kb_avail":282276856,"statfs":{"total":289910292480,"available":289051500544,"internally_reserved":0,"allocated":534011904,"data_stored":533237096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":324717570},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,2,5,11,6,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30056448,"num_objects":7338,"num_object_clones":0,"num_object_copies":14676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10775,"num_objects_misplaced":-10986,"num_objects_unfound":0,"num_objects_dirty":7338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7338,"num_write_kb":29352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":289,"num_bytes_recovered":1183744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003773"},"pg_stats":[{"pgid":"2.f","version":"43'4035","reported_seq":4108,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:16.084511+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:30:16.084511+0000","last_peered":"2024-09-10T20:30:16.084511+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:30:16.084511+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":4035,"log_dups_size":0,"ondisk_log_size":4035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16527360,"num_objects":4035,"num_object_clones":0,"num_object_copies":8070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4035,"num_write_kb":16140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":428},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4199","reported_seq":4304,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:18.084516+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:30:18.084516+0000","last_peered":"2024-09-10T20:30:18.084516+0000","last_clean":"2024-09-10T20:30:18.084516+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:30:18.084516+0000","last_undegraded":"2024-09-10T20:30:18.084516+0000","last_fullsized":"2024-09-10T20:30:18.084516+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":4199,"log_dups_size":0,"ondisk_log_size":4199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17174528,"num_objects":4193,"num_object_clones":0,"num_object_copies":8386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2385,"num_write_kb":9540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4103","reported_seq":4805,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:30:18.049431+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:30:18.049431+0000","last_peered":"2024-09-10T20:30:18.049431+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:30:18.049431+0000","last_undegraded":"2024-09-10T20:30:18.049431+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":4103,"log_dups_size":0,"ondisk_log_size":4103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16789504,"num_objects":4099,"num_object_clones":0,"num_object_copies":8198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3314,"num_objects_unfound":0,"num_objects_dirty":4099,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2231,"num_write_kb":8924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":289,"num_bytes_recovered":1183744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3314},{"shards":"2","objects":785}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4059","reported_seq":4134,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:18.045806+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:30:18.045806+0000","last_peered":"2024-09-10T20:30:18.045806+0000","last_clean":"2024-09-10T20:30:18.045806+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:30:18.045806+0000","last_undegraded":"2024-09-10T20:30:18.045806+0000","last_fullsized":"2024-09-10T20:30:18.045806+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":4059,"log_dups_size":0,"ondisk_log_size":4059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16625664,"num_objects":4059,"num_object_clones":0,"num_object_copies":8118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4059,"num_write_kb":16236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4126","reported_seq":4247,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:16.073893+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:30:16.073893+0000","last_peered":"2024-09-10T20:30:16.073893+0000","last_clean":"2024-09-10T20:30:16.073893+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:30:16.073893+0000","last_undegraded":"2024-09-10T20:30:16.073893+0000","last_fullsized":"2024-09-10T20:30:16.073893+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":4126,"log_dups_size":0,"ondisk_log_size":4126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16871424,"num_objects":4119,"num_object_clones":0,"num_object_copies":8238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1122,"num_write_kb":4488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4057","reported_seq":4186,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:16.102830+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:30:16.102830+0000","last_peered":"2024-09-10T20:30:16.102830+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:30:16.102830+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":4057,"log_dups_size":0,"ondisk_log_size":4057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16596992,"num_objects":4052,"num_object_clones":0,"num_object_copies":8104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":921,"num_write_kb":3684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":438},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4000","reported_seq":4133,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:16.081927+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:30:16.081927+0000","last_peered":"2024-09-10T20:30:16.081927+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:30:16.081927+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":4000,"log_dups_size":0,"ondisk_log_size":4000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16359424,"num_objects":3994,"num_object_clones":0,"num_object_copies":7988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":595,"num_write_kb":2380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":439},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4064","reported_seq":4183,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:16.107474+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:30:16.107474+0000","last_peered":"2024-09-10T20:30:16.107474+0000","last_clean":"2024-09-10T20:30:16.107474+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:30:16.107474+0000","last_undegraded":"2024-09-10T20:30:16.107474+0000","last_fullsized":"2024-09-10T20:30:16.107474+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":4064,"log_dups_size":0,"ondisk_log_size":4064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16637952,"num_objects":4062,"num_object_clones":0,"num_object_copies":8124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1135,"num_write_kb":4540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4051","reported_seq":4178,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:18.073468+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:30:18.073468+0000","last_peered":"2024-09-10T20:30:18.073468+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:30:18.073468+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":4051,"log_dups_size":0,"ondisk_log_size":4051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16564224,"num_objects":4044,"num_object_clones":0,"num_object_copies":8088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2044,"num_write_kb":8176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":518}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355438+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355438+0000","last_peered":"2024-09-10T20:30:05.355438+0000","last_clean":"2024-09-10T20:30:05.355438+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355438+0000","last_undegraded":"2024-09-10T20:30:05.355438+0000","last_fullsized":"2024-09-10T20:30:05.355438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4066","reported_seq":4168,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:16.100281+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:30:16.100281+0000","last_peered":"2024-09-10T20:30:16.100281+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:30:16.100281+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":4066,"log_dups_size":0,"ondisk_log_size":4066,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16622534,"num_objects":4060,"num_object_clones":0,"num_object_copies":8120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4060,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4064,"num_write_kb":16236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":445},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.413176+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.413176+0000","last_peered":"2024-09-10T20:30:05.413176+0000","last_clean":"2024-09-10T20:30:05.413176+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.413176+0000","last_undegraded":"2024-09-10T20:30:05.413176+0000","last_fullsized":"2024-09-10T20:30:05.413176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4129","reported_seq":4252,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:17.998338+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:30:17.998338+0000","last_peered":"2024-09-10T20:30:17.998338+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:30:17.998338+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":4129,"log_dups_size":0,"ondisk_log_size":4129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16891904,"num_objects":4124,"num_object_clones":0,"num_object_copies":8248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1957,"num_write_kb":7828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":491}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4125","reported_seq":4248,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:17.956360+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:30:17.956360+0000","last_peered":"2024-09-10T20:30:17.956360+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:30:17.956360+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":4125,"log_dups_size":0,"ondisk_log_size":4125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16871424,"num_objects":4119,"num_object_clones":0,"num_object_copies":8238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1911,"num_write_kb":7644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":509}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4020","reported_seq":4150,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:16.084551+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:30:16.084551+0000","last_peered":"2024-09-10T20:30:16.084551+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:30:16.084551+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":4020,"log_dups_size":0,"ondisk_log_size":4020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16457728,"num_objects":4018,"num_object_clones":0,"num_object_copies":8036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":743,"num_write_kb":2972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":370},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'3993","reported_seq":4126,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:16.073328+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:30:16.073328+0000","last_peered":"2024-09-10T20:30:16.073328+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:30:16.073328+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":3993,"log_dups_size":0,"ondisk_log_size":3993,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16322560,"num_objects":3985,"num_object_clones":0,"num_object_copies":7970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":687,"num_write_kb":2748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":453},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345705+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345705+0000","last_peered":"2024-09-10T20:30:05.345705+0000","last_clean":"2024-09-10T20:30:05.345705+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345705+0000","last_undegraded":"2024-09-10T20:30:05.345705+0000","last_fullsized":"2024-09-10T20:30:05.345705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4149","reported_seq":4275,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:16.048893+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:30:16.048893+0000","last_peered":"2024-09-10T20:30:16.048893+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:30:16.048893+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":4149,"log_dups_size":0,"ondisk_log_size":4149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16977920,"num_objects":4145,"num_object_clones":0,"num_object_copies":8290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":976,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":432},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'3914","reported_seq":4074,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:16.037728+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:30:16.037728+0000","last_peered":"2024-09-10T20:30:16.037728+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:30:16.037728+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":3914,"log_dups_size":0,"ondisk_log_size":3914,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15983075,"num_objects":3903,"num_object_clones":0,"num_object_copies":7806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":521,"num_write_kb":2084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":435},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":266274217,"num_objects":65011,"num_object_clones":0,"num_object_copies":130022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":3314,"num_objects_unfound":0,"num_objects_dirty":65011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29386,"num_write_kb":117524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":1187840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":535080960,"data_stored":534990635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65090,"ondisk_log_size":65090,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182421,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":386416,"kb_used_data":250656,"kb_used_omap":4,"kb_used_meta":135739,"kb_avail":93985424,"statfs":{"total":96636764160,"available":96241074176,"internally_reserved":0,"allocated":256671744,"data_stored":256438611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":138997418},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,6,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986193,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":382708,"kb_used_data":246884,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":93989132,"statfs":{"total":96636764160,"available":96244871168,"internally_reserved":0,"allocated":252809216,"data_stored":252580160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":139062954},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,0,0,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659140,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69540,"kb_used_data":23956,"kb_used_omap":4,"kb_used_meta":45563,"kb_avail":94302300,"statfs":{"total":96636764160,"available":96565555200,"internally_reserved":0,"allocated":24530944,"data_stored":24218325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":46657198},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,5,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":257290240,"data_stored":257286627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24862720,"data_stored":24779621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":252928000,"data_stored":252924387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:30:19.406 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:30:19.407 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'4035", 'reported_seq': 4108, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:16.084511+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:30:16.084511+0000', 'last_peered': '2024-09-10T20:30:16.084511+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:30:16.084511+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 4035, 'log_dups_size': 0, 'ondisk_log_size': 4035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16527360, 'num_objects': 4035, 'num_object_clones': 0, 'num_object_copies': 8070, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3607, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4035, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4035, 'num_write_kb': 16140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 428}, {'shards': '0,2', 'objects': 3607}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:19.407 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T20:30:19.407 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'4103", 'reported_seq': 4805, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:30:18.049431+0000', 'last_change': '2024-09-10T20:30:05.653560+0000', 'last_active': '2024-09-10T20:30:18.049431+0000', 'last_peered': '2024-09-10T20:30:18.049431+0000', 'last_clean': '2024-09-10T20:30:03.279259+0000', 'last_became_active': '2024-09-10T20:30:05.629664+0000', 'last_became_peered': '2024-09-10T20:30:05.629664+0000', 'last_unstale': '2024-09-10T20:30:18.049431+0000', 'last_undegraded': '2024-09-10T20:30:18.049431+0000', 'last_fullsized': '2024-09-10T20:30:05.312061+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1910", 'last_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'objects_scrubbed': 1868, 'log_size': 4103, 'log_dups_size': 0, 'ondisk_log_size': 4103, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 673, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16789504, 'num_objects': 4099, 'num_object_clones': 0, 'num_object_copies': 8198, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3314, 'num_objects_unfound': 0, 'num_objects_dirty': 4099, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2231, 'num_write_kb': 8924, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 289, 'num_bytes_recovered': 1183744, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3314}, {'shards': '2', 'objects': 785}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:19.407 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:30:19.407 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'4057", 'reported_seq': 4186, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:16.102830+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:30:16.102830+0000', 'last_peered': '2024-09-10T20:30:16.102830+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:30:16.102830+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 4057, 'log_dups_size': 0, 'ondisk_log_size': 4057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16596992, 'num_objects': 4052, 'num_object_clones': 0, 'num_object_copies': 8104, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3616, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4052, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 921, 'num_write_kb': 3684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 438}, {'shards': '0,2', 'objects': 3614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:19.408 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:30:19.408 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'4000", 'reported_seq': 4133, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:16.081927+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:30:16.081927+0000', 'last_peered': '2024-09-10T20:30:16.081927+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:30:16.081927+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 4000, 'log_dups_size': 0, 'ondisk_log_size': 4000, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16359424, 'num_objects': 3994, 'num_object_clones': 0, 'num_object_copies': 7988, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3557, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3994, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 595, 'num_write_kb': 2380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 439}, {'shards': '0,2', 'objects': 3555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:19.408 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:30:19.408 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'4051", 'reported_seq': 4178, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:18.073468+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:30:18.073468+0000', 'last_peered': '2024-09-10T20:30:18.073468+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:30:18.073468+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 4051, 'log_dups_size': 0, 'ondisk_log_size': 4051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16564224, 'num_objects': 4044, 'num_object_clones': 0, 'num_object_copies': 8088, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3528, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4044, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2044, 'num_write_kb': 8176, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 518}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:19.408 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:30:19.408 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'4066", 'reported_seq': 4168, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:16.100281+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:30:16.100281+0000', 'last_peered': '2024-09-10T20:30:16.100281+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:30:16.100281+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 4066, 'log_dups_size': 0, 'ondisk_log_size': 4066, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16622534, 'num_objects': 4060, 'num_object_clones': 0, 'num_object_copies': 8120, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3617, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4060, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4064, 'num_write_kb': 16236, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 445}, {'shards': '0,2', 'objects': 3615}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:19.408 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:30:19.408 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'4129", 'reported_seq': 4252, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:17.998338+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:30:17.998338+0000', 'last_peered': '2024-09-10T20:30:17.998338+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:30:17.998338+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 4129, 'log_dups_size': 0, 'ondisk_log_size': 4129, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16891904, 'num_objects': 4124, 'num_object_clones': 0, 'num_object_copies': 8248, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3635, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4124, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1957, 'num_write_kb': 7828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3633}, {'shards': '2', 'objects': 491}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:19.409 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:30:19.409 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'4125", 'reported_seq': 4248, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:17.956360+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:30:17.956360+0000', 'last_peered': '2024-09-10T20:30:17.956360+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:30:17.956360+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 4125, 'log_dups_size': 0, 'ondisk_log_size': 4125, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16871424, 'num_objects': 4119, 'num_object_clones': 0, 'num_object_copies': 8238, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4119, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1911, 'num_write_kb': 7644, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3610}, {'shards': '2', 'objects': 509}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:19.409 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:30:19.409 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'4020", 'reported_seq': 4150, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:16.084551+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:30:16.084551+0000', 'last_peered': '2024-09-10T20:30:16.084551+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:30:16.084551+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 4020, 'log_dups_size': 0, 'ondisk_log_size': 4020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16457728, 'num_objects': 4018, 'num_object_clones': 0, 'num_object_copies': 8036, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3649, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4018, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 743, 'num_write_kb': 2972, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 370}, {'shards': '0,2', 'objects': 3648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:19.409 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:30:19.409 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'3993", 'reported_seq': 4126, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:16.073328+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:30:16.073328+0000', 'last_peered': '2024-09-10T20:30:16.073328+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:30:16.073328+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 3993, 'log_dups_size': 0, 'ondisk_log_size': 3993, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16322560, 'num_objects': 3985, 'num_object_clones': 0, 'num_object_copies': 7970, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3535, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3985, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 687, 'num_write_kb': 2748, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 453}, {'shards': '0,2', 'objects': 3532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:19.409 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:30:19.409 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'4149", 'reported_seq': 4275, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:16.048893+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:30:16.048893+0000', 'last_peered': '2024-09-10T20:30:16.048893+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:30:16.048893+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 4149, 'log_dups_size': 0, 'ondisk_log_size': 4149, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16977920, 'num_objects': 4145, 'num_object_clones': 0, 'num_object_copies': 8290, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3715, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4145, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 976, 'num_write_kb': 3904, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 432}, {'shards': '0,2', 'objects': 3713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:19.410 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:30:19.410 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'3914", 'reported_seq': 4074, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:16.037728+0000', 'last_change': '2024-09-10T20:30:05.690055+0000', 'last_active': '2024-09-10T20:30:16.037728+0000', 'last_peered': '2024-09-10T20:30:16.037728+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:30:16.037728+0000', 'last_undegraded': '2024-09-10T20:30:05.378941+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 3914, 'log_dups_size': 0, 'ondisk_log_size': 3914, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15983075, 'num_objects': 3903, 'num_object_clones': 0, 'num_object_copies': 7806, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3471, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3903, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 521, 'num_write_kb': 2084, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 435}, {'shards': '0,2', 'objects': 3468}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:19.410 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:30:20.037 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 52 20 67104 67084 5.03865 2.42188 0.032282 0.0154971 2024-09-10T20:30:20.094 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:20.095 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39542/130026 objects degraded (30.411%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":131,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":65013,"data_bytes":266274236,"bytes_used":858791936,"bytes_avail":289051500544,"bytes_total":289910292480,"degraded_objects":39542,"degraded_total":130026,"degraded_ratio":0.30410840908741327,"misplaced_objects":3314,"misplaced_total":130026,"misplaced_ratio":0.025487210250257641,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":98614,"recovering_keys_per_sec":0,"num_objects_recovered":289,"num_bytes_recovered":1183744,"num_keys_recovered":0,"write_bytes_sec":2503916,"read_op_per_sec":0,"write_op_per_sec":611},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-10T20:30:14.834797+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T20:30:20.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:30:20.120 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:30:20.678 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:20.679 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:30:20.708 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":102,"stamp":"2024-09-10T20:30:18.835602+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":266274236,"num_objects":65013,"num_object_clones":0,"num_object_copies":130026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":3314,"num_objects_unfound":0,"num_objects_dirty":65013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29388,"num_write_kb":117526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":294,"num_bytes_recovered":1187859,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65093,"ondisk_log_size":65093,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":838664,"kb_used_data":521496,"kb_used_omap":12,"kb_used_meta":317107,"kb_avail":282276856,"statfs":{"total":289910292480,"available":289051500544,"internally_reserved":0,"allocated":534011904,"data_stored":533237096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":324717570},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,2,5,11,6,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30056448,"num_objects":7338,"num_object_clones":0,"num_object_copies":14676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10775,"num_objects_misplaced":-10986,"num_objects_unfound":0,"num_objects_dirty":7338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7338,"num_write_kb":29352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":289,"num_bytes_recovered":1183744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003773"},"pg_stats":[{"pgid":"2.f","version":"43'4035","reported_seq":4108,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:16.084511+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:30:16.084511+0000","last_peered":"2024-09-10T20:30:16.084511+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:30:16.084511+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":4035,"log_dups_size":0,"ondisk_log_size":4035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16527360,"num_objects":4035,"num_object_clones":0,"num_object_copies":8070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4035,"num_write_kb":16140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":428},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4199","reported_seq":4304,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:18.084516+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:30:18.084516+0000","last_peered":"2024-09-10T20:30:18.084516+0000","last_clean":"2024-09-10T20:30:18.084516+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:30:18.084516+0000","last_undegraded":"2024-09-10T20:30:18.084516+0000","last_fullsized":"2024-09-10T20:30:18.084516+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":4199,"log_dups_size":0,"ondisk_log_size":4199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17174528,"num_objects":4193,"num_object_clones":0,"num_object_copies":8386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2385,"num_write_kb":9540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4103","reported_seq":4805,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:30:18.049431+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:30:18.049431+0000","last_peered":"2024-09-10T20:30:18.049431+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:30:18.049431+0000","last_undegraded":"2024-09-10T20:30:18.049431+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":4103,"log_dups_size":0,"ondisk_log_size":4103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16789504,"num_objects":4099,"num_object_clones":0,"num_object_copies":8198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3314,"num_objects_unfound":0,"num_objects_dirty":4099,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2231,"num_write_kb":8924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":289,"num_bytes_recovered":1183744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3314},{"shards":"2","objects":785}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4059","reported_seq":4134,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:18.045806+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:30:18.045806+0000","last_peered":"2024-09-10T20:30:18.045806+0000","last_clean":"2024-09-10T20:30:18.045806+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:30:18.045806+0000","last_undegraded":"2024-09-10T20:30:18.045806+0000","last_fullsized":"2024-09-10T20:30:18.045806+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":4059,"log_dups_size":0,"ondisk_log_size":4059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16625664,"num_objects":4059,"num_object_clones":0,"num_object_copies":8118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4059,"num_write_kb":16236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4126","reported_seq":4247,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:16.073893+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:30:16.073893+0000","last_peered":"2024-09-10T20:30:16.073893+0000","last_clean":"2024-09-10T20:30:16.073893+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:30:16.073893+0000","last_undegraded":"2024-09-10T20:30:16.073893+0000","last_fullsized":"2024-09-10T20:30:16.073893+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":4126,"log_dups_size":0,"ondisk_log_size":4126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16871424,"num_objects":4119,"num_object_clones":0,"num_object_copies":8238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1122,"num_write_kb":4488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4057","reported_seq":4186,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:16.102830+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:30:16.102830+0000","last_peered":"2024-09-10T20:30:16.102830+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:30:16.102830+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":4057,"log_dups_size":0,"ondisk_log_size":4057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16596992,"num_objects":4052,"num_object_clones":0,"num_object_copies":8104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":921,"num_write_kb":3684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":438},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4000","reported_seq":4133,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:16.081927+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:30:16.081927+0000","last_peered":"2024-09-10T20:30:16.081927+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:30:16.081927+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":4000,"log_dups_size":0,"ondisk_log_size":4000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16359424,"num_objects":3994,"num_object_clones":0,"num_object_copies":7988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":595,"num_write_kb":2380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":439},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4064","reported_seq":4183,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:16.107474+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:30:16.107474+0000","last_peered":"2024-09-10T20:30:16.107474+0000","last_clean":"2024-09-10T20:30:16.107474+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:30:16.107474+0000","last_undegraded":"2024-09-10T20:30:16.107474+0000","last_fullsized":"2024-09-10T20:30:16.107474+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":4064,"log_dups_size":0,"ondisk_log_size":4064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16637952,"num_objects":4062,"num_object_clones":0,"num_object_copies":8124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1135,"num_write_kb":4540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4051","reported_seq":4178,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:18.073468+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:30:18.073468+0000","last_peered":"2024-09-10T20:30:18.073468+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:30:18.073468+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":4051,"log_dups_size":0,"ondisk_log_size":4051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16564224,"num_objects":4044,"num_object_clones":0,"num_object_copies":8088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2044,"num_write_kb":8176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":518}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355438+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355438+0000","last_peered":"2024-09-10T20:30:05.355438+0000","last_clean":"2024-09-10T20:30:05.355438+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355438+0000","last_undegraded":"2024-09-10T20:30:05.355438+0000","last_fullsized":"2024-09-10T20:30:05.355438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4066","reported_seq":4168,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:16.100281+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:30:16.100281+0000","last_peered":"2024-09-10T20:30:16.100281+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:30:16.100281+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":4066,"log_dups_size":0,"ondisk_log_size":4066,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16622534,"num_objects":4060,"num_object_clones":0,"num_object_copies":8120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4060,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4064,"num_write_kb":16236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":445},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.413176+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.413176+0000","last_peered":"2024-09-10T20:30:05.413176+0000","last_clean":"2024-09-10T20:30:05.413176+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.413176+0000","last_undegraded":"2024-09-10T20:30:05.413176+0000","last_fullsized":"2024-09-10T20:30:05.413176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4129","reported_seq":4252,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:17.998338+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:30:17.998338+0000","last_peered":"2024-09-10T20:30:17.998338+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:30:17.998338+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":4129,"log_dups_size":0,"ondisk_log_size":4129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16891904,"num_objects":4124,"num_object_clones":0,"num_object_copies":8248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1957,"num_write_kb":7828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":491}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4125","reported_seq":4248,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:17.956360+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:30:17.956360+0000","last_peered":"2024-09-10T20:30:17.956360+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:30:17.956360+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":4125,"log_dups_size":0,"ondisk_log_size":4125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16871424,"num_objects":4119,"num_object_clones":0,"num_object_copies":8238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1911,"num_write_kb":7644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":509}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4020","reported_seq":4150,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:16.084551+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:30:16.084551+0000","last_peered":"2024-09-10T20:30:16.084551+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:30:16.084551+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":4020,"log_dups_size":0,"ondisk_log_size":4020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16457728,"num_objects":4018,"num_object_clones":0,"num_object_copies":8036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":743,"num_write_kb":2972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":370},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'3993","reported_seq":4126,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:16.073328+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:30:16.073328+0000","last_peered":"2024-09-10T20:30:16.073328+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:30:16.073328+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":3993,"log_dups_size":0,"ondisk_log_size":3993,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16322560,"num_objects":3985,"num_object_clones":0,"num_object_copies":7970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":687,"num_write_kb":2748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":453},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345705+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345705+0000","last_peered":"2024-09-10T20:30:05.345705+0000","last_clean":"2024-09-10T20:30:05.345705+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345705+0000","last_undegraded":"2024-09-10T20:30:05.345705+0000","last_fullsized":"2024-09-10T20:30:05.345705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4149","reported_seq":4275,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:16.048893+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:30:16.048893+0000","last_peered":"2024-09-10T20:30:16.048893+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:30:16.048893+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":4149,"log_dups_size":0,"ondisk_log_size":4149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16977920,"num_objects":4145,"num_object_clones":0,"num_object_copies":8290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":976,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":432},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'3914","reported_seq":4074,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:16.037728+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:30:16.037728+0000","last_peered":"2024-09-10T20:30:16.037728+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:30:16.037728+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":3914,"log_dups_size":0,"ondisk_log_size":3914,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15983075,"num_objects":3903,"num_object_clones":0,"num_object_copies":7806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":521,"num_write_kb":2084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":435},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":266274217,"num_objects":65011,"num_object_clones":0,"num_object_copies":130022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":3314,"num_objects_unfound":0,"num_objects_dirty":65011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29386,"num_write_kb":117524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":1187840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":535080960,"data_stored":534990635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65090,"ondisk_log_size":65090,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182421,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":386416,"kb_used_data":250656,"kb_used_omap":4,"kb_used_meta":135739,"kb_avail":93985424,"statfs":{"total":96636764160,"available":96241074176,"internally_reserved":0,"allocated":256671744,"data_stored":256438611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":138997418},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,6,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986193,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":382708,"kb_used_data":246884,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":93989132,"statfs":{"total":96636764160,"available":96244871168,"internally_reserved":0,"allocated":252809216,"data_stored":252580160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":139062954},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,0,0,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659140,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69540,"kb_used_data":23956,"kb_used_omap":4,"kb_used_meta":45563,"kb_avail":94302300,"statfs":{"total":96636764160,"available":96565555200,"internally_reserved":0,"allocated":24530944,"data_stored":24218325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":46657198},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,5,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":257290240,"data_stored":257286627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24862720,"data_stored":24779621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":252928000,"data_stored":252924387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:30:21.039 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 53 20 67744 67724 4.99057 2.5 0.0317504 0.0156467 2024-09-10T20:30:22.039 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 54 20 68441 68421 4.94857 2.72266 0.0219858 0.0157789 2024-09-10T20:30:23.039 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 55 20 69132 69112 4.90767 2.69922 0.0221261 0.0159103 2024-09-10T20:30:23.710 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:30:24.039 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 56 20 69795 69775 4.86628 2.58984 0.0328161 0.0160462 2024-09-10T20:30:24.267 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:24.268 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:30:24.298 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2024-09-10T20:30:22.836521+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":274875836,"num_objects":67113,"num_object_clones":0,"num_object_copies":134226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":3314,"num_objects_unfound":0,"num_objects_dirty":67113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31488,"num_write_kb":125926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":294,"num_bytes_recovered":1187859,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":67193,"ondisk_log_size":67193,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":867172,"kb_used_data":541684,"kb_used_omap":12,"kb_used_meta":325427,"kb_avail":282248348,"statfs":{"total":289910292480,"available":289022308352,"internally_reserved":0,"allocated":554684416,"data_stored":553909608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":333237250},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,3,5,10,10,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":35893248,"num_objects":8763,"num_object_clones":0,"num_object_copies":17526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-234,"num_objects_unfound":0,"num_objects_dirty":8763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8763,"num_write_kb":35052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":958464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003700"},"pg_stats":[{"pgid":"2.f","version":"43'4228","reported_seq":4297,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:21.102963+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:30:21.102963+0000","last_peered":"2024-09-10T20:30:21.102963+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:30:21.102963+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":4228,"log_dups_size":0,"ondisk_log_size":4228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17317888,"num_objects":4228,"num_object_clones":0,"num_object_copies":8456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4228,"num_write_kb":16912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":621},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4199","reported_seq":4304,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:18.084516+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:30:18.084516+0000","last_peered":"2024-09-10T20:30:18.084516+0000","last_clean":"2024-09-10T20:30:18.084516+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:30:18.084516+0000","last_undegraded":"2024-09-10T20:30:18.084516+0000","last_fullsized":"2024-09-10T20:30:18.084516+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":4199,"log_dups_size":0,"ondisk_log_size":4199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17174528,"num_objects":4193,"num_object_clones":0,"num_object_copies":8386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2385,"num_write_kb":9540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4103","reported_seq":4805,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:30:18.049431+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:30:18.049431+0000","last_peered":"2024-09-10T20:30:18.049431+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:30:18.049431+0000","last_undegraded":"2024-09-10T20:30:18.049431+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":4103,"log_dups_size":0,"ondisk_log_size":4103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16789504,"num_objects":4099,"num_object_clones":0,"num_object_copies":8198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3314,"num_objects_unfound":0,"num_objects_dirty":4099,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2231,"num_write_kb":8924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":289,"num_bytes_recovered":1183744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3314},{"shards":"2","objects":785}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4059","reported_seq":4134,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:18.045806+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:30:18.045806+0000","last_peered":"2024-09-10T20:30:18.045806+0000","last_clean":"2024-09-10T20:30:18.045806+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:30:18.045806+0000","last_undegraded":"2024-09-10T20:30:18.045806+0000","last_fullsized":"2024-09-10T20:30:18.045806+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":4059,"log_dups_size":0,"ondisk_log_size":4059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16625664,"num_objects":4059,"num_object_clones":0,"num_object_copies":8118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4059,"num_write_kb":16236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4330","reported_seq":4451,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:21.081597+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:30:21.081597+0000","last_peered":"2024-09-10T20:30:21.081597+0000","last_clean":"2024-09-10T20:30:21.081597+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:30:21.081597+0000","last_undegraded":"2024-09-10T20:30:21.081597+0000","last_fullsized":"2024-09-10T20:30:21.081597+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":4330,"log_dups_size":0,"ondisk_log_size":4330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17707008,"num_objects":4323,"num_object_clones":0,"num_object_copies":8646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1326,"num_write_kb":5304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4259","reported_seq":4390,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:21.091770+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:30:21.091770+0000","last_peered":"2024-09-10T20:30:21.091770+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:30:21.091770+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":4259,"log_dups_size":0,"ondisk_log_size":4259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17424384,"num_objects":4254,"num_object_clones":0,"num_object_copies":8508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1123,"num_write_kb":4492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":640},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4188","reported_seq":4321,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:21.063308+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:30:21.063308+0000","last_peered":"2024-09-10T20:30:21.063308+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:30:21.063308+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":4188,"log_dups_size":0,"ondisk_log_size":4188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17129472,"num_objects":4182,"num_object_clones":0,"num_object_copies":8364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":783,"num_write_kb":3132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":627},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4279","reported_seq":4398,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:21.052460+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:30:21.052460+0000","last_peered":"2024-09-10T20:30:21.052460+0000","last_clean":"2024-09-10T20:30:21.052460+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:30:21.052460+0000","last_undegraded":"2024-09-10T20:30:21.052460+0000","last_fullsized":"2024-09-10T20:30:21.052460+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":4279,"log_dups_size":0,"ondisk_log_size":4279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17518592,"num_objects":4277,"num_object_clones":0,"num_object_copies":8554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1350,"num_write_kb":5400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4051","reported_seq":4178,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:18.073468+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:30:18.073468+0000","last_peered":"2024-09-10T20:30:18.073468+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:30:18.073468+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":4051,"log_dups_size":0,"ondisk_log_size":4051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16564224,"num_objects":4044,"num_object_clones":0,"num_object_copies":8088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2044,"num_write_kb":8176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":518}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355438+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355438+0000","last_peered":"2024-09-10T20:30:05.355438+0000","last_clean":"2024-09-10T20:30:05.355438+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355438+0000","last_undegraded":"2024-09-10T20:30:05.355438+0000","last_fullsized":"2024-09-10T20:30:05.355438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4300","reported_seq":4403,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:21.092419+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:30:21.092419+0000","last_peered":"2024-09-10T20:30:21.092419+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:30:21.092419+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":4300,"log_dups_size":0,"ondisk_log_size":4300,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17580998,"num_objects":4294,"num_object_clones":0,"num_object_copies":8588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4298,"num_write_kb":17172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":679},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.413176+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.413176+0000","last_peered":"2024-09-10T20:30:05.413176+0000","last_clean":"2024-09-10T20:30:05.413176+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.413176+0000","last_undegraded":"2024-09-10T20:30:05.413176+0000","last_fullsized":"2024-09-10T20:30:05.413176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4129","reported_seq":4252,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:17.998338+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:30:17.998338+0000","last_peered":"2024-09-10T20:30:17.998338+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:30:17.998338+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":4129,"log_dups_size":0,"ondisk_log_size":4129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16891904,"num_objects":4124,"num_object_clones":0,"num_object_copies":8248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1957,"num_write_kb":7828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":491}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4125","reported_seq":4248,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:17.956360+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:30:17.956360+0000","last_peered":"2024-09-10T20:30:17.956360+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:30:17.956360+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":4125,"log_dups_size":0,"ondisk_log_size":4125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16871424,"num_objects":4119,"num_object_clones":0,"num_object_copies":8238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1911,"num_write_kb":7644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":509}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4242","reported_seq":4372,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:21.005182+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:30:21.005182+0000","last_peered":"2024-09-10T20:30:21.005182+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:30:21.005182+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":4242,"log_dups_size":0,"ondisk_log_size":4242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17367040,"num_objects":4240,"num_object_clones":0,"num_object_copies":8480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":965,"num_write_kb":3860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":592},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4208","reported_seq":4341,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:21.095217+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:30:21.095217+0000","last_peered":"2024-09-10T20:30:21.095217+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:30:21.095217+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":4208,"log_dups_size":0,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17203200,"num_objects":4200,"num_object_clones":0,"num_object_copies":8400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":902,"num_write_kb":3608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":668},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345705+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345705+0000","last_peered":"2024-09-10T20:30:05.345705+0000","last_clean":"2024-09-10T20:30:05.345705+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345705+0000","last_undegraded":"2024-09-10T20:30:05.345705+0000","last_fullsized":"2024-09-10T20:30:05.345705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4349","reported_seq":4474,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:21.079149+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:30:21.079149+0000","last_peered":"2024-09-10T20:30:21.079149+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:30:21.079149+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":4349,"log_dups_size":0,"ondisk_log_size":4349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17797120,"num_objects":4345,"num_object_clones":0,"num_object_copies":8690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1176,"num_write_kb":4704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":632},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4141","reported_seq":4301,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:21.110202+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:30:21.110202+0000","last_peered":"2024-09-10T20:30:21.110202+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:30:21.110202+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":4141,"log_dups_size":0,"ondisk_log_size":4141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16912867,"num_objects":4130,"num_object_clones":0,"num_object_copies":8260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":748,"num_write_kb":2992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":662},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":274875817,"num_objects":67111,"num_object_clones":0,"num_object_copies":134222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":3314,"num_objects_unfound":0,"num_objects_dirty":67111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31486,"num_write_kb":125924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":1187840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":556007424,"data_stored":555917099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":67190,"ondisk_log_size":67190,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182422,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":400644,"kb_used_data":260724,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":93971196,"statfs":{"total":96636764160,"available":96226504704,"internally_reserved":0,"allocated":266981376,"data_stored":266748243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":143257258},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,6,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986193,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":382708,"kb_used_data":246884,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":93989132,"statfs":{"total":96636764160,"available":96244871168,"internally_reserved":0,"allocated":252809216,"data_stored":252580160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":139062954},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,0,0,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659141,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83820,"kb_used_data":34076,"kb_used_omap":4,"kb_used_meta":49723,"kb_avail":94288020,"statfs":{"total":96636764160,"available":96550932480,"internally_reserved":0,"allocated":34893824,"data_stored":34581205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":50917038},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,2,4,7],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":267603968,"data_stored":267600355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":35475456,"data_stored":35392357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":252928000,"data_stored":252924387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:30:24.299 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:30:24.883 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:24.883 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:30:24.914 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2024-09-10T20:30:24.837024+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":279811516,"num_objects":68318,"num_object_clones":0,"num_object_copies":136636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":3200,"num_objects_unfound":0,"num_objects_dirty":68318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32693,"num_write_kb":130746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":408,"num_bytes_recovered":1654803,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68398,"ondisk_log_size":68398,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":877808,"kb_used_data":548160,"kb_used_omap":12,"kb_used_meta":329587,"kb_avail":282237712,"statfs":{"total":289910292480,"available":289011417088,"internally_reserved":0,"allocated":561315840,"data_stored":560541032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":337497090},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,7,10,8],"upper_bound":64},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32337920,"num_objects":7895,"num_object_clones":0,"num_object_copies":15790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-348,"num_objects_unfound":0,"num_objects_dirty":7895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7895,"num_write_kb":31580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":348,"num_bytes_recovered":1425408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002784"},"pg_stats":[{"pgid":"2.f","version":"43'4228","reported_seq":4297,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:21.102963+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:30:21.102963+0000","last_peered":"2024-09-10T20:30:21.102963+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:30:21.102963+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":4228,"log_dups_size":0,"ondisk_log_size":4228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17317888,"num_objects":4228,"num_object_clones":0,"num_object_copies":8456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4228,"num_write_kb":16912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":621},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4392","reported_seq":4497,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:23.090920+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:30:23.090920+0000","last_peered":"2024-09-10T20:30:23.090920+0000","last_clean":"2024-09-10T20:30:23.090920+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:30:23.090920+0000","last_undegraded":"2024-09-10T20:30:23.090920+0000","last_fullsized":"2024-09-10T20:30:23.090920+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":4392,"log_dups_size":0,"ondisk_log_size":4392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17965056,"num_objects":4386,"num_object_clones":0,"num_object_copies":8772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2578,"num_write_kb":10312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4312","reported_seq":5243,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:30:23.097315+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:30:23.097315+0000","last_peered":"2024-09-10T20:30:23.097315+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:30:23.097315+0000","last_undegraded":"2024-09-10T20:30:23.097315+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":4312,"log_dups_size":0,"ondisk_log_size":4312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17645568,"num_objects":4308,"num_object_clones":0,"num_object_copies":8616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3200,"num_objects_unfound":0,"num_objects_dirty":4308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2440,"num_write_kb":9760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":403,"num_bytes_recovered":1650688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3200},{"shards":"2","objects":1108}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4269","reported_seq":4344,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:23.062904+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:30:23.062904+0000","last_peered":"2024-09-10T20:30:23.062904+0000","last_clean":"2024-09-10T20:30:23.062904+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:30:23.062904+0000","last_undegraded":"2024-09-10T20:30:23.062904+0000","last_fullsized":"2024-09-10T20:30:23.062904+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":4269,"log_dups_size":0,"ondisk_log_size":4269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17485824,"num_objects":4269,"num_object_clones":0,"num_object_copies":8538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4269,"num_write_kb":17076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4330","reported_seq":4451,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:21.081597+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:30:21.081597+0000","last_peered":"2024-09-10T20:30:21.081597+0000","last_clean":"2024-09-10T20:30:21.081597+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:30:21.081597+0000","last_undegraded":"2024-09-10T20:30:21.081597+0000","last_fullsized":"2024-09-10T20:30:21.081597+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":4330,"log_dups_size":0,"ondisk_log_size":4330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17707008,"num_objects":4323,"num_object_clones":0,"num_object_copies":8646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1326,"num_write_kb":5304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4259","reported_seq":4390,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:21.091770+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:30:21.091770+0000","last_peered":"2024-09-10T20:30:21.091770+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:30:21.091770+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":4259,"log_dups_size":0,"ondisk_log_size":4259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17424384,"num_objects":4254,"num_object_clones":0,"num_object_copies":8508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1123,"num_write_kb":4492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":640},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4188","reported_seq":4321,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:21.063308+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:30:21.063308+0000","last_peered":"2024-09-10T20:30:21.063308+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:30:21.063308+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":4188,"log_dups_size":0,"ondisk_log_size":4188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17129472,"num_objects":4182,"num_object_clones":0,"num_object_copies":8364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":783,"num_write_kb":3132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":627},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4279","reported_seq":4398,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:21.052460+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:30:21.052460+0000","last_peered":"2024-09-10T20:30:21.052460+0000","last_clean":"2024-09-10T20:30:21.052460+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:30:21.052460+0000","last_undegraded":"2024-09-10T20:30:21.052460+0000","last_fullsized":"2024-09-10T20:30:21.052460+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":4279,"log_dups_size":0,"ondisk_log_size":4279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17518592,"num_objects":4277,"num_object_clones":0,"num_object_copies":8554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1350,"num_write_kb":5400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4246","reported_seq":4371,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:23.038996+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:30:23.038996+0000","last_peered":"2024-09-10T20:30:23.038996+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:30:23.038996+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":4246,"log_dups_size":0,"ondisk_log_size":4246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17362944,"num_objects":4239,"num_object_clones":0,"num_object_copies":8478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2239,"num_write_kb":8956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":713}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355438+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355438+0000","last_peered":"2024-09-10T20:30:05.355438+0000","last_clean":"2024-09-10T20:30:05.355438+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355438+0000","last_undegraded":"2024-09-10T20:30:05.355438+0000","last_fullsized":"2024-09-10T20:30:05.355438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4300","reported_seq":4403,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:21.092419+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:30:21.092419+0000","last_peered":"2024-09-10T20:30:21.092419+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:30:21.092419+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":4300,"log_dups_size":0,"ondisk_log_size":4300,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17580998,"num_objects":4294,"num_object_clones":0,"num_object_copies":8588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4298,"num_write_kb":17172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":679},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.413176+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.413176+0000","last_peered":"2024-09-10T20:30:05.413176+0000","last_clean":"2024-09-10T20:30:05.413176+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.413176+0000","last_undegraded":"2024-09-10T20:30:05.413176+0000","last_fullsized":"2024-09-10T20:30:05.413176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4332","reported_seq":4455,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:23.024895+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:30:23.024895+0000","last_peered":"2024-09-10T20:30:23.024895+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:30:23.024895+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":4332,"log_dups_size":0,"ondisk_log_size":4332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17723392,"num_objects":4327,"num_object_clones":0,"num_object_copies":8654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2160,"num_write_kb":8640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":694}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4320","reported_seq":4443,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:23.095187+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:30:23.095187+0000","last_peered":"2024-09-10T20:30:23.095187+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:30:23.095187+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":4320,"log_dups_size":0,"ondisk_log_size":4320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17670144,"num_objects":4314,"num_object_clones":0,"num_object_copies":8628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2106,"num_write_kb":8424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":704}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4242","reported_seq":4372,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:21.005182+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:30:21.005182+0000","last_peered":"2024-09-10T20:30:21.005182+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:30:21.005182+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":4242,"log_dups_size":0,"ondisk_log_size":4242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17367040,"num_objects":4240,"num_object_clones":0,"num_object_copies":8480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":965,"num_write_kb":3860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":592},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4208","reported_seq":4341,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:21.095217+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:30:21.095217+0000","last_peered":"2024-09-10T20:30:21.095217+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:30:21.095217+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":4208,"log_dups_size":0,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17203200,"num_objects":4200,"num_object_clones":0,"num_object_copies":8400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":902,"num_write_kb":3608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":668},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345705+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345705+0000","last_peered":"2024-09-10T20:30:05.345705+0000","last_clean":"2024-09-10T20:30:05.345705+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345705+0000","last_undegraded":"2024-09-10T20:30:05.345705+0000","last_fullsized":"2024-09-10T20:30:05.345705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4349","reported_seq":4474,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:21.079149+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:30:21.079149+0000","last_peered":"2024-09-10T20:30:21.079149+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:30:21.079149+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":4349,"log_dups_size":0,"ondisk_log_size":4349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17797120,"num_objects":4345,"num_object_clones":0,"num_object_copies":8690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1176,"num_write_kb":4704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":632},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4141","reported_seq":4301,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:21.110202+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:30:21.110202+0000","last_peered":"2024-09-10T20:30:21.110202+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:30:21.110202+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":4141,"log_dups_size":0,"ondisk_log_size":4141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16912867,"num_objects":4130,"num_object_clones":0,"num_object_copies":8260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":748,"num_write_kb":2992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":662},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":279811497,"num_objects":68316,"num_object_clones":0,"num_object_copies":136632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":3200,"num_objects_unfound":0,"num_objects_dirty":68316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32691,"num_write_kb":130744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":406,"num_bytes_recovered":1654784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":562671616,"data_stored":562581291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68395,"ondisk_log_size":68395,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182422,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":400644,"kb_used_data":260724,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":93971196,"statfs":{"total":96636764160,"available":96226504704,"internally_reserved":0,"allocated":266981376,"data_stored":266748243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":143257258},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,6,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986194,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":393344,"kb_used_data":253360,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93978496,"statfs":{"total":96636764160,"available":96233979904,"internally_reserved":0,"allocated":259440640,"data_stored":259211584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":143322794},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659141,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83820,"kb_used_data":34076,"kb_used_omap":4,"kb_used_meta":49723,"kb_avail":94288020,"statfs":{"total":96636764160,"available":96550932480,"internally_reserved":0,"allocated":34893824,"data_stored":34581205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":50917038},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,2,4,7],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":267603968,"data_stored":267600355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":35475456,"data_stored":35392357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":259592192,"data_stored":259588579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:30:24.916 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:30:24.916 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'4228", 'reported_seq': 4297, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:21.102963+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:30:21.102963+0000', 'last_peered': '2024-09-10T20:30:21.102963+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:30:21.102963+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 4228, 'log_dups_size': 0, 'ondisk_log_size': 4228, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17317888, 'num_objects': 4228, 'num_object_clones': 0, 'num_object_copies': 8456, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3607, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4228, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4228, 'num_write_kb': 16912, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 621}, {'shards': '0,2', 'objects': 3607}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:24.916 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T20:30:24.916 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'4312", 'reported_seq': 5243, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:30:23.097315+0000', 'last_change': '2024-09-10T20:30:05.653560+0000', 'last_active': '2024-09-10T20:30:23.097315+0000', 'last_peered': '2024-09-10T20:30:23.097315+0000', 'last_clean': '2024-09-10T20:30:03.279259+0000', 'last_became_active': '2024-09-10T20:30:05.629664+0000', 'last_became_peered': '2024-09-10T20:30:05.629664+0000', 'last_unstale': '2024-09-10T20:30:23.097315+0000', 'last_undegraded': '2024-09-10T20:30:23.097315+0000', 'last_fullsized': '2024-09-10T20:30:05.312061+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1910", 'last_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'objects_scrubbed': 1868, 'log_size': 4312, 'log_dups_size': 0, 'ondisk_log_size': 4312, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 673, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17645568, 'num_objects': 4308, 'num_object_clones': 0, 'num_object_copies': 8616, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3200, 'num_objects_unfound': 0, 'num_objects_dirty': 4308, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2440, 'num_write_kb': 9760, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 403, 'num_bytes_recovered': 1650688, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3200}, {'shards': '2', 'objects': 1108}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:24.917 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:30:24.917 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'4259", 'reported_seq': 4390, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:21.091770+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:30:21.091770+0000', 'last_peered': '2024-09-10T20:30:21.091770+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:30:21.091770+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 4259, 'log_dups_size': 0, 'ondisk_log_size': 4259, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17424384, 'num_objects': 4254, 'num_object_clones': 0, 'num_object_copies': 8508, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3616, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4254, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1123, 'num_write_kb': 4492, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 640}, {'shards': '0,2', 'objects': 3614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:24.917 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:30:24.917 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'4188", 'reported_seq': 4321, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:21.063308+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:30:21.063308+0000', 'last_peered': '2024-09-10T20:30:21.063308+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:30:21.063308+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 4188, 'log_dups_size': 0, 'ondisk_log_size': 4188, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17129472, 'num_objects': 4182, 'num_object_clones': 0, 'num_object_copies': 8364, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3557, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4182, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 783, 'num_write_kb': 3132, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 627}, {'shards': '0,2', 'objects': 3555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:24.917 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:30:24.917 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'4246", 'reported_seq': 4371, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:23.038996+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:30:23.038996+0000', 'last_peered': '2024-09-10T20:30:23.038996+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:30:23.038996+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 4246, 'log_dups_size': 0, 'ondisk_log_size': 4246, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17362944, 'num_objects': 4239, 'num_object_clones': 0, 'num_object_copies': 8478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3528, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4239, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2239, 'num_write_kb': 8956, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:24.917 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:30:24.917 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'4300", 'reported_seq': 4403, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:21.092419+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:30:21.092419+0000', 'last_peered': '2024-09-10T20:30:21.092419+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:30:21.092419+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 4300, 'log_dups_size': 0, 'ondisk_log_size': 4300, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17580998, 'num_objects': 4294, 'num_object_clones': 0, 'num_object_copies': 8588, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3617, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4294, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4298, 'num_write_kb': 17172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 679}, {'shards': '0,2', 'objects': 3615}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:24.917 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:30:24.917 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'4332", 'reported_seq': 4455, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:23.024895+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:30:23.024895+0000', 'last_peered': '2024-09-10T20:30:23.024895+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:30:23.024895+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 4332, 'log_dups_size': 0, 'ondisk_log_size': 4332, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17723392, 'num_objects': 4327, 'num_object_clones': 0, 'num_object_copies': 8654, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3635, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4327, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2160, 'num_write_kb': 8640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3633}, {'shards': '2', 'objects': 694}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:24.918 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:30:24.918 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'4320", 'reported_seq': 4443, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:23.095187+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:30:23.095187+0000', 'last_peered': '2024-09-10T20:30:23.095187+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:30:23.095187+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 4320, 'log_dups_size': 0, 'ondisk_log_size': 4320, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17670144, 'num_objects': 4314, 'num_object_clones': 0, 'num_object_copies': 8628, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4314, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2106, 'num_write_kb': 8424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3610}, {'shards': '2', 'objects': 704}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:24.918 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:30:24.918 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'4242", 'reported_seq': 4372, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:21.005182+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:30:21.005182+0000', 'last_peered': '2024-09-10T20:30:21.005182+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:30:21.005182+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 4242, 'log_dups_size': 0, 'ondisk_log_size': 4242, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17367040, 'num_objects': 4240, 'num_object_clones': 0, 'num_object_copies': 8480, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3649, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4240, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 965, 'num_write_kb': 3860, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 592}, {'shards': '0,2', 'objects': 3648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:24.918 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:30:24.918 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'4208", 'reported_seq': 4341, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:21.095217+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:30:21.095217+0000', 'last_peered': '2024-09-10T20:30:21.095217+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:30:21.095217+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 4208, 'log_dups_size': 0, 'ondisk_log_size': 4208, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17203200, 'num_objects': 4200, 'num_object_clones': 0, 'num_object_copies': 8400, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3535, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4200, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 902, 'num_write_kb': 3608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 668}, {'shards': '0,2', 'objects': 3532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:24.918 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:30:24.918 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'4349", 'reported_seq': 4474, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:21.079149+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:30:21.079149+0000', 'last_peered': '2024-09-10T20:30:21.079149+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:30:21.079149+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 4349, 'log_dups_size': 0, 'ondisk_log_size': 4349, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17797120, 'num_objects': 4345, 'num_object_clones': 0, 'num_object_copies': 8690, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3715, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4345, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1176, 'num_write_kb': 4704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 632}, {'shards': '0,2', 'objects': 3713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:24.918 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:30:24.918 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'4141", 'reported_seq': 4301, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:21.110202+0000', 'last_change': '2024-09-10T20:30:05.690055+0000', 'last_active': '2024-09-10T20:30:21.110202+0000', 'last_peered': '2024-09-10T20:30:21.110202+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:30:21.110202+0000', 'last_undegraded': '2024-09-10T20:30:05.378941+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 4141, 'log_dups_size': 0, 'ondisk_log_size': 4141, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16912867, 'num_objects': 4130, 'num_object_clones': 0, 'num_object_copies': 8260, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3471, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4130, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 748, 'num_write_kb': 2992, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 662}, {'shards': '0,2', 'objects': 3468}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:24.919 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:30:25.039 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 57 20 70403 70383 4.82257 2.375 0.0202916 0.0161894 2024-09-10T20:30:25.597 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:25.597 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39542/136636 objects degraded (28.940%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":137,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":68318,"data_bytes":279811516,"bytes_used":898875392,"bytes_avail":289011417088,"bytes_total":289910292480,"degraded_objects":39542,"degraded_total":136636,"degraded_ratio":0.28939664510085189,"misplaced_objects":3200,"misplaced_total":136636,"misplaced_ratio":0.023419889341022862,"recovering_objects_per_sec":28,"recovering_bytes_per_sec":118756,"recovering_keys_per_sec":0,"num_objects_recovered":348,"num_bytes_recovered":1425408,"num_keys_recovered":0,"write_bytes_sec":2694201,"read_op_per_sec":0,"write_op_per_sec":657},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-10T20:30:14.834797+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T20:30:25.618 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:30:25.619 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:30:26.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T20:30:25.998+0000 7f1ab7fff640 0 --2- 172.21.15.131:0/1779306636 >> v2:172.21.15.131:3300/0 conn(0x7f1ab808d0c0 0x7f1ab808d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-10T20:30:26.039 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 58 20 71023 71003 4.78118 2.42188 0.0368464 0.0163333 2024-09-10T20:30:26.376 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:26.376 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:30:26.401 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2024-09-10T20:30:24.837024+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":279811516,"num_objects":68318,"num_object_clones":0,"num_object_copies":136636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":3200,"num_objects_unfound":0,"num_objects_dirty":68318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32693,"num_write_kb":130746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":408,"num_bytes_recovered":1654803,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68398,"ondisk_log_size":68398,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":877808,"kb_used_data":548160,"kb_used_omap":12,"kb_used_meta":329587,"kb_avail":282237712,"statfs":{"total":289910292480,"available":289011417088,"internally_reserved":0,"allocated":561315840,"data_stored":560541032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":337497090},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,7,10,8],"upper_bound":64},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32337920,"num_objects":7895,"num_object_clones":0,"num_object_copies":15790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-348,"num_objects_unfound":0,"num_objects_dirty":7895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7895,"num_write_kb":31580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":348,"num_bytes_recovered":1425408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002784"},"pg_stats":[{"pgid":"2.f","version":"43'4228","reported_seq":4297,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:21.102963+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:30:21.102963+0000","last_peered":"2024-09-10T20:30:21.102963+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:30:21.102963+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":4228,"log_dups_size":0,"ondisk_log_size":4228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17317888,"num_objects":4228,"num_object_clones":0,"num_object_copies":8456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4228,"num_write_kb":16912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":621},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4392","reported_seq":4497,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:23.090920+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:30:23.090920+0000","last_peered":"2024-09-10T20:30:23.090920+0000","last_clean":"2024-09-10T20:30:23.090920+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:30:23.090920+0000","last_undegraded":"2024-09-10T20:30:23.090920+0000","last_fullsized":"2024-09-10T20:30:23.090920+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":4392,"log_dups_size":0,"ondisk_log_size":4392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17965056,"num_objects":4386,"num_object_clones":0,"num_object_copies":8772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2578,"num_write_kb":10312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4312","reported_seq":5243,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:30:23.097315+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:30:23.097315+0000","last_peered":"2024-09-10T20:30:23.097315+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:30:23.097315+0000","last_undegraded":"2024-09-10T20:30:23.097315+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":4312,"log_dups_size":0,"ondisk_log_size":4312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17645568,"num_objects":4308,"num_object_clones":0,"num_object_copies":8616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3200,"num_objects_unfound":0,"num_objects_dirty":4308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2440,"num_write_kb":9760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":403,"num_bytes_recovered":1650688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3200},{"shards":"2","objects":1108}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4269","reported_seq":4344,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:23.062904+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:30:23.062904+0000","last_peered":"2024-09-10T20:30:23.062904+0000","last_clean":"2024-09-10T20:30:23.062904+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:30:23.062904+0000","last_undegraded":"2024-09-10T20:30:23.062904+0000","last_fullsized":"2024-09-10T20:30:23.062904+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":4269,"log_dups_size":0,"ondisk_log_size":4269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17485824,"num_objects":4269,"num_object_clones":0,"num_object_copies":8538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4269,"num_write_kb":17076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4330","reported_seq":4451,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:21.081597+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:30:21.081597+0000","last_peered":"2024-09-10T20:30:21.081597+0000","last_clean":"2024-09-10T20:30:21.081597+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:30:21.081597+0000","last_undegraded":"2024-09-10T20:30:21.081597+0000","last_fullsized":"2024-09-10T20:30:21.081597+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":4330,"log_dups_size":0,"ondisk_log_size":4330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17707008,"num_objects":4323,"num_object_clones":0,"num_object_copies":8646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1326,"num_write_kb":5304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4259","reported_seq":4390,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:21.091770+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:30:21.091770+0000","last_peered":"2024-09-10T20:30:21.091770+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:30:21.091770+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":4259,"log_dups_size":0,"ondisk_log_size":4259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17424384,"num_objects":4254,"num_object_clones":0,"num_object_copies":8508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1123,"num_write_kb":4492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":640},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4188","reported_seq":4321,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:21.063308+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:30:21.063308+0000","last_peered":"2024-09-10T20:30:21.063308+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:30:21.063308+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":4188,"log_dups_size":0,"ondisk_log_size":4188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17129472,"num_objects":4182,"num_object_clones":0,"num_object_copies":8364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":783,"num_write_kb":3132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":627},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4279","reported_seq":4398,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:21.052460+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:30:21.052460+0000","last_peered":"2024-09-10T20:30:21.052460+0000","last_clean":"2024-09-10T20:30:21.052460+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:30:21.052460+0000","last_undegraded":"2024-09-10T20:30:21.052460+0000","last_fullsized":"2024-09-10T20:30:21.052460+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":4279,"log_dups_size":0,"ondisk_log_size":4279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17518592,"num_objects":4277,"num_object_clones":0,"num_object_copies":8554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1350,"num_write_kb":5400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4246","reported_seq":4371,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:23.038996+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:30:23.038996+0000","last_peered":"2024-09-10T20:30:23.038996+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:30:23.038996+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":4246,"log_dups_size":0,"ondisk_log_size":4246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17362944,"num_objects":4239,"num_object_clones":0,"num_object_copies":8478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2239,"num_write_kb":8956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":713}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355438+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355438+0000","last_peered":"2024-09-10T20:30:05.355438+0000","last_clean":"2024-09-10T20:30:05.355438+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355438+0000","last_undegraded":"2024-09-10T20:30:05.355438+0000","last_fullsized":"2024-09-10T20:30:05.355438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4300","reported_seq":4403,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:21.092419+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:30:21.092419+0000","last_peered":"2024-09-10T20:30:21.092419+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:30:21.092419+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":4300,"log_dups_size":0,"ondisk_log_size":4300,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17580998,"num_objects":4294,"num_object_clones":0,"num_object_copies":8588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4298,"num_write_kb":17172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":679},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.413176+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.413176+0000","last_peered":"2024-09-10T20:30:05.413176+0000","last_clean":"2024-09-10T20:30:05.413176+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.413176+0000","last_undegraded":"2024-09-10T20:30:05.413176+0000","last_fullsized":"2024-09-10T20:30:05.413176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4332","reported_seq":4455,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:23.024895+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:30:23.024895+0000","last_peered":"2024-09-10T20:30:23.024895+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:30:23.024895+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":4332,"log_dups_size":0,"ondisk_log_size":4332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17723392,"num_objects":4327,"num_object_clones":0,"num_object_copies":8654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2160,"num_write_kb":8640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":694}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4320","reported_seq":4443,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:23.095187+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:30:23.095187+0000","last_peered":"2024-09-10T20:30:23.095187+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:30:23.095187+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":4320,"log_dups_size":0,"ondisk_log_size":4320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17670144,"num_objects":4314,"num_object_clones":0,"num_object_copies":8628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2106,"num_write_kb":8424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":704}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4242","reported_seq":4372,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:21.005182+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:30:21.005182+0000","last_peered":"2024-09-10T20:30:21.005182+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:30:21.005182+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":4242,"log_dups_size":0,"ondisk_log_size":4242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17367040,"num_objects":4240,"num_object_clones":0,"num_object_copies":8480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":965,"num_write_kb":3860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":592},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4208","reported_seq":4341,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:21.095217+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:30:21.095217+0000","last_peered":"2024-09-10T20:30:21.095217+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:30:21.095217+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":4208,"log_dups_size":0,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17203200,"num_objects":4200,"num_object_clones":0,"num_object_copies":8400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":902,"num_write_kb":3608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":668},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345705+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345705+0000","last_peered":"2024-09-10T20:30:05.345705+0000","last_clean":"2024-09-10T20:30:05.345705+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345705+0000","last_undegraded":"2024-09-10T20:30:05.345705+0000","last_fullsized":"2024-09-10T20:30:05.345705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4349","reported_seq":4474,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:21.079149+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:30:21.079149+0000","last_peered":"2024-09-10T20:30:21.079149+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:30:21.079149+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":4349,"log_dups_size":0,"ondisk_log_size":4349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17797120,"num_objects":4345,"num_object_clones":0,"num_object_copies":8690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1176,"num_write_kb":4704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":632},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4141","reported_seq":4301,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:21.110202+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:30:21.110202+0000","last_peered":"2024-09-10T20:30:21.110202+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:30:21.110202+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":4141,"log_dups_size":0,"ondisk_log_size":4141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16912867,"num_objects":4130,"num_object_clones":0,"num_object_copies":8260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":748,"num_write_kb":2992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":662},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":279811497,"num_objects":68316,"num_object_clones":0,"num_object_copies":136632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":3200,"num_objects_unfound":0,"num_objects_dirty":68316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32691,"num_write_kb":130744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":406,"num_bytes_recovered":1654784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":562671616,"data_stored":562581291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68395,"ondisk_log_size":68395,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182422,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":400644,"kb_used_data":260724,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":93971196,"statfs":{"total":96636764160,"available":96226504704,"internally_reserved":0,"allocated":266981376,"data_stored":266748243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":143257258},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,6,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986194,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":393344,"kb_used_data":253360,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93978496,"statfs":{"total":96636764160,"available":96233979904,"internally_reserved":0,"allocated":259440640,"data_stored":259211584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":143322794},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659141,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83820,"kb_used_data":34076,"kb_used_omap":4,"kb_used_meta":49723,"kb_avail":94288020,"statfs":{"total":96636764160,"available":96550932480,"internally_reserved":0,"allocated":34893824,"data_stored":34581205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":50917038},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,2,4,7],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":267603968,"data_stored":267600355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":35475456,"data_stored":35392357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":259592192,"data_stored":259588579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:30:27.040 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 59 20 71669 71649 4.74292 2.52344 0.023211 0.0164626 2024-09-10T20:30:28.040 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout:2024-09-10T20:30:28.037106+0000 min lat: 0.00191414 max lat: 2.51074 avg lat: 0.0165847 2024-09-10T20:30:28.040 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T20:30:28.040 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 60 20 72359 72339 4.70879 2.69531 0.0413933 0.0165847 2024-09-10T20:30:29.040 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 61 20 73034 73014 4.67482 2.63672 0.0104659 0.0167046 2024-09-10T20:30:29.403 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:30:29.963 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:29.963 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:30:29.986 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2024-09-10T20:30:28.837987+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":293086652,"num_objects":71559,"num_object_clones":0,"num_object_copies":143118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":3085,"num_objects_unfound":0,"num_objects_dirty":71559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35934,"num_write_kb":143710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":523,"num_bytes_recovered":2125843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71639,"ondisk_log_size":71639,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":965388,"kb_used_data":574684,"kb_used_omap":12,"kb_used_meta":390643,"kb_avail":282150132,"statfs":{"total":289910292480,"available":288921735168,"internally_reserved":0,"allocated":588476416,"data_stored":587701608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":400018434},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,3,3,5,6,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":31932416,"num_objects":7796,"num_object_clones":0,"num_object_copies":15592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-346,"num_objects_unfound":0,"num_objects_dirty":7796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7796,"num_write_kb":31184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":346,"num_bytes_recovered":1417216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002836"},"pg_stats":[{"pgid":"2.f","version":"43'4409","reported_seq":4484,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:26.089594+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:30:26.089594+0000","last_peered":"2024-09-10T20:30:26.089594+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:30:26.089594+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":4409,"log_dups_size":0,"ondisk_log_size":4409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18059264,"num_objects":4409,"num_object_clones":0,"num_object_copies":8818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4409,"num_write_kb":17636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":802},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4629","reported_seq":4734,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:28.096829+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:30:28.096829+0000","last_peered":"2024-09-10T20:30:28.096829+0000","last_clean":"2024-09-10T20:30:28.096829+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:30:28.096829+0000","last_undegraded":"2024-09-10T20:30:28.096829+0000","last_fullsized":"2024-09-10T20:30:28.096829+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":4629,"log_dups_size":0,"ondisk_log_size":4629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18935808,"num_objects":4623,"num_object_clones":0,"num_object_copies":9246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2815,"num_write_kb":11260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4512","reported_seq":5673,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:30:28.078749+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:30:28.078749+0000","last_peered":"2024-09-10T20:30:28.078749+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:30:28.078749+0000","last_undegraded":"2024-09-10T20:30:28.078749+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":4512,"log_dups_size":0,"ondisk_log_size":4512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18464768,"num_objects":4508,"num_object_clones":0,"num_object_copies":9016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3085,"num_objects_unfound":0,"num_objects_dirty":4508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2640,"num_write_kb":10560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":518,"num_bytes_recovered":2121728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3085},{"shards":"2","objects":1423}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4461","reported_seq":4536,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:28.097446+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:30:28.097446+0000","last_peered":"2024-09-10T20:30:28.097446+0000","last_clean":"2024-09-10T20:30:28.097446+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:30:28.097446+0000","last_undegraded":"2024-09-10T20:30:28.097446+0000","last_fullsized":"2024-09-10T20:30:28.097446+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":4461,"log_dups_size":0,"ondisk_log_size":4461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18272256,"num_objects":4461,"num_object_clones":0,"num_object_copies":8922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4461,"num_write_kb":17844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4529","reported_seq":4650,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:26.038222+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:30:26.038222+0000","last_peered":"2024-09-10T20:30:26.038222+0000","last_clean":"2024-09-10T20:30:26.038222+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:30:26.038222+0000","last_undegraded":"2024-09-10T20:30:26.038222+0000","last_fullsized":"2024-09-10T20:30:26.038222+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":4529,"log_dups_size":0,"ondisk_log_size":4529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18522112,"num_objects":4522,"num_object_clones":0,"num_object_copies":9044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1525,"num_write_kb":6100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4476","reported_seq":4607,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:26.086018+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:30:26.086018+0000","last_peered":"2024-09-10T20:30:26.086018+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:30:26.086018+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":4476,"log_dups_size":0,"ondisk_log_size":4476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18313216,"num_objects":4471,"num_object_clones":0,"num_object_copies":8942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4471,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1340,"num_write_kb":5360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":857},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4393","reported_seq":4526,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:26.116539+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:30:26.116539+0000","last_peered":"2024-09-10T20:30:26.116539+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:30:26.116539+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":4393,"log_dups_size":0,"ondisk_log_size":4393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17969152,"num_objects":4387,"num_object_clones":0,"num_object_copies":8774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":988,"num_write_kb":3952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":832},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4481","reported_seq":4600,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:26.073414+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:30:26.073414+0000","last_peered":"2024-09-10T20:30:26.073414+0000","last_clean":"2024-09-10T20:30:26.073414+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:30:26.073414+0000","last_undegraded":"2024-09-10T20:30:26.073414+0000","last_fullsized":"2024-09-10T20:30:26.073414+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":4481,"log_dups_size":0,"ondisk_log_size":4481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18345984,"num_objects":4479,"num_object_clones":0,"num_object_copies":8958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1552,"num_write_kb":6208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4413","reported_seq":4540,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:28.024081+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:30:28.024081+0000","last_peered":"2024-09-10T20:30:28.024081+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:30:28.024081+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":4413,"log_dups_size":0,"ondisk_log_size":4413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18046976,"num_objects":4406,"num_object_clones":0,"num_object_copies":8812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2406,"num_write_kb":9624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":880}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355438+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355438+0000","last_peered":"2024-09-10T20:30:05.355438+0000","last_clean":"2024-09-10T20:30:05.355438+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355438+0000","last_undegraded":"2024-09-10T20:30:05.355438+0000","last_fullsized":"2024-09-10T20:30:05.355438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4498","reported_seq":4601,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:26.100936+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:30:26.100936+0000","last_peered":"2024-09-10T20:30:26.100936+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:30:26.100936+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":4498,"log_dups_size":0,"ondisk_log_size":4498,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18392006,"num_objects":4492,"num_object_clones":0,"num_object_copies":8984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4496,"num_write_kb":17964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":877},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.413176+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.413176+0000","last_peered":"2024-09-10T20:30:05.413176+0000","last_clean":"2024-09-10T20:30:05.413176+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.413176+0000","last_undegraded":"2024-09-10T20:30:05.413176+0000","last_fullsized":"2024-09-10T20:30:05.413176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4526","reported_seq":4649,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:28.053559+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:30:28.053559+0000","last_peered":"2024-09-10T20:30:28.053559+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:30:28.053559+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":4526,"log_dups_size":0,"ondisk_log_size":4526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18518016,"num_objects":4521,"num_object_clones":0,"num_object_copies":9042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2354,"num_write_kb":9416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":888}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4524","reported_seq":4647,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:28.116447+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:30:28.116447+0000","last_peered":"2024-09-10T20:30:28.116447+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:30:28.116447+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":4524,"log_dups_size":0,"ondisk_log_size":4524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18505728,"num_objects":4518,"num_object_clones":0,"num_object_copies":9036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2310,"num_write_kb":9240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":908}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4453","reported_seq":4583,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:26.027287+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:30:26.027287+0000","last_peered":"2024-09-10T20:30:26.027287+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:30:26.027287+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":4453,"log_dups_size":0,"ondisk_log_size":4453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18231296,"num_objects":4451,"num_object_clones":0,"num_object_copies":8902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1176,"num_write_kb":4704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":803},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4391","reported_seq":4524,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:26.110378+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:30:26.110378+0000","last_peered":"2024-09-10T20:30:26.110378+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:30:26.110378+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":4391,"log_dups_size":0,"ondisk_log_size":4391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17952768,"num_objects":4383,"num_object_clones":0,"num_object_copies":8766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4383,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1085,"num_write_kb":4340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":851},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345705+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345705+0000","last_peered":"2024-09-10T20:30:05.345705+0000","last_clean":"2024-09-10T20:30:05.345705+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345705+0000","last_undegraded":"2024-09-10T20:30:05.345705+0000","last_fullsized":"2024-09-10T20:30:05.345705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4572","reported_seq":4698,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:26.098977+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:30:26.098977+0000","last_peered":"2024-09-10T20:30:26.098977+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:30:26.098977+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":4572,"log_dups_size":0,"ondisk_log_size":4572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18710528,"num_objects":4568,"num_object_clones":0,"num_object_copies":9136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1399,"num_write_kb":5596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":855},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4369","reported_seq":4529,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:26.112480+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:30:26.112480+0000","last_peered":"2024-09-10T20:30:26.112480+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:30:26.112480+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":4369,"log_dups_size":0,"ondisk_log_size":4369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17846755,"num_objects":4358,"num_object_clones":0,"num_object_copies":8716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":976,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":890},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":293086633,"num_objects":71557,"num_object_clones":0,"num_object_copies":143114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":3085,"num_objects_unfound":0,"num_objects_dirty":71557,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35932,"num_write_kb":143708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":521,"num_bytes_recovered":2125824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589864960,"data_stored":589774635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71636,"ondisk_log_size":71636,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182423,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":419052,"kb_used_data":270812,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93952788,"statfs":{"total":96636764160,"available":96207654912,"internally_reserved":0,"allocated":277311488,"data_stored":277078355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":151776938},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,0,2,2,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986195,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":408108,"kb_used_data":259804,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":93963732,"statfs":{"total":96636764160,"available":96218861568,"internally_reserved":0,"allocated":266039296,"data_stored":265810240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":151842474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,2,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659142,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":138228,"kb_used_data":44068,"kb_used_omap":4,"kb_used_meta":94139,"kb_avail":94233612,"statfs":{"total":96636764160,"available":96495218688,"internally_reserved":0,"allocated":45125632,"data_stored":44813013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":96399022},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,1,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":277798912,"data_stored":277795299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45895680,"data_stored":45812581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266170368,"data_stored":266166755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:30:29.988 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:30:30.040 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 62 20 73657 73637 4.63867 2.43359 0.0386627 0.0168339 2024-09-10T20:30:30.556 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:30.556 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:30:30.583 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2024-09-10T20:30:28.837987+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":293086652,"num_objects":71559,"num_object_clones":0,"num_object_copies":143118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":3085,"num_objects_unfound":0,"num_objects_dirty":71559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35934,"num_write_kb":143710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":523,"num_bytes_recovered":2125843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71639,"ondisk_log_size":71639,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":965388,"kb_used_data":574684,"kb_used_omap":12,"kb_used_meta":390643,"kb_avail":282150132,"statfs":{"total":289910292480,"available":288921735168,"internally_reserved":0,"allocated":588476416,"data_stored":587701608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":400018434},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,3,3,5,6,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":31932416,"num_objects":7796,"num_object_clones":0,"num_object_copies":15592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-346,"num_objects_unfound":0,"num_objects_dirty":7796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7796,"num_write_kb":31184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":346,"num_bytes_recovered":1417216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002836"},"pg_stats":[{"pgid":"2.f","version":"43'4409","reported_seq":4484,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:26.089594+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:30:26.089594+0000","last_peered":"2024-09-10T20:30:26.089594+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:30:26.089594+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":4409,"log_dups_size":0,"ondisk_log_size":4409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18059264,"num_objects":4409,"num_object_clones":0,"num_object_copies":8818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4409,"num_write_kb":17636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":802},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4629","reported_seq":4734,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:28.096829+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:30:28.096829+0000","last_peered":"2024-09-10T20:30:28.096829+0000","last_clean":"2024-09-10T20:30:28.096829+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:30:28.096829+0000","last_undegraded":"2024-09-10T20:30:28.096829+0000","last_fullsized":"2024-09-10T20:30:28.096829+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":4629,"log_dups_size":0,"ondisk_log_size":4629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18935808,"num_objects":4623,"num_object_clones":0,"num_object_copies":9246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2815,"num_write_kb":11260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4512","reported_seq":5673,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:30:28.078749+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:30:28.078749+0000","last_peered":"2024-09-10T20:30:28.078749+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:30:28.078749+0000","last_undegraded":"2024-09-10T20:30:28.078749+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":4512,"log_dups_size":0,"ondisk_log_size":4512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18464768,"num_objects":4508,"num_object_clones":0,"num_object_copies":9016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3085,"num_objects_unfound":0,"num_objects_dirty":4508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2640,"num_write_kb":10560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":518,"num_bytes_recovered":2121728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3085},{"shards":"2","objects":1423}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4461","reported_seq":4536,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:28.097446+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:30:28.097446+0000","last_peered":"2024-09-10T20:30:28.097446+0000","last_clean":"2024-09-10T20:30:28.097446+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:30:28.097446+0000","last_undegraded":"2024-09-10T20:30:28.097446+0000","last_fullsized":"2024-09-10T20:30:28.097446+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":4461,"log_dups_size":0,"ondisk_log_size":4461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18272256,"num_objects":4461,"num_object_clones":0,"num_object_copies":8922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4461,"num_write_kb":17844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4529","reported_seq":4650,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:26.038222+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:30:26.038222+0000","last_peered":"2024-09-10T20:30:26.038222+0000","last_clean":"2024-09-10T20:30:26.038222+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:30:26.038222+0000","last_undegraded":"2024-09-10T20:30:26.038222+0000","last_fullsized":"2024-09-10T20:30:26.038222+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":4529,"log_dups_size":0,"ondisk_log_size":4529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18522112,"num_objects":4522,"num_object_clones":0,"num_object_copies":9044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1525,"num_write_kb":6100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4476","reported_seq":4607,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:26.086018+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:30:26.086018+0000","last_peered":"2024-09-10T20:30:26.086018+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:30:26.086018+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":4476,"log_dups_size":0,"ondisk_log_size":4476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18313216,"num_objects":4471,"num_object_clones":0,"num_object_copies":8942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4471,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1340,"num_write_kb":5360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":857},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4393","reported_seq":4526,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:26.116539+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:30:26.116539+0000","last_peered":"2024-09-10T20:30:26.116539+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:30:26.116539+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":4393,"log_dups_size":0,"ondisk_log_size":4393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17969152,"num_objects":4387,"num_object_clones":0,"num_object_copies":8774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":988,"num_write_kb":3952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":832},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4481","reported_seq":4600,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:26.073414+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:30:26.073414+0000","last_peered":"2024-09-10T20:30:26.073414+0000","last_clean":"2024-09-10T20:30:26.073414+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:30:26.073414+0000","last_undegraded":"2024-09-10T20:30:26.073414+0000","last_fullsized":"2024-09-10T20:30:26.073414+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":4481,"log_dups_size":0,"ondisk_log_size":4481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18345984,"num_objects":4479,"num_object_clones":0,"num_object_copies":8958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1552,"num_write_kb":6208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4413","reported_seq":4540,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:28.024081+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:30:28.024081+0000","last_peered":"2024-09-10T20:30:28.024081+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:30:28.024081+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":4413,"log_dups_size":0,"ondisk_log_size":4413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18046976,"num_objects":4406,"num_object_clones":0,"num_object_copies":8812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2406,"num_write_kb":9624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":880}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355438+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355438+0000","last_peered":"2024-09-10T20:30:05.355438+0000","last_clean":"2024-09-10T20:30:05.355438+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355438+0000","last_undegraded":"2024-09-10T20:30:05.355438+0000","last_fullsized":"2024-09-10T20:30:05.355438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4498","reported_seq":4601,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:26.100936+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:30:26.100936+0000","last_peered":"2024-09-10T20:30:26.100936+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:30:26.100936+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":4498,"log_dups_size":0,"ondisk_log_size":4498,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18392006,"num_objects":4492,"num_object_clones":0,"num_object_copies":8984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4496,"num_write_kb":17964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":877},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.413176+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.413176+0000","last_peered":"2024-09-10T20:30:05.413176+0000","last_clean":"2024-09-10T20:30:05.413176+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.413176+0000","last_undegraded":"2024-09-10T20:30:05.413176+0000","last_fullsized":"2024-09-10T20:30:05.413176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4526","reported_seq":4649,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:28.053559+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:30:28.053559+0000","last_peered":"2024-09-10T20:30:28.053559+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:30:28.053559+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":4526,"log_dups_size":0,"ondisk_log_size":4526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18518016,"num_objects":4521,"num_object_clones":0,"num_object_copies":9042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2354,"num_write_kb":9416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":888}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4524","reported_seq":4647,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:28.116447+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:30:28.116447+0000","last_peered":"2024-09-10T20:30:28.116447+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:30:28.116447+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":4524,"log_dups_size":0,"ondisk_log_size":4524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18505728,"num_objects":4518,"num_object_clones":0,"num_object_copies":9036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2310,"num_write_kb":9240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":908}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4453","reported_seq":4583,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:26.027287+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:30:26.027287+0000","last_peered":"2024-09-10T20:30:26.027287+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:30:26.027287+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":4453,"log_dups_size":0,"ondisk_log_size":4453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18231296,"num_objects":4451,"num_object_clones":0,"num_object_copies":8902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1176,"num_write_kb":4704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":803},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4391","reported_seq":4524,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:26.110378+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:30:26.110378+0000","last_peered":"2024-09-10T20:30:26.110378+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:30:26.110378+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":4391,"log_dups_size":0,"ondisk_log_size":4391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17952768,"num_objects":4383,"num_object_clones":0,"num_object_copies":8766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4383,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1085,"num_write_kb":4340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":851},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345705+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345705+0000","last_peered":"2024-09-10T20:30:05.345705+0000","last_clean":"2024-09-10T20:30:05.345705+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345705+0000","last_undegraded":"2024-09-10T20:30:05.345705+0000","last_fullsized":"2024-09-10T20:30:05.345705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4572","reported_seq":4698,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:26.098977+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:30:26.098977+0000","last_peered":"2024-09-10T20:30:26.098977+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:30:26.098977+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":4572,"log_dups_size":0,"ondisk_log_size":4572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18710528,"num_objects":4568,"num_object_clones":0,"num_object_copies":9136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1399,"num_write_kb":5596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":855},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4369","reported_seq":4529,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:26.112480+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:30:26.112480+0000","last_peered":"2024-09-10T20:30:26.112480+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:30:26.112480+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":4369,"log_dups_size":0,"ondisk_log_size":4369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17846755,"num_objects":4358,"num_object_clones":0,"num_object_copies":8716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":976,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":890},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":293086633,"num_objects":71557,"num_object_clones":0,"num_object_copies":143114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":3085,"num_objects_unfound":0,"num_objects_dirty":71557,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35932,"num_write_kb":143708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":521,"num_bytes_recovered":2125824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589864960,"data_stored":589774635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71636,"ondisk_log_size":71636,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182423,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":419052,"kb_used_data":270812,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93952788,"statfs":{"total":96636764160,"available":96207654912,"internally_reserved":0,"allocated":277311488,"data_stored":277078355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":151776938},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,0,2,2,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986195,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":408108,"kb_used_data":259804,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":93963732,"statfs":{"total":96636764160,"available":96218861568,"internally_reserved":0,"allocated":266039296,"data_stored":265810240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":151842474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,2,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659142,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":138228,"kb_used_data":44068,"kb_used_omap":4,"kb_used_meta":94139,"kb_avail":94233612,"statfs":{"total":96636764160,"available":96495218688,"internally_reserved":0,"allocated":45125632,"data_stored":44813013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":96399022},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,1,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":277798912,"data_stored":277795299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45895680,"data_stored":45812581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266170368,"data_stored":266166755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:30:30.585 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:30:30.585 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'4409", 'reported_seq': 4484, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:26.089594+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:30:26.089594+0000', 'last_peered': '2024-09-10T20:30:26.089594+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:30:26.089594+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 4409, 'log_dups_size': 0, 'ondisk_log_size': 4409, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18059264, 'num_objects': 4409, 'num_object_clones': 0, 'num_object_copies': 8818, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3607, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4409, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4409, 'num_write_kb': 17636, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 802}, {'shards': '0,2', 'objects': 3607}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:30.585 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T20:30:30.585 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'4512", 'reported_seq': 5673, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:30:28.078749+0000', 'last_change': '2024-09-10T20:30:05.653560+0000', 'last_active': '2024-09-10T20:30:28.078749+0000', 'last_peered': '2024-09-10T20:30:28.078749+0000', 'last_clean': '2024-09-10T20:30:03.279259+0000', 'last_became_active': '2024-09-10T20:30:05.629664+0000', 'last_became_peered': '2024-09-10T20:30:05.629664+0000', 'last_unstale': '2024-09-10T20:30:28.078749+0000', 'last_undegraded': '2024-09-10T20:30:28.078749+0000', 'last_fullsized': '2024-09-10T20:30:05.312061+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1910", 'last_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'objects_scrubbed': 1868, 'log_size': 4512, 'log_dups_size': 0, 'ondisk_log_size': 4512, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 673, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18464768, 'num_objects': 4508, 'num_object_clones': 0, 'num_object_copies': 9016, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3085, 'num_objects_unfound': 0, 'num_objects_dirty': 4508, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2640, 'num_write_kb': 10560, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 518, 'num_bytes_recovered': 2121728, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3085}, {'shards': '2', 'objects': 1423}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:30.585 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:30:30.585 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'4476", 'reported_seq': 4607, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:26.086018+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:30:26.086018+0000', 'last_peered': '2024-09-10T20:30:26.086018+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:30:26.086018+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 4476, 'log_dups_size': 0, 'ondisk_log_size': 4476, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18313216, 'num_objects': 4471, 'num_object_clones': 0, 'num_object_copies': 8942, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3616, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4471, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1340, 'num_write_kb': 5360, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 857}, {'shards': '0,2', 'objects': 3614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:30.585 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:30:30.586 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'4393", 'reported_seq': 4526, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:26.116539+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:30:26.116539+0000', 'last_peered': '2024-09-10T20:30:26.116539+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:30:26.116539+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 4393, 'log_dups_size': 0, 'ondisk_log_size': 4393, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17969152, 'num_objects': 4387, 'num_object_clones': 0, 'num_object_copies': 8774, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3557, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4387, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 988, 'num_write_kb': 3952, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 832}, {'shards': '0,2', 'objects': 3555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:30.586 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:30:30.586 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'4413", 'reported_seq': 4540, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:28.024081+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:30:28.024081+0000', 'last_peered': '2024-09-10T20:30:28.024081+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:30:28.024081+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 4413, 'log_dups_size': 0, 'ondisk_log_size': 4413, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18046976, 'num_objects': 4406, 'num_object_clones': 0, 'num_object_copies': 8812, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3528, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4406, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2406, 'num_write_kb': 9624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 880}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:30.586 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:30:30.586 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'4498", 'reported_seq': 4601, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:26.100936+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:30:26.100936+0000', 'last_peered': '2024-09-10T20:30:26.100936+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:30:26.100936+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 4498, 'log_dups_size': 0, 'ondisk_log_size': 4498, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18392006, 'num_objects': 4492, 'num_object_clones': 0, 'num_object_copies': 8984, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3617, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4492, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4496, 'num_write_kb': 17964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 877}, {'shards': '0,2', 'objects': 3615}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:30.586 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:30:30.586 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'4526", 'reported_seq': 4649, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:28.053559+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:30:28.053559+0000', 'last_peered': '2024-09-10T20:30:28.053559+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:30:28.053559+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 4526, 'log_dups_size': 0, 'ondisk_log_size': 4526, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18518016, 'num_objects': 4521, 'num_object_clones': 0, 'num_object_copies': 9042, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3635, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4521, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2354, 'num_write_kb': 9416, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3633}, {'shards': '2', 'objects': 888}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:30.586 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:30:30.586 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'4524", 'reported_seq': 4647, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:28.116447+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:30:28.116447+0000', 'last_peered': '2024-09-10T20:30:28.116447+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:30:28.116447+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 4524, 'log_dups_size': 0, 'ondisk_log_size': 4524, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18505728, 'num_objects': 4518, 'num_object_clones': 0, 'num_object_copies': 9036, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4518, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2310, 'num_write_kb': 9240, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3610}, {'shards': '2', 'objects': 908}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:30.587 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:30:30.587 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'4453", 'reported_seq': 4583, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:26.027287+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:30:26.027287+0000', 'last_peered': '2024-09-10T20:30:26.027287+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:30:26.027287+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 4453, 'log_dups_size': 0, 'ondisk_log_size': 4453, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18231296, 'num_objects': 4451, 'num_object_clones': 0, 'num_object_copies': 8902, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3649, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4451, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1176, 'num_write_kb': 4704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 803}, {'shards': '0,2', 'objects': 3648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:30.587 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:30:30.587 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'4391", 'reported_seq': 4524, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:26.110378+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:30:26.110378+0000', 'last_peered': '2024-09-10T20:30:26.110378+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:30:26.110378+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 4391, 'log_dups_size': 0, 'ondisk_log_size': 4391, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17952768, 'num_objects': 4383, 'num_object_clones': 0, 'num_object_copies': 8766, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3535, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4383, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1085, 'num_write_kb': 4340, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 851}, {'shards': '0,2', 'objects': 3532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:30.587 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:30:30.587 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'4572", 'reported_seq': 4698, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:26.098977+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:30:26.098977+0000', 'last_peered': '2024-09-10T20:30:26.098977+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:30:26.098977+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 4572, 'log_dups_size': 0, 'ondisk_log_size': 4572, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18710528, 'num_objects': 4568, 'num_object_clones': 0, 'num_object_copies': 9136, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3715, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4568, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1399, 'num_write_kb': 5596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 855}, {'shards': '0,2', 'objects': 3713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:30.587 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:30:30.587 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'4369", 'reported_seq': 4529, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:26.112480+0000', 'last_change': '2024-09-10T20:30:05.690055+0000', 'last_active': '2024-09-10T20:30:26.112480+0000', 'last_peered': '2024-09-10T20:30:26.112480+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:30:26.112480+0000', 'last_undegraded': '2024-09-10T20:30:05.378941+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 4369, 'log_dups_size': 0, 'ondisk_log_size': 4369, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17846755, 'num_objects': 4358, 'num_object_clones': 0, 'num_object_copies': 8716, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3471, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4358, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 976, 'num_write_kb': 3904, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 890}, {'shards': '0,2', 'objects': 3468}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:30.588 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:30:31.040 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 63 20 74254 74234 4.60206 2.33203 0.0303487 0.0169657 2024-09-10T20:30:31.295 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:31.295 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39542/143118 objects degraded (27.629%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":142,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":71559,"data_bytes":293086652,"bytes_used":988557312,"bytes_avail":288921735168,"bytes_total":289910292480,"degraded_objects":39542,"degraded_total":143118,"degraded_ratio":0.27628949538143349,"misplaced_objects":3085,"misplaced_total":143118,"misplaced_ratio":0.021555639402451129,"recovering_objects_per_sec":28,"recovering_bytes_per_sec":118073,"recovering_keys_per_sec":0,"num_objects_recovered":346,"num_bytes_recovered":1417216,"num_keys_recovered":0,"write_bytes_sec":2660405,"read_op_per_sec":0,"write_op_per_sec":649},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-10T20:30:14.834797+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T20:30:31.316 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:30:31.316 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:30:31.873 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:31.874 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:30:31.897 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":108,"stamp":"2024-09-10T20:30:30.838446+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":293086652,"num_objects":71559,"num_object_clones":0,"num_object_copies":143118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":3085,"num_objects_unfound":0,"num_objects_dirty":71559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35934,"num_write_kb":143710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":523,"num_bytes_recovered":2125843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71639,"ondisk_log_size":71639,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":965388,"kb_used_data":574684,"kb_used_omap":12,"kb_used_meta":390643,"kb_avail":282150132,"statfs":{"total":289910292480,"available":288921735168,"internally_reserved":0,"allocated":588476416,"data_stored":587701608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":400018434},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,3,3,5,6,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26812416,"num_objects":6546,"num_object_clones":0,"num_object_copies":13092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-229,"num_objects_unfound":0,"num_objects_dirty":6546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6546,"num_write_kb":26184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":937984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002843"},"pg_stats":[{"pgid":"2.f","version":"43'4409","reported_seq":4484,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:26.089594+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:30:26.089594+0000","last_peered":"2024-09-10T20:30:26.089594+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:30:26.089594+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":4409,"log_dups_size":0,"ondisk_log_size":4409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18059264,"num_objects":4409,"num_object_clones":0,"num_object_copies":8818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4409,"num_write_kb":17636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":802},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4629","reported_seq":4734,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:28.096829+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:30:28.096829+0000","last_peered":"2024-09-10T20:30:28.096829+0000","last_clean":"2024-09-10T20:30:28.096829+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:30:28.096829+0000","last_undegraded":"2024-09-10T20:30:28.096829+0000","last_fullsized":"2024-09-10T20:30:28.096829+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":4629,"log_dups_size":0,"ondisk_log_size":4629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18935808,"num_objects":4623,"num_object_clones":0,"num_object_copies":9246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2815,"num_write_kb":11260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4512","reported_seq":5673,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:30:28.078749+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:30:28.078749+0000","last_peered":"2024-09-10T20:30:28.078749+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:30:28.078749+0000","last_undegraded":"2024-09-10T20:30:28.078749+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":4512,"log_dups_size":0,"ondisk_log_size":4512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18464768,"num_objects":4508,"num_object_clones":0,"num_object_copies":9016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3085,"num_objects_unfound":0,"num_objects_dirty":4508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2640,"num_write_kb":10560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":518,"num_bytes_recovered":2121728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3085},{"shards":"2","objects":1423}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4461","reported_seq":4536,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:28.097446+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:30:28.097446+0000","last_peered":"2024-09-10T20:30:28.097446+0000","last_clean":"2024-09-10T20:30:28.097446+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:30:28.097446+0000","last_undegraded":"2024-09-10T20:30:28.097446+0000","last_fullsized":"2024-09-10T20:30:28.097446+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":4461,"log_dups_size":0,"ondisk_log_size":4461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18272256,"num_objects":4461,"num_object_clones":0,"num_object_copies":8922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4461,"num_write_kb":17844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4529","reported_seq":4650,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:26.038222+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:30:26.038222+0000","last_peered":"2024-09-10T20:30:26.038222+0000","last_clean":"2024-09-10T20:30:26.038222+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:30:26.038222+0000","last_undegraded":"2024-09-10T20:30:26.038222+0000","last_fullsized":"2024-09-10T20:30:26.038222+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":4529,"log_dups_size":0,"ondisk_log_size":4529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18522112,"num_objects":4522,"num_object_clones":0,"num_object_copies":9044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1525,"num_write_kb":6100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4476","reported_seq":4607,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:26.086018+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:30:26.086018+0000","last_peered":"2024-09-10T20:30:26.086018+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:30:26.086018+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":4476,"log_dups_size":0,"ondisk_log_size":4476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18313216,"num_objects":4471,"num_object_clones":0,"num_object_copies":8942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4471,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1340,"num_write_kb":5360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":857},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4393","reported_seq":4526,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:26.116539+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:30:26.116539+0000","last_peered":"2024-09-10T20:30:26.116539+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:30:26.116539+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":4393,"log_dups_size":0,"ondisk_log_size":4393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17969152,"num_objects":4387,"num_object_clones":0,"num_object_copies":8774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":988,"num_write_kb":3952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":832},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4481","reported_seq":4600,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:26.073414+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:30:26.073414+0000","last_peered":"2024-09-10T20:30:26.073414+0000","last_clean":"2024-09-10T20:30:26.073414+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:30:26.073414+0000","last_undegraded":"2024-09-10T20:30:26.073414+0000","last_fullsized":"2024-09-10T20:30:26.073414+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":4481,"log_dups_size":0,"ondisk_log_size":4481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18345984,"num_objects":4479,"num_object_clones":0,"num_object_copies":8958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1552,"num_write_kb":6208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4413","reported_seq":4540,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:28.024081+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:30:28.024081+0000","last_peered":"2024-09-10T20:30:28.024081+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:30:28.024081+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":4413,"log_dups_size":0,"ondisk_log_size":4413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18046976,"num_objects":4406,"num_object_clones":0,"num_object_copies":8812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2406,"num_write_kb":9624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":880}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355438+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355438+0000","last_peered":"2024-09-10T20:30:05.355438+0000","last_clean":"2024-09-10T20:30:05.355438+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355438+0000","last_undegraded":"2024-09-10T20:30:05.355438+0000","last_fullsized":"2024-09-10T20:30:05.355438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4498","reported_seq":4601,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:26.100936+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:30:26.100936+0000","last_peered":"2024-09-10T20:30:26.100936+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:30:26.100936+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":4498,"log_dups_size":0,"ondisk_log_size":4498,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18392006,"num_objects":4492,"num_object_clones":0,"num_object_copies":8984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4496,"num_write_kb":17964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":877},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.413176+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.413176+0000","last_peered":"2024-09-10T20:30:05.413176+0000","last_clean":"2024-09-10T20:30:05.413176+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.413176+0000","last_undegraded":"2024-09-10T20:30:05.413176+0000","last_fullsized":"2024-09-10T20:30:05.413176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4526","reported_seq":4649,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:28.053559+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:30:28.053559+0000","last_peered":"2024-09-10T20:30:28.053559+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:30:28.053559+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":4526,"log_dups_size":0,"ondisk_log_size":4526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18518016,"num_objects":4521,"num_object_clones":0,"num_object_copies":9042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2354,"num_write_kb":9416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":888}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4524","reported_seq":4647,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:28.116447+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:30:28.116447+0000","last_peered":"2024-09-10T20:30:28.116447+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:30:28.116447+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":4524,"log_dups_size":0,"ondisk_log_size":4524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18505728,"num_objects":4518,"num_object_clones":0,"num_object_copies":9036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2310,"num_write_kb":9240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":908}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4453","reported_seq":4583,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:26.027287+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:30:26.027287+0000","last_peered":"2024-09-10T20:30:26.027287+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:30:26.027287+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":4453,"log_dups_size":0,"ondisk_log_size":4453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18231296,"num_objects":4451,"num_object_clones":0,"num_object_copies":8902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1176,"num_write_kb":4704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":803},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4391","reported_seq":4524,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:26.110378+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:30:26.110378+0000","last_peered":"2024-09-10T20:30:26.110378+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:30:26.110378+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":4391,"log_dups_size":0,"ondisk_log_size":4391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17952768,"num_objects":4383,"num_object_clones":0,"num_object_copies":8766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4383,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1085,"num_write_kb":4340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":851},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345705+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345705+0000","last_peered":"2024-09-10T20:30:05.345705+0000","last_clean":"2024-09-10T20:30:05.345705+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345705+0000","last_undegraded":"2024-09-10T20:30:05.345705+0000","last_fullsized":"2024-09-10T20:30:05.345705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4572","reported_seq":4698,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:26.098977+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:30:26.098977+0000","last_peered":"2024-09-10T20:30:26.098977+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:30:26.098977+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":4572,"log_dups_size":0,"ondisk_log_size":4572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18710528,"num_objects":4568,"num_object_clones":0,"num_object_copies":9136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1399,"num_write_kb":5596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":855},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4369","reported_seq":4529,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:26.112480+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:30:26.112480+0000","last_peered":"2024-09-10T20:30:26.112480+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:30:26.112480+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":4369,"log_dups_size":0,"ondisk_log_size":4369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17846755,"num_objects":4358,"num_object_clones":0,"num_object_copies":8716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":976,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":890},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":293086633,"num_objects":71557,"num_object_clones":0,"num_object_copies":143114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":3085,"num_objects_unfound":0,"num_objects_dirty":71557,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35932,"num_write_kb":143708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":521,"num_bytes_recovered":2125824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589864960,"data_stored":589774635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71636,"ondisk_log_size":71636,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182423,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":419052,"kb_used_data":270812,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93952788,"statfs":{"total":96636764160,"available":96207654912,"internally_reserved":0,"allocated":277311488,"data_stored":277078355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":151776938},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,0,2,2,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986195,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":408108,"kb_used_data":259804,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":93963732,"statfs":{"total":96636764160,"available":96218861568,"internally_reserved":0,"allocated":266039296,"data_stored":265810240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":151842474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,2,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659142,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":138228,"kb_used_data":44068,"kb_used_omap":4,"kb_used_meta":94139,"kb_avail":94233612,"statfs":{"total":96636764160,"available":96495218688,"internally_reserved":0,"allocated":45125632,"data_stored":44813013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":96399022},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,1,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":277798912,"data_stored":277795299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45895680,"data_stored":45812581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266170368,"data_stored":266166755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:30:32.042 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 64 20 74860 74840 4.567 2.36719 0.015807 0.0170999 2024-09-10T20:30:33.042 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 65 20 75397 75377 4.52901 2.09766 0.0843359 0.0172374 2024-09-10T20:30:34.042 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 66 20 76061 76041 4.49969 2.59375 0.0335191 0.0173557 2024-09-10T20:30:34.899 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:30:35.042 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 67 20 76718 76698 4.47083 2.56641 0.0518406 0.0174641 2024-09-10T20:30:35.461 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:35.463 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:30:35.488 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2024-09-10T20:30:34.839307+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":306001340,"num_objects":74712,"num_object_clones":0,"num_object_copies":149424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2941,"num_objects_unfound":0,"num_objects_dirty":74712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39087,"num_write_kb":156322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":667,"num_bytes_recovered":2715667,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74792,"ondisk_log_size":74792,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":953732,"kb_used_data":600468,"kb_used_omap":12,"kb_used_meta":353203,"kb_avail":282161788,"statfs":{"total":289910292480,"available":288933670912,"internally_reserved":0,"allocated":614879232,"data_stored":614104424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":361679874},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,4,6,6,5,12],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":31125504,"num_objects":7599,"num_object_clones":0,"num_object_copies":15198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-373,"num_objects_unfound":0,"num_objects_dirty":7599,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7599,"num_write_kb":30396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":373,"num_bytes_recovered":1527808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002786"},"pg_stats":[{"pgid":"2.f","version":"43'4612","reported_seq":4686,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:31.108046+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:30:31.108046+0000","last_peered":"2024-09-10T20:30:31.108046+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:30:31.108046+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":4612,"log_dups_size":0,"ondisk_log_size":4612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18890752,"num_objects":4612,"num_object_clones":0,"num_object_copies":9224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4612,"num_write_kb":18448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1005},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4831","reported_seq":4936,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:33.103198+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:30:33.103198+0000","last_peered":"2024-09-10T20:30:33.103198+0000","last_clean":"2024-09-10T20:30:33.103198+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:30:33.103198+0000","last_undegraded":"2024-09-10T20:30:33.103198+0000","last_fullsized":"2024-09-10T20:30:33.103198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":4831,"log_dups_size":0,"ondisk_log_size":4831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19763200,"num_objects":4825,"num_object_clones":0,"num_object_copies":9650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3017,"num_write_kb":12068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4686","reported_seq":6135,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:30:33.110792+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:30:33.110792+0000","last_peered":"2024-09-10T20:30:33.110792+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:30:33.110792+0000","last_undegraded":"2024-09-10T20:30:33.110792+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":4686,"log_dups_size":0,"ondisk_log_size":4686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19177472,"num_objects":4682,"num_object_clones":0,"num_object_copies":9364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2941,"num_objects_unfound":0,"num_objects_dirty":4682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2814,"num_write_kb":11256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":662,"num_bytes_recovered":2711552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2941},{"shards":"2","objects":1741}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4650","reported_seq":4725,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:33.082694+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:30:33.082694+0000","last_peered":"2024-09-10T20:30:33.082694+0000","last_clean":"2024-09-10T20:30:33.082694+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:30:33.082694+0000","last_undegraded":"2024-09-10T20:30:33.082694+0000","last_fullsized":"2024-09-10T20:30:33.082694+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":4650,"log_dups_size":0,"ondisk_log_size":4650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19046400,"num_objects":4650,"num_object_clones":0,"num_object_copies":9300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4650,"num_write_kb":18600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4738","reported_seq":4859,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:31.127330+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:30:31.127330+0000","last_peered":"2024-09-10T20:30:31.127330+0000","last_clean":"2024-09-10T20:30:31.127330+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:30:31.127330+0000","last_undegraded":"2024-09-10T20:30:31.127330+0000","last_fullsized":"2024-09-10T20:30:31.127330+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":4738,"log_dups_size":0,"ondisk_log_size":4738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19378176,"num_objects":4731,"num_object_clones":0,"num_object_copies":9462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1734,"num_write_kb":6936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4690","reported_seq":4820,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:31.133431+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:30:31.133431+0000","last_peered":"2024-09-10T20:30:31.133431+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:30:31.133431+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":4690,"log_dups_size":0,"ondisk_log_size":4690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19189760,"num_objects":4685,"num_object_clones":0,"num_object_copies":9370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4685,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1554,"num_write_kb":6216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1071},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4583","reported_seq":4716,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:31.092914+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:30:31.092914+0000","last_peered":"2024-09-10T20:30:31.092914+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:30:31.092914+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":4583,"log_dups_size":0,"ondisk_log_size":4583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18747392,"num_objects":4577,"num_object_clones":0,"num_object_copies":9154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1178,"num_write_kb":4712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1022},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4694","reported_seq":4812,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:31.138820+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:30:31.138820+0000","last_peered":"2024-09-10T20:30:31.138820+0000","last_clean":"2024-09-10T20:30:31.138820+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:30:31.138820+0000","last_undegraded":"2024-09-10T20:30:31.138820+0000","last_fullsized":"2024-09-10T20:30:31.138820+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":4694,"log_dups_size":0,"ondisk_log_size":4694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19218432,"num_objects":4692,"num_object_clones":0,"num_object_copies":9384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1765,"num_write_kb":7060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4598","reported_seq":4725,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:33.062408+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:30:33.062408+0000","last_peered":"2024-09-10T20:30:33.062408+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:30:33.062408+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":4598,"log_dups_size":0,"ondisk_log_size":4598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18804736,"num_objects":4591,"num_object_clones":0,"num_object_copies":9182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2591,"num_write_kb":10364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":1065}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355438+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355438+0000","last_peered":"2024-09-10T20:30:05.355438+0000","last_clean":"2024-09-10T20:30:05.355438+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355438+0000","last_undegraded":"2024-09-10T20:30:05.355438+0000","last_fullsized":"2024-09-10T20:30:05.355438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4670","reported_seq":4773,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:31.087166+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:30:31.087166+0000","last_peered":"2024-09-10T20:30:31.087166+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:30:31.087166+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":4670,"log_dups_size":0,"ondisk_log_size":4670,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19096518,"num_objects":4664,"num_object_clones":0,"num_object_copies":9328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4668,"num_write_kb":18652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1049},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.413176+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.413176+0000","last_peered":"2024-09-10T20:30:05.413176+0000","last_clean":"2024-09-10T20:30:05.413176+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.413176+0000","last_undegraded":"2024-09-10T20:30:05.413176+0000","last_fullsized":"2024-09-10T20:30:05.413176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4678","reported_seq":4801,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:32.996016+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:30:32.996016+0000","last_peered":"2024-09-10T20:30:32.996016+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:30:32.996016+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":4678,"log_dups_size":0,"ondisk_log_size":4678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19140608,"num_objects":4673,"num_object_clones":0,"num_object_copies":9346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2506,"num_write_kb":10024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":1040}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4735","reported_seq":4858,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:33.021692+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:30:33.021692+0000","last_peered":"2024-09-10T20:30:33.021692+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:30:33.021692+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":4735,"log_dups_size":0,"ondisk_log_size":4735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19369984,"num_objects":4729,"num_object_clones":0,"num_object_copies":9458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2521,"num_write_kb":10084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":1119}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4645","reported_seq":4775,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:31.119033+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:30:31.119033+0000","last_peered":"2024-09-10T20:30:31.119033+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:30:31.119033+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":4645,"log_dups_size":0,"ondisk_log_size":4645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19017728,"num_objects":4643,"num_object_clones":0,"num_object_copies":9286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1368,"num_write_kb":5472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":995},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4605","reported_seq":4738,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:31.127208+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:30:31.127208+0000","last_peered":"2024-09-10T20:30:31.127208+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:30:31.127208+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":4605,"log_dups_size":0,"ondisk_log_size":4605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18829312,"num_objects":4597,"num_object_clones":0,"num_object_copies":9194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1299,"num_write_kb":5196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1065},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345705+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345705+0000","last_peered":"2024-09-10T20:30:05.345705+0000","last_clean":"2024-09-10T20:30:05.345705+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345705+0000","last_undegraded":"2024-09-10T20:30:05.345705+0000","last_fullsized":"2024-09-10T20:30:05.345705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4780","reported_seq":4906,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:31.081372+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:30:31.081372+0000","last_peered":"2024-09-10T20:30:31.081372+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:30:31.081372+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":4780,"log_dups_size":0,"ondisk_log_size":4780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19562496,"num_objects":4776,"num_object_clones":0,"num_object_copies":9552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1607,"num_write_kb":6428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1063},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4594","reported_seq":4754,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:31.075831+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:30:31.075831+0000","last_peered":"2024-09-10T20:30:31.075831+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:30:31.075831+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":4594,"log_dups_size":0,"ondisk_log_size":4594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18768355,"num_objects":4583,"num_object_clones":0,"num_object_copies":9166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1201,"num_write_kb":4804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1115},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":306001321,"num_objects":74710,"num_object_clones":0,"num_object_copies":149420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2941,"num_objects_unfound":0,"num_objects_dirty":74710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39085,"num_write_kb":156320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":665,"num_bytes_recovered":2715648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":616214528,"data_stored":616124203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74789,"ondisk_log_size":74789,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182424,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":416288,"kb_used_data":280528,"kb_used_omap":4,"kb_used_meta":135739,"kb_avail":93955552,"statfs":{"total":96636764160,"available":96210485248,"internally_reserved":0,"allocated":287260672,"data_stored":287027539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":138997418},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,1,0,3,1,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986196,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":401852,"kb_used_data":266028,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":93969988,"statfs":{"total":96636764160,"available":96225267712,"internally_reserved":0,"allocated":272412672,"data_stored":272183616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":139062954},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,4,0,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659143,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":135592,"kb_used_data":53912,"kb_used_omap":4,"kb_used_meta":81659,"kb_avail":94236248,"statfs":{"total":96636764160,"available":96497917952,"internally_reserved":0,"allocated":55205888,"data_stored":54893269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":83619502},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":287850496,"data_stored":287846883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":56066048,"data_stored":55982949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":272297984,"data_stored":272294371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:30:35.490 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:30:36.039 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:36.040 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:30:36.044 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 68 20 77305 77285 4.43881 2.29297 0.0230206 0.017591 2024-09-10T20:30:36.071 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2024-09-10T20:30:34.839307+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":306001340,"num_objects":74712,"num_object_clones":0,"num_object_copies":149424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2941,"num_objects_unfound":0,"num_objects_dirty":74712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39087,"num_write_kb":156322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":667,"num_bytes_recovered":2715667,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74792,"ondisk_log_size":74792,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":953732,"kb_used_data":600468,"kb_used_omap":12,"kb_used_meta":353203,"kb_avail":282161788,"statfs":{"total":289910292480,"available":288933670912,"internally_reserved":0,"allocated":614879232,"data_stored":614104424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":361679874},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,4,6,6,5,12],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":31125504,"num_objects":7599,"num_object_clones":0,"num_object_copies":15198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-373,"num_objects_unfound":0,"num_objects_dirty":7599,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7599,"num_write_kb":30396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":373,"num_bytes_recovered":1527808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002786"},"pg_stats":[{"pgid":"2.f","version":"43'4612","reported_seq":4686,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:31.108046+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:30:31.108046+0000","last_peered":"2024-09-10T20:30:31.108046+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:30:31.108046+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":4612,"log_dups_size":0,"ondisk_log_size":4612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18890752,"num_objects":4612,"num_object_clones":0,"num_object_copies":9224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4612,"num_write_kb":18448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1005},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4831","reported_seq":4936,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:33.103198+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:30:33.103198+0000","last_peered":"2024-09-10T20:30:33.103198+0000","last_clean":"2024-09-10T20:30:33.103198+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:30:33.103198+0000","last_undegraded":"2024-09-10T20:30:33.103198+0000","last_fullsized":"2024-09-10T20:30:33.103198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":4831,"log_dups_size":0,"ondisk_log_size":4831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19763200,"num_objects":4825,"num_object_clones":0,"num_object_copies":9650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3017,"num_write_kb":12068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4686","reported_seq":6135,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:30:33.110792+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:30:33.110792+0000","last_peered":"2024-09-10T20:30:33.110792+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:30:33.110792+0000","last_undegraded":"2024-09-10T20:30:33.110792+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":4686,"log_dups_size":0,"ondisk_log_size":4686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19177472,"num_objects":4682,"num_object_clones":0,"num_object_copies":9364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2941,"num_objects_unfound":0,"num_objects_dirty":4682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2814,"num_write_kb":11256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":662,"num_bytes_recovered":2711552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2941},{"shards":"2","objects":1741}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4650","reported_seq":4725,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:33.082694+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:30:33.082694+0000","last_peered":"2024-09-10T20:30:33.082694+0000","last_clean":"2024-09-10T20:30:33.082694+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:30:33.082694+0000","last_undegraded":"2024-09-10T20:30:33.082694+0000","last_fullsized":"2024-09-10T20:30:33.082694+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":4650,"log_dups_size":0,"ondisk_log_size":4650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19046400,"num_objects":4650,"num_object_clones":0,"num_object_copies":9300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4650,"num_write_kb":18600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4738","reported_seq":4859,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:31.127330+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:30:31.127330+0000","last_peered":"2024-09-10T20:30:31.127330+0000","last_clean":"2024-09-10T20:30:31.127330+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:30:31.127330+0000","last_undegraded":"2024-09-10T20:30:31.127330+0000","last_fullsized":"2024-09-10T20:30:31.127330+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":4738,"log_dups_size":0,"ondisk_log_size":4738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19378176,"num_objects":4731,"num_object_clones":0,"num_object_copies":9462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1734,"num_write_kb":6936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4690","reported_seq":4820,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:31.133431+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:30:31.133431+0000","last_peered":"2024-09-10T20:30:31.133431+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:30:31.133431+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":4690,"log_dups_size":0,"ondisk_log_size":4690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19189760,"num_objects":4685,"num_object_clones":0,"num_object_copies":9370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4685,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1554,"num_write_kb":6216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1071},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4583","reported_seq":4716,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:31.092914+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:30:31.092914+0000","last_peered":"2024-09-10T20:30:31.092914+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:30:31.092914+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":4583,"log_dups_size":0,"ondisk_log_size":4583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18747392,"num_objects":4577,"num_object_clones":0,"num_object_copies":9154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1178,"num_write_kb":4712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1022},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4694","reported_seq":4812,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:31.138820+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:30:31.138820+0000","last_peered":"2024-09-10T20:30:31.138820+0000","last_clean":"2024-09-10T20:30:31.138820+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:30:31.138820+0000","last_undegraded":"2024-09-10T20:30:31.138820+0000","last_fullsized":"2024-09-10T20:30:31.138820+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":4694,"log_dups_size":0,"ondisk_log_size":4694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19218432,"num_objects":4692,"num_object_clones":0,"num_object_copies":9384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1765,"num_write_kb":7060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4598","reported_seq":4725,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:33.062408+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:30:33.062408+0000","last_peered":"2024-09-10T20:30:33.062408+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:30:33.062408+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":4598,"log_dups_size":0,"ondisk_log_size":4598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18804736,"num_objects":4591,"num_object_clones":0,"num_object_copies":9182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2591,"num_write_kb":10364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":1065}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355438+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355438+0000","last_peered":"2024-09-10T20:30:05.355438+0000","last_clean":"2024-09-10T20:30:05.355438+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355438+0000","last_undegraded":"2024-09-10T20:30:05.355438+0000","last_fullsized":"2024-09-10T20:30:05.355438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4670","reported_seq":4773,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:31.087166+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:30:31.087166+0000","last_peered":"2024-09-10T20:30:31.087166+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:30:31.087166+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":4670,"log_dups_size":0,"ondisk_log_size":4670,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19096518,"num_objects":4664,"num_object_clones":0,"num_object_copies":9328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4668,"num_write_kb":18652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1049},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.413176+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.413176+0000","last_peered":"2024-09-10T20:30:05.413176+0000","last_clean":"2024-09-10T20:30:05.413176+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.413176+0000","last_undegraded":"2024-09-10T20:30:05.413176+0000","last_fullsized":"2024-09-10T20:30:05.413176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4678","reported_seq":4801,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:32.996016+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:30:32.996016+0000","last_peered":"2024-09-10T20:30:32.996016+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:30:32.996016+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":4678,"log_dups_size":0,"ondisk_log_size":4678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19140608,"num_objects":4673,"num_object_clones":0,"num_object_copies":9346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2506,"num_write_kb":10024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":1040}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4735","reported_seq":4858,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:33.021692+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:30:33.021692+0000","last_peered":"2024-09-10T20:30:33.021692+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:30:33.021692+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":4735,"log_dups_size":0,"ondisk_log_size":4735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19369984,"num_objects":4729,"num_object_clones":0,"num_object_copies":9458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2521,"num_write_kb":10084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":1119}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4645","reported_seq":4775,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:31.119033+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:30:31.119033+0000","last_peered":"2024-09-10T20:30:31.119033+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:30:31.119033+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":4645,"log_dups_size":0,"ondisk_log_size":4645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19017728,"num_objects":4643,"num_object_clones":0,"num_object_copies":9286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1368,"num_write_kb":5472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":995},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4605","reported_seq":4738,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:31.127208+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:30:31.127208+0000","last_peered":"2024-09-10T20:30:31.127208+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:30:31.127208+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":4605,"log_dups_size":0,"ondisk_log_size":4605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18829312,"num_objects":4597,"num_object_clones":0,"num_object_copies":9194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1299,"num_write_kb":5196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1065},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345705+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345705+0000","last_peered":"2024-09-10T20:30:05.345705+0000","last_clean":"2024-09-10T20:30:05.345705+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345705+0000","last_undegraded":"2024-09-10T20:30:05.345705+0000","last_fullsized":"2024-09-10T20:30:05.345705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4780","reported_seq":4906,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:31.081372+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:30:31.081372+0000","last_peered":"2024-09-10T20:30:31.081372+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:30:31.081372+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":4780,"log_dups_size":0,"ondisk_log_size":4780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19562496,"num_objects":4776,"num_object_clones":0,"num_object_copies":9552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1607,"num_write_kb":6428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1063},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4594","reported_seq":4754,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:31.075831+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:30:31.075831+0000","last_peered":"2024-09-10T20:30:31.075831+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:30:31.075831+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":4594,"log_dups_size":0,"ondisk_log_size":4594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18768355,"num_objects":4583,"num_object_clones":0,"num_object_copies":9166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1201,"num_write_kb":4804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1115},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":306001321,"num_objects":74710,"num_object_clones":0,"num_object_copies":149420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2941,"num_objects_unfound":0,"num_objects_dirty":74710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39085,"num_write_kb":156320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":665,"num_bytes_recovered":2715648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":616214528,"data_stored":616124203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74789,"ondisk_log_size":74789,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182424,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":416288,"kb_used_data":280528,"kb_used_omap":4,"kb_used_meta":135739,"kb_avail":93955552,"statfs":{"total":96636764160,"available":96210485248,"internally_reserved":0,"allocated":287260672,"data_stored":287027539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":138997418},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,1,0,3,1,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986196,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":401852,"kb_used_data":266028,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":93969988,"statfs":{"total":96636764160,"available":96225267712,"internally_reserved":0,"allocated":272412672,"data_stored":272183616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":139062954},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,4,0,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659143,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":135592,"kb_used_data":53912,"kb_used_omap":4,"kb_used_meta":81659,"kb_avail":94236248,"statfs":{"total":96636764160,"available":96497917952,"internally_reserved":0,"allocated":55205888,"data_stored":54893269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":83619502},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":287850496,"data_stored":287846883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":56066048,"data_stored":55982949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":272297984,"data_stored":272294371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:30:36.073 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:30:36.073 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'4612", 'reported_seq': 4686, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:31.108046+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:30:31.108046+0000', 'last_peered': '2024-09-10T20:30:31.108046+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:30:31.108046+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 4612, 'log_dups_size': 0, 'ondisk_log_size': 4612, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18890752, 'num_objects': 4612, 'num_object_clones': 0, 'num_object_copies': 9224, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3607, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4612, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4612, 'num_write_kb': 18448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1005}, {'shards': '0,2', 'objects': 3607}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:36.073 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T20:30:36.073 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'4686", 'reported_seq': 6135, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:30:33.110792+0000', 'last_change': '2024-09-10T20:30:05.653560+0000', 'last_active': '2024-09-10T20:30:33.110792+0000', 'last_peered': '2024-09-10T20:30:33.110792+0000', 'last_clean': '2024-09-10T20:30:03.279259+0000', 'last_became_active': '2024-09-10T20:30:05.629664+0000', 'last_became_peered': '2024-09-10T20:30:05.629664+0000', 'last_unstale': '2024-09-10T20:30:33.110792+0000', 'last_undegraded': '2024-09-10T20:30:33.110792+0000', 'last_fullsized': '2024-09-10T20:30:05.312061+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1910", 'last_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'objects_scrubbed': 1868, 'log_size': 4686, 'log_dups_size': 0, 'ondisk_log_size': 4686, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 673, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19177472, 'num_objects': 4682, 'num_object_clones': 0, 'num_object_copies': 9364, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2941, 'num_objects_unfound': 0, 'num_objects_dirty': 4682, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2814, 'num_write_kb': 11256, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 662, 'num_bytes_recovered': 2711552, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2941}, {'shards': '2', 'objects': 1741}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:36.073 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:30:36.073 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'4690", 'reported_seq': 4820, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:31.133431+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:30:31.133431+0000', 'last_peered': '2024-09-10T20:30:31.133431+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:30:31.133431+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 4690, 'log_dups_size': 0, 'ondisk_log_size': 4690, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19189760, 'num_objects': 4685, 'num_object_clones': 0, 'num_object_copies': 9370, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3616, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4685, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1554, 'num_write_kb': 6216, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1071}, {'shards': '0,2', 'objects': 3614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:36.073 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:30:36.073 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'4583", 'reported_seq': 4716, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:31.092914+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:30:31.092914+0000', 'last_peered': '2024-09-10T20:30:31.092914+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:30:31.092914+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 4583, 'log_dups_size': 0, 'ondisk_log_size': 4583, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18747392, 'num_objects': 4577, 'num_object_clones': 0, 'num_object_copies': 9154, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3557, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4577, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1178, 'num_write_kb': 4712, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1022}, {'shards': '0,2', 'objects': 3555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:36.074 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:30:36.074 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'4598", 'reported_seq': 4725, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:33.062408+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:30:33.062408+0000', 'last_peered': '2024-09-10T20:30:33.062408+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:30:33.062408+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 4598, 'log_dups_size': 0, 'ondisk_log_size': 4598, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18804736, 'num_objects': 4591, 'num_object_clones': 0, 'num_object_copies': 9182, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3528, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4591, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2591, 'num_write_kb': 10364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 1065}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:36.074 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:30:36.074 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'4670", 'reported_seq': 4773, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:31.087166+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:30:31.087166+0000', 'last_peered': '2024-09-10T20:30:31.087166+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:30:31.087166+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 4670, 'log_dups_size': 0, 'ondisk_log_size': 4670, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19096518, 'num_objects': 4664, 'num_object_clones': 0, 'num_object_copies': 9328, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3617, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4664, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4668, 'num_write_kb': 18652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1049}, {'shards': '0,2', 'objects': 3615}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:36.074 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:30:36.074 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'4678", 'reported_seq': 4801, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:32.996016+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:30:32.996016+0000', 'last_peered': '2024-09-10T20:30:32.996016+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:30:32.996016+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 4678, 'log_dups_size': 0, 'ondisk_log_size': 4678, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19140608, 'num_objects': 4673, 'num_object_clones': 0, 'num_object_copies': 9346, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3635, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4673, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2506, 'num_write_kb': 10024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3633}, {'shards': '2', 'objects': 1040}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:36.074 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:30:36.074 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'4735", 'reported_seq': 4858, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:33.021692+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:30:33.021692+0000', 'last_peered': '2024-09-10T20:30:33.021692+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:30:33.021692+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 4735, 'log_dups_size': 0, 'ondisk_log_size': 4735, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19369984, 'num_objects': 4729, 'num_object_clones': 0, 'num_object_copies': 9458, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4729, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2521, 'num_write_kb': 10084, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3610}, {'shards': '2', 'objects': 1119}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:36.074 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:30:36.074 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'4645", 'reported_seq': 4775, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:31.119033+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:30:31.119033+0000', 'last_peered': '2024-09-10T20:30:31.119033+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:30:31.119033+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 4645, 'log_dups_size': 0, 'ondisk_log_size': 4645, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19017728, 'num_objects': 4643, 'num_object_clones': 0, 'num_object_copies': 9286, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3649, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4643, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1368, 'num_write_kb': 5472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 995}, {'shards': '0,2', 'objects': 3648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:36.074 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:30:36.075 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'4605", 'reported_seq': 4738, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:31.127208+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:30:31.127208+0000', 'last_peered': '2024-09-10T20:30:31.127208+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:30:31.127208+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 4605, 'log_dups_size': 0, 'ondisk_log_size': 4605, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18829312, 'num_objects': 4597, 'num_object_clones': 0, 'num_object_copies': 9194, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3535, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4597, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1299, 'num_write_kb': 5196, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1065}, {'shards': '0,2', 'objects': 3532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:36.075 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:30:36.075 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'4780", 'reported_seq': 4906, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:31.081372+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:30:31.081372+0000', 'last_peered': '2024-09-10T20:30:31.081372+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:30:31.081372+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 4780, 'log_dups_size': 0, 'ondisk_log_size': 4780, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19562496, 'num_objects': 4776, 'num_object_clones': 0, 'num_object_copies': 9552, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3715, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4776, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1607, 'num_write_kb': 6428, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1063}, {'shards': '0,2', 'objects': 3713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:36.075 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:30:36.075 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'4594", 'reported_seq': 4754, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:31.075831+0000', 'last_change': '2024-09-10T20:30:05.690055+0000', 'last_active': '2024-09-10T20:30:31.075831+0000', 'last_peered': '2024-09-10T20:30:31.075831+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:30:31.075831+0000', 'last_undegraded': '2024-09-10T20:30:05.378941+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 4594, 'log_dups_size': 0, 'ondisk_log_size': 4594, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18768355, 'num_objects': 4583, 'num_object_clones': 0, 'num_object_copies': 9166, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3471, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4583, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1201, 'num_write_kb': 4804, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1115}, {'shards': '0,2', 'objects': 3468}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:36.075 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:30:36.772 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:36.772 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39542/149424 objects degraded (26.463%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":148,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":74712,"data_bytes":306001340,"bytes_used":976621568,"bytes_avail":288933670912,"bytes_total":289910292480,"degraded_objects":39542,"degraded_total":149424,"degraded_ratio":0.26462951065424561,"misplaced_objects":2941,"misplaced_total":149424,"misplaced_ratio":0.019682246493200557,"recovering_objects_per_sec":31,"recovering_bytes_per_sec":127287,"recovering_keys_per_sec":0,"num_objects_recovered":373,"num_bytes_recovered":1527808,"num_keys_recovered":0,"write_bytes_sec":2593189,"read_op_per_sec":0,"write_op_per_sec":633},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-10T20:30:14.834797+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T20:30:36.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:30:36.803 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:30:37.042 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 69 20 77879 77859 4.40697 2.24219 0.0249264 0.0177185 2024-09-10T20:30:37.371 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:37.371 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:30:37.393 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2024-09-10T20:30:36.839761+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":313976252,"num_objects":76659,"num_object_clones":0,"num_object_copies":153318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2941,"num_objects_unfound":0,"num_objects_dirty":76659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41034,"num_write_kb":164110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":667,"num_bytes_recovered":2715667,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76739,"ondisk_log_size":76739,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":985416,"kb_used_data":609944,"kb_used_omap":12,"kb_used_meta":375411,"kb_avail":282130104,"statfs":{"total":289910292480,"available":288901226496,"internally_reserved":0,"allocated":624582656,"data_stored":623807848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":384420866},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,3,8,8,8,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":34164736,"num_objects":8341,"num_object_clones":0,"num_object_copies":16682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-259,"num_objects_unfound":0,"num_objects_dirty":8341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8341,"num_write_kb":33364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":1060864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002737"},"pg_stats":[{"pgid":"2.f","version":"43'4806","reported_seq":4881,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:36.123696+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:30:36.123696+0000","last_peered":"2024-09-10T20:30:36.123696+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:30:36.123696+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":4806,"log_dups_size":0,"ondisk_log_size":4806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19685376,"num_objects":4806,"num_object_clones":0,"num_object_copies":9612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4806,"num_write_kb":19224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1199},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4831","reported_seq":4936,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:33.103198+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:30:33.103198+0000","last_peered":"2024-09-10T20:30:33.103198+0000","last_clean":"2024-09-10T20:30:33.103198+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:30:33.103198+0000","last_undegraded":"2024-09-10T20:30:33.103198+0000","last_fullsized":"2024-09-10T20:30:33.103198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":4831,"log_dups_size":0,"ondisk_log_size":4831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19763200,"num_objects":4825,"num_object_clones":0,"num_object_copies":9650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3017,"num_write_kb":12068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4686","reported_seq":6135,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:30:33.110792+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:30:33.110792+0000","last_peered":"2024-09-10T20:30:33.110792+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:30:33.110792+0000","last_undegraded":"2024-09-10T20:30:33.110792+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":4686,"log_dups_size":0,"ondisk_log_size":4686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19177472,"num_objects":4682,"num_object_clones":0,"num_object_copies":9364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2941,"num_objects_unfound":0,"num_objects_dirty":4682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2814,"num_write_kb":11256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":662,"num_bytes_recovered":2711552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2941},{"shards":"2","objects":1741}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4650","reported_seq":4725,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:33.082694+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:30:33.082694+0000","last_peered":"2024-09-10T20:30:33.082694+0000","last_clean":"2024-09-10T20:30:33.082694+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:30:33.082694+0000","last_undegraded":"2024-09-10T20:30:33.082694+0000","last_fullsized":"2024-09-10T20:30:33.082694+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":4650,"log_dups_size":0,"ondisk_log_size":4650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19046400,"num_objects":4650,"num_object_clones":0,"num_object_copies":9300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4650,"num_write_kb":18600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4921","reported_seq":5042,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:36.182123+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:30:36.182123+0000","last_peered":"2024-09-10T20:30:36.182123+0000","last_clean":"2024-09-10T20:30:36.182123+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:30:36.182123+0000","last_undegraded":"2024-09-10T20:30:36.182123+0000","last_fullsized":"2024-09-10T20:30:36.182123+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":4921,"log_dups_size":0,"ondisk_log_size":4921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20127744,"num_objects":4914,"num_object_clones":0,"num_object_copies":9828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1917,"num_write_kb":7668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4872","reported_seq":5003,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:36.013464+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:30:36.013464+0000","last_peered":"2024-09-10T20:30:36.013464+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:30:36.013464+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":4872,"log_dups_size":0,"ondisk_log_size":4872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19935232,"num_objects":4867,"num_object_clones":0,"num_object_copies":9734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1736,"num_write_kb":6944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1253},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4779","reported_seq":4912,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:36.167644+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:30:36.167644+0000","last_peered":"2024-09-10T20:30:36.167644+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:30:36.167644+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":4779,"log_dups_size":0,"ondisk_log_size":4779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19550208,"num_objects":4773,"num_object_clones":0,"num_object_copies":9546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1374,"num_write_kb":5496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1218},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4909","reported_seq":5028,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:36.158871+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:30:36.158871+0000","last_peered":"2024-09-10T20:30:36.158871+0000","last_clean":"2024-09-10T20:30:36.158871+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:30:36.158871+0000","last_undegraded":"2024-09-10T20:30:36.158871+0000","last_fullsized":"2024-09-10T20:30:36.158871+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":4909,"log_dups_size":0,"ondisk_log_size":4909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20099072,"num_objects":4907,"num_object_clones":0,"num_object_copies":9814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1980,"num_write_kb":7920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4598","reported_seq":4725,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:33.062408+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:30:33.062408+0000","last_peered":"2024-09-10T20:30:33.062408+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:30:33.062408+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":4598,"log_dups_size":0,"ondisk_log_size":4598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18804736,"num_objects":4591,"num_object_clones":0,"num_object_copies":9182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2591,"num_write_kb":10364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":1065}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355438+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355438+0000","last_peered":"2024-09-10T20:30:05.355438+0000","last_clean":"2024-09-10T20:30:05.355438+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355438+0000","last_undegraded":"2024-09-10T20:30:05.355438+0000","last_fullsized":"2024-09-10T20:30:05.355438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4868","reported_seq":4971,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:36.103506+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:30:36.103506+0000","last_peered":"2024-09-10T20:30:36.103506+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:30:36.103506+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":4868,"log_dups_size":0,"ondisk_log_size":4868,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19907526,"num_objects":4862,"num_object_clones":0,"num_object_copies":9724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4866,"num_write_kb":19444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1247},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.413176+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.413176+0000","last_peered":"2024-09-10T20:30:05.413176+0000","last_clean":"2024-09-10T20:30:05.413176+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.413176+0000","last_undegraded":"2024-09-10T20:30:05.413176+0000","last_fullsized":"2024-09-10T20:30:05.413176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4678","reported_seq":4801,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:32.996016+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:30:32.996016+0000","last_peered":"2024-09-10T20:30:32.996016+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:30:32.996016+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":4678,"log_dups_size":0,"ondisk_log_size":4678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19140608,"num_objects":4673,"num_object_clones":0,"num_object_copies":9346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2506,"num_write_kb":10024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":1040}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4735","reported_seq":4858,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:33.021692+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:30:33.021692+0000","last_peered":"2024-09-10T20:30:33.021692+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:30:33.021692+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":4735,"log_dups_size":0,"ondisk_log_size":4735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19369984,"num_objects":4729,"num_object_clones":0,"num_object_copies":9458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2521,"num_write_kb":10084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":1119}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4837","reported_seq":4967,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:36.112577+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:30:36.112577+0000","last_peered":"2024-09-10T20:30:36.112577+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:30:36.112577+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":4837,"log_dups_size":0,"ondisk_log_size":4837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19804160,"num_objects":4835,"num_object_clones":0,"num_object_copies":9670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1560,"num_write_kb":6240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1187},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4803","reported_seq":4936,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:36.103154+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:30:36.103154+0000","last_peered":"2024-09-10T20:30:36.103154+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:30:36.103154+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":4803,"log_dups_size":0,"ondisk_log_size":4803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19640320,"num_objects":4795,"num_object_clones":0,"num_object_copies":9590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1497,"num_write_kb":5988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1263},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345705+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345705+0000","last_peered":"2024-09-10T20:30:05.345705+0000","last_clean":"2024-09-10T20:30:05.345705+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345705+0000","last_undegraded":"2024-09-10T20:30:05.345705+0000","last_fullsized":"2024-09-10T20:30:05.345705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4970","reported_seq":5096,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:36.181993+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:30:36.181993+0000","last_peered":"2024-09-10T20:30:36.181993+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:30:36.181993+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":4970,"log_dups_size":0,"ondisk_log_size":4970,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20340736,"num_objects":4966,"num_object_clones":0,"num_object_copies":9932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1797,"num_write_kb":7188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1253},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4793","reported_seq":4953,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:36.137133+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:30:36.137133+0000","last_peered":"2024-09-10T20:30:36.137133+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:30:36.137133+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":4793,"log_dups_size":0,"ondisk_log_size":4793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19583459,"num_objects":4782,"num_object_clones":0,"num_object_copies":9564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1400,"num_write_kb":5600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1314},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":313976233,"num_objects":76657,"num_object_clones":0,"num_object_copies":153314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2941,"num_objects_unfound":0,"num_objects_dirty":76657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41032,"num_write_kb":164108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":665,"num_bytes_recovered":2715648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":625811456,"data_stored":625721131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76736,"ondisk_log_size":76736,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182425,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447972,"kb_used_data":290004,"kb_used_omap":4,"kb_used_meta":157947,"kb_avail":93923868,"statfs":{"total":96636764160,"available":96178040832,"internally_reserved":0,"allocated":296964096,"data_stored":296730963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":161738410},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,0,2,5,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986196,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":401852,"kb_used_data":266028,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":93969988,"statfs":{"total":96636764160,"available":96225267712,"internally_reserved":0,"allocated":272412672,"data_stored":272183616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":139062954},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,4,0,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659143,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":135592,"kb_used_data":53912,"kb_used_omap":4,"kb_used_meta":81659,"kb_avail":94236248,"statfs":{"total":96636764160,"available":96497917952,"internally_reserved":0,"allocated":55205888,"data_stored":54893269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":83619502},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":297447424,"data_stored":297443811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":56066048,"data_stored":55982949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":272297984,"data_stored":272294371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:30:38.042 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 70 20 78507 78487 4.37906 2.45312 0.0160404 0.0178311 2024-09-10T20:30:39.043 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 71 20 79150 79130 4.35276 2.51172 0.0127854 0.0179415 2024-09-10T20:30:40.043 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 72 20 79813 79793 4.32827 2.58984 0.0167672 0.0180418 2024-09-10T20:30:40.395 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:30:40.952 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:40.955 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:30:40.978 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2024-09-10T20:30:40.841034+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":318649788,"num_objects":77800,"num_object_clones":0,"num_object_copies":155600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2815,"num_objects_unfound":0,"num_objects_dirty":77800,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42175,"num_write_kb":168674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":793,"num_bytes_recovered":3231763,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77880,"ondisk_log_size":77880,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1013532,"kb_used_data":625576,"kb_used_omap":12,"kb_used_meta":387891,"kb_avail":282101988,"statfs":{"total":289910292480,"available":288872435712,"internally_reserved":0,"allocated":640589824,"data_stored":639815016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":397200386},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,3,2,7,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25563136,"num_objects":6241,"num_object_clones":0,"num_object_copies":12482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-270,"num_objects_unfound":0,"num_objects_dirty":6241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6241,"num_write_kb":24964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":1105920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003046"},"pg_stats":[{"pgid":"2.f","version":"43'4806","reported_seq":4881,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:36.123696+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:30:36.123696+0000","last_peered":"2024-09-10T20:30:36.123696+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:30:36.123696+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":4806,"log_dups_size":0,"ondisk_log_size":4806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19685376,"num_objects":4806,"num_object_clones":0,"num_object_copies":9612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4806,"num_write_kb":19224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1199},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5036","reported_seq":5141,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:38.111253+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:30:38.111253+0000","last_peered":"2024-09-10T20:30:38.111253+0000","last_clean":"2024-09-10T20:30:38.111253+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:30:38.111253+0000","last_undegraded":"2024-09-10T20:30:38.111253+0000","last_fullsized":"2024-09-10T20:30:38.111253+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":5036,"log_dups_size":0,"ondisk_log_size":5036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20602880,"num_objects":5030,"num_object_clones":0,"num_object_copies":10060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3222,"num_write_kb":12888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4861","reported_seq":6562,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:30:38.128022+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:30:38.128022+0000","last_peered":"2024-09-10T20:30:38.128022+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:30:38.128022+0000","last_undegraded":"2024-09-10T20:30:38.128022+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":4861,"log_dups_size":0,"ondisk_log_size":4861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19894272,"num_objects":4857,"num_object_clones":0,"num_object_copies":9714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2815,"num_objects_unfound":0,"num_objects_dirty":4857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2989,"num_write_kb":11956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":788,"num_bytes_recovered":3227648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2815},{"shards":"2","objects":2042}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4841","reported_seq":4916,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:38.078081+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:30:38.078081+0000","last_peered":"2024-09-10T20:30:38.078081+0000","last_clean":"2024-09-10T20:30:38.078081+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:30:38.078081+0000","last_undegraded":"2024-09-10T20:30:38.078081+0000","last_fullsized":"2024-09-10T20:30:38.078081+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":4841,"log_dups_size":0,"ondisk_log_size":4841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19828736,"num_objects":4841,"num_object_clones":0,"num_object_copies":9682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4841,"num_write_kb":19364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4921","reported_seq":5042,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:36.182123+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:30:36.182123+0000","last_peered":"2024-09-10T20:30:36.182123+0000","last_clean":"2024-09-10T20:30:36.182123+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:30:36.182123+0000","last_undegraded":"2024-09-10T20:30:36.182123+0000","last_fullsized":"2024-09-10T20:30:36.182123+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":4921,"log_dups_size":0,"ondisk_log_size":4921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20127744,"num_objects":4914,"num_object_clones":0,"num_object_copies":9828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1917,"num_write_kb":7668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4872","reported_seq":5003,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:36.013464+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:30:36.013464+0000","last_peered":"2024-09-10T20:30:36.013464+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:30:36.013464+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":4872,"log_dups_size":0,"ondisk_log_size":4872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19935232,"num_objects":4867,"num_object_clones":0,"num_object_copies":9734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1736,"num_write_kb":6944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1253},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4779","reported_seq":4912,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:36.167644+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:30:36.167644+0000","last_peered":"2024-09-10T20:30:36.167644+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:30:36.167644+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":4779,"log_dups_size":0,"ondisk_log_size":4779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19550208,"num_objects":4773,"num_object_clones":0,"num_object_copies":9546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1374,"num_write_kb":5496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1218},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4909","reported_seq":5028,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:36.158871+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:30:36.158871+0000","last_peered":"2024-09-10T20:30:36.158871+0000","last_clean":"2024-09-10T20:30:36.158871+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:30:36.158871+0000","last_undegraded":"2024-09-10T20:30:36.158871+0000","last_fullsized":"2024-09-10T20:30:36.158871+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":4909,"log_dups_size":0,"ondisk_log_size":4909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20099072,"num_objects":4907,"num_object_clones":0,"num_object_copies":9814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1980,"num_write_kb":7920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4787","reported_seq":4914,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:38.121317+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:30:38.121317+0000","last_peered":"2024-09-10T20:30:38.121317+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:30:38.121317+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":4787,"log_dups_size":0,"ondisk_log_size":4787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19578880,"num_objects":4780,"num_object_clones":0,"num_object_copies":9560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2780,"num_write_kb":11120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":1254}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355438+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355438+0000","last_peered":"2024-09-10T20:30:05.355438+0000","last_clean":"2024-09-10T20:30:05.355438+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355438+0000","last_undegraded":"2024-09-10T20:30:05.355438+0000","last_fullsized":"2024-09-10T20:30:05.355438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4868","reported_seq":4971,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:36.103506+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:30:36.103506+0000","last_peered":"2024-09-10T20:30:36.103506+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:30:36.103506+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":4868,"log_dups_size":0,"ondisk_log_size":4868,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19907526,"num_objects":4862,"num_object_clones":0,"num_object_copies":9724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4866,"num_write_kb":19444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1247},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.413176+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.413176+0000","last_peered":"2024-09-10T20:30:05.413176+0000","last_clean":"2024-09-10T20:30:05.413176+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.413176+0000","last_undegraded":"2024-09-10T20:30:05.413176+0000","last_fullsized":"2024-09-10T20:30:05.413176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4853","reported_seq":4976,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:38.078902+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:30:38.078902+0000","last_peered":"2024-09-10T20:30:38.078902+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:30:38.078902+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":4853,"log_dups_size":0,"ondisk_log_size":4853,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19857408,"num_objects":4848,"num_object_clones":0,"num_object_copies":9696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2681,"num_write_kb":10724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":1215}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4941","reported_seq":5063,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:38.116755+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:30:38.116755+0000","last_peered":"2024-09-10T20:30:38.116755+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:30:38.116755+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":4941,"log_dups_size":0,"ondisk_log_size":4941,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20213760,"num_objects":4935,"num_object_clones":0,"num_object_copies":9870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2727,"num_write_kb":10908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":1325}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4837","reported_seq":4967,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:36.112577+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:30:36.112577+0000","last_peered":"2024-09-10T20:30:36.112577+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:30:36.112577+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":4837,"log_dups_size":0,"ondisk_log_size":4837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19804160,"num_objects":4835,"num_object_clones":0,"num_object_copies":9670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1560,"num_write_kb":6240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1187},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4803","reported_seq":4936,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:36.103154+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:30:36.103154+0000","last_peered":"2024-09-10T20:30:36.103154+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:30:36.103154+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":4803,"log_dups_size":0,"ondisk_log_size":4803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19640320,"num_objects":4795,"num_object_clones":0,"num_object_copies":9590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1497,"num_write_kb":5988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1263},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345705+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345705+0000","last_peered":"2024-09-10T20:30:05.345705+0000","last_clean":"2024-09-10T20:30:05.345705+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345705+0000","last_undegraded":"2024-09-10T20:30:05.345705+0000","last_fullsized":"2024-09-10T20:30:05.345705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4970","reported_seq":5096,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:36.181993+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:30:36.181993+0000","last_peered":"2024-09-10T20:30:36.181993+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:30:36.181993+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":4970,"log_dups_size":0,"ondisk_log_size":4970,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20340736,"num_objects":4966,"num_object_clones":0,"num_object_copies":9932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1797,"num_write_kb":7188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1253},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4793","reported_seq":4953,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:36.137133+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:30:36.137133+0000","last_peered":"2024-09-10T20:30:36.137133+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:30:36.137133+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":4793,"log_dups_size":0,"ondisk_log_size":4793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19583459,"num_objects":4782,"num_object_clones":0,"num_object_copies":9564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1400,"num_write_kb":5600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1314},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":318649769,"num_objects":77798,"num_object_clones":0,"num_object_copies":155596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2815,"num_objects_unfound":0,"num_objects_dirty":77798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42173,"num_write_kb":168672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":791,"num_bytes_recovered":3231744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":641904640,"data_stored":641814315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77877,"ondisk_log_size":77877,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182425,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447972,"kb_used_data":290004,"kb_used_omap":4,"kb_used_meta":157947,"kb_avail":93923868,"statfs":{"total":96636764160,"available":96178040832,"internally_reserved":0,"allocated":296964096,"data_stored":296730963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":161738410},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,0,2,5,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986197,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":412012,"kb_used_data":272028,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93959828,"statfs":{"total":96636764160,"available":96214863872,"internally_reserved":0,"allocated":278556672,"data_stored":278327616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":143322794},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659144,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153548,"kb_used_data":63544,"kb_used_omap":4,"kb_used_meta":89979,"kb_avail":94218292,"statfs":{"total":96636764160,"available":96479531008,"internally_reserved":0,"allocated":65069056,"data_stored":64756437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":92139182},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":297447424,"data_stored":297443811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65847296,"data_stored":65764197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":278609920,"data_stored":278606307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:30:40.979 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:30:41.043 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 73 20 80423 80403 4.30162 2.38281 0.0178921 0.018151 2024-09-10T20:30:41.374 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:41.376 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:30:41.404 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2024-09-10T20:30:40.841034+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":318649788,"num_objects":77800,"num_object_clones":0,"num_object_copies":155600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2815,"num_objects_unfound":0,"num_objects_dirty":77800,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42175,"num_write_kb":168674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":793,"num_bytes_recovered":3231763,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77880,"ondisk_log_size":77880,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1013532,"kb_used_data":625576,"kb_used_omap":12,"kb_used_meta":387891,"kb_avail":282101988,"statfs":{"total":289910292480,"available":288872435712,"internally_reserved":0,"allocated":640589824,"data_stored":639815016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":397200386},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,3,2,7,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25563136,"num_objects":6241,"num_object_clones":0,"num_object_copies":12482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-270,"num_objects_unfound":0,"num_objects_dirty":6241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6241,"num_write_kb":24964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":1105920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003046"},"pg_stats":[{"pgid":"2.f","version":"43'4806","reported_seq":4881,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:36.123696+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:30:36.123696+0000","last_peered":"2024-09-10T20:30:36.123696+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:30:36.123696+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":4806,"log_dups_size":0,"ondisk_log_size":4806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19685376,"num_objects":4806,"num_object_clones":0,"num_object_copies":9612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4806,"num_write_kb":19224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1199},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5036","reported_seq":5141,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:38.111253+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:30:38.111253+0000","last_peered":"2024-09-10T20:30:38.111253+0000","last_clean":"2024-09-10T20:30:38.111253+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:30:38.111253+0000","last_undegraded":"2024-09-10T20:30:38.111253+0000","last_fullsized":"2024-09-10T20:30:38.111253+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":5036,"log_dups_size":0,"ondisk_log_size":5036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20602880,"num_objects":5030,"num_object_clones":0,"num_object_copies":10060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3222,"num_write_kb":12888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4861","reported_seq":6562,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:30:38.128022+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:30:38.128022+0000","last_peered":"2024-09-10T20:30:38.128022+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:30:38.128022+0000","last_undegraded":"2024-09-10T20:30:38.128022+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":4861,"log_dups_size":0,"ondisk_log_size":4861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19894272,"num_objects":4857,"num_object_clones":0,"num_object_copies":9714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2815,"num_objects_unfound":0,"num_objects_dirty":4857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2989,"num_write_kb":11956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":788,"num_bytes_recovered":3227648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2815},{"shards":"2","objects":2042}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4841","reported_seq":4916,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:38.078081+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:30:38.078081+0000","last_peered":"2024-09-10T20:30:38.078081+0000","last_clean":"2024-09-10T20:30:38.078081+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:30:38.078081+0000","last_undegraded":"2024-09-10T20:30:38.078081+0000","last_fullsized":"2024-09-10T20:30:38.078081+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":4841,"log_dups_size":0,"ondisk_log_size":4841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19828736,"num_objects":4841,"num_object_clones":0,"num_object_copies":9682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4841,"num_write_kb":19364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4921","reported_seq":5042,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:36.182123+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:30:36.182123+0000","last_peered":"2024-09-10T20:30:36.182123+0000","last_clean":"2024-09-10T20:30:36.182123+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:30:36.182123+0000","last_undegraded":"2024-09-10T20:30:36.182123+0000","last_fullsized":"2024-09-10T20:30:36.182123+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":4921,"log_dups_size":0,"ondisk_log_size":4921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20127744,"num_objects":4914,"num_object_clones":0,"num_object_copies":9828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1917,"num_write_kb":7668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4872","reported_seq":5003,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:36.013464+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:30:36.013464+0000","last_peered":"2024-09-10T20:30:36.013464+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:30:36.013464+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":4872,"log_dups_size":0,"ondisk_log_size":4872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19935232,"num_objects":4867,"num_object_clones":0,"num_object_copies":9734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1736,"num_write_kb":6944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1253},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4779","reported_seq":4912,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:36.167644+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:30:36.167644+0000","last_peered":"2024-09-10T20:30:36.167644+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:30:36.167644+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":4779,"log_dups_size":0,"ondisk_log_size":4779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19550208,"num_objects":4773,"num_object_clones":0,"num_object_copies":9546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1374,"num_write_kb":5496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1218},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4909","reported_seq":5028,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:36.158871+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:30:36.158871+0000","last_peered":"2024-09-10T20:30:36.158871+0000","last_clean":"2024-09-10T20:30:36.158871+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:30:36.158871+0000","last_undegraded":"2024-09-10T20:30:36.158871+0000","last_fullsized":"2024-09-10T20:30:36.158871+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":4909,"log_dups_size":0,"ondisk_log_size":4909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20099072,"num_objects":4907,"num_object_clones":0,"num_object_copies":9814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1980,"num_write_kb":7920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4787","reported_seq":4914,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:38.121317+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:30:38.121317+0000","last_peered":"2024-09-10T20:30:38.121317+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:30:38.121317+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":4787,"log_dups_size":0,"ondisk_log_size":4787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19578880,"num_objects":4780,"num_object_clones":0,"num_object_copies":9560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2780,"num_write_kb":11120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":1254}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355438+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355438+0000","last_peered":"2024-09-10T20:30:05.355438+0000","last_clean":"2024-09-10T20:30:05.355438+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355438+0000","last_undegraded":"2024-09-10T20:30:05.355438+0000","last_fullsized":"2024-09-10T20:30:05.355438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4868","reported_seq":4971,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:36.103506+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:30:36.103506+0000","last_peered":"2024-09-10T20:30:36.103506+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:30:36.103506+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":4868,"log_dups_size":0,"ondisk_log_size":4868,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19907526,"num_objects":4862,"num_object_clones":0,"num_object_copies":9724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4866,"num_write_kb":19444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1247},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.413176+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.413176+0000","last_peered":"2024-09-10T20:30:05.413176+0000","last_clean":"2024-09-10T20:30:05.413176+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.413176+0000","last_undegraded":"2024-09-10T20:30:05.413176+0000","last_fullsized":"2024-09-10T20:30:05.413176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4853","reported_seq":4976,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:38.078902+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:30:38.078902+0000","last_peered":"2024-09-10T20:30:38.078902+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:30:38.078902+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":4853,"log_dups_size":0,"ondisk_log_size":4853,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19857408,"num_objects":4848,"num_object_clones":0,"num_object_copies":9696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2681,"num_write_kb":10724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":1215}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4941","reported_seq":5063,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:38.116755+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:30:38.116755+0000","last_peered":"2024-09-10T20:30:38.116755+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:30:38.116755+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":4941,"log_dups_size":0,"ondisk_log_size":4941,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20213760,"num_objects":4935,"num_object_clones":0,"num_object_copies":9870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2727,"num_write_kb":10908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":1325}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4837","reported_seq":4967,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:36.112577+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:30:36.112577+0000","last_peered":"2024-09-10T20:30:36.112577+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:30:36.112577+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":4837,"log_dups_size":0,"ondisk_log_size":4837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19804160,"num_objects":4835,"num_object_clones":0,"num_object_copies":9670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1560,"num_write_kb":6240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1187},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4803","reported_seq":4936,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:36.103154+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:30:36.103154+0000","last_peered":"2024-09-10T20:30:36.103154+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:30:36.103154+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":4803,"log_dups_size":0,"ondisk_log_size":4803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19640320,"num_objects":4795,"num_object_clones":0,"num_object_copies":9590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1497,"num_write_kb":5988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1263},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345705+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345705+0000","last_peered":"2024-09-10T20:30:05.345705+0000","last_clean":"2024-09-10T20:30:05.345705+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345705+0000","last_undegraded":"2024-09-10T20:30:05.345705+0000","last_fullsized":"2024-09-10T20:30:05.345705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4970","reported_seq":5096,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:36.181993+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:30:36.181993+0000","last_peered":"2024-09-10T20:30:36.181993+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:30:36.181993+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":4970,"log_dups_size":0,"ondisk_log_size":4970,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20340736,"num_objects":4966,"num_object_clones":0,"num_object_copies":9932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1797,"num_write_kb":7188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1253},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4793","reported_seq":4953,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:36.137133+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:30:36.137133+0000","last_peered":"2024-09-10T20:30:36.137133+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:30:36.137133+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":4793,"log_dups_size":0,"ondisk_log_size":4793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19583459,"num_objects":4782,"num_object_clones":0,"num_object_copies":9564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1400,"num_write_kb":5600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1314},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":318649769,"num_objects":77798,"num_object_clones":0,"num_object_copies":155596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2815,"num_objects_unfound":0,"num_objects_dirty":77798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42173,"num_write_kb":168672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":791,"num_bytes_recovered":3231744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":641904640,"data_stored":641814315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77877,"ondisk_log_size":77877,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182425,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447972,"kb_used_data":290004,"kb_used_omap":4,"kb_used_meta":157947,"kb_avail":93923868,"statfs":{"total":96636764160,"available":96178040832,"internally_reserved":0,"allocated":296964096,"data_stored":296730963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":161738410},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,0,2,5,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986197,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":412012,"kb_used_data":272028,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93959828,"statfs":{"total":96636764160,"available":96214863872,"internally_reserved":0,"allocated":278556672,"data_stored":278327616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":143322794},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659144,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153548,"kb_used_data":63544,"kb_used_omap":4,"kb_used_meta":89979,"kb_avail":94218292,"statfs":{"total":96636764160,"available":96479531008,"internally_reserved":0,"allocated":65069056,"data_stored":64756437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":92139182},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":297447424,"data_stored":297443811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65847296,"data_stored":65764197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":278609920,"data_stored":278606307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:30:41.405 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:30:41.405 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'4806", 'reported_seq': 4881, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:36.123696+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:30:36.123696+0000', 'last_peered': '2024-09-10T20:30:36.123696+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:30:36.123696+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 4806, 'log_dups_size': 0, 'ondisk_log_size': 4806, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19685376, 'num_objects': 4806, 'num_object_clones': 0, 'num_object_copies': 9612, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3607, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4806, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4806, 'num_write_kb': 19224, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1199}, {'shards': '0,2', 'objects': 3607}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:41.405 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T20:30:41.405 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'4861", 'reported_seq': 6562, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:30:38.128022+0000', 'last_change': '2024-09-10T20:30:05.653560+0000', 'last_active': '2024-09-10T20:30:38.128022+0000', 'last_peered': '2024-09-10T20:30:38.128022+0000', 'last_clean': '2024-09-10T20:30:03.279259+0000', 'last_became_active': '2024-09-10T20:30:05.629664+0000', 'last_became_peered': '2024-09-10T20:30:05.629664+0000', 'last_unstale': '2024-09-10T20:30:38.128022+0000', 'last_undegraded': '2024-09-10T20:30:38.128022+0000', 'last_fullsized': '2024-09-10T20:30:05.312061+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1910", 'last_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'objects_scrubbed': 1868, 'log_size': 4861, 'log_dups_size': 0, 'ondisk_log_size': 4861, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 673, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19894272, 'num_objects': 4857, 'num_object_clones': 0, 'num_object_copies': 9714, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2815, 'num_objects_unfound': 0, 'num_objects_dirty': 4857, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2989, 'num_write_kb': 11956, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 788, 'num_bytes_recovered': 3227648, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2815}, {'shards': '2', 'objects': 2042}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:41.405 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:30:41.405 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'4872", 'reported_seq': 5003, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:36.013464+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:30:36.013464+0000', 'last_peered': '2024-09-10T20:30:36.013464+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:30:36.013464+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 4872, 'log_dups_size': 0, 'ondisk_log_size': 4872, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19935232, 'num_objects': 4867, 'num_object_clones': 0, 'num_object_copies': 9734, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3616, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4867, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1736, 'num_write_kb': 6944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1253}, {'shards': '0,2', 'objects': 3614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:41.405 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:30:41.406 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'4779", 'reported_seq': 4912, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:36.167644+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:30:36.167644+0000', 'last_peered': '2024-09-10T20:30:36.167644+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:30:36.167644+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 4779, 'log_dups_size': 0, 'ondisk_log_size': 4779, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19550208, 'num_objects': 4773, 'num_object_clones': 0, 'num_object_copies': 9546, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3557, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4773, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1374, 'num_write_kb': 5496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1218}, {'shards': '0,2', 'objects': 3555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:41.406 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:30:41.406 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'4787", 'reported_seq': 4914, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:38.121317+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:30:38.121317+0000', 'last_peered': '2024-09-10T20:30:38.121317+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:30:38.121317+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 4787, 'log_dups_size': 0, 'ondisk_log_size': 4787, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19578880, 'num_objects': 4780, 'num_object_clones': 0, 'num_object_copies': 9560, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3528, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4780, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2780, 'num_write_kb': 11120, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 1254}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:41.406 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:30:41.406 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'4868", 'reported_seq': 4971, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:36.103506+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:30:36.103506+0000', 'last_peered': '2024-09-10T20:30:36.103506+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:30:36.103506+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 4868, 'log_dups_size': 0, 'ondisk_log_size': 4868, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19907526, 'num_objects': 4862, 'num_object_clones': 0, 'num_object_copies': 9724, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3617, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4862, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4866, 'num_write_kb': 19444, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1247}, {'shards': '0,2', 'objects': 3615}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:41.406 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:30:41.406 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'4853", 'reported_seq': 4976, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:38.078902+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:30:38.078902+0000', 'last_peered': '2024-09-10T20:30:38.078902+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:30:38.078902+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 4853, 'log_dups_size': 0, 'ondisk_log_size': 4853, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19857408, 'num_objects': 4848, 'num_object_clones': 0, 'num_object_copies': 9696, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3635, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4848, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2681, 'num_write_kb': 10724, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3633}, {'shards': '2', 'objects': 1215}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:41.406 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:30:41.406 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'4941", 'reported_seq': 5063, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:38.116755+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:30:38.116755+0000', 'last_peered': '2024-09-10T20:30:38.116755+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:30:38.116755+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 4941, 'log_dups_size': 0, 'ondisk_log_size': 4941, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20213760, 'num_objects': 4935, 'num_object_clones': 0, 'num_object_copies': 9870, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4935, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2727, 'num_write_kb': 10908, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3610}, {'shards': '2', 'objects': 1325}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:41.406 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:30:41.407 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'4837", 'reported_seq': 4967, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:36.112577+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:30:36.112577+0000', 'last_peered': '2024-09-10T20:30:36.112577+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:30:36.112577+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 4837, 'log_dups_size': 0, 'ondisk_log_size': 4837, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19804160, 'num_objects': 4835, 'num_object_clones': 0, 'num_object_copies': 9670, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3649, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4835, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1560, 'num_write_kb': 6240, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1187}, {'shards': '0,2', 'objects': 3648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:41.407 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:30:41.407 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'4803", 'reported_seq': 4936, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:36.103154+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:30:36.103154+0000', 'last_peered': '2024-09-10T20:30:36.103154+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:30:36.103154+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 4803, 'log_dups_size': 0, 'ondisk_log_size': 4803, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19640320, 'num_objects': 4795, 'num_object_clones': 0, 'num_object_copies': 9590, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3535, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4795, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1497, 'num_write_kb': 5988, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1263}, {'shards': '0,2', 'objects': 3532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:41.407 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:30:41.407 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'4970", 'reported_seq': 5096, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:36.181993+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:30:36.181993+0000', 'last_peered': '2024-09-10T20:30:36.181993+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:30:36.181993+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 4970, 'log_dups_size': 0, 'ondisk_log_size': 4970, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20340736, 'num_objects': 4966, 'num_object_clones': 0, 'num_object_copies': 9932, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3715, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4966, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1797, 'num_write_kb': 7188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1253}, {'shards': '0,2', 'objects': 3713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:41.407 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:30:41.407 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'4793", 'reported_seq': 4953, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:36.137133+0000', 'last_change': '2024-09-10T20:30:05.690055+0000', 'last_active': '2024-09-10T20:30:36.137133+0000', 'last_peered': '2024-09-10T20:30:36.137133+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:30:36.137133+0000', 'last_undegraded': '2024-09-10T20:30:05.378941+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 4793, 'log_dups_size': 0, 'ondisk_log_size': 4793, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19583459, 'num_objects': 4782, 'num_object_clones': 0, 'num_object_copies': 9564, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3471, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4782, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1400, 'num_write_kb': 5600, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1314}, {'shards': '0,2', 'objects': 3468}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:41.407 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:30:42.043 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 74 20 80877 80857 4.26745 1.77344 0.0436722 0.0182987 2024-09-10T20:30:42.099 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:42.100 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39542/155600 objects degraded (25.413%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":153,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":77800,"data_bytes":318649788,"bytes_used":1037856768,"bytes_avail":288872435712,"bytes_total":289910292480,"degraded_objects":39542,"degraded_total":155600,"degraded_ratio":0.25412596401028276,"misplaced_objects":2815,"misplaced_total":155600,"misplaced_ratio":0.018091259640102827,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":92136,"recovering_keys_per_sec":0,"num_objects_recovered":270,"num_bytes_recovered":1105920,"num_keys_recovered":0,"write_bytes_sec":2129720,"read_op_per_sec":0,"write_op_per_sec":519},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-10T20:30:14.834797+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T20:30:42.124 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:30:42.125 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:30:42.683 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:42.684 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:30:42.710 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2024-09-10T20:30:40.841034+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":318649788,"num_objects":77800,"num_object_clones":0,"num_object_copies":155600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2815,"num_objects_unfound":0,"num_objects_dirty":77800,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42175,"num_write_kb":168674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":793,"num_bytes_recovered":3231763,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77880,"ondisk_log_size":77880,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1013532,"kb_used_data":625576,"kb_used_omap":12,"kb_used_meta":387891,"kb_avail":282101988,"statfs":{"total":289910292480,"available":288872435712,"internally_reserved":0,"allocated":640589824,"data_stored":639815016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":397200386},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,3,2,7,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25563136,"num_objects":6241,"num_object_clones":0,"num_object_copies":12482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-270,"num_objects_unfound":0,"num_objects_dirty":6241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6241,"num_write_kb":24964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":1105920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003046"},"pg_stats":[{"pgid":"2.f","version":"43'4806","reported_seq":4881,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:36.123696+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:30:36.123696+0000","last_peered":"2024-09-10T20:30:36.123696+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:30:36.123696+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":4806,"log_dups_size":0,"ondisk_log_size":4806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19685376,"num_objects":4806,"num_object_clones":0,"num_object_copies":9612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4806,"num_write_kb":19224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1199},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5036","reported_seq":5141,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:38.111253+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:30:38.111253+0000","last_peered":"2024-09-10T20:30:38.111253+0000","last_clean":"2024-09-10T20:30:38.111253+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:30:38.111253+0000","last_undegraded":"2024-09-10T20:30:38.111253+0000","last_fullsized":"2024-09-10T20:30:38.111253+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":5036,"log_dups_size":0,"ondisk_log_size":5036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20602880,"num_objects":5030,"num_object_clones":0,"num_object_copies":10060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3222,"num_write_kb":12888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4861","reported_seq":6562,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:30:38.128022+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:30:38.128022+0000","last_peered":"2024-09-10T20:30:38.128022+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:30:38.128022+0000","last_undegraded":"2024-09-10T20:30:38.128022+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":4861,"log_dups_size":0,"ondisk_log_size":4861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19894272,"num_objects":4857,"num_object_clones":0,"num_object_copies":9714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2815,"num_objects_unfound":0,"num_objects_dirty":4857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2989,"num_write_kb":11956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":788,"num_bytes_recovered":3227648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2815},{"shards":"2","objects":2042}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4841","reported_seq":4916,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:38.078081+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:30:38.078081+0000","last_peered":"2024-09-10T20:30:38.078081+0000","last_clean":"2024-09-10T20:30:38.078081+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:30:38.078081+0000","last_undegraded":"2024-09-10T20:30:38.078081+0000","last_fullsized":"2024-09-10T20:30:38.078081+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":4841,"log_dups_size":0,"ondisk_log_size":4841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19828736,"num_objects":4841,"num_object_clones":0,"num_object_copies":9682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4841,"num_write_kb":19364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4921","reported_seq":5042,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:36.182123+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:30:36.182123+0000","last_peered":"2024-09-10T20:30:36.182123+0000","last_clean":"2024-09-10T20:30:36.182123+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:30:36.182123+0000","last_undegraded":"2024-09-10T20:30:36.182123+0000","last_fullsized":"2024-09-10T20:30:36.182123+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":4921,"log_dups_size":0,"ondisk_log_size":4921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20127744,"num_objects":4914,"num_object_clones":0,"num_object_copies":9828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1917,"num_write_kb":7668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4872","reported_seq":5003,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:36.013464+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:30:36.013464+0000","last_peered":"2024-09-10T20:30:36.013464+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:30:36.013464+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":4872,"log_dups_size":0,"ondisk_log_size":4872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19935232,"num_objects":4867,"num_object_clones":0,"num_object_copies":9734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1736,"num_write_kb":6944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1253},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4779","reported_seq":4912,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:36.167644+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:30:36.167644+0000","last_peered":"2024-09-10T20:30:36.167644+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:30:36.167644+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":4779,"log_dups_size":0,"ondisk_log_size":4779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19550208,"num_objects":4773,"num_object_clones":0,"num_object_copies":9546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1374,"num_write_kb":5496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1218},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4909","reported_seq":5028,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:36.158871+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:30:36.158871+0000","last_peered":"2024-09-10T20:30:36.158871+0000","last_clean":"2024-09-10T20:30:36.158871+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:30:36.158871+0000","last_undegraded":"2024-09-10T20:30:36.158871+0000","last_fullsized":"2024-09-10T20:30:36.158871+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":4909,"log_dups_size":0,"ondisk_log_size":4909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20099072,"num_objects":4907,"num_object_clones":0,"num_object_copies":9814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1980,"num_write_kb":7920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4787","reported_seq":4914,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:38.121317+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:30:38.121317+0000","last_peered":"2024-09-10T20:30:38.121317+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:30:38.121317+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":4787,"log_dups_size":0,"ondisk_log_size":4787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19578880,"num_objects":4780,"num_object_clones":0,"num_object_copies":9560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2780,"num_write_kb":11120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":1254}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355438+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355438+0000","last_peered":"2024-09-10T20:30:05.355438+0000","last_clean":"2024-09-10T20:30:05.355438+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355438+0000","last_undegraded":"2024-09-10T20:30:05.355438+0000","last_fullsized":"2024-09-10T20:30:05.355438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4868","reported_seq":4971,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:36.103506+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:30:36.103506+0000","last_peered":"2024-09-10T20:30:36.103506+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:30:36.103506+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":4868,"log_dups_size":0,"ondisk_log_size":4868,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19907526,"num_objects":4862,"num_object_clones":0,"num_object_copies":9724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4866,"num_write_kb":19444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1247},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.413176+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.413176+0000","last_peered":"2024-09-10T20:30:05.413176+0000","last_clean":"2024-09-10T20:30:05.413176+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.413176+0000","last_undegraded":"2024-09-10T20:30:05.413176+0000","last_fullsized":"2024-09-10T20:30:05.413176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4853","reported_seq":4976,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:38.078902+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:30:38.078902+0000","last_peered":"2024-09-10T20:30:38.078902+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:30:38.078902+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":4853,"log_dups_size":0,"ondisk_log_size":4853,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19857408,"num_objects":4848,"num_object_clones":0,"num_object_copies":9696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2681,"num_write_kb":10724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":1215}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4941","reported_seq":5063,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:38.116755+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:30:38.116755+0000","last_peered":"2024-09-10T20:30:38.116755+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:30:38.116755+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":4941,"log_dups_size":0,"ondisk_log_size":4941,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20213760,"num_objects":4935,"num_object_clones":0,"num_object_copies":9870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2727,"num_write_kb":10908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":1325}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4837","reported_seq":4967,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:36.112577+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:30:36.112577+0000","last_peered":"2024-09-10T20:30:36.112577+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:30:36.112577+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":4837,"log_dups_size":0,"ondisk_log_size":4837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19804160,"num_objects":4835,"num_object_clones":0,"num_object_copies":9670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1560,"num_write_kb":6240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1187},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4803","reported_seq":4936,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:36.103154+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:30:36.103154+0000","last_peered":"2024-09-10T20:30:36.103154+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:30:36.103154+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":4803,"log_dups_size":0,"ondisk_log_size":4803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19640320,"num_objects":4795,"num_object_clones":0,"num_object_copies":9590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1497,"num_write_kb":5988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1263},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345705+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345705+0000","last_peered":"2024-09-10T20:30:05.345705+0000","last_clean":"2024-09-10T20:30:05.345705+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345705+0000","last_undegraded":"2024-09-10T20:30:05.345705+0000","last_fullsized":"2024-09-10T20:30:05.345705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4970","reported_seq":5096,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:36.181993+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:30:36.181993+0000","last_peered":"2024-09-10T20:30:36.181993+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:30:36.181993+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":4970,"log_dups_size":0,"ondisk_log_size":4970,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20340736,"num_objects":4966,"num_object_clones":0,"num_object_copies":9932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1797,"num_write_kb":7188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1253},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4793","reported_seq":4953,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:36.137133+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:30:36.137133+0000","last_peered":"2024-09-10T20:30:36.137133+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:30:36.137133+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":4793,"log_dups_size":0,"ondisk_log_size":4793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19583459,"num_objects":4782,"num_object_clones":0,"num_object_copies":9564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1400,"num_write_kb":5600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1314},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":318649769,"num_objects":77798,"num_object_clones":0,"num_object_copies":155596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2815,"num_objects_unfound":0,"num_objects_dirty":77798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42173,"num_write_kb":168672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":791,"num_bytes_recovered":3231744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":641904640,"data_stored":641814315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77877,"ondisk_log_size":77877,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182425,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447972,"kb_used_data":290004,"kb_used_omap":4,"kb_used_meta":157947,"kb_avail":93923868,"statfs":{"total":96636764160,"available":96178040832,"internally_reserved":0,"allocated":296964096,"data_stored":296730963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":161738410},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,0,2,5,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986197,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":412012,"kb_used_data":272028,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93959828,"statfs":{"total":96636764160,"available":96214863872,"internally_reserved":0,"allocated":278556672,"data_stored":278327616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":143322794},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659144,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153548,"kb_used_data":63544,"kb_used_omap":4,"kb_used_meta":89979,"kb_avail":94218292,"statfs":{"total":96636764160,"available":96479531008,"internally_reserved":0,"allocated":65069056,"data_stored":64756437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":92139182},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":297447424,"data_stored":297443811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65847296,"data_stored":65764197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":278609920,"data_stored":278606307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:30:43.043 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 75 20 81460 81440 4.24092 2.27734 0.04666 0.0184125 2024-09-10T20:30:44.043 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 76 20 82113 82093 4.21868 2.55078 0.0249773 0.0185114 2024-09-10T20:30:45.043 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 77 20 82750 82730 4.19621 2.48828 0.016461 0.0186113 2024-09-10T20:30:45.712 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:30:46.043 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 78 20 83346 83326 4.17226 2.32812 0.0577887 0.0187159 2024-09-10T20:30:46.291 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:46.291 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:30:46.319 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":115,"stamp":"2024-09-10T20:30:44.842172+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":331011516,"num_objects":80818,"num_object_clones":0,"num_object_copies":161636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2681,"num_objects_unfound":0,"num_objects_dirty":80818,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45193,"num_write_kb":180746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":927,"num_bytes_recovered":3780627,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80898,"ondisk_log_size":80898,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1054876,"kb_used_data":650284,"kb_used_omap":12,"kb_used_meta":404531,"kb_avail":282060644,"statfs":{"total":289910292480,"available":288830099456,"internally_reserved":0,"allocated":665890816,"data_stored":665116008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":414239746},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,2,8,14,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29569024,"num_objects":7219,"num_object_clones":0,"num_object_copies":14438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-404,"num_objects_unfound":0,"num_objects_dirty":7219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":28876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":1654784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003316"},"pg_stats":[{"pgid":"2.f","version":"43'4993","reported_seq":5068,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:40.977457+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:30:40.977457+0000","last_peered":"2024-09-10T20:30:40.977457+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:30:40.977457+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":4993,"log_dups_size":0,"ondisk_log_size":4993,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20451328,"num_objects":4993,"num_object_clones":0,"num_object_copies":9986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4993,"num_write_kb":19972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1386},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5216","reported_seq":5321,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:43.075280+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:30:43.075280+0000","last_peered":"2024-09-10T20:30:43.075280+0000","last_clean":"2024-09-10T20:30:43.075280+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:30:43.075280+0000","last_undegraded":"2024-09-10T20:30:43.075280+0000","last_fullsized":"2024-09-10T20:30:43.075280+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":5216,"log_dups_size":0,"ondisk_log_size":5216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21340160,"num_objects":5210,"num_object_clones":0,"num_object_copies":10420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3402,"num_write_kb":13608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5067","reported_seq":7036,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:30:43.147646+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:30:43.147646+0000","last_peered":"2024-09-10T20:30:43.147646+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:30:43.147646+0000","last_undegraded":"2024-09-10T20:30:43.147646+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":5067,"log_dups_size":0,"ondisk_log_size":5067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20738048,"num_objects":5063,"num_object_clones":0,"num_object_copies":10126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2681,"num_objects_unfound":0,"num_objects_dirty":5063,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3195,"num_write_kb":12780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":922,"num_bytes_recovered":3776512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2681},{"shards":"2","objects":2382}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5014","reported_seq":5089,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:43.087182+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:30:43.087182+0000","last_peered":"2024-09-10T20:30:43.087182+0000","last_clean":"2024-09-10T20:30:43.087182+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:30:43.087182+0000","last_undegraded":"2024-09-10T20:30:43.087182+0000","last_fullsized":"2024-09-10T20:30:43.087182+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":5014,"log_dups_size":0,"ondisk_log_size":5014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20537344,"num_objects":5014,"num_object_clones":0,"num_object_copies":10028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5014,"num_write_kb":20056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5118","reported_seq":5239,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:41.002919+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:30:41.002919+0000","last_peered":"2024-09-10T20:30:41.002919+0000","last_clean":"2024-09-10T20:30:41.002919+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:30:41.002919+0000","last_undegraded":"2024-09-10T20:30:41.002919+0000","last_fullsized":"2024-09-10T20:30:41.002919+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":5118,"log_dups_size":0,"ondisk_log_size":5118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20934656,"num_objects":5111,"num_object_clones":0,"num_object_copies":10222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2114,"num_write_kb":8456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5070","reported_seq":5201,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:40.975322+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:30:40.975322+0000","last_peered":"2024-09-10T20:30:40.975322+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:30:40.975322+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":5070,"log_dups_size":0,"ondisk_log_size":5070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20746240,"num_objects":5065,"num_object_clones":0,"num_object_copies":10130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1934,"num_write_kb":7736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1451},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4951","reported_seq":5084,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:41.016036+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:30:41.016036+0000","last_peered":"2024-09-10T20:30:41.016036+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:30:41.016036+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":4951,"log_dups_size":0,"ondisk_log_size":4951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20254720,"num_objects":4945,"num_object_clones":0,"num_object_copies":9890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1546,"num_write_kb":6184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1390},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5091","reported_seq":5210,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:40.987524+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:30:40.987524+0000","last_peered":"2024-09-10T20:30:40.987524+0000","last_clean":"2024-09-10T20:30:40.987524+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:30:40.987524+0000","last_undegraded":"2024-09-10T20:30:40.987524+0000","last_fullsized":"2024-09-10T20:30:40.987524+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":5091,"log_dups_size":0,"ondisk_log_size":5091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20844544,"num_objects":5089,"num_object_clones":0,"num_object_copies":10178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2162,"num_write_kb":8648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4947","reported_seq":5074,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:43.056389+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:30:43.056389+0000","last_peered":"2024-09-10T20:30:43.056389+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:30:43.056389+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":4947,"log_dups_size":0,"ondisk_log_size":4947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20234240,"num_objects":4940,"num_object_clones":0,"num_object_copies":9880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2940,"num_write_kb":11760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":1414}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355438+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355438+0000","last_peered":"2024-09-10T20:30:05.355438+0000","last_clean":"2024-09-10T20:30:05.355438+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355438+0000","last_undegraded":"2024-09-10T20:30:05.355438+0000","last_fullsized":"2024-09-10T20:30:05.355438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5049","reported_seq":5152,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:40.952793+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:30:40.952793+0000","last_peered":"2024-09-10T20:30:40.952793+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:30:40.952793+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":5049,"log_dups_size":0,"ondisk_log_size":5049,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20648902,"num_objects":5043,"num_object_clones":0,"num_object_copies":10086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5047,"num_write_kb":20168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1428},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.413176+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.413176+0000","last_peered":"2024-09-10T20:30:05.413176+0000","last_clean":"2024-09-10T20:30:05.413176+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.413176+0000","last_undegraded":"2024-09-10T20:30:05.413176+0000","last_fullsized":"2024-09-10T20:30:05.413176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5054","reported_seq":5175,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:43.140260+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:30:43.140260+0000","last_peered":"2024-09-10T20:30:43.140260+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:30:43.140260+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":5054,"log_dups_size":0,"ondisk_log_size":5054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20680704,"num_objects":5049,"num_object_clones":0,"num_object_copies":10098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2882,"num_write_kb":11528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":1416}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5139","reported_seq":5262,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:43.111118+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:30:43.111118+0000","last_peered":"2024-09-10T20:30:43.111118+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:30:43.111118+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":5139,"log_dups_size":0,"ondisk_log_size":5139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21024768,"num_objects":5133,"num_object_clones":0,"num_object_copies":10266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2925,"num_write_kb":11700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":1523}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5052","reported_seq":5182,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:40.978506+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:30:40.978506+0000","last_peered":"2024-09-10T20:30:40.978506+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:30:40.978506+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":5052,"log_dups_size":0,"ondisk_log_size":5052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20684800,"num_objects":5050,"num_object_clones":0,"num_object_copies":10100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1775,"num_write_kb":7100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1402},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4987","reported_seq":5120,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:41.034963+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:30:41.034963+0000","last_peered":"2024-09-10T20:30:41.034963+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:30:41.034963+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":4987,"log_dups_size":0,"ondisk_log_size":4987,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20393984,"num_objects":4979,"num_object_clones":0,"num_object_copies":9958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1681,"num_write_kb":6724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1447},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345705+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345705+0000","last_peered":"2024-09-10T20:30:05.345705+0000","last_clean":"2024-09-10T20:30:05.345705+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345705+0000","last_undegraded":"2024-09-10T20:30:05.345705+0000","last_fullsized":"2024-09-10T20:30:05.345705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5167","reported_seq":5293,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:41.043048+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:30:41.043048+0000","last_peered":"2024-09-10T20:30:41.043048+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:30:41.043048+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":5167,"log_dups_size":0,"ondisk_log_size":5167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21147648,"num_objects":5163,"num_object_clones":0,"num_object_copies":10326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1994,"num_write_kb":7976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1450},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4980","reported_seq":5140,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:41.021828+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:30:41.021828+0000","last_peered":"2024-09-10T20:30:41.021828+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:30:41.021828+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":4980,"log_dups_size":0,"ondisk_log_size":4980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20349411,"num_objects":4969,"num_object_clones":0,"num_object_copies":9938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1587,"num_write_kb":6348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1501},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":331011497,"num_objects":80816,"num_object_clones":0,"num_object_copies":161632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2681,"num_objects_unfound":0,"num_objects_dirty":80816,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45191,"num_write_kb":180744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":925,"num_bytes_recovered":3780608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":667131904,"data_stored":667041579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80895,"ondisk_log_size":80895,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182426,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465512,"kb_used_data":299224,"kb_used_omap":4,"kb_used_meta":166267,"kb_avail":93906328,"statfs":{"total":96636764160,"available":96160079872,"internally_reserved":0,"allocated":306405376,"data_stored":306172243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":170258090},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,4,7],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986198,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422060,"kb_used_data":277916,"kb_used_omap":4,"kb_used_meta":144123,"kb_avail":93949780,"statfs":{"total":96636764160,"available":96204574720,"internally_reserved":0,"allocated":284585984,"data_stored":284356928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":147582634},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659145,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":167304,"kb_used_data":73144,"kb_used_omap":4,"kb_used_meta":94139,"kb_avail":94204536,"statfs":{"total":96636764160,"available":96465444864,"internally_reserved":0,"allocated":74899456,"data_stored":74586837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":96399022},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,1,3,4,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":306749440,"data_stored":306745827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":75710464,"data_stored":75627365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":284672000,"data_stored":284668387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:30:46.320 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:30:46.882 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:46.883 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:30:46.918 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":116,"stamp":"2024-09-10T20:30:46.842794+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":338793916,"num_objects":82718,"num_object_clones":0,"num_object_copies":165436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2681,"num_objects_unfound":0,"num_objects_dirty":82718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47093,"num_write_kb":188346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":927,"num_bytes_recovered":3780627,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82798,"ondisk_log_size":82798,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1051116,"kb_used_data":659004,"kb_used_omap":12,"kb_used_meta":392051,"kb_avail":282064404,"statfs":{"total":289910292480,"available":288833949696,"internally_reserved":0,"allocated":674820096,"data_stored":674045288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":401460226},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,3,4,5,13,11,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32792576,"num_objects":8006,"num_object_clones":0,"num_object_copies":16012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-260,"num_objects_unfound":0,"num_objects_dirty":8006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8006,"num_write_kb":32024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":1064960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003487"},"pg_stats":[{"pgid":"2.f","version":"43'5194","reported_seq":5269,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:46.161141+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:30:46.161141+0000","last_peered":"2024-09-10T20:30:46.161141+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:30:46.161141+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":5194,"log_dups_size":0,"ondisk_log_size":5194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21274624,"num_objects":5194,"num_object_clones":0,"num_object_copies":10388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5194,"num_write_kb":20776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1587},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5216","reported_seq":5321,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:43.075280+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:30:43.075280+0000","last_peered":"2024-09-10T20:30:43.075280+0000","last_clean":"2024-09-10T20:30:43.075280+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:30:43.075280+0000","last_undegraded":"2024-09-10T20:30:43.075280+0000","last_fullsized":"2024-09-10T20:30:43.075280+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":5216,"log_dups_size":0,"ondisk_log_size":5216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21340160,"num_objects":5210,"num_object_clones":0,"num_object_copies":10420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3402,"num_write_kb":13608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5067","reported_seq":7036,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:30:43.147646+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:30:43.147646+0000","last_peered":"2024-09-10T20:30:43.147646+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:30:43.147646+0000","last_undegraded":"2024-09-10T20:30:43.147646+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":5067,"log_dups_size":0,"ondisk_log_size":5067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20738048,"num_objects":5063,"num_object_clones":0,"num_object_copies":10126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2681,"num_objects_unfound":0,"num_objects_dirty":5063,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3195,"num_write_kb":12780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":922,"num_bytes_recovered":3776512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2681},{"shards":"2","objects":2382}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5014","reported_seq":5089,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:43.087182+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:30:43.087182+0000","last_peered":"2024-09-10T20:30:43.087182+0000","last_clean":"2024-09-10T20:30:43.087182+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:30:43.087182+0000","last_undegraded":"2024-09-10T20:30:43.087182+0000","last_fullsized":"2024-09-10T20:30:43.087182+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":5014,"log_dups_size":0,"ondisk_log_size":5014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20537344,"num_objects":5014,"num_object_clones":0,"num_object_copies":10028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5014,"num_write_kb":20056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5299","reported_seq":5420,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:46.101520+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:30:46.101520+0000","last_peered":"2024-09-10T20:30:46.101520+0000","last_clean":"2024-09-10T20:30:46.101520+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:30:46.101520+0000","last_undegraded":"2024-09-10T20:30:46.101520+0000","last_fullsized":"2024-09-10T20:30:46.101520+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":5299,"log_dups_size":0,"ondisk_log_size":5299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21676032,"num_objects":5292,"num_object_clones":0,"num_object_copies":10584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2295,"num_write_kb":9180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5258","reported_seq":5389,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:46.180737+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:30:46.180737+0000","last_peered":"2024-09-10T20:30:46.180737+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:30:46.180737+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":5258,"log_dups_size":0,"ondisk_log_size":5258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21516288,"num_objects":5253,"num_object_clones":0,"num_object_copies":10506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2122,"num_write_kb":8488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1639},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5143","reported_seq":5276,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:46.198218+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:30:46.198218+0000","last_peered":"2024-09-10T20:30:46.198218+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:30:46.198218+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":5143,"log_dups_size":0,"ondisk_log_size":5143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21041152,"num_objects":5137,"num_object_clones":0,"num_object_copies":10274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1738,"num_write_kb":6952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1582},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5272","reported_seq":5391,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:46.068046+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:30:46.068046+0000","last_peered":"2024-09-10T20:30:46.068046+0000","last_clean":"2024-09-10T20:30:46.068046+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:30:46.068046+0000","last_undegraded":"2024-09-10T20:30:46.068046+0000","last_fullsized":"2024-09-10T20:30:46.068046+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":5272,"log_dups_size":0,"ondisk_log_size":5272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21585920,"num_objects":5270,"num_object_clones":0,"num_object_copies":10540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2343,"num_write_kb":9372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4947","reported_seq":5074,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:43.056389+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:30:43.056389+0000","last_peered":"2024-09-10T20:30:43.056389+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:30:43.056389+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":4947,"log_dups_size":0,"ondisk_log_size":4947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20234240,"num_objects":4940,"num_object_clones":0,"num_object_copies":9880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2940,"num_write_kb":11760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":1414}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355438+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355438+0000","last_peered":"2024-09-10T20:30:05.355438+0000","last_clean":"2024-09-10T20:30:05.355438+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355438+0000","last_undegraded":"2024-09-10T20:30:05.355438+0000","last_fullsized":"2024-09-10T20:30:05.355438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5241","reported_seq":5344,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:46.179727+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:30:46.179727+0000","last_peered":"2024-09-10T20:30:46.179727+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:30:46.179727+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":5241,"log_dups_size":0,"ondisk_log_size":5241,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21435334,"num_objects":5235,"num_object_clones":0,"num_object_copies":10470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5239,"num_write_kb":20936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1620},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.413176+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.413176+0000","last_peered":"2024-09-10T20:30:05.413176+0000","last_clean":"2024-09-10T20:30:05.413176+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.413176+0000","last_undegraded":"2024-09-10T20:30:05.413176+0000","last_fullsized":"2024-09-10T20:30:05.413176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5054","reported_seq":5175,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:43.140260+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:30:43.140260+0000","last_peered":"2024-09-10T20:30:43.140260+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:30:43.140260+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":5054,"log_dups_size":0,"ondisk_log_size":5054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20680704,"num_objects":5049,"num_object_clones":0,"num_object_copies":10098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2882,"num_write_kb":11528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":1416}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5139","reported_seq":5262,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:43.111118+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:30:43.111118+0000","last_peered":"2024-09-10T20:30:43.111118+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:30:43.111118+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":5139,"log_dups_size":0,"ondisk_log_size":5139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21024768,"num_objects":5133,"num_object_clones":0,"num_object_copies":10266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2925,"num_write_kb":11700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":1523}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5233","reported_seq":5363,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:46.135854+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:30:46.135854+0000","last_peered":"2024-09-10T20:30:46.135854+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:30:46.135854+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":5233,"log_dups_size":0,"ondisk_log_size":5233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21426176,"num_objects":5231,"num_object_clones":0,"num_object_copies":10462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1956,"num_write_kb":7824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1583},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5193","reported_seq":5326,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:46.220876+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:30:46.220876+0000","last_peered":"2024-09-10T20:30:46.220876+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:30:46.220876+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":5193,"log_dups_size":0,"ondisk_log_size":5193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21237760,"num_objects":5185,"num_object_clones":0,"num_object_copies":10370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1887,"num_write_kb":7548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1653},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345705+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345705+0000","last_peered":"2024-09-10T20:30:05.345705+0000","last_clean":"2024-09-10T20:30:05.345705+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345705+0000","last_undegraded":"2024-09-10T20:30:05.345705+0000","last_fullsized":"2024-09-10T20:30:05.345705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5349","reported_seq":5474,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:46.206092+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:30:46.206092+0000","last_peered":"2024-09-10T20:30:46.206092+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:30:46.206092+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":5349,"log_dups_size":0,"ondisk_log_size":5349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21893120,"num_objects":5345,"num_object_clones":0,"num_object_copies":10690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2176,"num_write_kb":8704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1632},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5176","reported_seq":5335,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:46.214895+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:30:46.214895+0000","last_peered":"2024-09-10T20:30:46.214895+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:30:46.214895+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":5176,"log_dups_size":0,"ondisk_log_size":5176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21152227,"num_objects":5165,"num_object_clones":0,"num_object_copies":10330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1783,"num_write_kb":7132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1697},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":338793897,"num_objects":82716,"num_object_clones":0,"num_object_copies":165432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2681,"num_objects_unfound":0,"num_objects_dirty":82716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47091,"num_write_kb":188344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":925,"num_bytes_recovered":3780608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":676392960,"data_stored":676302635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82795,"ondisk_log_size":82795,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182427,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461752,"kb_used_data":307944,"kb_used_omap":4,"kb_used_meta":153787,"kb_avail":93910088,"statfs":{"total":96636764160,"available":96163930112,"internally_reserved":0,"allocated":315334656,"data_stored":315101523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":157478570},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,2,1,6,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986198,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422060,"kb_used_data":277916,"kb_used_omap":4,"kb_used_meta":144123,"kb_avail":93949780,"statfs":{"total":96636764160,"available":96204574720,"internally_reserved":0,"allocated":284585984,"data_stored":284356928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":147582634},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659145,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":167304,"kb_used_data":73144,"kb_used_omap":4,"kb_used_meta":94139,"kb_avail":94204536,"statfs":{"total":96636764160,"available":96465444864,"internally_reserved":0,"allocated":74899456,"data_stored":74586837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":96399022},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,1,3,4,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":316010496,"data_stored":316006883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":75710464,"data_stored":75627365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":284672000,"data_stored":284668387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:30:46.919 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:30:46.919 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'5194", 'reported_seq': 5269, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:46.161141+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:30:46.161141+0000', 'last_peered': '2024-09-10T20:30:46.161141+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:30:46.161141+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 5194, 'log_dups_size': 0, 'ondisk_log_size': 5194, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21274624, 'num_objects': 5194, 'num_object_clones': 0, 'num_object_copies': 10388, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3607, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5194, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5194, 'num_write_kb': 20776, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1587}, {'shards': '0,2', 'objects': 3607}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:46.920 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T20:30:46.920 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'5067", 'reported_seq': 7036, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:30:43.147646+0000', 'last_change': '2024-09-10T20:30:05.653560+0000', 'last_active': '2024-09-10T20:30:43.147646+0000', 'last_peered': '2024-09-10T20:30:43.147646+0000', 'last_clean': '2024-09-10T20:30:03.279259+0000', 'last_became_active': '2024-09-10T20:30:05.629664+0000', 'last_became_peered': '2024-09-10T20:30:05.629664+0000', 'last_unstale': '2024-09-10T20:30:43.147646+0000', 'last_undegraded': '2024-09-10T20:30:43.147646+0000', 'last_fullsized': '2024-09-10T20:30:05.312061+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1910", 'last_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'objects_scrubbed': 1868, 'log_size': 5067, 'log_dups_size': 0, 'ondisk_log_size': 5067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 673, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20738048, 'num_objects': 5063, 'num_object_clones': 0, 'num_object_copies': 10126, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2681, 'num_objects_unfound': 0, 'num_objects_dirty': 5063, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3195, 'num_write_kb': 12780, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 922, 'num_bytes_recovered': 3776512, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2681}, {'shards': '2', 'objects': 2382}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:46.920 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:30:46.920 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'5258", 'reported_seq': 5389, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:46.180737+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:30:46.180737+0000', 'last_peered': '2024-09-10T20:30:46.180737+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:30:46.180737+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 5258, 'log_dups_size': 0, 'ondisk_log_size': 5258, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21516288, 'num_objects': 5253, 'num_object_clones': 0, 'num_object_copies': 10506, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3616, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5253, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2122, 'num_write_kb': 8488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1639}, {'shards': '0,2', 'objects': 3614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:46.920 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:30:46.920 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'5143", 'reported_seq': 5276, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:46.198218+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:30:46.198218+0000', 'last_peered': '2024-09-10T20:30:46.198218+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:30:46.198218+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 5143, 'log_dups_size': 0, 'ondisk_log_size': 5143, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21041152, 'num_objects': 5137, 'num_object_clones': 0, 'num_object_copies': 10274, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3557, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5137, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1738, 'num_write_kb': 6952, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1582}, {'shards': '0,2', 'objects': 3555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:46.920 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:30:46.920 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'4947", 'reported_seq': 5074, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:43.056389+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:30:43.056389+0000', 'last_peered': '2024-09-10T20:30:43.056389+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:30:43.056389+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 4947, 'log_dups_size': 0, 'ondisk_log_size': 4947, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20234240, 'num_objects': 4940, 'num_object_clones': 0, 'num_object_copies': 9880, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3528, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4940, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2940, 'num_write_kb': 11760, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 1414}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:46.920 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:30:46.921 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'5241", 'reported_seq': 5344, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:46.179727+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:30:46.179727+0000', 'last_peered': '2024-09-10T20:30:46.179727+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:30:46.179727+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 5241, 'log_dups_size': 0, 'ondisk_log_size': 5241, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21435334, 'num_objects': 5235, 'num_object_clones': 0, 'num_object_copies': 10470, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3617, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5235, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5239, 'num_write_kb': 20936, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1620}, {'shards': '0,2', 'objects': 3615}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:46.921 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:30:46.921 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'5054", 'reported_seq': 5175, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:43.140260+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:30:43.140260+0000', 'last_peered': '2024-09-10T20:30:43.140260+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:30:43.140260+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 5054, 'log_dups_size': 0, 'ondisk_log_size': 5054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20680704, 'num_objects': 5049, 'num_object_clones': 0, 'num_object_copies': 10098, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3635, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5049, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2882, 'num_write_kb': 11528, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3633}, {'shards': '2', 'objects': 1416}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:46.921 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:30:46.921 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'5139", 'reported_seq': 5262, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:43.111118+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:30:43.111118+0000', 'last_peered': '2024-09-10T20:30:43.111118+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:30:43.111118+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 5139, 'log_dups_size': 0, 'ondisk_log_size': 5139, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21024768, 'num_objects': 5133, 'num_object_clones': 0, 'num_object_copies': 10266, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5133, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2925, 'num_write_kb': 11700, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3610}, {'shards': '2', 'objects': 1523}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:46.921 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:30:46.921 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'5233", 'reported_seq': 5363, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:46.135854+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:30:46.135854+0000', 'last_peered': '2024-09-10T20:30:46.135854+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:30:46.135854+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 5233, 'log_dups_size': 0, 'ondisk_log_size': 5233, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21426176, 'num_objects': 5231, 'num_object_clones': 0, 'num_object_copies': 10462, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3649, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5231, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1956, 'num_write_kb': 7824, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1583}, {'shards': '0,2', 'objects': 3648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:46.921 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:30:46.921 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'5193", 'reported_seq': 5326, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:46.220876+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:30:46.220876+0000', 'last_peered': '2024-09-10T20:30:46.220876+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:30:46.220876+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 5193, 'log_dups_size': 0, 'ondisk_log_size': 5193, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21237760, 'num_objects': 5185, 'num_object_clones': 0, 'num_object_copies': 10370, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3535, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5185, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1887, 'num_write_kb': 7548, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1653}, {'shards': '0,2', 'objects': 3532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:46.921 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:30:46.921 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'5349", 'reported_seq': 5474, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:46.206092+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:30:46.206092+0000', 'last_peered': '2024-09-10T20:30:46.206092+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:30:46.206092+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 5349, 'log_dups_size': 0, 'ondisk_log_size': 5349, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21893120, 'num_objects': 5345, 'num_object_clones': 0, 'num_object_copies': 10690, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3715, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5345, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2176, 'num_write_kb': 8704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1632}, {'shards': '0,2', 'objects': 3713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:46.922 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:30:46.922 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'5176", 'reported_seq': 5335, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:46.214895+0000', 'last_change': '2024-09-10T20:30:05.690055+0000', 'last_active': '2024-09-10T20:30:46.214895+0000', 'last_peered': '2024-09-10T20:30:46.214895+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:30:46.214895+0000', 'last_undegraded': '2024-09-10T20:30:05.378941+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 5176, 'log_dups_size': 0, 'ondisk_log_size': 5176, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21152227, 'num_objects': 5165, 'num_object_clones': 0, 'num_object_copies': 10330, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3471, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5165, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1783, 'num_write_kb': 7132, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1697}, {'shards': '0,2', 'objects': 3468}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:46.922 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:30:47.043 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 79 20 83904 83884 4.14703 2.17969 0.0328175 0.0188308 2024-09-10T20:30:47.602 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:47.602 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39542/165436 objects degraded (23.902%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":159,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":82718,"data_bytes":338793916,"bytes_used":1076342784,"bytes_avail":288833949696,"bytes_total":289910292480,"degraded_objects":39542,"degraded_total":165436,"degraded_ratio":0.23901690079547377,"misplaced_objects":2681,"misplaced_total":165436,"misplaced_ratio":0.016205662612732416,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":88720,"recovering_keys_per_sec":0,"num_objects_recovered":260,"num_bytes_recovered":1064960,"num_keys_recovered":0,"write_bytes_sec":2731920,"read_op_per_sec":0,"write_op_per_sec":666},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-10T20:30:14.834797+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T20:30:47.623 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:30:47.623 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:30:48.043 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout:2024-09-10T20:30:48.040761+0000 min lat: 0.00191414 max lat: 2.51074 avg lat: 0.0189418 2024-09-10T20:30:48.043 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T20:30:48.043 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 80 20 84462 84442 4.12244 2.17969 0.0656148 0.0189418 2024-09-10T20:30:48.199 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:48.202 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:30:48.225 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":116,"stamp":"2024-09-10T20:30:46.842794+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":338793916,"num_objects":82718,"num_object_clones":0,"num_object_copies":165436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2681,"num_objects_unfound":0,"num_objects_dirty":82718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47093,"num_write_kb":188346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":927,"num_bytes_recovered":3780627,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82798,"ondisk_log_size":82798,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1051116,"kb_used_data":659004,"kb_used_omap":12,"kb_used_meta":392051,"kb_avail":282064404,"statfs":{"total":289910292480,"available":288833949696,"internally_reserved":0,"allocated":674820096,"data_stored":674045288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":401460226},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,3,4,5,13,11,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32792576,"num_objects":8006,"num_object_clones":0,"num_object_copies":16012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-260,"num_objects_unfound":0,"num_objects_dirty":8006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8006,"num_write_kb":32024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":1064960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003487"},"pg_stats":[{"pgid":"2.f","version":"43'5194","reported_seq":5269,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:46.161141+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:30:46.161141+0000","last_peered":"2024-09-10T20:30:46.161141+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:30:46.161141+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":5194,"log_dups_size":0,"ondisk_log_size":5194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21274624,"num_objects":5194,"num_object_clones":0,"num_object_copies":10388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5194,"num_write_kb":20776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1587},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5216","reported_seq":5321,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:43.075280+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:30:43.075280+0000","last_peered":"2024-09-10T20:30:43.075280+0000","last_clean":"2024-09-10T20:30:43.075280+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:30:43.075280+0000","last_undegraded":"2024-09-10T20:30:43.075280+0000","last_fullsized":"2024-09-10T20:30:43.075280+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":5216,"log_dups_size":0,"ondisk_log_size":5216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21340160,"num_objects":5210,"num_object_clones":0,"num_object_copies":10420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3402,"num_write_kb":13608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5067","reported_seq":7036,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:30:43.147646+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:30:43.147646+0000","last_peered":"2024-09-10T20:30:43.147646+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:30:43.147646+0000","last_undegraded":"2024-09-10T20:30:43.147646+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":5067,"log_dups_size":0,"ondisk_log_size":5067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20738048,"num_objects":5063,"num_object_clones":0,"num_object_copies":10126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2681,"num_objects_unfound":0,"num_objects_dirty":5063,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3195,"num_write_kb":12780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":922,"num_bytes_recovered":3776512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2681},{"shards":"2","objects":2382}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5014","reported_seq":5089,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:43.087182+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:30:43.087182+0000","last_peered":"2024-09-10T20:30:43.087182+0000","last_clean":"2024-09-10T20:30:43.087182+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:30:43.087182+0000","last_undegraded":"2024-09-10T20:30:43.087182+0000","last_fullsized":"2024-09-10T20:30:43.087182+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":5014,"log_dups_size":0,"ondisk_log_size":5014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20537344,"num_objects":5014,"num_object_clones":0,"num_object_copies":10028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5014,"num_write_kb":20056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5299","reported_seq":5420,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:46.101520+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:30:46.101520+0000","last_peered":"2024-09-10T20:30:46.101520+0000","last_clean":"2024-09-10T20:30:46.101520+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:30:46.101520+0000","last_undegraded":"2024-09-10T20:30:46.101520+0000","last_fullsized":"2024-09-10T20:30:46.101520+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":5299,"log_dups_size":0,"ondisk_log_size":5299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21676032,"num_objects":5292,"num_object_clones":0,"num_object_copies":10584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2295,"num_write_kb":9180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5258","reported_seq":5389,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:46.180737+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:30:46.180737+0000","last_peered":"2024-09-10T20:30:46.180737+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:30:46.180737+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":5258,"log_dups_size":0,"ondisk_log_size":5258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21516288,"num_objects":5253,"num_object_clones":0,"num_object_copies":10506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2122,"num_write_kb":8488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1639},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5143","reported_seq":5276,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:46.198218+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:30:46.198218+0000","last_peered":"2024-09-10T20:30:46.198218+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:30:46.198218+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":5143,"log_dups_size":0,"ondisk_log_size":5143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21041152,"num_objects":5137,"num_object_clones":0,"num_object_copies":10274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1738,"num_write_kb":6952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1582},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5272","reported_seq":5391,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:46.068046+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:30:46.068046+0000","last_peered":"2024-09-10T20:30:46.068046+0000","last_clean":"2024-09-10T20:30:46.068046+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:30:46.068046+0000","last_undegraded":"2024-09-10T20:30:46.068046+0000","last_fullsized":"2024-09-10T20:30:46.068046+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":5272,"log_dups_size":0,"ondisk_log_size":5272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21585920,"num_objects":5270,"num_object_clones":0,"num_object_copies":10540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2343,"num_write_kb":9372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4947","reported_seq":5074,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:43.056389+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:30:43.056389+0000","last_peered":"2024-09-10T20:30:43.056389+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:30:43.056389+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":4947,"log_dups_size":0,"ondisk_log_size":4947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20234240,"num_objects":4940,"num_object_clones":0,"num_object_copies":9880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2940,"num_write_kb":11760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":1414}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355438+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355438+0000","last_peered":"2024-09-10T20:30:05.355438+0000","last_clean":"2024-09-10T20:30:05.355438+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355438+0000","last_undegraded":"2024-09-10T20:30:05.355438+0000","last_fullsized":"2024-09-10T20:30:05.355438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5241","reported_seq":5344,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:46.179727+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:30:46.179727+0000","last_peered":"2024-09-10T20:30:46.179727+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:30:46.179727+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":5241,"log_dups_size":0,"ondisk_log_size":5241,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21435334,"num_objects":5235,"num_object_clones":0,"num_object_copies":10470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5239,"num_write_kb":20936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1620},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.413176+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.413176+0000","last_peered":"2024-09-10T20:30:05.413176+0000","last_clean":"2024-09-10T20:30:05.413176+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.413176+0000","last_undegraded":"2024-09-10T20:30:05.413176+0000","last_fullsized":"2024-09-10T20:30:05.413176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5054","reported_seq":5175,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:43.140260+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:30:43.140260+0000","last_peered":"2024-09-10T20:30:43.140260+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:30:43.140260+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":5054,"log_dups_size":0,"ondisk_log_size":5054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20680704,"num_objects":5049,"num_object_clones":0,"num_object_copies":10098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2882,"num_write_kb":11528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":1416}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5139","reported_seq":5262,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:43.111118+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:30:43.111118+0000","last_peered":"2024-09-10T20:30:43.111118+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:30:43.111118+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":5139,"log_dups_size":0,"ondisk_log_size":5139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21024768,"num_objects":5133,"num_object_clones":0,"num_object_copies":10266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2925,"num_write_kb":11700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":1523}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5233","reported_seq":5363,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:46.135854+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:30:46.135854+0000","last_peered":"2024-09-10T20:30:46.135854+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:30:46.135854+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":5233,"log_dups_size":0,"ondisk_log_size":5233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21426176,"num_objects":5231,"num_object_clones":0,"num_object_copies":10462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1956,"num_write_kb":7824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1583},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5193","reported_seq":5326,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:46.220876+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:30:46.220876+0000","last_peered":"2024-09-10T20:30:46.220876+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:30:46.220876+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":5193,"log_dups_size":0,"ondisk_log_size":5193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21237760,"num_objects":5185,"num_object_clones":0,"num_object_copies":10370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1887,"num_write_kb":7548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1653},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345705+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345705+0000","last_peered":"2024-09-10T20:30:05.345705+0000","last_clean":"2024-09-10T20:30:05.345705+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345705+0000","last_undegraded":"2024-09-10T20:30:05.345705+0000","last_fullsized":"2024-09-10T20:30:05.345705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5349","reported_seq":5474,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:46.206092+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:30:46.206092+0000","last_peered":"2024-09-10T20:30:46.206092+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:30:46.206092+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":5349,"log_dups_size":0,"ondisk_log_size":5349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21893120,"num_objects":5345,"num_object_clones":0,"num_object_copies":10690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2176,"num_write_kb":8704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1632},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5176","reported_seq":5335,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:46.214895+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:30:46.214895+0000","last_peered":"2024-09-10T20:30:46.214895+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:30:46.214895+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":5176,"log_dups_size":0,"ondisk_log_size":5176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21152227,"num_objects":5165,"num_object_clones":0,"num_object_copies":10330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1783,"num_write_kb":7132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1697},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":338793897,"num_objects":82716,"num_object_clones":0,"num_object_copies":165432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2681,"num_objects_unfound":0,"num_objects_dirty":82716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47091,"num_write_kb":188344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":925,"num_bytes_recovered":3780608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":676392960,"data_stored":676302635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82795,"ondisk_log_size":82795,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182427,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461752,"kb_used_data":307944,"kb_used_omap":4,"kb_used_meta":153787,"kb_avail":93910088,"statfs":{"total":96636764160,"available":96163930112,"internally_reserved":0,"allocated":315334656,"data_stored":315101523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":157478570},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,2,1,6,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986198,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422060,"kb_used_data":277916,"kb_used_omap":4,"kb_used_meta":144123,"kb_avail":93949780,"statfs":{"total":96636764160,"available":96204574720,"internally_reserved":0,"allocated":284585984,"data_stored":284356928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":147582634},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659145,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":167304,"kb_used_data":73144,"kb_used_omap":4,"kb_used_meta":94139,"kb_avail":94204536,"statfs":{"total":96636764160,"available":96465444864,"internally_reserved":0,"allocated":74899456,"data_stored":74586837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":96399022},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,1,3,4,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":316010496,"data_stored":316006883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":75710464,"data_stored":75627365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":284672000,"data_stored":284668387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:30:49.043 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 81 20 85082 85062 4.10145 2.42188 0.0174101 0.0190407 2024-09-10T20:30:50.043 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 82 20 85703 85683 4.08101 2.42578 0.0134943 0.0191355 2024-09-10T20:30:51.044 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 83 20 86336 86316 4.06163 2.47266 0.0396104 0.0192264 2024-09-10T20:30:51.227 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:30:51.788 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:51.788 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:30:51.820 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":118,"stamp":"2024-09-10T20:30:50.844211+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":343225788,"num_objects":83800,"num_object_clones":0,"num_object_copies":167600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2558,"num_objects_unfound":0,"num_objects_dirty":83800,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48175,"num_write_kb":192674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1050,"num_bytes_recovered":4284435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83880,"ondisk_log_size":83880,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1062668,"kb_used_data":674712,"kb_used_omap":12,"kb_used_meta":387891,"kb_avail":282052852,"statfs":{"total":289910292480,"available":288822120448,"internally_reserved":0,"allocated":690905088,"data_stored":690130280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":397200386},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,5,5,4,8,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24576000,"num_objects":6000,"num_object_clones":0,"num_object_copies":12000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-257,"num_objects_unfound":0,"num_objects_dirty":6000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6000,"num_write_kb":24000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":1052672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003878"},"pg_stats":[{"pgid":"2.f","version":"43'5194","reported_seq":5269,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:46.161141+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:30:46.161141+0000","last_peered":"2024-09-10T20:30:46.161141+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:30:46.161141+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":5194,"log_dups_size":0,"ondisk_log_size":5194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21274624,"num_objects":5194,"num_object_clones":0,"num_object_copies":10388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5194,"num_write_kb":20776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1587},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5391","reported_seq":5496,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:48.059557+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:30:48.059557+0000","last_peered":"2024-09-10T20:30:48.059557+0000","last_clean":"2024-09-10T20:30:48.059557+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:30:48.059557+0000","last_undegraded":"2024-09-10T20:30:48.059557+0000","last_fullsized":"2024-09-10T20:30:48.059557+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":5391,"log_dups_size":0,"ondisk_log_size":5391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22056960,"num_objects":5385,"num_object_clones":0,"num_object_copies":10770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3577,"num_write_kb":14308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5258","reported_seq":7472,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:30:48.160463+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:30:48.160463+0000","last_peered":"2024-09-10T20:30:48.160463+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:30:48.160463+0000","last_undegraded":"2024-09-10T20:30:48.160463+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":5258,"log_dups_size":0,"ondisk_log_size":5258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21520384,"num_objects":5254,"num_object_clones":0,"num_object_copies":10508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2558,"num_objects_unfound":0,"num_objects_dirty":5254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3386,"num_write_kb":13544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1045,"num_bytes_recovered":4280320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2558},{"shards":"2","objects":2696}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5196","reported_seq":5271,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:48.101235+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:30:48.101235+0000","last_peered":"2024-09-10T20:30:48.101235+0000","last_clean":"2024-09-10T20:30:48.101235+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:30:48.101235+0000","last_undegraded":"2024-09-10T20:30:48.101235+0000","last_fullsized":"2024-09-10T20:30:48.101235+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":5196,"log_dups_size":0,"ondisk_log_size":5196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21282816,"num_objects":5196,"num_object_clones":0,"num_object_copies":10392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5196,"num_write_kb":20784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5299","reported_seq":5420,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:46.101520+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:30:46.101520+0000","last_peered":"2024-09-10T20:30:46.101520+0000","last_clean":"2024-09-10T20:30:46.101520+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:30:46.101520+0000","last_undegraded":"2024-09-10T20:30:46.101520+0000","last_fullsized":"2024-09-10T20:30:46.101520+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":5299,"log_dups_size":0,"ondisk_log_size":5299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21676032,"num_objects":5292,"num_object_clones":0,"num_object_copies":10584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2295,"num_write_kb":9180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5258","reported_seq":5389,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:46.180737+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:30:46.180737+0000","last_peered":"2024-09-10T20:30:46.180737+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:30:46.180737+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":5258,"log_dups_size":0,"ondisk_log_size":5258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21516288,"num_objects":5253,"num_object_clones":0,"num_object_copies":10506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2122,"num_write_kb":8488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1639},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5143","reported_seq":5276,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:46.198218+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:30:46.198218+0000","last_peered":"2024-09-10T20:30:46.198218+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:30:46.198218+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":5143,"log_dups_size":0,"ondisk_log_size":5143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21041152,"num_objects":5137,"num_object_clones":0,"num_object_copies":10274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1738,"num_write_kb":6952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1582},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5272","reported_seq":5391,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:46.068046+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:30:46.068046+0000","last_peered":"2024-09-10T20:30:46.068046+0000","last_clean":"2024-09-10T20:30:46.068046+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:30:46.068046+0000","last_undegraded":"2024-09-10T20:30:46.068046+0000","last_fullsized":"2024-09-10T20:30:46.068046+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":5272,"log_dups_size":0,"ondisk_log_size":5272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21585920,"num_objects":5270,"num_object_clones":0,"num_object_copies":10540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2343,"num_write_kb":9372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5107","reported_seq":5234,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:48.086845+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:30:48.086845+0000","last_peered":"2024-09-10T20:30:48.086845+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:30:48.086845+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":5107,"log_dups_size":0,"ondisk_log_size":5107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20889600,"num_objects":5100,"num_object_clones":0,"num_object_copies":10200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3100,"num_write_kb":12400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":1574}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355438+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355438+0000","last_peered":"2024-09-10T20:30:05.355438+0000","last_clean":"2024-09-10T20:30:05.355438+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355438+0000","last_undegraded":"2024-09-10T20:30:05.355438+0000","last_fullsized":"2024-09-10T20:30:05.355438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5241","reported_seq":5344,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:46.179727+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:30:46.179727+0000","last_peered":"2024-09-10T20:30:46.179727+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:30:46.179727+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":5241,"log_dups_size":0,"ondisk_log_size":5241,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21435334,"num_objects":5235,"num_object_clones":0,"num_object_copies":10470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5239,"num_write_kb":20936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1620},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.413176+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.413176+0000","last_peered":"2024-09-10T20:30:05.413176+0000","last_clean":"2024-09-10T20:30:05.413176+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.413176+0000","last_undegraded":"2024-09-10T20:30:05.413176+0000","last_fullsized":"2024-09-10T20:30:05.413176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5242","reported_seq":5365,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:48.136075+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:30:48.136075+0000","last_peered":"2024-09-10T20:30:48.136075+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:30:48.136075+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":5242,"log_dups_size":0,"ondisk_log_size":5242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21450752,"num_objects":5237,"num_object_clones":0,"num_object_copies":10474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3070,"num_write_kb":12280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":1604}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5325","reported_seq":5448,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:48.148860+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:30:48.148860+0000","last_peered":"2024-09-10T20:30:48.148860+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:30:48.148860+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":5325,"log_dups_size":0,"ondisk_log_size":5325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21786624,"num_objects":5319,"num_object_clones":0,"num_object_copies":10638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3111,"num_write_kb":12444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":1709}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5233","reported_seq":5363,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:46.135854+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:30:46.135854+0000","last_peered":"2024-09-10T20:30:46.135854+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:30:46.135854+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":5233,"log_dups_size":0,"ondisk_log_size":5233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21426176,"num_objects":5231,"num_object_clones":0,"num_object_copies":10462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1956,"num_write_kb":7824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1583},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5193","reported_seq":5326,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:46.220876+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:30:46.220876+0000","last_peered":"2024-09-10T20:30:46.220876+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:30:46.220876+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":5193,"log_dups_size":0,"ondisk_log_size":5193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21237760,"num_objects":5185,"num_object_clones":0,"num_object_copies":10370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1887,"num_write_kb":7548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1653},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345705+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345705+0000","last_peered":"2024-09-10T20:30:05.345705+0000","last_clean":"2024-09-10T20:30:05.345705+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345705+0000","last_undegraded":"2024-09-10T20:30:05.345705+0000","last_fullsized":"2024-09-10T20:30:05.345705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5349","reported_seq":5474,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:46.206092+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:30:46.206092+0000","last_peered":"2024-09-10T20:30:46.206092+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:30:46.206092+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":5349,"log_dups_size":0,"ondisk_log_size":5349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21893120,"num_objects":5345,"num_object_clones":0,"num_object_copies":10690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2176,"num_write_kb":8704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1632},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5176","reported_seq":5335,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:46.214895+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:30:46.214895+0000","last_peered":"2024-09-10T20:30:46.214895+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:30:46.214895+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":5176,"log_dups_size":0,"ondisk_log_size":5176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21152227,"num_objects":5165,"num_object_clones":0,"num_object_copies":10330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1783,"num_write_kb":7132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1697},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":343225769,"num_objects":83798,"num_object_clones":0,"num_object_copies":167596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2558,"num_objects_unfound":0,"num_objects_dirty":83798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48173,"num_write_kb":192672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1048,"num_bytes_recovered":4284416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":692269056,"data_stored":692178731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83877,"ondisk_log_size":83877,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182427,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461752,"kb_used_data":307944,"kb_used_omap":4,"kb_used_meta":153787,"kb_avail":93910088,"statfs":{"total":96636764160,"available":96163930112,"internally_reserved":0,"allocated":315334656,"data_stored":315101523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":157478570},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,2,1,6,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986199,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432280,"kb_used_data":283976,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":93939560,"statfs":{"total":96636764160,"available":96194109440,"internally_reserved":0,"allocated":290791424,"data_stored":290562368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":151842474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659146,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":168636,"kb_used_data":82792,"kb_used_omap":4,"kb_used_meta":85819,"kb_avail":94203204,"statfs":{"total":96636764160,"available":96464080896,"internally_reserved":0,"allocated":84779008,"data_stored":84466389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":87879342},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":316010496,"data_stored":316006883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":85602304,"data_stored":85519205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":290656256,"data_stored":290652643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:30:51.821 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:30:52.043 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 84 20 86906 86886 4.03979 2.22656 0.043154 0.019332 2024-09-10T20:30:52.423 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:52.998 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:30:52.999 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":118,"stamp":"2024-09-10T20:30:50.844211+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":343225788,"num_objects":83800,"num_object_clones":0,"num_object_copies":167600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2558,"num_objects_unfound":0,"num_objects_dirty":83800,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48175,"num_write_kb":192674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1050,"num_bytes_recovered":4284435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83880,"ondisk_log_size":83880,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1062668,"kb_used_data":674712,"kb_used_omap":12,"kb_used_meta":387891,"kb_avail":282052852,"statfs":{"total":289910292480,"available":288822120448,"internally_reserved":0,"allocated":690905088,"data_stored":690130280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":397200386},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,5,5,4,8,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24576000,"num_objects":6000,"num_object_clones":0,"num_object_copies":12000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-257,"num_objects_unfound":0,"num_objects_dirty":6000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6000,"num_write_kb":24000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":1052672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003878"},"pg_stats":[{"pgid":"2.f","version":"43'5194","reported_seq":5269,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:46.161141+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:30:46.161141+0000","last_peered":"2024-09-10T20:30:46.161141+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:30:46.161141+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":5194,"log_dups_size":0,"ondisk_log_size":5194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21274624,"num_objects":5194,"num_object_clones":0,"num_object_copies":10388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5194,"num_write_kb":20776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1587},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5391","reported_seq":5496,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:48.059557+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:30:48.059557+0000","last_peered":"2024-09-10T20:30:48.059557+0000","last_clean":"2024-09-10T20:30:48.059557+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:30:48.059557+0000","last_undegraded":"2024-09-10T20:30:48.059557+0000","last_fullsized":"2024-09-10T20:30:48.059557+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":5391,"log_dups_size":0,"ondisk_log_size":5391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22056960,"num_objects":5385,"num_object_clones":0,"num_object_copies":10770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3577,"num_write_kb":14308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5258","reported_seq":7472,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:30:48.160463+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:30:48.160463+0000","last_peered":"2024-09-10T20:30:48.160463+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:30:48.160463+0000","last_undegraded":"2024-09-10T20:30:48.160463+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":5258,"log_dups_size":0,"ondisk_log_size":5258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21520384,"num_objects":5254,"num_object_clones":0,"num_object_copies":10508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2558,"num_objects_unfound":0,"num_objects_dirty":5254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3386,"num_write_kb":13544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1045,"num_bytes_recovered":4280320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2558},{"shards":"2","objects":2696}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5196","reported_seq":5271,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:48.101235+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:30:48.101235+0000","last_peered":"2024-09-10T20:30:48.101235+0000","last_clean":"2024-09-10T20:30:48.101235+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:30:48.101235+0000","last_undegraded":"2024-09-10T20:30:48.101235+0000","last_fullsized":"2024-09-10T20:30:48.101235+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":5196,"log_dups_size":0,"ondisk_log_size":5196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21282816,"num_objects":5196,"num_object_clones":0,"num_object_copies":10392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5196,"num_write_kb":20784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5299","reported_seq":5420,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:46.101520+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:30:46.101520+0000","last_peered":"2024-09-10T20:30:46.101520+0000","last_clean":"2024-09-10T20:30:46.101520+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:30:46.101520+0000","last_undegraded":"2024-09-10T20:30:46.101520+0000","last_fullsized":"2024-09-10T20:30:46.101520+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":5299,"log_dups_size":0,"ondisk_log_size":5299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21676032,"num_objects":5292,"num_object_clones":0,"num_object_copies":10584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2295,"num_write_kb":9180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5258","reported_seq":5389,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:46.180737+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:30:46.180737+0000","last_peered":"2024-09-10T20:30:46.180737+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:30:46.180737+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":5258,"log_dups_size":0,"ondisk_log_size":5258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21516288,"num_objects":5253,"num_object_clones":0,"num_object_copies":10506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2122,"num_write_kb":8488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1639},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5143","reported_seq":5276,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:46.198218+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:30:46.198218+0000","last_peered":"2024-09-10T20:30:46.198218+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:30:46.198218+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":5143,"log_dups_size":0,"ondisk_log_size":5143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21041152,"num_objects":5137,"num_object_clones":0,"num_object_copies":10274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1738,"num_write_kb":6952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1582},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5272","reported_seq":5391,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:46.068046+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:30:46.068046+0000","last_peered":"2024-09-10T20:30:46.068046+0000","last_clean":"2024-09-10T20:30:46.068046+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:30:46.068046+0000","last_undegraded":"2024-09-10T20:30:46.068046+0000","last_fullsized":"2024-09-10T20:30:46.068046+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":5272,"log_dups_size":0,"ondisk_log_size":5272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21585920,"num_objects":5270,"num_object_clones":0,"num_object_copies":10540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2343,"num_write_kb":9372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5107","reported_seq":5234,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:48.086845+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:30:48.086845+0000","last_peered":"2024-09-10T20:30:48.086845+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:30:48.086845+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":5107,"log_dups_size":0,"ondisk_log_size":5107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20889600,"num_objects":5100,"num_object_clones":0,"num_object_copies":10200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3100,"num_write_kb":12400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":1574}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355438+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355438+0000","last_peered":"2024-09-10T20:30:05.355438+0000","last_clean":"2024-09-10T20:30:05.355438+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355438+0000","last_undegraded":"2024-09-10T20:30:05.355438+0000","last_fullsized":"2024-09-10T20:30:05.355438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5241","reported_seq":5344,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:46.179727+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:30:46.179727+0000","last_peered":"2024-09-10T20:30:46.179727+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:30:46.179727+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":5241,"log_dups_size":0,"ondisk_log_size":5241,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21435334,"num_objects":5235,"num_object_clones":0,"num_object_copies":10470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5239,"num_write_kb":20936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1620},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.413176+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.413176+0000","last_peered":"2024-09-10T20:30:05.413176+0000","last_clean":"2024-09-10T20:30:05.413176+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.413176+0000","last_undegraded":"2024-09-10T20:30:05.413176+0000","last_fullsized":"2024-09-10T20:30:05.413176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5242","reported_seq":5365,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:48.136075+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:30:48.136075+0000","last_peered":"2024-09-10T20:30:48.136075+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:30:48.136075+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":5242,"log_dups_size":0,"ondisk_log_size":5242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21450752,"num_objects":5237,"num_object_clones":0,"num_object_copies":10474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3070,"num_write_kb":12280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":1604}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5325","reported_seq":5448,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:48.148860+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:30:48.148860+0000","last_peered":"2024-09-10T20:30:48.148860+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:30:48.148860+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":5325,"log_dups_size":0,"ondisk_log_size":5325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21786624,"num_objects":5319,"num_object_clones":0,"num_object_copies":10638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3111,"num_write_kb":12444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":1709}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5233","reported_seq":5363,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:46.135854+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:30:46.135854+0000","last_peered":"2024-09-10T20:30:46.135854+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:30:46.135854+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":5233,"log_dups_size":0,"ondisk_log_size":5233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21426176,"num_objects":5231,"num_object_clones":0,"num_object_copies":10462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1956,"num_write_kb":7824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1583},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5193","reported_seq":5326,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:46.220876+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:30:46.220876+0000","last_peered":"2024-09-10T20:30:46.220876+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:30:46.220876+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":5193,"log_dups_size":0,"ondisk_log_size":5193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21237760,"num_objects":5185,"num_object_clones":0,"num_object_copies":10370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1887,"num_write_kb":7548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1653},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345705+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345705+0000","last_peered":"2024-09-10T20:30:05.345705+0000","last_clean":"2024-09-10T20:30:05.345705+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345705+0000","last_undegraded":"2024-09-10T20:30:05.345705+0000","last_fullsized":"2024-09-10T20:30:05.345705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5349","reported_seq":5474,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:46.206092+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:30:46.206092+0000","last_peered":"2024-09-10T20:30:46.206092+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:30:46.206092+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":5349,"log_dups_size":0,"ondisk_log_size":5349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21893120,"num_objects":5345,"num_object_clones":0,"num_object_copies":10690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2176,"num_write_kb":8704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1632},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5176","reported_seq":5335,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:46.214895+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:30:46.214895+0000","last_peered":"2024-09-10T20:30:46.214895+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:30:46.214895+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":5176,"log_dups_size":0,"ondisk_log_size":5176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21152227,"num_objects":5165,"num_object_clones":0,"num_object_copies":10330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1783,"num_write_kb":7132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1697},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":343225769,"num_objects":83798,"num_object_clones":0,"num_object_copies":167596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2558,"num_objects_unfound":0,"num_objects_dirty":83798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48173,"num_write_kb":192672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1048,"num_bytes_recovered":4284416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":692269056,"data_stored":692178731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83877,"ondisk_log_size":83877,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182427,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461752,"kb_used_data":307944,"kb_used_omap":4,"kb_used_meta":153787,"kb_avail":93910088,"statfs":{"total":96636764160,"available":96163930112,"internally_reserved":0,"allocated":315334656,"data_stored":315101523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":157478570},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,2,1,6,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986199,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432280,"kb_used_data":283976,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":93939560,"statfs":{"total":96636764160,"available":96194109440,"internally_reserved":0,"allocated":290791424,"data_stored":290562368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":151842474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659146,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":168636,"kb_used_data":82792,"kb_used_omap":4,"kb_used_meta":85819,"kb_avail":94203204,"statfs":{"total":96636764160,"available":96464080896,"internally_reserved":0,"allocated":84779008,"data_stored":84466389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":87879342},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":316010496,"data_stored":316006883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":85602304,"data_stored":85519205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":290656256,"data_stored":290652643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:30:53.000 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:30:53.000 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'5194", 'reported_seq': 5269, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:46.161141+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:30:46.161141+0000', 'last_peered': '2024-09-10T20:30:46.161141+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:30:46.161141+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 5194, 'log_dups_size': 0, 'ondisk_log_size': 5194, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21274624, 'num_objects': 5194, 'num_object_clones': 0, 'num_object_copies': 10388, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3607, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5194, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5194, 'num_write_kb': 20776, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1587}, {'shards': '0,2', 'objects': 3607}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:53.000 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T20:30:53.000 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'5258", 'reported_seq': 7472, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:30:48.160463+0000', 'last_change': '2024-09-10T20:30:05.653560+0000', 'last_active': '2024-09-10T20:30:48.160463+0000', 'last_peered': '2024-09-10T20:30:48.160463+0000', 'last_clean': '2024-09-10T20:30:03.279259+0000', 'last_became_active': '2024-09-10T20:30:05.629664+0000', 'last_became_peered': '2024-09-10T20:30:05.629664+0000', 'last_unstale': '2024-09-10T20:30:48.160463+0000', 'last_undegraded': '2024-09-10T20:30:48.160463+0000', 'last_fullsized': '2024-09-10T20:30:05.312061+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1910", 'last_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'objects_scrubbed': 1868, 'log_size': 5258, 'log_dups_size': 0, 'ondisk_log_size': 5258, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 673, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21520384, 'num_objects': 5254, 'num_object_clones': 0, 'num_object_copies': 10508, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2558, 'num_objects_unfound': 0, 'num_objects_dirty': 5254, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3386, 'num_write_kb': 13544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1045, 'num_bytes_recovered': 4280320, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2558}, {'shards': '2', 'objects': 2696}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:53.000 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:30:53.000 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'5258", 'reported_seq': 5389, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:46.180737+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:30:46.180737+0000', 'last_peered': '2024-09-10T20:30:46.180737+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:30:46.180737+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 5258, 'log_dups_size': 0, 'ondisk_log_size': 5258, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21516288, 'num_objects': 5253, 'num_object_clones': 0, 'num_object_copies': 10506, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3616, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5253, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2122, 'num_write_kb': 8488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1639}, {'shards': '0,2', 'objects': 3614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:53.001 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:30:53.001 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'5143", 'reported_seq': 5276, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:46.198218+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:30:46.198218+0000', 'last_peered': '2024-09-10T20:30:46.198218+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:30:46.198218+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 5143, 'log_dups_size': 0, 'ondisk_log_size': 5143, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21041152, 'num_objects': 5137, 'num_object_clones': 0, 'num_object_copies': 10274, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3557, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5137, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1738, 'num_write_kb': 6952, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1582}, {'shards': '0,2', 'objects': 3555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:53.001 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:30:53.001 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'5107", 'reported_seq': 5234, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:48.086845+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:30:48.086845+0000', 'last_peered': '2024-09-10T20:30:48.086845+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:30:48.086845+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 5107, 'log_dups_size': 0, 'ondisk_log_size': 5107, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20889600, 'num_objects': 5100, 'num_object_clones': 0, 'num_object_copies': 10200, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3528, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5100, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3100, 'num_write_kb': 12400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 1574}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:53.001 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:30:53.001 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'5241", 'reported_seq': 5344, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:46.179727+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:30:46.179727+0000', 'last_peered': '2024-09-10T20:30:46.179727+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:30:46.179727+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 5241, 'log_dups_size': 0, 'ondisk_log_size': 5241, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21435334, 'num_objects': 5235, 'num_object_clones': 0, 'num_object_copies': 10470, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3617, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5235, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5239, 'num_write_kb': 20936, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1620}, {'shards': '0,2', 'objects': 3615}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:53.001 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:30:53.001 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'5242", 'reported_seq': 5365, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:48.136075+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:30:48.136075+0000', 'last_peered': '2024-09-10T20:30:48.136075+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:30:48.136075+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 5242, 'log_dups_size': 0, 'ondisk_log_size': 5242, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21450752, 'num_objects': 5237, 'num_object_clones': 0, 'num_object_copies': 10474, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3635, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5237, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3070, 'num_write_kb': 12280, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3633}, {'shards': '2', 'objects': 1604}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:53.001 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:30:53.001 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'5325", 'reported_seq': 5448, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:48.148860+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:30:48.148860+0000', 'last_peered': '2024-09-10T20:30:48.148860+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:30:48.148860+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 5325, 'log_dups_size': 0, 'ondisk_log_size': 5325, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21786624, 'num_objects': 5319, 'num_object_clones': 0, 'num_object_copies': 10638, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5319, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3111, 'num_write_kb': 12444, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3610}, {'shards': '2', 'objects': 1709}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:53.002 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:30:53.002 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'5233", 'reported_seq': 5363, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:46.135854+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:30:46.135854+0000', 'last_peered': '2024-09-10T20:30:46.135854+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:30:46.135854+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 5233, 'log_dups_size': 0, 'ondisk_log_size': 5233, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21426176, 'num_objects': 5231, 'num_object_clones': 0, 'num_object_copies': 10462, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3649, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5231, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1956, 'num_write_kb': 7824, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1583}, {'shards': '0,2', 'objects': 3648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:53.002 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:30:53.002 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'5193", 'reported_seq': 5326, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:46.220876+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:30:46.220876+0000', 'last_peered': '2024-09-10T20:30:46.220876+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:30:46.220876+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 5193, 'log_dups_size': 0, 'ondisk_log_size': 5193, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21237760, 'num_objects': 5185, 'num_object_clones': 0, 'num_object_copies': 10370, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3535, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5185, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1887, 'num_write_kb': 7548, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1653}, {'shards': '0,2', 'objects': 3532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:53.002 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:30:53.002 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'5349", 'reported_seq': 5474, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:46.206092+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:30:46.206092+0000', 'last_peered': '2024-09-10T20:30:46.206092+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:30:46.206092+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 5349, 'log_dups_size': 0, 'ondisk_log_size': 5349, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21893120, 'num_objects': 5345, 'num_object_clones': 0, 'num_object_copies': 10690, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3715, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5345, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2176, 'num_write_kb': 8704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1632}, {'shards': '0,2', 'objects': 3713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:53.002 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:30:53.002 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'5176", 'reported_seq': 5335, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:46.214895+0000', 'last_change': '2024-09-10T20:30:05.690055+0000', 'last_active': '2024-09-10T20:30:46.214895+0000', 'last_peered': '2024-09-10T20:30:46.214895+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:30:46.214895+0000', 'last_undegraded': '2024-09-10T20:30:05.378941+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 5176, 'log_dups_size': 0, 'ondisk_log_size': 5176, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21152227, 'num_objects': 5165, 'num_object_clones': 0, 'num_object_copies': 10330, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3471, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5165, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1783, 'num_write_kb': 7132, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1697}, {'shards': '0,2', 'objects': 3468}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:53.002 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:30:53.044 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 85 20 87510 87490 4.02002 2.35938 0.0146505 0.0194262 2024-09-10T20:30:53.718 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:53.718 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39542/171528 objects degraded (23.053%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":165,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":85764,"data_bytes":351270332,"bytes_used":1152716800,"bytes_avail":288757575680,"bytes_total":289910292480,"degraded_objects":39542,"degraded_total":171528,"degraded_ratio":0.23052796044960588,"misplaced_objects":2558,"misplaced_total":171528,"misplaced_ratio":0.014913017116738958,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":87696,"recovering_keys_per_sec":0,"num_objects_recovered":257,"num_bytes_recovered":1052672,"num_keys_recovered":0,"write_bytes_sec":2717564,"read_op_per_sec":0,"write_op_per_sec":663},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-10T20:30:14.834797+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T20:30:53.746 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:30:53.746 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:30:54.044 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 86 20 88080 88060 3.99916 2.22656 0.0336368 0.0195275 2024-09-10T20:30:54.337 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:54.337 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:30:54.360 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2024-09-10T20:30:52.844630+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":351270332,"num_objects":85764,"num_object_clones":0,"num_object_copies":171528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2558,"num_objects_unfound":0,"num_objects_dirty":85764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50139,"num_write_kb":200530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1050,"num_bytes_recovered":4284435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85844,"ondisk_log_size":85844,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1125700,"kb_used_data":693332,"kb_used_omap":12,"kb_used_meta":432307,"kb_avail":281989820,"statfs":{"total":289910292480,"available":288757575680,"internally_reserved":0,"allocated":709971968,"data_stored":709197160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":442682370},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,6,9,6,10,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32620544,"num_objects":7964,"num_object_clones":0,"num_object_copies":15928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-257,"num_objects_unfound":0,"num_objects_dirty":7964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7964,"num_write_kb":31856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":1052672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003596"},"pg_stats":[{"pgid":"2.f","version":"43'5392","reported_seq":5467,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:51.230289+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:30:51.230289+0000","last_peered":"2024-09-10T20:30:51.230289+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:30:51.230289+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":5392,"log_dups_size":0,"ondisk_log_size":5392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22085632,"num_objects":5392,"num_object_clones":0,"num_object_copies":10784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5392,"num_write_kb":21568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1785},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5391","reported_seq":5496,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:48.059557+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:30:48.059557+0000","last_peered":"2024-09-10T20:30:48.059557+0000","last_clean":"2024-09-10T20:30:48.059557+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:30:48.059557+0000","last_undegraded":"2024-09-10T20:30:48.059557+0000","last_fullsized":"2024-09-10T20:30:48.059557+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":5391,"log_dups_size":0,"ondisk_log_size":5391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22056960,"num_objects":5385,"num_object_clones":0,"num_object_copies":10770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3577,"num_write_kb":14308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5258","reported_seq":7472,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:30:48.160463+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:30:48.160463+0000","last_peered":"2024-09-10T20:30:48.160463+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:30:48.160463+0000","last_undegraded":"2024-09-10T20:30:48.160463+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":5258,"log_dups_size":0,"ondisk_log_size":5258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21520384,"num_objects":5254,"num_object_clones":0,"num_object_copies":10508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2558,"num_objects_unfound":0,"num_objects_dirty":5254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3386,"num_write_kb":13544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1045,"num_bytes_recovered":4280320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2558},{"shards":"2","objects":2696}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5196","reported_seq":5271,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:48.101235+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:30:48.101235+0000","last_peered":"2024-09-10T20:30:48.101235+0000","last_clean":"2024-09-10T20:30:48.101235+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:30:48.101235+0000","last_undegraded":"2024-09-10T20:30:48.101235+0000","last_fullsized":"2024-09-10T20:30:48.101235+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":5196,"log_dups_size":0,"ondisk_log_size":5196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21282816,"num_objects":5196,"num_object_clones":0,"num_object_copies":10392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5196,"num_write_kb":20784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5500","reported_seq":5621,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:51.183285+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:30:51.183285+0000","last_peered":"2024-09-10T20:30:51.183285+0000","last_clean":"2024-09-10T20:30:51.183285+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:30:51.183285+0000","last_undegraded":"2024-09-10T20:30:51.183285+0000","last_fullsized":"2024-09-10T20:30:51.183285+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":5500,"log_dups_size":0,"ondisk_log_size":5500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22499328,"num_objects":5493,"num_object_clones":0,"num_object_copies":10986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2496,"num_write_kb":9984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5436","reported_seq":5567,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:51.228002+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:30:51.228002+0000","last_peered":"2024-09-10T20:30:51.228002+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:30:51.228002+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":5436,"log_dups_size":0,"ondisk_log_size":5436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22245376,"num_objects":5431,"num_object_clones":0,"num_object_copies":10862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2300,"num_write_kb":9200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1817},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5336","reported_seq":5468,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:51.218538+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:30:51.218538+0000","last_peered":"2024-09-10T20:30:51.218538+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:30:51.218538+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":5336,"log_dups_size":0,"ondisk_log_size":5336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21831680,"num_objects":5330,"num_object_clones":0,"num_object_copies":10660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1931,"num_write_kb":7724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1775},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5461","reported_seq":5578,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:51.186751+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:30:51.186751+0000","last_peered":"2024-09-10T20:30:51.186751+0000","last_clean":"2024-09-10T20:30:51.186751+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:30:51.186751+0000","last_undegraded":"2024-09-10T20:30:51.186751+0000","last_fullsized":"2024-09-10T20:30:51.186751+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":5461,"log_dups_size":0,"ondisk_log_size":5461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22360064,"num_objects":5459,"num_object_clones":0,"num_object_copies":10918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5459,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2532,"num_write_kb":10128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5107","reported_seq":5234,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:48.086845+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:30:48.086845+0000","last_peered":"2024-09-10T20:30:48.086845+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:30:48.086845+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":5107,"log_dups_size":0,"ondisk_log_size":5107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20889600,"num_objects":5100,"num_object_clones":0,"num_object_copies":10200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3100,"num_write_kb":12400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":1574}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355438+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355438+0000","last_peered":"2024-09-10T20:30:05.355438+0000","last_clean":"2024-09-10T20:30:05.355438+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355438+0000","last_undegraded":"2024-09-10T20:30:05.355438+0000","last_fullsized":"2024-09-10T20:30:05.355438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5446","reported_seq":5549,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:51.216950+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:30:51.216950+0000","last_peered":"2024-09-10T20:30:51.216950+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:30:51.216950+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":5446,"log_dups_size":0,"ondisk_log_size":5446,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22275014,"num_objects":5440,"num_object_clones":0,"num_object_copies":10880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5444,"num_write_kb":21756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1825},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.413176+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.413176+0000","last_peered":"2024-09-10T20:30:05.413176+0000","last_clean":"2024-09-10T20:30:05.413176+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.413176+0000","last_undegraded":"2024-09-10T20:30:05.413176+0000","last_fullsized":"2024-09-10T20:30:05.413176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5242","reported_seq":5365,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:48.136075+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:30:48.136075+0000","last_peered":"2024-09-10T20:30:48.136075+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:30:48.136075+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":5242,"log_dups_size":0,"ondisk_log_size":5242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21450752,"num_objects":5237,"num_object_clones":0,"num_object_copies":10474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3070,"num_write_kb":12280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":1604}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5325","reported_seq":5448,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:48.148860+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:30:48.148860+0000","last_peered":"2024-09-10T20:30:48.148860+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:30:48.148860+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":5325,"log_dups_size":0,"ondisk_log_size":5325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21786624,"num_objects":5319,"num_object_clones":0,"num_object_copies":10638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3111,"num_write_kb":12444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":1709}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5449","reported_seq":5578,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:51.234460+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:30:51.234460+0000","last_peered":"2024-09-10T20:30:51.234460+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:30:51.234460+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":5449,"log_dups_size":0,"ondisk_log_size":5449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22310912,"num_objects":5447,"num_object_clones":0,"num_object_copies":10894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2172,"num_write_kb":8688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1799},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5394","reported_seq":5527,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:51.237743+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:30:51.237743+0000","last_peered":"2024-09-10T20:30:51.237743+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:30:51.237743+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":5394,"log_dups_size":0,"ondisk_log_size":5394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22061056,"num_objects":5386,"num_object_clones":0,"num_object_copies":10772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2088,"num_write_kb":8352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1854},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345705+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345705+0000","last_peered":"2024-09-10T20:30:05.345705+0000","last_clean":"2024-09-10T20:30:05.345705+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345705+0000","last_undegraded":"2024-09-10T20:30:05.345705+0000","last_fullsized":"2024-09-10T20:30:05.345705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5547","reported_seq":5672,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:51.251804+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:30:51.251804+0000","last_peered":"2024-09-10T20:30:51.251804+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:30:51.251804+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":5547,"log_dups_size":0,"ondisk_log_size":5547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22704128,"num_objects":5543,"num_object_clones":0,"num_object_copies":11086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2374,"num_write_kb":9496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1830},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5361","reported_seq":5521,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:51.202319+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:30:51.202319+0000","last_peered":"2024-09-10T20:30:51.202319+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:30:51.202319+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":5361,"log_dups_size":0,"ondisk_log_size":5361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21909987,"num_objects":5350,"num_object_clones":0,"num_object_copies":10700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1968,"num_write_kb":7872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1882},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":351270313,"num_objects":85762,"num_object_clones":0,"num_object_copies":171524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2558,"num_objects_unfound":0,"num_objects_dirty":85762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50137,"num_write_kb":200528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1048,"num_bytes_recovered":4284416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":711704576,"data_stored":711614251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85841,"ondisk_log_size":85841,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182428,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492936,"kb_used_data":316920,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93878904,"statfs":{"total":96636764160,"available":96131997696,"internally_reserved":0,"allocated":324526080,"data_stored":324292947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180219562},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,3,2,8,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986199,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432280,"kb_used_data":283976,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":93939560,"statfs":{"total":96636764160,"available":96194109440,"internally_reserved":0,"allocated":290791424,"data_stored":290562368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":151842474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659147,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":200484,"kb_used_data":92436,"kb_used_omap":4,"kb_used_meta":108027,"kb_avail":94171356,"statfs":{"total":96636764160,"available":96431468544,"internally_reserved":0,"allocated":94654464,"data_stored":94341845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":110620334},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,3,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":325480448,"data_stored":325476835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":95567872,"data_stored":95484773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":290656256,"data_stored":290652643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:30:55.044 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 87 20 88683 88663 3.98027 2.35547 0.0263588 0.0196199 2024-09-10T20:30:56.044 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 88 20 89313 89293 3.963 2.46094 0.0470063 0.0197053 2024-09-10T20:30:57.044 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 89 20 89946 89926 3.94626 2.47266 0.0218849 0.0197892 2024-09-10T20:30:57.363 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:30:57.906 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:57.907 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:30:57.932 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":121,"stamp":"2024-09-10T20:30:56.845707+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":363468220,"num_objects":88742,"num_object_clones":0,"num_object_copies":177484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2435,"num_objects_unfound":0,"num_objects_dirty":88742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53117,"num_write_kb":212442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1173,"num_bytes_recovered":4788243,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88822,"ondisk_log_size":88822,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1136372,"kb_used_data":708164,"kb_used_omap":12,"kb_used_meta":428147,"kb_avail":281979148,"statfs":{"total":289910292480,"available":288746647552,"internally_reserved":0,"allocated":725159936,"data_stored":724385128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":438422530},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,6,7,6,15,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32456704,"num_objects":7924,"num_object_clones":0,"num_object_copies":15848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-246,"num_objects_unfound":0,"num_objects_dirty":7924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7924,"num_write_kb":31696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":1007616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003535"},"pg_stats":[{"pgid":"2.f","version":"43'5580","reported_seq":5655,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:56.175046+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:30:56.175046+0000","last_peered":"2024-09-10T20:30:56.175046+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:30:56.175046+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":5580,"log_dups_size":0,"ondisk_log_size":5580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22855680,"num_objects":5580,"num_object_clones":0,"num_object_copies":11160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5580,"num_write_kb":22320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1973},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5577","reported_seq":5682,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:53.124711+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:30:53.124711+0000","last_peered":"2024-09-10T20:30:53.124711+0000","last_clean":"2024-09-10T20:30:53.124711+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:30:53.124711+0000","last_undegraded":"2024-09-10T20:30:53.124711+0000","last_fullsized":"2024-09-10T20:30:53.124711+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":5577,"log_dups_size":0,"ondisk_log_size":5577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22818816,"num_objects":5571,"num_object_clones":0,"num_object_copies":11142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3763,"num_write_kb":15052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5476","reported_seq":7937,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:30:53.164933+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:30:53.164933+0000","last_peered":"2024-09-10T20:30:53.164933+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:30:53.164933+0000","last_undegraded":"2024-09-10T20:30:53.164933+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":5476,"log_dups_size":0,"ondisk_log_size":5476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22413312,"num_objects":5472,"num_object_clones":0,"num_object_copies":10944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2435,"num_objects_unfound":0,"num_objects_dirty":5472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3604,"num_write_kb":14416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1168,"num_bytes_recovered":4784128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2435},{"shards":"2","objects":3037}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5381","reported_seq":5456,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:53.105520+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:30:53.105520+0000","last_peered":"2024-09-10T20:30:53.105520+0000","last_clean":"2024-09-10T20:30:53.105520+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:30:53.105520+0000","last_undegraded":"2024-09-10T20:30:53.105520+0000","last_fullsized":"2024-09-10T20:30:53.105520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":5381,"log_dups_size":0,"ondisk_log_size":5381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22040576,"num_objects":5381,"num_object_clones":0,"num_object_copies":10762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5381,"num_write_kb":21524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5689","reported_seq":5809,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:56.274737+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:30:56.274737+0000","last_peered":"2024-09-10T20:30:56.274737+0000","last_clean":"2024-09-10T20:30:56.274737+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:30:56.274737+0000","last_undegraded":"2024-09-10T20:30:56.274737+0000","last_fullsized":"2024-09-10T20:30:56.274737+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":5689,"log_dups_size":0,"ondisk_log_size":5689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23273472,"num_objects":5682,"num_object_clones":0,"num_object_copies":11364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2685,"num_write_kb":10740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5617","reported_seq":5748,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:56.173019+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:30:56.173019+0000","last_peered":"2024-09-10T20:30:56.173019+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:30:56.173019+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":5617,"log_dups_size":0,"ondisk_log_size":5617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22986752,"num_objects":5612,"num_object_clones":0,"num_object_copies":11224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2481,"num_write_kb":9924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1998},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5512","reported_seq":5645,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:56.213752+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:30:56.213752+0000","last_peered":"2024-09-10T20:30:56.213752+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:30:56.213752+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":5512,"log_dups_size":0,"ondisk_log_size":5512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22552576,"num_objects":5506,"num_object_clones":0,"num_object_copies":11012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5506,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2107,"num_write_kb":8428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1951},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5643","reported_seq":5762,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:56.241415+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:30:56.241415+0000","last_peered":"2024-09-10T20:30:56.241415+0000","last_clean":"2024-09-10T20:30:56.241415+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:30:56.241415+0000","last_undegraded":"2024-09-10T20:30:56.241415+0000","last_fullsized":"2024-09-10T20:30:56.241415+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":5643,"log_dups_size":0,"ondisk_log_size":5643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23105536,"num_objects":5641,"num_object_clones":0,"num_object_copies":11282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2714,"num_write_kb":10856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5296","reported_seq":5423,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:53.162162+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:30:53.162162+0000","last_peered":"2024-09-10T20:30:53.162162+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:30:53.162162+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":5296,"log_dups_size":0,"ondisk_log_size":5296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21663744,"num_objects":5289,"num_object_clones":0,"num_object_copies":10578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3289,"num_write_kb":13156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":1763}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355438+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355438+0000","last_peered":"2024-09-10T20:30:05.355438+0000","last_clean":"2024-09-10T20:30:05.355438+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355438+0000","last_undegraded":"2024-09-10T20:30:05.355438+0000","last_fullsized":"2024-09-10T20:30:05.355438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5640","reported_seq":5743,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:56.232096+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:30:56.232096+0000","last_peered":"2024-09-10T20:30:56.232096+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:30:56.232096+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":5640,"log_dups_size":0,"ondisk_log_size":5640,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23069638,"num_objects":5634,"num_object_clones":0,"num_object_copies":11268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5638,"num_write_kb":22532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2019},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.413176+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.413176+0000","last_peered":"2024-09-10T20:30:05.413176+0000","last_clean":"2024-09-10T20:30:05.413176+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.413176+0000","last_undegraded":"2024-09-10T20:30:05.413176+0000","last_fullsized":"2024-09-10T20:30:05.413176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5411","reported_seq":5533,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:53.131059+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:30:53.131059+0000","last_peered":"2024-09-10T20:30:53.131059+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:30:53.131059+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":5411,"log_dups_size":0,"ondisk_log_size":5411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22142976,"num_objects":5406,"num_object_clones":0,"num_object_copies":10812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3239,"num_write_kb":12956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":1773}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5498","reported_seq":5621,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:53.161088+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:30:53.161088+0000","last_peered":"2024-09-10T20:30:53.161088+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:30:53.161088+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":5498,"log_dups_size":0,"ondisk_log_size":5498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22495232,"num_objects":5492,"num_object_clones":0,"num_object_copies":10984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3284,"num_write_kb":13136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":1882}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5631","reported_seq":5761,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:56.242141+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:30:56.242141+0000","last_peered":"2024-09-10T20:30:56.242141+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:30:56.242141+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":5631,"log_dups_size":0,"ondisk_log_size":5631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23056384,"num_objects":5629,"num_object_clones":0,"num_object_copies":11258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2354,"num_write_kb":9416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1981},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5596","reported_seq":5729,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:56.269313+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:30:56.269313+0000","last_peered":"2024-09-10T20:30:56.269313+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:30:56.269313+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":5596,"log_dups_size":0,"ondisk_log_size":5596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22888448,"num_objects":5588,"num_object_clones":0,"num_object_copies":11176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2290,"num_write_kb":9160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2056},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345705+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345705+0000","last_peered":"2024-09-10T20:30:05.345705+0000","last_clean":"2024-09-10T20:30:05.345705+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345705+0000","last_undegraded":"2024-09-10T20:30:05.345705+0000","last_fullsized":"2024-09-10T20:30:05.345705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5730","reported_seq":5856,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:56.174212+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:30:56.174212+0000","last_peered":"2024-09-10T20:30:56.174212+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:30:56.174212+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":5730,"log_dups_size":0,"ondisk_log_size":5730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23453696,"num_objects":5726,"num_object_clones":0,"num_object_copies":11452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2557,"num_write_kb":10228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2013},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5542","reported_seq":5702,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:56.228096+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:30:56.228096+0000","last_peered":"2024-09-10T20:30:56.228096+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:30:56.228096+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":5542,"log_dups_size":0,"ondisk_log_size":5542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22651363,"num_objects":5531,"num_object_clones":0,"num_object_copies":11062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5531,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2149,"num_write_kb":8596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2063},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":363468201,"num_objects":88740,"num_object_clones":0,"num_object_copies":177480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2435,"num_objects_unfound":0,"num_objects_dirty":88740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53115,"num_write_kb":212440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1171,"num_bytes_recovered":4788224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":726892544,"data_stored":726802219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88819,"ondisk_log_size":88819,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182429,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":510308,"kb_used_data":325972,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93861532,"statfs":{"total":96636764160,"available":96114208768,"internally_reserved":0,"allocated":333795328,"data_stored":333562195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188739242},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,0,3,1,7,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986200,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425580,"kb_used_data":289756,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":93946260,"statfs":{"total":96636764160,"available":96200970240,"internally_reserved":0,"allocated":296710144,"data_stored":296481088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":139062954},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659147,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":200484,"kb_used_data":92436,"kb_used_omap":4,"kb_used_meta":108027,"kb_avail":94171356,"statfs":{"total":96636764160,"available":96431468544,"internally_reserved":0,"allocated":94654464,"data_stored":94341845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":110620334},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,3,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":334589952,"data_stored":334586339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":95567872,"data_stored":95484773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":296734720,"data_stored":296731107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:30:57.935 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:30:58.044 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 90 20 90523 90503 3.92745 2.25391 0.0690052 0.0198823 2024-09-10T20:30:58.520 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:58.523 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:30:58.545 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":121,"stamp":"2024-09-10T20:30:56.845707+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":363468220,"num_objects":88742,"num_object_clones":0,"num_object_copies":177484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2435,"num_objects_unfound":0,"num_objects_dirty":88742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53117,"num_write_kb":212442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1173,"num_bytes_recovered":4788243,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88822,"ondisk_log_size":88822,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1136372,"kb_used_data":708164,"kb_used_omap":12,"kb_used_meta":428147,"kb_avail":281979148,"statfs":{"total":289910292480,"available":288746647552,"internally_reserved":0,"allocated":725159936,"data_stored":724385128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":438422530},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,6,7,6,15,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32456704,"num_objects":7924,"num_object_clones":0,"num_object_copies":15848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-246,"num_objects_unfound":0,"num_objects_dirty":7924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7924,"num_write_kb":31696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":1007616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003535"},"pg_stats":[{"pgid":"2.f","version":"43'5580","reported_seq":5655,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:56.175046+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:30:56.175046+0000","last_peered":"2024-09-10T20:30:56.175046+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:30:56.175046+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":5580,"log_dups_size":0,"ondisk_log_size":5580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22855680,"num_objects":5580,"num_object_clones":0,"num_object_copies":11160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5580,"num_write_kb":22320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1973},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5577","reported_seq":5682,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:53.124711+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:30:53.124711+0000","last_peered":"2024-09-10T20:30:53.124711+0000","last_clean":"2024-09-10T20:30:53.124711+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:30:53.124711+0000","last_undegraded":"2024-09-10T20:30:53.124711+0000","last_fullsized":"2024-09-10T20:30:53.124711+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":5577,"log_dups_size":0,"ondisk_log_size":5577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22818816,"num_objects":5571,"num_object_clones":0,"num_object_copies":11142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3763,"num_write_kb":15052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5476","reported_seq":7937,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:30:53.164933+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:30:53.164933+0000","last_peered":"2024-09-10T20:30:53.164933+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:30:53.164933+0000","last_undegraded":"2024-09-10T20:30:53.164933+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":5476,"log_dups_size":0,"ondisk_log_size":5476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22413312,"num_objects":5472,"num_object_clones":0,"num_object_copies":10944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2435,"num_objects_unfound":0,"num_objects_dirty":5472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3604,"num_write_kb":14416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1168,"num_bytes_recovered":4784128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2435},{"shards":"2","objects":3037}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5381","reported_seq":5456,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:53.105520+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:30:53.105520+0000","last_peered":"2024-09-10T20:30:53.105520+0000","last_clean":"2024-09-10T20:30:53.105520+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:30:53.105520+0000","last_undegraded":"2024-09-10T20:30:53.105520+0000","last_fullsized":"2024-09-10T20:30:53.105520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":5381,"log_dups_size":0,"ondisk_log_size":5381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22040576,"num_objects":5381,"num_object_clones":0,"num_object_copies":10762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5381,"num_write_kb":21524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5689","reported_seq":5809,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:56.274737+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:30:56.274737+0000","last_peered":"2024-09-10T20:30:56.274737+0000","last_clean":"2024-09-10T20:30:56.274737+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:30:56.274737+0000","last_undegraded":"2024-09-10T20:30:56.274737+0000","last_fullsized":"2024-09-10T20:30:56.274737+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":5689,"log_dups_size":0,"ondisk_log_size":5689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23273472,"num_objects":5682,"num_object_clones":0,"num_object_copies":11364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2685,"num_write_kb":10740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5617","reported_seq":5748,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:56.173019+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:30:56.173019+0000","last_peered":"2024-09-10T20:30:56.173019+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:30:56.173019+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":5617,"log_dups_size":0,"ondisk_log_size":5617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22986752,"num_objects":5612,"num_object_clones":0,"num_object_copies":11224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2481,"num_write_kb":9924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1998},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5512","reported_seq":5645,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:56.213752+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:30:56.213752+0000","last_peered":"2024-09-10T20:30:56.213752+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:30:56.213752+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":5512,"log_dups_size":0,"ondisk_log_size":5512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22552576,"num_objects":5506,"num_object_clones":0,"num_object_copies":11012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5506,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2107,"num_write_kb":8428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1951},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5643","reported_seq":5762,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:56.241415+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:30:56.241415+0000","last_peered":"2024-09-10T20:30:56.241415+0000","last_clean":"2024-09-10T20:30:56.241415+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:30:56.241415+0000","last_undegraded":"2024-09-10T20:30:56.241415+0000","last_fullsized":"2024-09-10T20:30:56.241415+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":5643,"log_dups_size":0,"ondisk_log_size":5643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23105536,"num_objects":5641,"num_object_clones":0,"num_object_copies":11282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2714,"num_write_kb":10856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5296","reported_seq":5423,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:53.162162+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:30:53.162162+0000","last_peered":"2024-09-10T20:30:53.162162+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:30:53.162162+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":5296,"log_dups_size":0,"ondisk_log_size":5296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21663744,"num_objects":5289,"num_object_clones":0,"num_object_copies":10578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3289,"num_write_kb":13156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":1763}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355438+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355438+0000","last_peered":"2024-09-10T20:30:05.355438+0000","last_clean":"2024-09-10T20:30:05.355438+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355438+0000","last_undegraded":"2024-09-10T20:30:05.355438+0000","last_fullsized":"2024-09-10T20:30:05.355438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5640","reported_seq":5743,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:56.232096+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:30:56.232096+0000","last_peered":"2024-09-10T20:30:56.232096+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:30:56.232096+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":5640,"log_dups_size":0,"ondisk_log_size":5640,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23069638,"num_objects":5634,"num_object_clones":0,"num_object_copies":11268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5638,"num_write_kb":22532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2019},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.413176+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.413176+0000","last_peered":"2024-09-10T20:30:05.413176+0000","last_clean":"2024-09-10T20:30:05.413176+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.413176+0000","last_undegraded":"2024-09-10T20:30:05.413176+0000","last_fullsized":"2024-09-10T20:30:05.413176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5411","reported_seq":5533,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:53.131059+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:30:53.131059+0000","last_peered":"2024-09-10T20:30:53.131059+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:30:53.131059+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":5411,"log_dups_size":0,"ondisk_log_size":5411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22142976,"num_objects":5406,"num_object_clones":0,"num_object_copies":10812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3239,"num_write_kb":12956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":1773}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5498","reported_seq":5621,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:53.161088+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:30:53.161088+0000","last_peered":"2024-09-10T20:30:53.161088+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:30:53.161088+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":5498,"log_dups_size":0,"ondisk_log_size":5498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22495232,"num_objects":5492,"num_object_clones":0,"num_object_copies":10984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3284,"num_write_kb":13136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":1882}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5631","reported_seq":5761,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:56.242141+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:30:56.242141+0000","last_peered":"2024-09-10T20:30:56.242141+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:30:56.242141+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":5631,"log_dups_size":0,"ondisk_log_size":5631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23056384,"num_objects":5629,"num_object_clones":0,"num_object_copies":11258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2354,"num_write_kb":9416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1981},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5596","reported_seq":5729,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:56.269313+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:30:56.269313+0000","last_peered":"2024-09-10T20:30:56.269313+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:30:56.269313+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":5596,"log_dups_size":0,"ondisk_log_size":5596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22888448,"num_objects":5588,"num_object_clones":0,"num_object_copies":11176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2290,"num_write_kb":9160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2056},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345705+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345705+0000","last_peered":"2024-09-10T20:30:05.345705+0000","last_clean":"2024-09-10T20:30:05.345705+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345705+0000","last_undegraded":"2024-09-10T20:30:05.345705+0000","last_fullsized":"2024-09-10T20:30:05.345705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5730","reported_seq":5856,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:56.174212+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:30:56.174212+0000","last_peered":"2024-09-10T20:30:56.174212+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:30:56.174212+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":5730,"log_dups_size":0,"ondisk_log_size":5730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23453696,"num_objects":5726,"num_object_clones":0,"num_object_copies":11452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2557,"num_write_kb":10228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2013},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5542","reported_seq":5702,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:56.228096+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:30:56.228096+0000","last_peered":"2024-09-10T20:30:56.228096+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:30:56.228096+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":5542,"log_dups_size":0,"ondisk_log_size":5542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22651363,"num_objects":5531,"num_object_clones":0,"num_object_copies":11062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5531,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2149,"num_write_kb":8596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2063},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":363468201,"num_objects":88740,"num_object_clones":0,"num_object_copies":177480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2435,"num_objects_unfound":0,"num_objects_dirty":88740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53115,"num_write_kb":212440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1171,"num_bytes_recovered":4788224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":726892544,"data_stored":726802219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88819,"ondisk_log_size":88819,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182429,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":510308,"kb_used_data":325972,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93861532,"statfs":{"total":96636764160,"available":96114208768,"internally_reserved":0,"allocated":333795328,"data_stored":333562195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188739242},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,0,3,1,7,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986200,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425580,"kb_used_data":289756,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":93946260,"statfs":{"total":96636764160,"available":96200970240,"internally_reserved":0,"allocated":296710144,"data_stored":296481088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":139062954},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659147,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":200484,"kb_used_data":92436,"kb_used_omap":4,"kb_used_meta":108027,"kb_avail":94171356,"statfs":{"total":96636764160,"available":96431468544,"internally_reserved":0,"allocated":94654464,"data_stored":94341845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":110620334},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,3,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":334589952,"data_stored":334586339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":95567872,"data_stored":95484773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":296734720,"data_stored":296731107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:30:58.546 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:30:58.546 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'5580", 'reported_seq': 5655, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:56.175046+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:30:56.175046+0000', 'last_peered': '2024-09-10T20:30:56.175046+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:30:56.175046+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 5580, 'log_dups_size': 0, 'ondisk_log_size': 5580, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22855680, 'num_objects': 5580, 'num_object_clones': 0, 'num_object_copies': 11160, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3607, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5580, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5580, 'num_write_kb': 22320, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1973}, {'shards': '0,2', 'objects': 3607}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:58.546 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T20:30:58.546 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'5476", 'reported_seq': 7937, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:30:53.164933+0000', 'last_change': '2024-09-10T20:30:05.653560+0000', 'last_active': '2024-09-10T20:30:53.164933+0000', 'last_peered': '2024-09-10T20:30:53.164933+0000', 'last_clean': '2024-09-10T20:30:03.279259+0000', 'last_became_active': '2024-09-10T20:30:05.629664+0000', 'last_became_peered': '2024-09-10T20:30:05.629664+0000', 'last_unstale': '2024-09-10T20:30:53.164933+0000', 'last_undegraded': '2024-09-10T20:30:53.164933+0000', 'last_fullsized': '2024-09-10T20:30:05.312061+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1910", 'last_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'objects_scrubbed': 1868, 'log_size': 5476, 'log_dups_size': 0, 'ondisk_log_size': 5476, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 673, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22413312, 'num_objects': 5472, 'num_object_clones': 0, 'num_object_copies': 10944, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2435, 'num_objects_unfound': 0, 'num_objects_dirty': 5472, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3604, 'num_write_kb': 14416, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1168, 'num_bytes_recovered': 4784128, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2435}, {'shards': '2', 'objects': 3037}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:58.546 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:30:58.547 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'5617", 'reported_seq': 5748, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:56.173019+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:30:56.173019+0000', 'last_peered': '2024-09-10T20:30:56.173019+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:30:56.173019+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 5617, 'log_dups_size': 0, 'ondisk_log_size': 5617, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22986752, 'num_objects': 5612, 'num_object_clones': 0, 'num_object_copies': 11224, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3616, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5612, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2481, 'num_write_kb': 9924, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1998}, {'shards': '0,2', 'objects': 3614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:58.547 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:30:58.547 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'5512", 'reported_seq': 5645, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:56.213752+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:30:56.213752+0000', 'last_peered': '2024-09-10T20:30:56.213752+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:30:56.213752+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 5512, 'log_dups_size': 0, 'ondisk_log_size': 5512, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22552576, 'num_objects': 5506, 'num_object_clones': 0, 'num_object_copies': 11012, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3557, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5506, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2107, 'num_write_kb': 8428, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1951}, {'shards': '0,2', 'objects': 3555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:58.547 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:30:58.547 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'5296", 'reported_seq': 5423, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:53.162162+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:30:53.162162+0000', 'last_peered': '2024-09-10T20:30:53.162162+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:30:53.162162+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 5296, 'log_dups_size': 0, 'ondisk_log_size': 5296, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21663744, 'num_objects': 5289, 'num_object_clones': 0, 'num_object_copies': 10578, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3528, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5289, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3289, 'num_write_kb': 13156, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 1763}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:58.547 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:30:58.547 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'5640", 'reported_seq': 5743, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:56.232096+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:30:56.232096+0000', 'last_peered': '2024-09-10T20:30:56.232096+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:30:56.232096+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 5640, 'log_dups_size': 0, 'ondisk_log_size': 5640, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23069638, 'num_objects': 5634, 'num_object_clones': 0, 'num_object_copies': 11268, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3617, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5634, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5638, 'num_write_kb': 22532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2019}, {'shards': '0,2', 'objects': 3615}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:58.547 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:30:58.547 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'5411", 'reported_seq': 5533, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:53.131059+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:30:53.131059+0000', 'last_peered': '2024-09-10T20:30:53.131059+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:30:53.131059+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 5411, 'log_dups_size': 0, 'ondisk_log_size': 5411, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22142976, 'num_objects': 5406, 'num_object_clones': 0, 'num_object_copies': 10812, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3635, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5406, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3239, 'num_write_kb': 12956, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3633}, {'shards': '2', 'objects': 1773}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:58.547 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:30:58.547 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'5498", 'reported_seq': 5621, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:53.161088+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:30:53.161088+0000', 'last_peered': '2024-09-10T20:30:53.161088+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:30:53.161088+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 5498, 'log_dups_size': 0, 'ondisk_log_size': 5498, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22495232, 'num_objects': 5492, 'num_object_clones': 0, 'num_object_copies': 10984, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5492, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3284, 'num_write_kb': 13136, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3610}, {'shards': '2', 'objects': 1882}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:30:58.548 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:30:58.548 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'5631", 'reported_seq': 5761, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:56.242141+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:30:56.242141+0000', 'last_peered': '2024-09-10T20:30:56.242141+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:30:56.242141+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 5631, 'log_dups_size': 0, 'ondisk_log_size': 5631, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23056384, 'num_objects': 5629, 'num_object_clones': 0, 'num_object_copies': 11258, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3649, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5629, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2354, 'num_write_kb': 9416, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1981}, {'shards': '0,2', 'objects': 3648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:58.548 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:30:58.548 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'5596", 'reported_seq': 5729, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:56.269313+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:30:56.269313+0000', 'last_peered': '2024-09-10T20:30:56.269313+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:30:56.269313+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 5596, 'log_dups_size': 0, 'ondisk_log_size': 5596, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22888448, 'num_objects': 5588, 'num_object_clones': 0, 'num_object_copies': 11176, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3535, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5588, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2290, 'num_write_kb': 9160, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2056}, {'shards': '0,2', 'objects': 3532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:58.548 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:30:58.548 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'5730", 'reported_seq': 5856, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:56.174212+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:30:56.174212+0000', 'last_peered': '2024-09-10T20:30:56.174212+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:30:56.174212+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 5730, 'log_dups_size': 0, 'ondisk_log_size': 5730, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23453696, 'num_objects': 5726, 'num_object_clones': 0, 'num_object_copies': 11452, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3715, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5726, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2557, 'num_write_kb': 10228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2013}, {'shards': '0,2', 'objects': 3713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:58.548 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:30:58.548 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'5542", 'reported_seq': 5702, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:56.228096+0000', 'last_change': '2024-09-10T20:30:05.690055+0000', 'last_active': '2024-09-10T20:30:56.228096+0000', 'last_peered': '2024-09-10T20:30:56.228096+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:30:56.228096+0000', 'last_undegraded': '2024-09-10T20:30:05.378941+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 5542, 'log_dups_size': 0, 'ondisk_log_size': 5542, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22651363, 'num_objects': 5531, 'num_object_clones': 0, 'num_object_copies': 11062, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3471, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5531, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2149, 'num_write_kb': 8596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2063}, {'shards': '0,2', 'objects': 3468}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:30:58.548 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:30:59.044 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 91 20 91078 91058 3.90812 2.16797 0.0432431 0.0199819 2024-09-10T20:30:59.239 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:59.240 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39542/179780 objects degraded (21.995%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":170,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":89890,"data_bytes":368170428,"bytes_used":1210847232,"bytes_avail":288699445248,"bytes_total":289910292480,"degraded_objects":39542,"degraded_total":179780,"degraded_ratio":0.21994660140171318,"misplaced_objects":2327,"misplaced_total":179780,"misplaced_ratio":0.012943597730559572,"recovering_objects_per_sec":29,"recovering_bytes_per_sec":120794,"recovering_keys_per_sec":0,"num_objects_recovered":354,"num_bytes_recovered":1449984,"num_keys_recovered":0,"write_bytes_sec":2447289,"read_op_per_sec":0,"write_op_per_sec":597},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-10T20:30:14.834797+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T20:30:59.267 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:30:59.268 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:30:59.814 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:30:59.815 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:30:59.846 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2024-09-10T20:30:58.846486+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":368170428,"num_objects":89890,"num_object_clones":0,"num_object_copies":179780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2327,"num_objects_unfound":0,"num_objects_dirty":89890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":54265,"num_write_kb":217034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1281,"num_bytes_recovered":5230611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":89970,"ondisk_log_size":89970,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1182468,"kb_used_data":723732,"kb_used_omap":12,"kb_used_meta":458675,"kb_avail":281933052,"statfs":{"total":289910292480,"available":288699445248,"internally_reserved":0,"allocated":741101568,"data_stored":740326760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":469683202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,3,2,5,6,10,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29376512,"num_objects":7172,"num_object_clones":0,"num_object_copies":14344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-354,"num_objects_unfound":0,"num_objects_dirty":7172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7172,"num_write_kb":28688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":354,"num_bytes_recovered":1449984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003691"},"pg_stats":[{"pgid":"2.f","version":"43'5580","reported_seq":5655,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:56.175046+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:30:56.175046+0000","last_peered":"2024-09-10T20:30:56.175046+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:30:56.175046+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":5580,"log_dups_size":0,"ondisk_log_size":5580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22855680,"num_objects":5580,"num_object_clones":0,"num_object_copies":11160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5580,"num_write_kb":22320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1973},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5766","reported_seq":5870,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:58.127916+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:30:58.127916+0000","last_peered":"2024-09-10T20:30:58.127916+0000","last_clean":"2024-09-10T20:30:58.127916+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:30:58.127916+0000","last_undegraded":"2024-09-10T20:30:58.127916+0000","last_fullsized":"2024-09-10T20:30:58.127916+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":5766,"log_dups_size":0,"ondisk_log_size":5766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23592960,"num_objects":5760,"num_object_clones":0,"num_object_copies":11520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3952,"num_write_kb":15808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5699","reported_seq":8373,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:30:58.186401+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:30:58.186401+0000","last_peered":"2024-09-10T20:30:58.186401+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:30:58.186401+0000","last_undegraded":"2024-09-10T20:30:58.186401+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":5699,"log_dups_size":0,"ondisk_log_size":5699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23326720,"num_objects":5695,"num_object_clones":0,"num_object_copies":11390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2327,"num_objects_unfound":0,"num_objects_dirty":5695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3827,"num_write_kb":15308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1276,"num_bytes_recovered":5226496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2327},{"shards":"2","objects":3368}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5552","reported_seq":5627,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:58.070942+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:30:58.070942+0000","last_peered":"2024-09-10T20:30:58.070942+0000","last_clean":"2024-09-10T20:30:58.070942+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:30:58.070942+0000","last_undegraded":"2024-09-10T20:30:58.070942+0000","last_fullsized":"2024-09-10T20:30:58.070942+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":5552,"log_dups_size":0,"ondisk_log_size":5552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22740992,"num_objects":5552,"num_object_clones":0,"num_object_copies":11104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5552,"num_write_kb":22208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5689","reported_seq":5809,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:56.274737+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:30:56.274737+0000","last_peered":"2024-09-10T20:30:56.274737+0000","last_clean":"2024-09-10T20:30:56.274737+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:30:56.274737+0000","last_undegraded":"2024-09-10T20:30:56.274737+0000","last_fullsized":"2024-09-10T20:30:56.274737+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":5689,"log_dups_size":0,"ondisk_log_size":5689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23273472,"num_objects":5682,"num_object_clones":0,"num_object_copies":11364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2685,"num_write_kb":10740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5617","reported_seq":5748,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:56.173019+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:30:56.173019+0000","last_peered":"2024-09-10T20:30:56.173019+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:30:56.173019+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":5617,"log_dups_size":0,"ondisk_log_size":5617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22986752,"num_objects":5612,"num_object_clones":0,"num_object_copies":11224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2481,"num_write_kb":9924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1998},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5512","reported_seq":5645,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:56.213752+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:30:56.213752+0000","last_peered":"2024-09-10T20:30:56.213752+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:30:56.213752+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":5512,"log_dups_size":0,"ondisk_log_size":5512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22552576,"num_objects":5506,"num_object_clones":0,"num_object_copies":11012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5506,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2107,"num_write_kb":8428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1951},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5643","reported_seq":5762,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:56.241415+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:30:56.241415+0000","last_peered":"2024-09-10T20:30:56.241415+0000","last_clean":"2024-09-10T20:30:56.241415+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:30:56.241415+0000","last_undegraded":"2024-09-10T20:30:56.241415+0000","last_fullsized":"2024-09-10T20:30:56.241415+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":5643,"log_dups_size":0,"ondisk_log_size":5643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23105536,"num_objects":5641,"num_object_clones":0,"num_object_copies":11282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2714,"num_write_kb":10856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5481","reported_seq":5608,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:58.169125+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:30:58.169125+0000","last_peered":"2024-09-10T20:30:58.169125+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:30:58.169125+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":5481,"log_dups_size":0,"ondisk_log_size":5481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22421504,"num_objects":5474,"num_object_clones":0,"num_object_copies":10948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3474,"num_write_kb":13896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":1948}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355438+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355438+0000","last_peered":"2024-09-10T20:30:05.355438+0000","last_clean":"2024-09-10T20:30:05.355438+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355438+0000","last_undegraded":"2024-09-10T20:30:05.355438+0000","last_fullsized":"2024-09-10T20:30:05.355438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5640","reported_seq":5743,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:56.232096+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:30:56.232096+0000","last_peered":"2024-09-10T20:30:56.232096+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:30:56.232096+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":5640,"log_dups_size":0,"ondisk_log_size":5640,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23069638,"num_objects":5634,"num_object_clones":0,"num_object_copies":11268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5638,"num_write_kb":22532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2019},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.413176+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.413176+0000","last_peered":"2024-09-10T20:30:05.413176+0000","last_clean":"2024-09-10T20:30:05.413176+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.413176+0000","last_undegraded":"2024-09-10T20:30:05.413176+0000","last_fullsized":"2024-09-10T20:30:05.413176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5599","reported_seq":5722,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:58.174826+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:30:58.174826+0000","last_peered":"2024-09-10T20:30:58.174826+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:30:58.174826+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":5599,"log_dups_size":0,"ondisk_log_size":5599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22913024,"num_objects":5594,"num_object_clones":0,"num_object_copies":11188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5594,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3427,"num_write_kb":13708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":1961}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5690","reported_seq":5813,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:58.110070+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:30:58.110070+0000","last_peered":"2024-09-10T20:30:58.110070+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:30:58.110070+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":5690,"log_dups_size":0,"ondisk_log_size":5690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23281664,"num_objects":5684,"num_object_clones":0,"num_object_copies":11368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5684,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3476,"num_write_kb":13904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":2074}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5631","reported_seq":5761,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:56.242141+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:30:56.242141+0000","last_peered":"2024-09-10T20:30:56.242141+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:30:56.242141+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":5631,"log_dups_size":0,"ondisk_log_size":5631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23056384,"num_objects":5629,"num_object_clones":0,"num_object_copies":11258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2354,"num_write_kb":9416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1981},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5596","reported_seq":5729,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:56.269313+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:30:56.269313+0000","last_peered":"2024-09-10T20:30:56.269313+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:30:56.269313+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":5596,"log_dups_size":0,"ondisk_log_size":5596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22888448,"num_objects":5588,"num_object_clones":0,"num_object_copies":11176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2290,"num_write_kb":9160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2056},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345705+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345705+0000","last_peered":"2024-09-10T20:30:05.345705+0000","last_clean":"2024-09-10T20:30:05.345705+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345705+0000","last_undegraded":"2024-09-10T20:30:05.345705+0000","last_fullsized":"2024-09-10T20:30:05.345705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5730","reported_seq":5856,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:56.174212+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:30:56.174212+0000","last_peered":"2024-09-10T20:30:56.174212+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:30:56.174212+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":5730,"log_dups_size":0,"ondisk_log_size":5730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23453696,"num_objects":5726,"num_object_clones":0,"num_object_copies":11452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2557,"num_write_kb":10228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2013},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5542","reported_seq":5702,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:56.228096+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:30:56.228096+0000","last_peered":"2024-09-10T20:30:56.228096+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:30:56.228096+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":5542,"log_dups_size":0,"ondisk_log_size":5542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22651363,"num_objects":5531,"num_object_clones":0,"num_object_copies":11062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5531,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2149,"num_write_kb":8596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2063},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":368170409,"num_objects":89888,"num_object_clones":0,"num_object_copies":179776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2327,"num_objects_unfound":0,"num_objects_dirty":89888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":54263,"num_write_kb":217032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1279,"num_bytes_recovered":5230592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":743088128,"data_stored":742997803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":89967,"ondisk_log_size":89967,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182429,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":510308,"kb_used_data":325972,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93861532,"statfs":{"total":96636764160,"available":96114208768,"internally_reserved":0,"allocated":333795328,"data_stored":333562195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188739242},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,0,3,1,7,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986201,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":458072,"kb_used_data":295880,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":93913768,"statfs":{"total":96636764160,"available":96167698432,"internally_reserved":0,"allocated":302981120,"data_stored":302752064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":166063786},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659148,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":214088,"kb_used_data":101880,"kb_used_omap":4,"kb_used_meta":112187,"kb_avail":94157752,"statfs":{"total":96636764160,"available":96417538048,"internally_reserved":0,"allocated":104325120,"data_stored":104012501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":114880174},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":334589952,"data_stored":334586339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":105455616,"data_stored":105372517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":303042560,"data_stored":303038947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:31:00.046 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 92 20 91637 91617 3.8893 2.18359 0.0322472 0.0200787 2024-09-10T20:31:01.046 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 93 20 92261 92241 3.87369 2.4375 0.0286982 0.0201599 2024-09-10T20:31:02.046 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 94 20 92876 92856 3.85803 2.40234 0.0176798 0.0202401 2024-09-10T20:31:02.849 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:31:03.046 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 95 20 93448 93428 3.84094 2.23438 0.0268273 0.020331 2024-09-10T20:31:03.399 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:03.399 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:31:03.427 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":124,"stamp":"2024-09-10T20:31:02.848024+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":375702972,"num_objects":91729,"num_object_clones":0,"num_object_copies":183458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2327,"num_objects_unfound":0,"num_objects_dirty":91729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56104,"num_write_kb":224390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1281,"num_bytes_recovered":5230611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91809,"ondisk_log_size":91809,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1197844,"kb_used_data":741860,"kb_used_omap":12,"kb_used_meta":455923,"kb_avail":281917676,"statfs":{"total":289910292480,"available":288683700224,"internally_reserved":0,"allocated":759664640,"data_stored":758889832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":466865154},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,3,7,6,11,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32477184,"num_objects":7929,"num_object_clones":0,"num_object_copies":15858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-231,"num_objects_unfound":0,"num_objects_dirty":7929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7929,"num_write_kb":31716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":231,"num_bytes_recovered":946176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003812"},"pg_stats":[{"pgid":"2.f","version":"43'5774","reported_seq":5849,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:01.274800+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:31:01.274800+0000","last_peered":"2024-09-10T20:31:01.274800+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:31:01.274800+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":5774,"log_dups_size":0,"ondisk_log_size":5774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23650304,"num_objects":5774,"num_object_clones":0,"num_object_copies":11548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5774,"num_write_kb":23096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2167},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5766","reported_seq":5870,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:58.127916+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:30:58.127916+0000","last_peered":"2024-09-10T20:30:58.127916+0000","last_clean":"2024-09-10T20:30:58.127916+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:30:58.127916+0000","last_undegraded":"2024-09-10T20:30:58.127916+0000","last_fullsized":"2024-09-10T20:30:58.127916+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":5766,"log_dups_size":0,"ondisk_log_size":5766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23592960,"num_objects":5760,"num_object_clones":0,"num_object_copies":11520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3952,"num_write_kb":15808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5699","reported_seq":8373,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:30:58.186401+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:30:58.186401+0000","last_peered":"2024-09-10T20:30:58.186401+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:30:58.186401+0000","last_undegraded":"2024-09-10T20:30:58.186401+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":5699,"log_dups_size":0,"ondisk_log_size":5699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23326720,"num_objects":5695,"num_object_clones":0,"num_object_copies":11390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2327,"num_objects_unfound":0,"num_objects_dirty":5695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3827,"num_write_kb":15308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1276,"num_bytes_recovered":5226496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2327},{"shards":"2","objects":3368}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5552","reported_seq":5627,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:58.070942+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:30:58.070942+0000","last_peered":"2024-09-10T20:30:58.070942+0000","last_clean":"2024-09-10T20:30:58.070942+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:30:58.070942+0000","last_undegraded":"2024-09-10T20:30:58.070942+0000","last_fullsized":"2024-09-10T20:30:58.070942+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":5552,"log_dups_size":0,"ondisk_log_size":5552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22740992,"num_objects":5552,"num_object_clones":0,"num_object_copies":11104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5552,"num_write_kb":22208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5890","reported_seq":6011,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:01.267248+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:31:01.267248+0000","last_peered":"2024-09-10T20:31:01.267248+0000","last_clean":"2024-09-10T20:31:01.267248+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:31:01.267248+0000","last_undegraded":"2024-09-10T20:31:01.267248+0000","last_fullsized":"2024-09-10T20:31:01.267248+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":5890,"log_dups_size":0,"ondisk_log_size":5890,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24096768,"num_objects":5883,"num_object_clones":0,"num_object_copies":11766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2886,"num_write_kb":11544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5781","reported_seq":5912,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:01.292501+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:31:01.292501+0000","last_peered":"2024-09-10T20:31:01.292501+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:31:01.292501+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":5781,"log_dups_size":0,"ondisk_log_size":5781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23658496,"num_objects":5776,"num_object_clones":0,"num_object_copies":11552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2645,"num_write_kb":10580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2162},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5683","reported_seq":5816,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:01.289085+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:31:01.289085+0000","last_peered":"2024-09-10T20:31:01.289085+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:31:01.289085+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":5683,"log_dups_size":0,"ondisk_log_size":5683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23252992,"num_objects":5677,"num_object_clones":0,"num_object_copies":11354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2278,"num_write_kb":9112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2122},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5831","reported_seq":5949,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:01.241193+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:31:01.241193+0000","last_peered":"2024-09-10T20:31:01.241193+0000","last_clean":"2024-09-10T20:31:01.241193+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:31:01.241193+0000","last_undegraded":"2024-09-10T20:31:01.241193+0000","last_fullsized":"2024-09-10T20:31:01.241193+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":5831,"log_dups_size":0,"ondisk_log_size":5831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23875584,"num_objects":5829,"num_object_clones":0,"num_object_copies":11658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2902,"num_write_kb":11608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5481","reported_seq":5608,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:58.169125+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:30:58.169125+0000","last_peered":"2024-09-10T20:30:58.169125+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:30:58.169125+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":5481,"log_dups_size":0,"ondisk_log_size":5481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22421504,"num_objects":5474,"num_object_clones":0,"num_object_copies":10948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3474,"num_write_kb":13896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":1948}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355438+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355438+0000","last_peered":"2024-09-10T20:30:05.355438+0000","last_clean":"2024-09-10T20:30:05.355438+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355438+0000","last_undegraded":"2024-09-10T20:30:05.355438+0000","last_fullsized":"2024-09-10T20:30:05.355438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5830","reported_seq":5933,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:01.292590+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:31:01.292590+0000","last_peered":"2024-09-10T20:31:01.292590+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:31:01.292590+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":5830,"log_dups_size":0,"ondisk_log_size":5830,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23847878,"num_objects":5824,"num_object_clones":0,"num_object_copies":11648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5828,"num_write_kb":23292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2209},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5599","reported_seq":5722,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:58.174826+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:30:58.174826+0000","last_peered":"2024-09-10T20:30:58.174826+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:30:58.174826+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":5599,"log_dups_size":0,"ondisk_log_size":5599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22913024,"num_objects":5594,"num_object_clones":0,"num_object_copies":11188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5594,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3427,"num_write_kb":13708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":1961}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5690","reported_seq":5813,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:58.110070+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:30:58.110070+0000","last_peered":"2024-09-10T20:30:58.110070+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:30:58.110070+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":5690,"log_dups_size":0,"ondisk_log_size":5690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23281664,"num_objects":5684,"num_object_clones":0,"num_object_copies":11368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5684,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3476,"num_write_kb":13904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":2074}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5802","reported_seq":5932,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:01.155930+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:31:01.155930+0000","last_peered":"2024-09-10T20:31:01.155930+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:31:01.155930+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":5802,"log_dups_size":0,"ondisk_log_size":5802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23756800,"num_objects":5800,"num_object_clones":0,"num_object_copies":11600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5800,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2525,"num_write_kb":10100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2152},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5766","reported_seq":5896,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:01.230416+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:31:01.230416+0000","last_peered":"2024-09-10T20:31:01.230416+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:31:01.230416+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":5766,"log_dups_size":0,"ondisk_log_size":5766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23584768,"num_objects":5758,"num_object_clones":0,"num_object_copies":11516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2460,"num_write_kb":9840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2226},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345705+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345705+0000","last_peered":"2024-09-10T20:30:05.345705+0000","last_clean":"2024-09-10T20:30:05.345705+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345705+0000","last_undegraded":"2024-09-10T20:30:05.345705+0000","last_fullsized":"2024-09-10T20:30:05.345705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5917","reported_seq":6043,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:01.218655+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:31:01.218655+0000","last_peered":"2024-09-10T20:31:01.218655+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:31:01.218655+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":5917,"log_dups_size":0,"ondisk_log_size":5917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24219648,"num_objects":5913,"num_object_clones":0,"num_object_copies":11826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2744,"num_write_kb":10976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2200},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5745","reported_seq":5905,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:01.188813+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:31:01.188813+0000","last_peered":"2024-09-10T20:31:01.188813+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:31:01.188813+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":5745,"log_dups_size":0,"ondisk_log_size":5745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23482851,"num_objects":5734,"num_object_clones":0,"num_object_copies":11468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2352,"num_write_kb":9408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2266},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":375702953,"num_objects":91727,"num_object_clones":0,"num_object_copies":183454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2327,"num_objects_unfound":0,"num_objects_dirty":91727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56102,"num_write_kb":224388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1279,"num_bytes_recovered":5230592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":761589760,"data_stored":761499435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91806,"ondisk_log_size":91806,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182430,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506588,"kb_used_data":334732,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93865252,"statfs":{"total":96636764160,"available":96118018048,"internally_reserved":0,"allocated":342765568,"data_stored":342532435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":175959722},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,3,4,3,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986201,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":458072,"kb_used_data":295880,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":93913768,"statfs":{"total":96636764160,"available":96167698432,"internally_reserved":0,"allocated":302981120,"data_stored":302752064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":166063786},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659149,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":233184,"kb_used_data":111248,"kb_used_omap":4,"kb_used_meta":121915,"kb_avail":94138656,"statfs":{"total":96636764160,"available":96397983744,"internally_reserved":0,"allocated":113917952,"data_stored":113605333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":124841646},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":343539712,"data_stored":343536099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":115007488,"data_stored":114924389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":303042560,"data_stored":303038947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:31:03.428 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:31:03.955 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:03.958 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:31:03.983 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":124,"stamp":"2024-09-10T20:31:02.848024+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":375702972,"num_objects":91729,"num_object_clones":0,"num_object_copies":183458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2327,"num_objects_unfound":0,"num_objects_dirty":91729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56104,"num_write_kb":224390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1281,"num_bytes_recovered":5230611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91809,"ondisk_log_size":91809,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1197844,"kb_used_data":741860,"kb_used_omap":12,"kb_used_meta":455923,"kb_avail":281917676,"statfs":{"total":289910292480,"available":288683700224,"internally_reserved":0,"allocated":759664640,"data_stored":758889832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":466865154},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,3,7,6,11,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32477184,"num_objects":7929,"num_object_clones":0,"num_object_copies":15858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-231,"num_objects_unfound":0,"num_objects_dirty":7929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7929,"num_write_kb":31716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":231,"num_bytes_recovered":946176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003812"},"pg_stats":[{"pgid":"2.f","version":"43'5774","reported_seq":5849,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:01.274800+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:31:01.274800+0000","last_peered":"2024-09-10T20:31:01.274800+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:31:01.274800+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":5774,"log_dups_size":0,"ondisk_log_size":5774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23650304,"num_objects":5774,"num_object_clones":0,"num_object_copies":11548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5774,"num_write_kb":23096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2167},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5766","reported_seq":5870,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:58.127916+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:30:58.127916+0000","last_peered":"2024-09-10T20:30:58.127916+0000","last_clean":"2024-09-10T20:30:58.127916+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:30:58.127916+0000","last_undegraded":"2024-09-10T20:30:58.127916+0000","last_fullsized":"2024-09-10T20:30:58.127916+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":5766,"log_dups_size":0,"ondisk_log_size":5766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23592960,"num_objects":5760,"num_object_clones":0,"num_object_copies":11520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3952,"num_write_kb":15808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5699","reported_seq":8373,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:30:58.186401+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:30:58.186401+0000","last_peered":"2024-09-10T20:30:58.186401+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:30:58.186401+0000","last_undegraded":"2024-09-10T20:30:58.186401+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":5699,"log_dups_size":0,"ondisk_log_size":5699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23326720,"num_objects":5695,"num_object_clones":0,"num_object_copies":11390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2327,"num_objects_unfound":0,"num_objects_dirty":5695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3827,"num_write_kb":15308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1276,"num_bytes_recovered":5226496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2327},{"shards":"2","objects":3368}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5552","reported_seq":5627,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:58.070942+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:30:58.070942+0000","last_peered":"2024-09-10T20:30:58.070942+0000","last_clean":"2024-09-10T20:30:58.070942+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:30:58.070942+0000","last_undegraded":"2024-09-10T20:30:58.070942+0000","last_fullsized":"2024-09-10T20:30:58.070942+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":5552,"log_dups_size":0,"ondisk_log_size":5552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22740992,"num_objects":5552,"num_object_clones":0,"num_object_copies":11104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5552,"num_write_kb":22208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5890","reported_seq":6011,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:01.267248+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:31:01.267248+0000","last_peered":"2024-09-10T20:31:01.267248+0000","last_clean":"2024-09-10T20:31:01.267248+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:31:01.267248+0000","last_undegraded":"2024-09-10T20:31:01.267248+0000","last_fullsized":"2024-09-10T20:31:01.267248+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":5890,"log_dups_size":0,"ondisk_log_size":5890,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24096768,"num_objects":5883,"num_object_clones":0,"num_object_copies":11766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2886,"num_write_kb":11544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5781","reported_seq":5912,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:01.292501+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:31:01.292501+0000","last_peered":"2024-09-10T20:31:01.292501+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:31:01.292501+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":5781,"log_dups_size":0,"ondisk_log_size":5781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23658496,"num_objects":5776,"num_object_clones":0,"num_object_copies":11552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2645,"num_write_kb":10580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2162},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5683","reported_seq":5816,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:01.289085+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:31:01.289085+0000","last_peered":"2024-09-10T20:31:01.289085+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:31:01.289085+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":5683,"log_dups_size":0,"ondisk_log_size":5683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23252992,"num_objects":5677,"num_object_clones":0,"num_object_copies":11354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2278,"num_write_kb":9112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2122},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5831","reported_seq":5949,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:01.241193+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:31:01.241193+0000","last_peered":"2024-09-10T20:31:01.241193+0000","last_clean":"2024-09-10T20:31:01.241193+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:31:01.241193+0000","last_undegraded":"2024-09-10T20:31:01.241193+0000","last_fullsized":"2024-09-10T20:31:01.241193+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":5831,"log_dups_size":0,"ondisk_log_size":5831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23875584,"num_objects":5829,"num_object_clones":0,"num_object_copies":11658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2902,"num_write_kb":11608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5481","reported_seq":5608,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:58.169125+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:30:58.169125+0000","last_peered":"2024-09-10T20:30:58.169125+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:30:58.169125+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":5481,"log_dups_size":0,"ondisk_log_size":5481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22421504,"num_objects":5474,"num_object_clones":0,"num_object_copies":10948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3474,"num_write_kb":13896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":1948}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355438+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355438+0000","last_peered":"2024-09-10T20:30:05.355438+0000","last_clean":"2024-09-10T20:30:05.355438+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355438+0000","last_undegraded":"2024-09-10T20:30:05.355438+0000","last_fullsized":"2024-09-10T20:30:05.355438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5830","reported_seq":5933,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:01.292590+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:31:01.292590+0000","last_peered":"2024-09-10T20:31:01.292590+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:31:01.292590+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":5830,"log_dups_size":0,"ondisk_log_size":5830,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23847878,"num_objects":5824,"num_object_clones":0,"num_object_copies":11648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5828,"num_write_kb":23292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2209},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5599","reported_seq":5722,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:58.174826+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:30:58.174826+0000","last_peered":"2024-09-10T20:30:58.174826+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:30:58.174826+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":5599,"log_dups_size":0,"ondisk_log_size":5599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22913024,"num_objects":5594,"num_object_clones":0,"num_object_copies":11188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5594,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3427,"num_write_kb":13708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":1961}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5690","reported_seq":5813,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:30:58.110070+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:30:58.110070+0000","last_peered":"2024-09-10T20:30:58.110070+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:30:58.110070+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":5690,"log_dups_size":0,"ondisk_log_size":5690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23281664,"num_objects":5684,"num_object_clones":0,"num_object_copies":11368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5684,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3476,"num_write_kb":13904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":2074}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5802","reported_seq":5932,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:01.155930+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:31:01.155930+0000","last_peered":"2024-09-10T20:31:01.155930+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:31:01.155930+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":5802,"log_dups_size":0,"ondisk_log_size":5802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23756800,"num_objects":5800,"num_object_clones":0,"num_object_copies":11600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5800,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2525,"num_write_kb":10100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2152},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5766","reported_seq":5896,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:01.230416+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:31:01.230416+0000","last_peered":"2024-09-10T20:31:01.230416+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:31:01.230416+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":5766,"log_dups_size":0,"ondisk_log_size":5766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23584768,"num_objects":5758,"num_object_clones":0,"num_object_copies":11516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2460,"num_write_kb":9840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2226},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345705+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345705+0000","last_peered":"2024-09-10T20:30:05.345705+0000","last_clean":"2024-09-10T20:30:05.345705+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345705+0000","last_undegraded":"2024-09-10T20:30:05.345705+0000","last_fullsized":"2024-09-10T20:30:05.345705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5917","reported_seq":6043,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:01.218655+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:31:01.218655+0000","last_peered":"2024-09-10T20:31:01.218655+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:31:01.218655+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":5917,"log_dups_size":0,"ondisk_log_size":5917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24219648,"num_objects":5913,"num_object_clones":0,"num_object_copies":11826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2744,"num_write_kb":10976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2200},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5745","reported_seq":5905,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:01.188813+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:31:01.188813+0000","last_peered":"2024-09-10T20:31:01.188813+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:31:01.188813+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":5745,"log_dups_size":0,"ondisk_log_size":5745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23482851,"num_objects":5734,"num_object_clones":0,"num_object_copies":11468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2352,"num_write_kb":9408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2266},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":375702953,"num_objects":91727,"num_object_clones":0,"num_object_copies":183454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2327,"num_objects_unfound":0,"num_objects_dirty":91727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56102,"num_write_kb":224388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1279,"num_bytes_recovered":5230592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":761589760,"data_stored":761499435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91806,"ondisk_log_size":91806,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182430,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506588,"kb_used_data":334732,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93865252,"statfs":{"total":96636764160,"available":96118018048,"internally_reserved":0,"allocated":342765568,"data_stored":342532435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":175959722},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,3,4,3,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986201,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":458072,"kb_used_data":295880,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":93913768,"statfs":{"total":96636764160,"available":96167698432,"internally_reserved":0,"allocated":302981120,"data_stored":302752064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":166063786},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659149,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":233184,"kb_used_data":111248,"kb_used_omap":4,"kb_used_meta":121915,"kb_avail":94138656,"statfs":{"total":96636764160,"available":96397983744,"internally_reserved":0,"allocated":113917952,"data_stored":113605333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":124841646},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":343539712,"data_stored":343536099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":115007488,"data_stored":114924389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":303042560,"data_stored":303038947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:31:03.984 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:31:03.984 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'5774", 'reported_seq': 5849, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:01.274800+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:31:01.274800+0000', 'last_peered': '2024-09-10T20:31:01.274800+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:31:01.274800+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 5774, 'log_dups_size': 0, 'ondisk_log_size': 5774, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23650304, 'num_objects': 5774, 'num_object_clones': 0, 'num_object_copies': 11548, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3607, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5774, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5774, 'num_write_kb': 23096, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2167}, {'shards': '0,2', 'objects': 3607}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:03.984 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T20:31:03.984 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'5699", 'reported_seq': 8373, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:30:58.186401+0000', 'last_change': '2024-09-10T20:30:05.653560+0000', 'last_active': '2024-09-10T20:30:58.186401+0000', 'last_peered': '2024-09-10T20:30:58.186401+0000', 'last_clean': '2024-09-10T20:30:03.279259+0000', 'last_became_active': '2024-09-10T20:30:05.629664+0000', 'last_became_peered': '2024-09-10T20:30:05.629664+0000', 'last_unstale': '2024-09-10T20:30:58.186401+0000', 'last_undegraded': '2024-09-10T20:30:58.186401+0000', 'last_fullsized': '2024-09-10T20:30:05.312061+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1910", 'last_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'objects_scrubbed': 1868, 'log_size': 5699, 'log_dups_size': 0, 'ondisk_log_size': 5699, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 673, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23326720, 'num_objects': 5695, 'num_object_clones': 0, 'num_object_copies': 11390, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2327, 'num_objects_unfound': 0, 'num_objects_dirty': 5695, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3827, 'num_write_kb': 15308, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1276, 'num_bytes_recovered': 5226496, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2327}, {'shards': '2', 'objects': 3368}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:03.984 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:31:03.984 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'5781", 'reported_seq': 5912, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:01.292501+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:31:01.292501+0000', 'last_peered': '2024-09-10T20:31:01.292501+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:31:01.292501+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 5781, 'log_dups_size': 0, 'ondisk_log_size': 5781, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23658496, 'num_objects': 5776, 'num_object_clones': 0, 'num_object_copies': 11552, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3616, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5776, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2645, 'num_write_kb': 10580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2162}, {'shards': '0,2', 'objects': 3614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:03.984 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:31:03.984 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'5683", 'reported_seq': 5816, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:01.289085+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:31:01.289085+0000', 'last_peered': '2024-09-10T20:31:01.289085+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:31:01.289085+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 5683, 'log_dups_size': 0, 'ondisk_log_size': 5683, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23252992, 'num_objects': 5677, 'num_object_clones': 0, 'num_object_copies': 11354, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3557, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5677, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2278, 'num_write_kb': 9112, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2122}, {'shards': '0,2', 'objects': 3555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:03.985 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:31:03.985 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'5481", 'reported_seq': 5608, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:58.169125+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:30:58.169125+0000', 'last_peered': '2024-09-10T20:30:58.169125+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:30:58.169125+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 5481, 'log_dups_size': 0, 'ondisk_log_size': 5481, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22421504, 'num_objects': 5474, 'num_object_clones': 0, 'num_object_copies': 10948, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3528, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5474, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3474, 'num_write_kb': 13896, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 1948}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:03.985 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:31:03.985 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'5830", 'reported_seq': 5933, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:01.292590+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:31:01.292590+0000', 'last_peered': '2024-09-10T20:31:01.292590+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:31:01.292590+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 5830, 'log_dups_size': 0, 'ondisk_log_size': 5830, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23847878, 'num_objects': 5824, 'num_object_clones': 0, 'num_object_copies': 11648, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3617, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5824, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5828, 'num_write_kb': 23292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2209}, {'shards': '0,2', 'objects': 3615}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:03.985 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:31:03.985 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'5599", 'reported_seq': 5722, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:58.174826+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:30:58.174826+0000', 'last_peered': '2024-09-10T20:30:58.174826+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:30:58.174826+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 5599, 'log_dups_size': 0, 'ondisk_log_size': 5599, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22913024, 'num_objects': 5594, 'num_object_clones': 0, 'num_object_copies': 11188, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3635, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5594, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3427, 'num_write_kb': 13708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3633}, {'shards': '2', 'objects': 1961}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:03.985 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:31:03.985 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'5690", 'reported_seq': 5813, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:30:58.110070+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:30:58.110070+0000', 'last_peered': '2024-09-10T20:30:58.110070+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:30:58.110070+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 5690, 'log_dups_size': 0, 'ondisk_log_size': 5690, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23281664, 'num_objects': 5684, 'num_object_clones': 0, 'num_object_copies': 11368, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5684, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3476, 'num_write_kb': 13904, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3610}, {'shards': '2', 'objects': 2074}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:03.985 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:31:03.985 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'5802", 'reported_seq': 5932, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:01.155930+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:31:01.155930+0000', 'last_peered': '2024-09-10T20:31:01.155930+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:31:01.155930+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 5802, 'log_dups_size': 0, 'ondisk_log_size': 5802, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23756800, 'num_objects': 5800, 'num_object_clones': 0, 'num_object_copies': 11600, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3649, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5800, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2525, 'num_write_kb': 10100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2152}, {'shards': '0,2', 'objects': 3648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:03.986 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:31:03.986 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'5766", 'reported_seq': 5896, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:01.230416+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:31:01.230416+0000', 'last_peered': '2024-09-10T20:31:01.230416+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:31:01.230416+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 5766, 'log_dups_size': 0, 'ondisk_log_size': 5766, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23584768, 'num_objects': 5758, 'num_object_clones': 0, 'num_object_copies': 11516, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3535, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5758, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2460, 'num_write_kb': 9840, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2226}, {'shards': '0,2', 'objects': 3532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:03.986 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:31:03.986 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'5917", 'reported_seq': 6043, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:01.218655+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:31:01.218655+0000', 'last_peered': '2024-09-10T20:31:01.218655+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:31:01.218655+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 5917, 'log_dups_size': 0, 'ondisk_log_size': 5917, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24219648, 'num_objects': 5913, 'num_object_clones': 0, 'num_object_copies': 11826, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3715, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5913, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2744, 'num_write_kb': 10976, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2200}, {'shards': '0,2', 'objects': 3713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:03.986 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:31:03.986 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'5745", 'reported_seq': 5905, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:01.188813+0000', 'last_change': '2024-09-10T20:30:05.690055+0000', 'last_active': '2024-09-10T20:31:01.188813+0000', 'last_peered': '2024-09-10T20:31:01.188813+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:31:01.188813+0000', 'last_undegraded': '2024-09-10T20:30:05.378941+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 5745, 'log_dups_size': 0, 'ondisk_log_size': 5745, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23482851, 'num_objects': 5734, 'num_object_clones': 0, 'num_object_copies': 11468, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3471, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5734, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2352, 'num_write_kb': 9408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2266}, {'shards': '0,2', 'objects': 3468}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:03.986 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:31:04.046 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 96 20 93981 93961 3.82262 2.08203 0.0564937 0.0204285 2024-09-10T20:31:04.668 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:04.669 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39542/183458 objects degraded (21.554%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":176,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":91729,"data_bytes":375702972,"bytes_used":1226592256,"bytes_avail":288683700224,"bytes_total":289910292480,"degraded_objects":39542,"degraded_total":183458,"degraded_ratio":0.21553707115525078,"misplaced_objects":2327,"misplaced_total":183458,"misplaced_ratio":0.012684102083310621,"recovering_objects_per_sec":19,"recovering_bytes_per_sec":78822,"recovering_keys_per_sec":0,"num_objects_recovered":231,"num_bytes_recovered":946176,"num_keys_recovered":0,"write_bytes_sec":2705572,"read_op_per_sec":0,"write_op_per_sec":660},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:31:04.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:31:04.706 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:31:05.046 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 97 20 94521 94501 3.80496 2.10938 0.0460341 0.0205249 2024-09-10T20:31:05.253 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:05.253 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:31:05.277 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2024-09-10T20:31:04.848850+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":380118460,"num_objects":92807,"num_object_clones":0,"num_object_copies":185614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2208,"num_objects_unfound":0,"num_objects_dirty":92807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57182,"num_write_kb":228702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1400,"num_bytes_recovered":5718035,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92887,"ondisk_log_size":92887,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1207936,"kb_used_data":747792,"kb_used_omap":12,"kb_used_meta":460083,"kb_avail":281907584,"statfs":{"total":289910292480,"available":288673366016,"internally_reserved":0,"allocated":765739008,"data_stored":764964200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":471124994},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,3,3,5,6,13,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28848128,"num_objects":7043,"num_object_clones":0,"num_object_copies":14086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-350,"num_objects_unfound":0,"num_objects_dirty":7043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7043,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":350,"num_bytes_recovered":1433600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004220"},"pg_stats":[{"pgid":"2.f","version":"43'5774","reported_seq":5849,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:01.274800+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:31:01.274800+0000","last_peered":"2024-09-10T20:31:01.274800+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:31:01.274800+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":5774,"log_dups_size":0,"ondisk_log_size":5774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23650304,"num_objects":5774,"num_object_clones":0,"num_object_copies":11548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5774,"num_write_kb":23096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2167},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5933","reported_seq":6035,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:03.124625+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:31:03.124625+0000","last_peered":"2024-09-10T20:31:03.124625+0000","last_clean":"2024-09-10T20:31:03.124625+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:31:03.124625+0000","last_undegraded":"2024-09-10T20:31:03.124625+0000","last_fullsized":"2024-09-10T20:31:03.124625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":5933,"log_dups_size":0,"ondisk_log_size":5933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24276992,"num_objects":5927,"num_object_clones":0,"num_object_copies":11854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4119,"num_write_kb":16476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5868","reported_seq":8780,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:31:03.140397+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:31:03.140397+0000","last_peered":"2024-09-10T20:31:03.140397+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:31:03.140397+0000","last_undegraded":"2024-09-10T20:31:03.140397+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":5868,"log_dups_size":0,"ondisk_log_size":5868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24018944,"num_objects":5864,"num_object_clones":0,"num_object_copies":11728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2208,"num_objects_unfound":0,"num_objects_dirty":5864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3996,"num_write_kb":15984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1395,"num_bytes_recovered":5713920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2208},{"shards":"2","objects":3656}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5737","reported_seq":5812,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:03.175512+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:31:03.175512+0000","last_peered":"2024-09-10T20:31:03.175512+0000","last_clean":"2024-09-10T20:31:03.175512+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:31:03.175512+0000","last_undegraded":"2024-09-10T20:31:03.175512+0000","last_fullsized":"2024-09-10T20:31:03.175512+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":5737,"log_dups_size":0,"ondisk_log_size":5737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23498752,"num_objects":5737,"num_object_clones":0,"num_object_copies":11474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5737,"num_write_kb":22948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5890","reported_seq":6011,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:01.267248+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:31:01.267248+0000","last_peered":"2024-09-10T20:31:01.267248+0000","last_clean":"2024-09-10T20:31:01.267248+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:31:01.267248+0000","last_undegraded":"2024-09-10T20:31:01.267248+0000","last_fullsized":"2024-09-10T20:31:01.267248+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":5890,"log_dups_size":0,"ondisk_log_size":5890,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24096768,"num_objects":5883,"num_object_clones":0,"num_object_copies":11766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2886,"num_write_kb":11544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5781","reported_seq":5912,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:01.292501+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:31:01.292501+0000","last_peered":"2024-09-10T20:31:01.292501+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:31:01.292501+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":5781,"log_dups_size":0,"ondisk_log_size":5781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23658496,"num_objects":5776,"num_object_clones":0,"num_object_copies":11552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2645,"num_write_kb":10580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2162},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5683","reported_seq":5816,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:01.289085+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:31:01.289085+0000","last_peered":"2024-09-10T20:31:01.289085+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:31:01.289085+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":5683,"log_dups_size":0,"ondisk_log_size":5683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23252992,"num_objects":5677,"num_object_clones":0,"num_object_copies":11354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2278,"num_write_kb":9112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2122},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5831","reported_seq":5949,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:01.241193+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:31:01.241193+0000","last_peered":"2024-09-10T20:31:01.241193+0000","last_clean":"2024-09-10T20:31:01.241193+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:31:01.241193+0000","last_undegraded":"2024-09-10T20:31:01.241193+0000","last_fullsized":"2024-09-10T20:31:01.241193+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":5831,"log_dups_size":0,"ondisk_log_size":5831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23875584,"num_objects":5829,"num_object_clones":0,"num_object_copies":11658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2902,"num_write_kb":11608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5678","reported_seq":5805,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:03.183133+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:31:03.183133+0000","last_peered":"2024-09-10T20:31:03.183133+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:31:03.183133+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":5678,"log_dups_size":0,"ondisk_log_size":5678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23228416,"num_objects":5671,"num_object_clones":0,"num_object_copies":11342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3671,"num_write_kb":14684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":2145}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355438+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355438+0000","last_peered":"2024-09-10T20:30:05.355438+0000","last_clean":"2024-09-10T20:30:05.355438+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355438+0000","last_undegraded":"2024-09-10T20:30:05.355438+0000","last_fullsized":"2024-09-10T20:30:05.355438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5830","reported_seq":5933,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:01.292590+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:31:01.292590+0000","last_peered":"2024-09-10T20:31:01.292590+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:31:01.292590+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":5830,"log_dups_size":0,"ondisk_log_size":5830,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23847878,"num_objects":5824,"num_object_clones":0,"num_object_copies":11648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5828,"num_write_kb":23292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2209},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5781","reported_seq":5904,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:03.125750+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:31:03.125750+0000","last_peered":"2024-09-10T20:31:03.125750+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:31:03.125750+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":5781,"log_dups_size":0,"ondisk_log_size":5781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23658496,"num_objects":5776,"num_object_clones":0,"num_object_copies":11552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3609,"num_write_kb":14436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":2143}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5868","reported_seq":5991,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:03.154875+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:31:03.154875+0000","last_peered":"2024-09-10T20:31:03.154875+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:31:03.154875+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":5868,"log_dups_size":0,"ondisk_log_size":5868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24010752,"num_objects":5862,"num_object_clones":0,"num_object_copies":11724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3654,"num_write_kb":14616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":2252}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5802","reported_seq":5932,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:01.155930+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:31:01.155930+0000","last_peered":"2024-09-10T20:31:01.155930+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:31:01.155930+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":5802,"log_dups_size":0,"ondisk_log_size":5802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23756800,"num_objects":5800,"num_object_clones":0,"num_object_copies":11600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5800,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2525,"num_write_kb":10100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2152},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5766","reported_seq":5896,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:01.230416+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:31:01.230416+0000","last_peered":"2024-09-10T20:31:01.230416+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:31:01.230416+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":5766,"log_dups_size":0,"ondisk_log_size":5766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23584768,"num_objects":5758,"num_object_clones":0,"num_object_copies":11516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2460,"num_write_kb":9840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2226},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345705+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345705+0000","last_peered":"2024-09-10T20:30:05.345705+0000","last_clean":"2024-09-10T20:30:05.345705+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345705+0000","last_undegraded":"2024-09-10T20:30:05.345705+0000","last_fullsized":"2024-09-10T20:30:05.345705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5917","reported_seq":6043,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:01.218655+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:31:01.218655+0000","last_peered":"2024-09-10T20:31:01.218655+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:31:01.218655+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":5917,"log_dups_size":0,"ondisk_log_size":5917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24219648,"num_objects":5913,"num_object_clones":0,"num_object_copies":11826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2744,"num_write_kb":10976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2200},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5745","reported_seq":5905,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:01.188813+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:31:01.188813+0000","last_peered":"2024-09-10T20:31:01.188813+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:31:01.188813+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":5745,"log_dups_size":0,"ondisk_log_size":5745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23482851,"num_objects":5734,"num_object_clones":0,"num_object_copies":11468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2352,"num_write_kb":9408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2266},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":380118441,"num_objects":92805,"num_object_clones":0,"num_object_copies":185610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2208,"num_objects_unfound":0,"num_objects_dirty":92805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57180,"num_write_kb":228700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1398,"num_bytes_recovered":5718016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":767639552,"data_stored":767549227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92884,"ondisk_log_size":92884,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182430,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506588,"kb_used_data":334732,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93865252,"statfs":{"total":96636764160,"available":96118018048,"internally_reserved":0,"allocated":342765568,"data_stored":342532435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":175959722},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,3,4,3,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986202,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":468164,"kb_used_data":301812,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":93903676,"statfs":{"total":96636764160,"available":96157364224,"internally_reserved":0,"allocated":309055488,"data_stored":308826432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":170323626},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,0,0,1,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659149,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":233184,"kb_used_data":111248,"kb_used_omap":4,"kb_used_meta":121915,"kb_avail":94138656,"statfs":{"total":96636764160,"available":96397983744,"internally_reserved":0,"allocated":113917952,"data_stored":113605333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":124841646},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":343539712,"data_stored":343536099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":115007488,"data_stored":114924389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":309092352,"data_stored":309088739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:31:06.050 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 98 20 95116 95096 3.78971 2.32422 0.0297526 0.0206085 2024-09-10T20:31:07.052 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 99 20 95732 95712 3.77565 2.40625 0.0286686 0.0206827 2024-09-10T20:31:08.055 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout:2024-09-10T20:31:08.052097+0000 min lat: 0.00191414 max lat: 2.51074 avg lat: 0.0207618 2024-09-10T20:31:08.055 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T20:31:08.055 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 100 20 96337 96317 3.76144 2.36328 0.0370621 0.0207618 2024-09-10T20:31:08.279 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:31:08.841 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:08.843 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:31:08.867 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2024-09-10T20:31:06.849609+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":387519932,"num_objects":94614,"num_object_clones":0,"num_object_copies":189228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2208,"num_objects_unfound":0,"num_objects_dirty":94614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58989,"num_write_kb":235930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1400,"num_bytes_recovered":5718035,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94694,"ondisk_log_size":94694,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1220652,"kb_used_data":756348,"kb_used_omap":12,"kb_used_meta":464243,"kb_avail":281894868,"statfs":{"total":289910292480,"available":288660344832,"internally_reserved":0,"allocated":774500352,"data_stored":773725544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":475384834},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,1,3,3,15,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":31662080,"num_objects":7730,"num_object_clones":0,"num_object_copies":15460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-227,"num_objects_unfound":0,"num_objects_dirty":7730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7730,"num_write_kb":30920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":929792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004490"},"pg_stats":[{"pgid":"2.f","version":"43'5957","reported_seq":6031,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:06.279599+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:31:06.279599+0000","last_peered":"2024-09-10T20:31:06.279599+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:31:06.279599+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":5957,"log_dups_size":0,"ondisk_log_size":5957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24399872,"num_objects":5957,"num_object_clones":0,"num_object_copies":11914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5957,"num_write_kb":23828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2350},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5933","reported_seq":6035,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:03.124625+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:31:03.124625+0000","last_peered":"2024-09-10T20:31:03.124625+0000","last_clean":"2024-09-10T20:31:03.124625+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:31:03.124625+0000","last_undegraded":"2024-09-10T20:31:03.124625+0000","last_fullsized":"2024-09-10T20:31:03.124625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":5933,"log_dups_size":0,"ondisk_log_size":5933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24276992,"num_objects":5927,"num_object_clones":0,"num_object_copies":11854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4119,"num_write_kb":16476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5868","reported_seq":8780,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:31:03.140397+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:31:03.140397+0000","last_peered":"2024-09-10T20:31:03.140397+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:31:03.140397+0000","last_undegraded":"2024-09-10T20:31:03.140397+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":5868,"log_dups_size":0,"ondisk_log_size":5868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24018944,"num_objects":5864,"num_object_clones":0,"num_object_copies":11728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2208,"num_objects_unfound":0,"num_objects_dirty":5864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3996,"num_write_kb":15984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1395,"num_bytes_recovered":5713920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2208},{"shards":"2","objects":3656}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5737","reported_seq":5812,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:03.175512+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:31:03.175512+0000","last_peered":"2024-09-10T20:31:03.175512+0000","last_clean":"2024-09-10T20:31:03.175512+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:31:03.175512+0000","last_undegraded":"2024-09-10T20:31:03.175512+0000","last_fullsized":"2024-09-10T20:31:03.175512+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":5737,"log_dups_size":0,"ondisk_log_size":5737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23498752,"num_objects":5737,"num_object_clones":0,"num_object_copies":11474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5737,"num_write_kb":22948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6091","reported_seq":6212,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:06.309058+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:31:06.309058+0000","last_peered":"2024-09-10T20:31:06.309058+0000","last_clean":"2024-09-10T20:31:06.309058+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:31:06.309058+0000","last_undegraded":"2024-09-10T20:31:06.309058+0000","last_fullsized":"2024-09-10T20:31:06.309058+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":6091,"log_dups_size":0,"ondisk_log_size":6091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24920064,"num_objects":6084,"num_object_clones":0,"num_object_copies":12168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3087,"num_write_kb":12348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5938","reported_seq":6069,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:06.255821+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:31:06.255821+0000","last_peered":"2024-09-10T20:31:06.255821+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:31:06.255821+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":5938,"log_dups_size":0,"ondisk_log_size":5938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24301568,"num_objects":5933,"num_object_clones":0,"num_object_copies":11866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5933,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2802,"num_write_kb":11208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2319},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5875","reported_seq":6008,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:06.287822+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:31:06.287822+0000","last_peered":"2024-09-10T20:31:06.287822+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:31:06.287822+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":5875,"log_dups_size":0,"ondisk_log_size":5875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24039424,"num_objects":5869,"num_object_clones":0,"num_object_copies":11738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2470,"num_write_kb":9880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2314},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6031","reported_seq":6150,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:06.320400+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:31:06.320400+0000","last_peered":"2024-09-10T20:31:06.320400+0000","last_clean":"2024-09-10T20:31:06.320400+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:31:06.320400+0000","last_undegraded":"2024-09-10T20:31:06.320400+0000","last_fullsized":"2024-09-10T20:31:06.320400+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":6031,"log_dups_size":0,"ondisk_log_size":6031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24694784,"num_objects":6029,"num_object_clones":0,"num_object_copies":12058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3102,"num_write_kb":12408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5678","reported_seq":5805,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:03.183133+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:31:03.183133+0000","last_peered":"2024-09-10T20:31:03.183133+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:31:03.183133+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":5678,"log_dups_size":0,"ondisk_log_size":5678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23228416,"num_objects":5671,"num_object_clones":0,"num_object_copies":11342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3671,"num_write_kb":14684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":2145}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355438+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355438+0000","last_peered":"2024-09-10T20:30:05.355438+0000","last_clean":"2024-09-10T20:30:05.355438+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355438+0000","last_undegraded":"2024-09-10T20:30:05.355438+0000","last_fullsized":"2024-09-10T20:30:05.355438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6010","reported_seq":6112,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:06.249104+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:31:06.249104+0000","last_peered":"2024-09-10T20:31:06.249104+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:31:06.249104+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":6010,"log_dups_size":0,"ondisk_log_size":6010,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24585158,"num_objects":6004,"num_object_clones":0,"num_object_copies":12008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6008,"num_write_kb":24012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2389},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5781","reported_seq":5904,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:03.125750+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:31:03.125750+0000","last_peered":"2024-09-10T20:31:03.125750+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:31:03.125750+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":5781,"log_dups_size":0,"ondisk_log_size":5781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23658496,"num_objects":5776,"num_object_clones":0,"num_object_copies":11552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3609,"num_write_kb":14436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":2143}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5868","reported_seq":5991,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:03.154875+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:31:03.154875+0000","last_peered":"2024-09-10T20:31:03.154875+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:31:03.154875+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":5868,"log_dups_size":0,"ondisk_log_size":5868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24010752,"num_objects":5862,"num_object_clones":0,"num_object_copies":11724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3654,"num_write_kb":14616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":2252}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5964","reported_seq":6094,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:06.227049+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:31:06.227049+0000","last_peered":"2024-09-10T20:31:06.227049+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:31:06.227049+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":5964,"log_dups_size":0,"ondisk_log_size":5964,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24420352,"num_objects":5962,"num_object_clones":0,"num_object_copies":11924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2687,"num_write_kb":10748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2314},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5932","reported_seq":6065,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:06.276357+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:31:06.276357+0000","last_peered":"2024-09-10T20:31:06.276357+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:31:06.276357+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":5932,"log_dups_size":0,"ondisk_log_size":5932,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24264704,"num_objects":5924,"num_object_clones":0,"num_object_copies":11848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2626,"num_write_kb":10504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2392},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345705+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345705+0000","last_peered":"2024-09-10T20:30:05.345705+0000","last_clean":"2024-09-10T20:30:05.345705+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345705+0000","last_undegraded":"2024-09-10T20:30:05.345705+0000","last_fullsized":"2024-09-10T20:30:05.345705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6117","reported_seq":6242,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:06.313327+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:31:06.313327+0000","last_peered":"2024-09-10T20:31:06.313327+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:31:06.313327+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":6117,"log_dups_size":0,"ondisk_log_size":6117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25038848,"num_objects":6113,"num_object_clones":0,"num_object_copies":12226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2944,"num_write_kb":11776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2400},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5911","reported_seq":6071,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:06.278825+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:31:06.278825+0000","last_peered":"2024-09-10T20:31:06.278825+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:31:06.278825+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":5911,"log_dups_size":0,"ondisk_log_size":5911,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24162787,"num_objects":5900,"num_object_clones":0,"num_object_copies":11800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2518,"num_write_kb":10072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2432},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":387519913,"num_objects":94612,"num_object_clones":0,"num_object_copies":189224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2208,"num_objects_unfound":0,"num_objects_dirty":94612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58987,"num_write_kb":235928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1398,"num_bytes_recovered":5718016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":776445952,"data_stored":776355627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94691,"ondisk_log_size":94691,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182431,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519304,"kb_used_data":343288,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93852536,"statfs":{"total":96636764160,"available":96104996864,"internally_reserved":0,"allocated":351526912,"data_stored":351293779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180219562},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986202,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":468164,"kb_used_data":301812,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":93903676,"statfs":{"total":96636764160,"available":96157364224,"internally_reserved":0,"allocated":309055488,"data_stored":308826432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":170323626},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,0,0,1,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659149,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":233184,"kb_used_data":111248,"kb_used_omap":4,"kb_used_meta":121915,"kb_avail":94138656,"statfs":{"total":96636764160,"available":96397983744,"internally_reserved":0,"allocated":113917952,"data_stored":113605333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":124841646},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":352346112,"data_stored":352342499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":115007488,"data_stored":114924389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":309092352,"data_stored":309088739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:31:08.869 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:31:09.055 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 101 20 96885 96865 3.7454 2.14062 0.0232428 0.0208512 2024-09-10T20:31:09.440 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:09.441 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:31:09.471 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":127,"stamp":"2024-09-10T20:31:08.850416+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":392078780,"num_objects":95727,"num_object_clones":0,"num_object_copies":191454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2076,"num_objects_unfound":0,"num_objects_dirty":95727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60102,"num_write_kb":240382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1532,"num_bytes_recovered":6258707,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":95807,"ondisk_log_size":95807,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1226992,"kb_used_data":771008,"kb_used_omap":12,"kb_used_meta":455923,"kb_avail":281888528,"statfs":{"total":289910292480,"available":288653852672,"internally_reserved":0,"allocated":789512192,"data_stored":788737384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":466865154},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,6,7,13,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28610560,"num_objects":6985,"num_object_clones":0,"num_object_copies":13970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-359,"num_objects_unfound":0,"num_objects_dirty":6985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6985,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":359,"num_bytes_recovered":1470464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004709"},"pg_stats":[{"pgid":"2.f","version":"43'5957","reported_seq":6031,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:06.279599+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:31:06.279599+0000","last_peered":"2024-09-10T20:31:06.279599+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:31:06.279599+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":5957,"log_dups_size":0,"ondisk_log_size":5957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24399872,"num_objects":5957,"num_object_clones":0,"num_object_copies":11914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5957,"num_write_kb":23828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2350},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6110","reported_seq":6215,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:08.153527+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:31:08.153527+0000","last_peered":"2024-09-10T20:31:08.153527+0000","last_clean":"2024-09-10T20:31:08.153527+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:31:08.153527+0000","last_undegraded":"2024-09-10T20:31:08.153527+0000","last_fullsized":"2024-09-10T20:31:08.153527+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":6110,"log_dups_size":0,"ondisk_log_size":6110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25001984,"num_objects":6104,"num_object_clones":0,"num_object_copies":12208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4296,"num_write_kb":17184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6051","reported_seq":9230,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:31:08.179761+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:31:08.179761+0000","last_peered":"2024-09-10T20:31:08.179761+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:31:08.179761+0000","last_undegraded":"2024-09-10T20:31:08.179761+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":6051,"log_dups_size":0,"ondisk_log_size":6051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24768512,"num_objects":6047,"num_object_clones":0,"num_object_copies":12094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2076,"num_objects_unfound":0,"num_objects_dirty":6047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4179,"num_write_kb":16716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1527,"num_bytes_recovered":6254592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2076},{"shards":"2","objects":3971}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5892","reported_seq":5967,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:08.136227+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:31:08.136227+0000","last_peered":"2024-09-10T20:31:08.136227+0000","last_clean":"2024-09-10T20:31:08.136227+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:31:08.136227+0000","last_undegraded":"2024-09-10T20:31:08.136227+0000","last_fullsized":"2024-09-10T20:31:08.136227+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":5892,"log_dups_size":0,"ondisk_log_size":5892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24133632,"num_objects":5892,"num_object_clones":0,"num_object_copies":11784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5892,"num_write_kb":23568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6091","reported_seq":6212,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:06.309058+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:31:06.309058+0000","last_peered":"2024-09-10T20:31:06.309058+0000","last_clean":"2024-09-10T20:31:06.309058+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:31:06.309058+0000","last_undegraded":"2024-09-10T20:31:06.309058+0000","last_fullsized":"2024-09-10T20:31:06.309058+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":6091,"log_dups_size":0,"ondisk_log_size":6091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24920064,"num_objects":6084,"num_object_clones":0,"num_object_copies":12168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3087,"num_write_kb":12348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5938","reported_seq":6069,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:06.255821+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:31:06.255821+0000","last_peered":"2024-09-10T20:31:06.255821+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:31:06.255821+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":5938,"log_dups_size":0,"ondisk_log_size":5938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24301568,"num_objects":5933,"num_object_clones":0,"num_object_copies":11866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5933,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2802,"num_write_kb":11208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2319},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5875","reported_seq":6008,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:06.287822+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:31:06.287822+0000","last_peered":"2024-09-10T20:31:06.287822+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:31:06.287822+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":5875,"log_dups_size":0,"ondisk_log_size":5875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24039424,"num_objects":5869,"num_object_clones":0,"num_object_copies":11738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2470,"num_write_kb":9880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2314},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6031","reported_seq":6150,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:06.320400+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:31:06.320400+0000","last_peered":"2024-09-10T20:31:06.320400+0000","last_clean":"2024-09-10T20:31:06.320400+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:31:06.320400+0000","last_undegraded":"2024-09-10T20:31:06.320400+0000","last_fullsized":"2024-09-10T20:31:06.320400+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":6031,"log_dups_size":0,"ondisk_log_size":6031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24694784,"num_objects":6029,"num_object_clones":0,"num_object_copies":12058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3102,"num_write_kb":12408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5879","reported_seq":6006,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:08.163724+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:31:08.163724+0000","last_peered":"2024-09-10T20:31:08.163724+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:31:08.163724+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":5879,"log_dups_size":0,"ondisk_log_size":5879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24051712,"num_objects":5872,"num_object_clones":0,"num_object_copies":11744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5872,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3872,"num_write_kb":15488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":2346}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355438+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355438+0000","last_peered":"2024-09-10T20:30:05.355438+0000","last_clean":"2024-09-10T20:30:05.355438+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355438+0000","last_undegraded":"2024-09-10T20:30:05.355438+0000","last_fullsized":"2024-09-10T20:30:05.355438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6010","reported_seq":6112,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:06.249104+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:31:06.249104+0000","last_peered":"2024-09-10T20:31:06.249104+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:31:06.249104+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":6010,"log_dups_size":0,"ondisk_log_size":6010,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24585158,"num_objects":6004,"num_object_clones":0,"num_object_copies":12008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6008,"num_write_kb":24012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2389},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5993","reported_seq":6116,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:08.187934+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:31:08.187934+0000","last_peered":"2024-09-10T20:31:08.187934+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:31:08.187934+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":5993,"log_dups_size":0,"ondisk_log_size":5993,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24526848,"num_objects":5988,"num_object_clones":0,"num_object_copies":11976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3821,"num_write_kb":15284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":2355}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6053","reported_seq":6176,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:08.180077+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:31:08.180077+0000","last_peered":"2024-09-10T20:31:08.180077+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:31:08.180077+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":6053,"log_dups_size":0,"ondisk_log_size":6053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24768512,"num_objects":6047,"num_object_clones":0,"num_object_copies":12094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3839,"num_write_kb":15356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":2437}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5964","reported_seq":6094,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:06.227049+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:31:06.227049+0000","last_peered":"2024-09-10T20:31:06.227049+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:31:06.227049+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":5964,"log_dups_size":0,"ondisk_log_size":5964,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24420352,"num_objects":5962,"num_object_clones":0,"num_object_copies":11924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2687,"num_write_kb":10748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2314},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5932","reported_seq":6065,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:06.276357+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:31:06.276357+0000","last_peered":"2024-09-10T20:31:06.276357+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:31:06.276357+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":5932,"log_dups_size":0,"ondisk_log_size":5932,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24264704,"num_objects":5924,"num_object_clones":0,"num_object_copies":11848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2626,"num_write_kb":10504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2392},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345705+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345705+0000","last_peered":"2024-09-10T20:30:05.345705+0000","last_clean":"2024-09-10T20:30:05.345705+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345705+0000","last_undegraded":"2024-09-10T20:30:05.345705+0000","last_fullsized":"2024-09-10T20:30:05.345705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6117","reported_seq":6242,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:06.313327+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:31:06.313327+0000","last_peered":"2024-09-10T20:31:06.313327+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:31:06.313327+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":6117,"log_dups_size":0,"ondisk_log_size":6117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25038848,"num_objects":6113,"num_object_clones":0,"num_object_copies":12226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2944,"num_write_kb":11776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2400},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5911","reported_seq":6071,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:06.278825+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:31:06.278825+0000","last_peered":"2024-09-10T20:31:06.278825+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:31:06.278825+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":5911,"log_dups_size":0,"ondisk_log_size":5911,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24162787,"num_objects":5900,"num_object_clones":0,"num_object_copies":11800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2518,"num_write_kb":10072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2432},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":392078761,"num_objects":95725,"num_object_clones":0,"num_object_copies":191450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2076,"num_objects_unfound":0,"num_objects_dirty":95725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60100,"num_write_kb":240380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1530,"num_bytes_recovered":6258688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":791883776,"data_stored":791793451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":95804,"ondisk_log_size":95804,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182431,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519304,"kb_used_data":343288,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93852536,"statfs":{"total":96636764160,"available":96104996864,"internally_reserved":0,"allocated":351526912,"data_stored":351293779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180219562},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986203,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461444,"kb_used_data":307572,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":93910396,"statfs":{"total":96636764160,"available":96164245504,"internally_reserved":0,"allocated":314953728,"data_stored":314724672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":157544106},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,4,1,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659150,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246244,"kb_used_data":120148,"kb_used_omap":4,"kb_used_meta":126075,"kb_avail":94125596,"statfs":{"total":96636764160,"available":96384610304,"internally_reserved":0,"allocated":123031552,"data_stored":122718933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":129101486},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":352346112,"data_stored":352342499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":124338176,"data_stored":124255077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":315199488,"data_stored":315195875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:31:09.472 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:31:09.472 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'5957", 'reported_seq': 6031, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:06.279599+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:31:06.279599+0000', 'last_peered': '2024-09-10T20:31:06.279599+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:31:06.279599+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 5957, 'log_dups_size': 0, 'ondisk_log_size': 5957, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24399872, 'num_objects': 5957, 'num_object_clones': 0, 'num_object_copies': 11914, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3607, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5957, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5957, 'num_write_kb': 23828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2350}, {'shards': '0,2', 'objects': 3607}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:09.472 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T20:31:09.472 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'6051", 'reported_seq': 9230, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:31:08.179761+0000', 'last_change': '2024-09-10T20:30:05.653560+0000', 'last_active': '2024-09-10T20:31:08.179761+0000', 'last_peered': '2024-09-10T20:31:08.179761+0000', 'last_clean': '2024-09-10T20:30:03.279259+0000', 'last_became_active': '2024-09-10T20:30:05.629664+0000', 'last_became_peered': '2024-09-10T20:30:05.629664+0000', 'last_unstale': '2024-09-10T20:31:08.179761+0000', 'last_undegraded': '2024-09-10T20:31:08.179761+0000', 'last_fullsized': '2024-09-10T20:30:05.312061+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1910", 'last_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'objects_scrubbed': 1868, 'log_size': 6051, 'log_dups_size': 0, 'ondisk_log_size': 6051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 673, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24768512, 'num_objects': 6047, 'num_object_clones': 0, 'num_object_copies': 12094, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2076, 'num_objects_unfound': 0, 'num_objects_dirty': 6047, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4179, 'num_write_kb': 16716, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1527, 'num_bytes_recovered': 6254592, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2076}, {'shards': '2', 'objects': 3971}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:09.473 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:31:09.473 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'5938", 'reported_seq': 6069, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:06.255821+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:31:06.255821+0000', 'last_peered': '2024-09-10T20:31:06.255821+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:31:06.255821+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 5938, 'log_dups_size': 0, 'ondisk_log_size': 5938, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24301568, 'num_objects': 5933, 'num_object_clones': 0, 'num_object_copies': 11866, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3616, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5933, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2802, 'num_write_kb': 11208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2319}, {'shards': '0,2', 'objects': 3614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:09.473 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:31:09.473 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'5875", 'reported_seq': 6008, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:06.287822+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:31:06.287822+0000', 'last_peered': '2024-09-10T20:31:06.287822+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:31:06.287822+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 5875, 'log_dups_size': 0, 'ondisk_log_size': 5875, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24039424, 'num_objects': 5869, 'num_object_clones': 0, 'num_object_copies': 11738, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3557, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5869, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2470, 'num_write_kb': 9880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2314}, {'shards': '0,2', 'objects': 3555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:09.473 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:31:09.473 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'5879", 'reported_seq': 6006, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:08.163724+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:31:08.163724+0000', 'last_peered': '2024-09-10T20:31:08.163724+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:31:08.163724+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 5879, 'log_dups_size': 0, 'ondisk_log_size': 5879, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24051712, 'num_objects': 5872, 'num_object_clones': 0, 'num_object_copies': 11744, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3528, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5872, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3872, 'num_write_kb': 15488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 2346}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:09.473 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:31:09.473 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'6010", 'reported_seq': 6112, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:06.249104+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:31:06.249104+0000', 'last_peered': '2024-09-10T20:31:06.249104+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:31:06.249104+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 6010, 'log_dups_size': 0, 'ondisk_log_size': 6010, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24585158, 'num_objects': 6004, 'num_object_clones': 0, 'num_object_copies': 12008, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3617, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6004, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6008, 'num_write_kb': 24012, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2389}, {'shards': '0,2', 'objects': 3615}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:09.473 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:31:09.473 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'5993", 'reported_seq': 6116, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:08.187934+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:31:08.187934+0000', 'last_peered': '2024-09-10T20:31:08.187934+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:31:08.187934+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 5993, 'log_dups_size': 0, 'ondisk_log_size': 5993, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24526848, 'num_objects': 5988, 'num_object_clones': 0, 'num_object_copies': 11976, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3635, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5988, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3821, 'num_write_kb': 15284, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3633}, {'shards': '2', 'objects': 2355}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:09.473 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:31:09.473 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'6053", 'reported_seq': 6176, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:08.180077+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:31:08.180077+0000', 'last_peered': '2024-09-10T20:31:08.180077+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:31:08.180077+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 6053, 'log_dups_size': 0, 'ondisk_log_size': 6053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24768512, 'num_objects': 6047, 'num_object_clones': 0, 'num_object_copies': 12094, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6047, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3839, 'num_write_kb': 15356, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3610}, {'shards': '2', 'objects': 2437}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:09.474 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:31:09.474 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'5964", 'reported_seq': 6094, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:06.227049+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:31:06.227049+0000', 'last_peered': '2024-09-10T20:31:06.227049+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:31:06.227049+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 5964, 'log_dups_size': 0, 'ondisk_log_size': 5964, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24420352, 'num_objects': 5962, 'num_object_clones': 0, 'num_object_copies': 11924, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3649, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5962, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2687, 'num_write_kb': 10748, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2314}, {'shards': '0,2', 'objects': 3648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:09.474 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:31:09.474 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'5932", 'reported_seq': 6065, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:06.276357+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:31:06.276357+0000', 'last_peered': '2024-09-10T20:31:06.276357+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:31:06.276357+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 5932, 'log_dups_size': 0, 'ondisk_log_size': 5932, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24264704, 'num_objects': 5924, 'num_object_clones': 0, 'num_object_copies': 11848, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3535, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5924, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2626, 'num_write_kb': 10504, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2392}, {'shards': '0,2', 'objects': 3532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:09.474 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:31:09.474 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'6117", 'reported_seq': 6242, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:06.313327+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:31:06.313327+0000', 'last_peered': '2024-09-10T20:31:06.313327+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:31:06.313327+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 6117, 'log_dups_size': 0, 'ondisk_log_size': 6117, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25038848, 'num_objects': 6113, 'num_object_clones': 0, 'num_object_copies': 12226, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3715, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6113, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2944, 'num_write_kb': 11776, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2400}, {'shards': '0,2', 'objects': 3713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:09.474 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:31:09.474 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'5911", 'reported_seq': 6071, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:06.278825+0000', 'last_change': '2024-09-10T20:30:05.690055+0000', 'last_active': '2024-09-10T20:31:06.278825+0000', 'last_peered': '2024-09-10T20:31:06.278825+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:31:06.278825+0000', 'last_undegraded': '2024-09-10T20:30:05.378941+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 5911, 'log_dups_size': 0, 'ondisk_log_size': 5911, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24162787, 'num_objects': 5900, 'num_object_clones': 0, 'num_object_copies': 11800, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3471, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5900, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2518, 'num_write_kb': 10072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2432}, {'shards': '0,2', 'objects': 3468}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:09.474 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:31:10.055 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 102 20 97412 97392 3.72884 2.05859 0.038762 0.0209452 2024-09-10T20:31:10.136 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:10.136 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39542/191454 objects degraded (20.654%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":181,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":95727,"data_bytes":392078780,"bytes_used":1256439808,"bytes_avail":288653852672,"bytes_total":289910292480,"degraded_objects":39542,"degraded_total":191454,"degraded_ratio":0.20653525128751554,"misplaced_objects":2076,"misplaced_total":191454,"misplaced_ratio":0.010843335735999248,"recovering_objects_per_sec":29,"recovering_bytes_per_sec":122490,"recovering_keys_per_sec":0,"num_objects_recovered":359,"num_bytes_recovered":1470464,"num_keys_recovered":0,"write_bytes_sec":2383278,"read_op_per_sec":0,"write_op_per_sec":581},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:31:10.166 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:31:10.167 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:31:10.702 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:10.704 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:31:10.726 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":127,"stamp":"2024-09-10T20:31:08.850416+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":392078780,"num_objects":95727,"num_object_clones":0,"num_object_copies":191454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2076,"num_objects_unfound":0,"num_objects_dirty":95727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60102,"num_write_kb":240382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1532,"num_bytes_recovered":6258707,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":95807,"ondisk_log_size":95807,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1226992,"kb_used_data":771008,"kb_used_omap":12,"kb_used_meta":455923,"kb_avail":281888528,"statfs":{"total":289910292480,"available":288653852672,"internally_reserved":0,"allocated":789512192,"data_stored":788737384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":466865154},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,6,7,13,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28610560,"num_objects":6985,"num_object_clones":0,"num_object_copies":13970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-359,"num_objects_unfound":0,"num_objects_dirty":6985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6985,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":359,"num_bytes_recovered":1470464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004709"},"pg_stats":[{"pgid":"2.f","version":"43'5957","reported_seq":6031,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:06.279599+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:31:06.279599+0000","last_peered":"2024-09-10T20:31:06.279599+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:31:06.279599+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":5957,"log_dups_size":0,"ondisk_log_size":5957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24399872,"num_objects":5957,"num_object_clones":0,"num_object_copies":11914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5957,"num_write_kb":23828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2350},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6110","reported_seq":6215,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:08.153527+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:31:08.153527+0000","last_peered":"2024-09-10T20:31:08.153527+0000","last_clean":"2024-09-10T20:31:08.153527+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:31:08.153527+0000","last_undegraded":"2024-09-10T20:31:08.153527+0000","last_fullsized":"2024-09-10T20:31:08.153527+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":6110,"log_dups_size":0,"ondisk_log_size":6110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25001984,"num_objects":6104,"num_object_clones":0,"num_object_copies":12208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4296,"num_write_kb":17184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6051","reported_seq":9230,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:31:08.179761+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:31:08.179761+0000","last_peered":"2024-09-10T20:31:08.179761+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:31:08.179761+0000","last_undegraded":"2024-09-10T20:31:08.179761+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":6051,"log_dups_size":0,"ondisk_log_size":6051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24768512,"num_objects":6047,"num_object_clones":0,"num_object_copies":12094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2076,"num_objects_unfound":0,"num_objects_dirty":6047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4179,"num_write_kb":16716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1527,"num_bytes_recovered":6254592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2076},{"shards":"2","objects":3971}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5892","reported_seq":5967,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:08.136227+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:31:08.136227+0000","last_peered":"2024-09-10T20:31:08.136227+0000","last_clean":"2024-09-10T20:31:08.136227+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:31:08.136227+0000","last_undegraded":"2024-09-10T20:31:08.136227+0000","last_fullsized":"2024-09-10T20:31:08.136227+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":5892,"log_dups_size":0,"ondisk_log_size":5892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24133632,"num_objects":5892,"num_object_clones":0,"num_object_copies":11784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5892,"num_write_kb":23568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6091","reported_seq":6212,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:06.309058+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:31:06.309058+0000","last_peered":"2024-09-10T20:31:06.309058+0000","last_clean":"2024-09-10T20:31:06.309058+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:31:06.309058+0000","last_undegraded":"2024-09-10T20:31:06.309058+0000","last_fullsized":"2024-09-10T20:31:06.309058+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":6091,"log_dups_size":0,"ondisk_log_size":6091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24920064,"num_objects":6084,"num_object_clones":0,"num_object_copies":12168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3087,"num_write_kb":12348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5938","reported_seq":6069,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:06.255821+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:31:06.255821+0000","last_peered":"2024-09-10T20:31:06.255821+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:31:06.255821+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":5938,"log_dups_size":0,"ondisk_log_size":5938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24301568,"num_objects":5933,"num_object_clones":0,"num_object_copies":11866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5933,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2802,"num_write_kb":11208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2319},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5875","reported_seq":6008,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:06.287822+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:31:06.287822+0000","last_peered":"2024-09-10T20:31:06.287822+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:31:06.287822+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":5875,"log_dups_size":0,"ondisk_log_size":5875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24039424,"num_objects":5869,"num_object_clones":0,"num_object_copies":11738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2470,"num_write_kb":9880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2314},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6031","reported_seq":6150,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:06.320400+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:31:06.320400+0000","last_peered":"2024-09-10T20:31:06.320400+0000","last_clean":"2024-09-10T20:31:06.320400+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:31:06.320400+0000","last_undegraded":"2024-09-10T20:31:06.320400+0000","last_fullsized":"2024-09-10T20:31:06.320400+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":6031,"log_dups_size":0,"ondisk_log_size":6031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24694784,"num_objects":6029,"num_object_clones":0,"num_object_copies":12058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3102,"num_write_kb":12408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5879","reported_seq":6006,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:08.163724+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:31:08.163724+0000","last_peered":"2024-09-10T20:31:08.163724+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:31:08.163724+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":5879,"log_dups_size":0,"ondisk_log_size":5879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24051712,"num_objects":5872,"num_object_clones":0,"num_object_copies":11744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5872,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3872,"num_write_kb":15488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":2346}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355438+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355438+0000","last_peered":"2024-09-10T20:30:05.355438+0000","last_clean":"2024-09-10T20:30:05.355438+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355438+0000","last_undegraded":"2024-09-10T20:30:05.355438+0000","last_fullsized":"2024-09-10T20:30:05.355438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6010","reported_seq":6112,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:06.249104+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:31:06.249104+0000","last_peered":"2024-09-10T20:31:06.249104+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:31:06.249104+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":6010,"log_dups_size":0,"ondisk_log_size":6010,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24585158,"num_objects":6004,"num_object_clones":0,"num_object_copies":12008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6008,"num_write_kb":24012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2389},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5993","reported_seq":6116,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:08.187934+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:31:08.187934+0000","last_peered":"2024-09-10T20:31:08.187934+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:31:08.187934+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":5993,"log_dups_size":0,"ondisk_log_size":5993,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24526848,"num_objects":5988,"num_object_clones":0,"num_object_copies":11976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3821,"num_write_kb":15284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":2355}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6053","reported_seq":6176,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:08.180077+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:31:08.180077+0000","last_peered":"2024-09-10T20:31:08.180077+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:31:08.180077+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":6053,"log_dups_size":0,"ondisk_log_size":6053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24768512,"num_objects":6047,"num_object_clones":0,"num_object_copies":12094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3839,"num_write_kb":15356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":2437}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5964","reported_seq":6094,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:06.227049+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:31:06.227049+0000","last_peered":"2024-09-10T20:31:06.227049+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:31:06.227049+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":5964,"log_dups_size":0,"ondisk_log_size":5964,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24420352,"num_objects":5962,"num_object_clones":0,"num_object_copies":11924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2687,"num_write_kb":10748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2314},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5932","reported_seq":6065,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:06.276357+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:31:06.276357+0000","last_peered":"2024-09-10T20:31:06.276357+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:31:06.276357+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":5932,"log_dups_size":0,"ondisk_log_size":5932,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24264704,"num_objects":5924,"num_object_clones":0,"num_object_copies":11848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2626,"num_write_kb":10504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2392},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345705+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345705+0000","last_peered":"2024-09-10T20:30:05.345705+0000","last_clean":"2024-09-10T20:30:05.345705+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345705+0000","last_undegraded":"2024-09-10T20:30:05.345705+0000","last_fullsized":"2024-09-10T20:30:05.345705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6117","reported_seq":6242,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:06.313327+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:31:06.313327+0000","last_peered":"2024-09-10T20:31:06.313327+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:31:06.313327+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":6117,"log_dups_size":0,"ondisk_log_size":6117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25038848,"num_objects":6113,"num_object_clones":0,"num_object_copies":12226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2944,"num_write_kb":11776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2400},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5911","reported_seq":6071,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:06.278825+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:31:06.278825+0000","last_peered":"2024-09-10T20:31:06.278825+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:31:06.278825+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":5911,"log_dups_size":0,"ondisk_log_size":5911,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24162787,"num_objects":5900,"num_object_clones":0,"num_object_copies":11800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2518,"num_write_kb":10072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2432},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":392078761,"num_objects":95725,"num_object_clones":0,"num_object_copies":191450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2076,"num_objects_unfound":0,"num_objects_dirty":95725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60100,"num_write_kb":240380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1530,"num_bytes_recovered":6258688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":791883776,"data_stored":791793451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":95804,"ondisk_log_size":95804,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182431,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519304,"kb_used_data":343288,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93852536,"statfs":{"total":96636764160,"available":96104996864,"internally_reserved":0,"allocated":351526912,"data_stored":351293779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180219562},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986203,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461444,"kb_used_data":307572,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":93910396,"statfs":{"total":96636764160,"available":96164245504,"internally_reserved":0,"allocated":314953728,"data_stored":314724672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":157544106},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,4,1,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659150,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246244,"kb_used_data":120148,"kb_used_omap":4,"kb_used_meta":126075,"kb_avail":94125596,"statfs":{"total":96636764160,"available":96384610304,"internally_reserved":0,"allocated":123031552,"data_stored":122718933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":129101486},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":352346112,"data_stored":352342499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":124338176,"data_stored":124255077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":315199488,"data_stored":315195875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:31:11.055 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 103 20 97965 97945 3.71361 2.16016 0.0353501 0.0210294 2024-09-10T20:31:12.056 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 104 20 98573 98553 3.70072 2.375 0.0503035 0.0211022 2024-09-10T20:31:13.056 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 105 20 99179 99159 3.68802 2.36719 0.0201638 0.0211753 2024-09-10T20:31:13.729 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:31:14.056 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 106 20 99762 99742 3.67471 2.27734 0.032758 0.021253 2024-09-10T20:31:14.283 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:14.283 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:31:14.316 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2024-09-10T20:31:12.851502+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":399316412,"num_objects":97494,"num_object_clones":0,"num_object_copies":194988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2076,"num_objects_unfound":0,"num_objects_dirty":97494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61869,"num_write_kb":247450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1532,"num_bytes_recovered":6258707,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97574,"ondisk_log_size":97574,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1261304,"kb_used_data":788680,"kb_used_omap":12,"kb_used_meta":472563,"kb_avail":281854216,"statfs":{"total":289910292480,"available":288618717184,"internally_reserved":0,"allocated":807608320,"data_stored":806833512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":483904514},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,6,11,8,11,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":31145984,"num_objects":7604,"num_object_clones":0,"num_object_copies":15208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-251,"num_objects_unfound":0,"num_objects_dirty":7604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7604,"num_write_kb":30416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":1028096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004215"},"pg_stats":[{"pgid":"2.f","version":"43'6130","reported_seq":6206,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:11.305981+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:31:11.305981+0000","last_peered":"2024-09-10T20:31:11.305981+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:31:11.305981+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":6130,"log_dups_size":0,"ondisk_log_size":6130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25108480,"num_objects":6130,"num_object_clones":0,"num_object_copies":12260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6130,"num_write_kb":24520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2523},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6110","reported_seq":6215,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:08.153527+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:31:08.153527+0000","last_peered":"2024-09-10T20:31:08.153527+0000","last_clean":"2024-09-10T20:31:08.153527+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:31:08.153527+0000","last_undegraded":"2024-09-10T20:31:08.153527+0000","last_fullsized":"2024-09-10T20:31:08.153527+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":6110,"log_dups_size":0,"ondisk_log_size":6110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25001984,"num_objects":6104,"num_object_clones":0,"num_object_copies":12208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4296,"num_write_kb":17184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6051","reported_seq":9230,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:31:08.179761+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:31:08.179761+0000","last_peered":"2024-09-10T20:31:08.179761+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:31:08.179761+0000","last_undegraded":"2024-09-10T20:31:08.179761+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":6051,"log_dups_size":0,"ondisk_log_size":6051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24768512,"num_objects":6047,"num_object_clones":0,"num_object_copies":12094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2076,"num_objects_unfound":0,"num_objects_dirty":6047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4179,"num_write_kb":16716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1527,"num_bytes_recovered":6254592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2076},{"shards":"2","objects":3971}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5892","reported_seq":5967,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:08.136227+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:31:08.136227+0000","last_peered":"2024-09-10T20:31:08.136227+0000","last_clean":"2024-09-10T20:31:08.136227+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:31:08.136227+0000","last_undegraded":"2024-09-10T20:31:08.136227+0000","last_fullsized":"2024-09-10T20:31:08.136227+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":5892,"log_dups_size":0,"ondisk_log_size":5892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24133632,"num_objects":5892,"num_object_clones":0,"num_object_copies":11784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5892,"num_write_kb":23568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6243","reported_seq":6365,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:11.354305+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:31:11.354305+0000","last_peered":"2024-09-10T20:31:11.354305+0000","last_clean":"2024-09-10T20:31:11.354305+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:31:11.354305+0000","last_undegraded":"2024-09-10T20:31:11.354305+0000","last_fullsized":"2024-09-10T20:31:11.354305+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":6243,"log_dups_size":0,"ondisk_log_size":6243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25542656,"num_objects":6236,"num_object_clones":0,"num_object_copies":12472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3239,"num_write_kb":12956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6128","reported_seq":6259,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:11.345073+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:31:11.345073+0000","last_peered":"2024-09-10T20:31:11.345073+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:31:11.345073+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":6128,"log_dups_size":0,"ondisk_log_size":6128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25079808,"num_objects":6123,"num_object_clones":0,"num_object_copies":12246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2992,"num_write_kb":11968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2509},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6050","reported_seq":6183,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:11.159304+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:31:11.159304+0000","last_peered":"2024-09-10T20:31:11.159304+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:31:11.159304+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":6050,"log_dups_size":0,"ondisk_log_size":6050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24756224,"num_objects":6044,"num_object_clones":0,"num_object_copies":12088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2645,"num_write_kb":10580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2489},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6205","reported_seq":6325,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:11.274960+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:31:11.274960+0000","last_peered":"2024-09-10T20:31:11.274960+0000","last_clean":"2024-09-10T20:31:11.274960+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:31:11.274960+0000","last_undegraded":"2024-09-10T20:31:11.274960+0000","last_fullsized":"2024-09-10T20:31:11.274960+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":6205,"log_dups_size":0,"ondisk_log_size":6205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25407488,"num_objects":6203,"num_object_clones":0,"num_object_copies":12406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3276,"num_write_kb":13104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5879","reported_seq":6006,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:08.163724+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:31:08.163724+0000","last_peered":"2024-09-10T20:31:08.163724+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:31:08.163724+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":5879,"log_dups_size":0,"ondisk_log_size":5879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24051712,"num_objects":5872,"num_object_clones":0,"num_object_copies":11744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5872,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3872,"num_write_kb":15488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":2346}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355438+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355438+0000","last_peered":"2024-09-10T20:30:05.355438+0000","last_clean":"2024-09-10T20:30:05.355438+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355438+0000","last_undegraded":"2024-09-10T20:30:05.355438+0000","last_fullsized":"2024-09-10T20:30:05.355438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6185","reported_seq":6289,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:11.261366+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:31:11.261366+0000","last_peered":"2024-09-10T20:31:11.261366+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:31:11.261366+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":6185,"log_dups_size":0,"ondisk_log_size":6185,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25301958,"num_objects":6179,"num_object_clones":0,"num_object_copies":12358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6183,"num_write_kb":24712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2564},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5993","reported_seq":6116,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:08.187934+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:31:08.187934+0000","last_peered":"2024-09-10T20:31:08.187934+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:31:08.187934+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":5993,"log_dups_size":0,"ondisk_log_size":5993,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24526848,"num_objects":5988,"num_object_clones":0,"num_object_copies":11976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3821,"num_write_kb":15284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":2355}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6053","reported_seq":6176,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:08.180077+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:31:08.180077+0000","last_peered":"2024-09-10T20:31:08.180077+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:31:08.180077+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":6053,"log_dups_size":0,"ondisk_log_size":6053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24768512,"num_objects":6047,"num_object_clones":0,"num_object_copies":12094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3839,"num_write_kb":15356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":2437}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6154","reported_seq":6284,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:11.358897+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:31:11.358897+0000","last_peered":"2024-09-10T20:31:11.358897+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:31:11.358897+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":6154,"log_dups_size":0,"ondisk_log_size":6154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25198592,"num_objects":6152,"num_object_clones":0,"num_object_copies":12304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2877,"num_write_kb":11508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2504},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6114","reported_seq":6247,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:11.316555+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:31:11.316555+0000","last_peered":"2024-09-10T20:31:11.316555+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:31:11.316555+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":6114,"log_dups_size":0,"ondisk_log_size":6114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25010176,"num_objects":6106,"num_object_clones":0,"num_object_copies":12212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2808,"num_write_kb":11232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2574},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345705+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345705+0000","last_peered":"2024-09-10T20:30:05.345705+0000","last_clean":"2024-09-10T20:30:05.345705+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345705+0000","last_undegraded":"2024-09-10T20:30:05.345705+0000","last_fullsized":"2024-09-10T20:30:05.345705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6293","reported_seq":6419,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:11.346207+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:31:11.346207+0000","last_peered":"2024-09-10T20:31:11.346207+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:31:11.346207+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":6293,"log_dups_size":0,"ondisk_log_size":6293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25759744,"num_objects":6289,"num_object_clones":0,"num_object_copies":12578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3120,"num_write_kb":12480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2576},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6091","reported_seq":6249,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:11.346516+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:31:11.346516+0000","last_peered":"2024-09-10T20:31:11.346516+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:31:11.346516+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":6091,"log_dups_size":0,"ondisk_log_size":6091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24900067,"num_objects":6080,"num_object_clones":0,"num_object_copies":12160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2698,"num_write_kb":10792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2612},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":399316393,"num_objects":97492,"num_object_clones":0,"num_object_copies":194984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2076,"num_objects_unfound":0,"num_objects_dirty":97492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61867,"num_write_kb":247448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1530,"num_bytes_recovered":6258688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":809881600,"data_stored":809791275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97571,"ondisk_log_size":97571,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182432,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":536092,"kb_used_data":351756,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93835748,"statfs":{"total":96636764160,"available":96087805952,"internally_reserved":0,"allocated":360198144,"data_stored":359965011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188739242},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,6,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986203,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461444,"kb_used_data":307572,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":93910396,"statfs":{"total":96636764160,"available":96164245504,"internally_reserved":0,"allocated":314953728,"data_stored":314724672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":157544106},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,4,1,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659151,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":263768,"kb_used_data":129352,"kb_used_omap":4,"kb_used_meta":134395,"kb_avail":94108072,"statfs":{"total":96636764160,"available":96366665728,"internally_reserved":0,"allocated":132456448,"data_stored":132143829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":137621166},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,6,1,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":360980480,"data_stored":360976867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":133701632,"data_stored":133618533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":315199488,"data_stored":315195875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:31:14.317 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:31:14.841 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:14.845 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:31:14.868 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2024-09-10T20:31:12.851502+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":399316412,"num_objects":97494,"num_object_clones":0,"num_object_copies":194988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2076,"num_objects_unfound":0,"num_objects_dirty":97494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61869,"num_write_kb":247450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1532,"num_bytes_recovered":6258707,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97574,"ondisk_log_size":97574,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1261304,"kb_used_data":788680,"kb_used_omap":12,"kb_used_meta":472563,"kb_avail":281854216,"statfs":{"total":289910292480,"available":288618717184,"internally_reserved":0,"allocated":807608320,"data_stored":806833512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":483904514},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,6,11,8,11,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":31145984,"num_objects":7604,"num_object_clones":0,"num_object_copies":15208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-251,"num_objects_unfound":0,"num_objects_dirty":7604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7604,"num_write_kb":30416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":1028096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004215"},"pg_stats":[{"pgid":"2.f","version":"43'6130","reported_seq":6206,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:11.305981+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:31:11.305981+0000","last_peered":"2024-09-10T20:31:11.305981+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:31:11.305981+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":6130,"log_dups_size":0,"ondisk_log_size":6130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25108480,"num_objects":6130,"num_object_clones":0,"num_object_copies":12260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6130,"num_write_kb":24520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2523},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6110","reported_seq":6215,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:08.153527+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:31:08.153527+0000","last_peered":"2024-09-10T20:31:08.153527+0000","last_clean":"2024-09-10T20:31:08.153527+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:31:08.153527+0000","last_undegraded":"2024-09-10T20:31:08.153527+0000","last_fullsized":"2024-09-10T20:31:08.153527+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":6110,"log_dups_size":0,"ondisk_log_size":6110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25001984,"num_objects":6104,"num_object_clones":0,"num_object_copies":12208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4296,"num_write_kb":17184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6051","reported_seq":9230,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:31:08.179761+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:31:08.179761+0000","last_peered":"2024-09-10T20:31:08.179761+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:31:08.179761+0000","last_undegraded":"2024-09-10T20:31:08.179761+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":6051,"log_dups_size":0,"ondisk_log_size":6051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24768512,"num_objects":6047,"num_object_clones":0,"num_object_copies":12094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2076,"num_objects_unfound":0,"num_objects_dirty":6047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4179,"num_write_kb":16716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1527,"num_bytes_recovered":6254592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2076},{"shards":"2","objects":3971}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5892","reported_seq":5967,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:08.136227+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:31:08.136227+0000","last_peered":"2024-09-10T20:31:08.136227+0000","last_clean":"2024-09-10T20:31:08.136227+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:31:08.136227+0000","last_undegraded":"2024-09-10T20:31:08.136227+0000","last_fullsized":"2024-09-10T20:31:08.136227+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":5892,"log_dups_size":0,"ondisk_log_size":5892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24133632,"num_objects":5892,"num_object_clones":0,"num_object_copies":11784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5892,"num_write_kb":23568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6243","reported_seq":6365,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:11.354305+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:31:11.354305+0000","last_peered":"2024-09-10T20:31:11.354305+0000","last_clean":"2024-09-10T20:31:11.354305+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:31:11.354305+0000","last_undegraded":"2024-09-10T20:31:11.354305+0000","last_fullsized":"2024-09-10T20:31:11.354305+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":6243,"log_dups_size":0,"ondisk_log_size":6243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25542656,"num_objects":6236,"num_object_clones":0,"num_object_copies":12472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3239,"num_write_kb":12956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6128","reported_seq":6259,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:11.345073+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:31:11.345073+0000","last_peered":"2024-09-10T20:31:11.345073+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:31:11.345073+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":6128,"log_dups_size":0,"ondisk_log_size":6128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25079808,"num_objects":6123,"num_object_clones":0,"num_object_copies":12246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2992,"num_write_kb":11968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2509},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6050","reported_seq":6183,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:11.159304+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:31:11.159304+0000","last_peered":"2024-09-10T20:31:11.159304+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:31:11.159304+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":6050,"log_dups_size":0,"ondisk_log_size":6050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24756224,"num_objects":6044,"num_object_clones":0,"num_object_copies":12088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2645,"num_write_kb":10580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2489},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6205","reported_seq":6325,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:11.274960+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:31:11.274960+0000","last_peered":"2024-09-10T20:31:11.274960+0000","last_clean":"2024-09-10T20:31:11.274960+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:31:11.274960+0000","last_undegraded":"2024-09-10T20:31:11.274960+0000","last_fullsized":"2024-09-10T20:31:11.274960+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":6205,"log_dups_size":0,"ondisk_log_size":6205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25407488,"num_objects":6203,"num_object_clones":0,"num_object_copies":12406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3276,"num_write_kb":13104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5879","reported_seq":6006,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:08.163724+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:31:08.163724+0000","last_peered":"2024-09-10T20:31:08.163724+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:31:08.163724+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":5879,"log_dups_size":0,"ondisk_log_size":5879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24051712,"num_objects":5872,"num_object_clones":0,"num_object_copies":11744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5872,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3872,"num_write_kb":15488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":2346}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355438+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355438+0000","last_peered":"2024-09-10T20:30:05.355438+0000","last_clean":"2024-09-10T20:30:05.355438+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355438+0000","last_undegraded":"2024-09-10T20:30:05.355438+0000","last_fullsized":"2024-09-10T20:30:05.355438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6185","reported_seq":6289,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:11.261366+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:31:11.261366+0000","last_peered":"2024-09-10T20:31:11.261366+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:31:11.261366+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":6185,"log_dups_size":0,"ondisk_log_size":6185,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25301958,"num_objects":6179,"num_object_clones":0,"num_object_copies":12358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6183,"num_write_kb":24712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2564},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5993","reported_seq":6116,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:08.187934+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:31:08.187934+0000","last_peered":"2024-09-10T20:31:08.187934+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:31:08.187934+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":5993,"log_dups_size":0,"ondisk_log_size":5993,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24526848,"num_objects":5988,"num_object_clones":0,"num_object_copies":11976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3821,"num_write_kb":15284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":2355}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6053","reported_seq":6176,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:08.180077+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:31:08.180077+0000","last_peered":"2024-09-10T20:31:08.180077+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:31:08.180077+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":6053,"log_dups_size":0,"ondisk_log_size":6053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24768512,"num_objects":6047,"num_object_clones":0,"num_object_copies":12094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3839,"num_write_kb":15356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":2437}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6154","reported_seq":6284,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:11.358897+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:31:11.358897+0000","last_peered":"2024-09-10T20:31:11.358897+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:31:11.358897+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":6154,"log_dups_size":0,"ondisk_log_size":6154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25198592,"num_objects":6152,"num_object_clones":0,"num_object_copies":12304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2877,"num_write_kb":11508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2504},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6114","reported_seq":6247,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:11.316555+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:31:11.316555+0000","last_peered":"2024-09-10T20:31:11.316555+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:31:11.316555+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":6114,"log_dups_size":0,"ondisk_log_size":6114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25010176,"num_objects":6106,"num_object_clones":0,"num_object_copies":12212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2808,"num_write_kb":11232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2574},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345705+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345705+0000","last_peered":"2024-09-10T20:30:05.345705+0000","last_clean":"2024-09-10T20:30:05.345705+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345705+0000","last_undegraded":"2024-09-10T20:30:05.345705+0000","last_fullsized":"2024-09-10T20:30:05.345705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6293","reported_seq":6419,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:11.346207+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:31:11.346207+0000","last_peered":"2024-09-10T20:31:11.346207+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:31:11.346207+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":6293,"log_dups_size":0,"ondisk_log_size":6293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25759744,"num_objects":6289,"num_object_clones":0,"num_object_copies":12578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3120,"num_write_kb":12480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2576},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6091","reported_seq":6249,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:11.346516+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:31:11.346516+0000","last_peered":"2024-09-10T20:31:11.346516+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:31:11.346516+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":6091,"log_dups_size":0,"ondisk_log_size":6091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24900067,"num_objects":6080,"num_object_clones":0,"num_object_copies":12160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2698,"num_write_kb":10792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2612},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":399316393,"num_objects":97492,"num_object_clones":0,"num_object_copies":194984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":2076,"num_objects_unfound":0,"num_objects_dirty":97492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61867,"num_write_kb":247448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1530,"num_bytes_recovered":6258688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":809881600,"data_stored":809791275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97571,"ondisk_log_size":97571,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182432,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":536092,"kb_used_data":351756,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93835748,"statfs":{"total":96636764160,"available":96087805952,"internally_reserved":0,"allocated":360198144,"data_stored":359965011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188739242},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,6,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986203,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461444,"kb_used_data":307572,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":93910396,"statfs":{"total":96636764160,"available":96164245504,"internally_reserved":0,"allocated":314953728,"data_stored":314724672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":157544106},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,4,1,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659151,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":263768,"kb_used_data":129352,"kb_used_omap":4,"kb_used_meta":134395,"kb_avail":94108072,"statfs":{"total":96636764160,"available":96366665728,"internally_reserved":0,"allocated":132456448,"data_stored":132143829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":137621166},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,6,1,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":360980480,"data_stored":360976867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":133701632,"data_stored":133618533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":315199488,"data_stored":315195875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:31:14.870 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:31:14.870 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'6130", 'reported_seq': 6206, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:11.305981+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:31:11.305981+0000', 'last_peered': '2024-09-10T20:31:11.305981+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:31:11.305981+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 6130, 'log_dups_size': 0, 'ondisk_log_size': 6130, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25108480, 'num_objects': 6130, 'num_object_clones': 0, 'num_object_copies': 12260, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3607, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6130, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6130, 'num_write_kb': 24520, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2523}, {'shards': '0,2', 'objects': 3607}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:14.870 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T20:31:14.870 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'6051", 'reported_seq': 9230, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:31:08.179761+0000', 'last_change': '2024-09-10T20:30:05.653560+0000', 'last_active': '2024-09-10T20:31:08.179761+0000', 'last_peered': '2024-09-10T20:31:08.179761+0000', 'last_clean': '2024-09-10T20:30:03.279259+0000', 'last_became_active': '2024-09-10T20:30:05.629664+0000', 'last_became_peered': '2024-09-10T20:30:05.629664+0000', 'last_unstale': '2024-09-10T20:31:08.179761+0000', 'last_undegraded': '2024-09-10T20:31:08.179761+0000', 'last_fullsized': '2024-09-10T20:30:05.312061+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1910", 'last_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'objects_scrubbed': 1868, 'log_size': 6051, 'log_dups_size': 0, 'ondisk_log_size': 6051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 673, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24768512, 'num_objects': 6047, 'num_object_clones': 0, 'num_object_copies': 12094, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2076, 'num_objects_unfound': 0, 'num_objects_dirty': 6047, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4179, 'num_write_kb': 16716, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1527, 'num_bytes_recovered': 6254592, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2076}, {'shards': '2', 'objects': 3971}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:14.870 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:31:14.870 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'6128", 'reported_seq': 6259, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:11.345073+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:31:11.345073+0000', 'last_peered': '2024-09-10T20:31:11.345073+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:31:11.345073+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 6128, 'log_dups_size': 0, 'ondisk_log_size': 6128, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25079808, 'num_objects': 6123, 'num_object_clones': 0, 'num_object_copies': 12246, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3616, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6123, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2992, 'num_write_kb': 11968, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2509}, {'shards': '0,2', 'objects': 3614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:14.870 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:31:14.870 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'6050", 'reported_seq': 6183, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:11.159304+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:31:11.159304+0000', 'last_peered': '2024-09-10T20:31:11.159304+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:31:11.159304+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 6050, 'log_dups_size': 0, 'ondisk_log_size': 6050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24756224, 'num_objects': 6044, 'num_object_clones': 0, 'num_object_copies': 12088, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3557, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6044, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2645, 'num_write_kb': 10580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2489}, {'shards': '0,2', 'objects': 3555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:14.870 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:31:14.871 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'5879", 'reported_seq': 6006, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:08.163724+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:31:08.163724+0000', 'last_peered': '2024-09-10T20:31:08.163724+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:31:08.163724+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 5879, 'log_dups_size': 0, 'ondisk_log_size': 5879, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24051712, 'num_objects': 5872, 'num_object_clones': 0, 'num_object_copies': 11744, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3528, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5872, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3872, 'num_write_kb': 15488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 2346}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:14.871 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:31:14.871 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'6185", 'reported_seq': 6289, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:11.261366+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:31:11.261366+0000', 'last_peered': '2024-09-10T20:31:11.261366+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:31:11.261366+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 6185, 'log_dups_size': 0, 'ondisk_log_size': 6185, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25301958, 'num_objects': 6179, 'num_object_clones': 0, 'num_object_copies': 12358, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3617, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6179, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6183, 'num_write_kb': 24712, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2564}, {'shards': '0,2', 'objects': 3615}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:14.871 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:31:14.871 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'5993", 'reported_seq': 6116, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:08.187934+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:31:08.187934+0000', 'last_peered': '2024-09-10T20:31:08.187934+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:31:08.187934+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 5993, 'log_dups_size': 0, 'ondisk_log_size': 5993, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24526848, 'num_objects': 5988, 'num_object_clones': 0, 'num_object_copies': 11976, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3635, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5988, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3821, 'num_write_kb': 15284, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3633}, {'shards': '2', 'objects': 2355}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:14.871 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:31:14.871 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'6053", 'reported_seq': 6176, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:08.180077+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:31:08.180077+0000', 'last_peered': '2024-09-10T20:31:08.180077+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:31:08.180077+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 6053, 'log_dups_size': 0, 'ondisk_log_size': 6053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24768512, 'num_objects': 6047, 'num_object_clones': 0, 'num_object_copies': 12094, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6047, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3839, 'num_write_kb': 15356, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3610}, {'shards': '2', 'objects': 2437}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:14.871 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:31:14.871 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'6154", 'reported_seq': 6284, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:11.358897+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:31:11.358897+0000', 'last_peered': '2024-09-10T20:31:11.358897+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:31:11.358897+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 6154, 'log_dups_size': 0, 'ondisk_log_size': 6154, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25198592, 'num_objects': 6152, 'num_object_clones': 0, 'num_object_copies': 12304, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3649, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6152, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2877, 'num_write_kb': 11508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2504}, {'shards': '0,2', 'objects': 3648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:14.871 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:31:14.871 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'6114", 'reported_seq': 6247, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:11.316555+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:31:11.316555+0000', 'last_peered': '2024-09-10T20:31:11.316555+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:31:11.316555+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 6114, 'log_dups_size': 0, 'ondisk_log_size': 6114, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25010176, 'num_objects': 6106, 'num_object_clones': 0, 'num_object_copies': 12212, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3535, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6106, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2808, 'num_write_kb': 11232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2574}, {'shards': '0,2', 'objects': 3532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:14.872 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:31:14.872 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'6293", 'reported_seq': 6419, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:11.346207+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:31:11.346207+0000', 'last_peered': '2024-09-10T20:31:11.346207+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:31:11.346207+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 6293, 'log_dups_size': 0, 'ondisk_log_size': 6293, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25759744, 'num_objects': 6289, 'num_object_clones': 0, 'num_object_copies': 12578, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3715, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6289, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3120, 'num_write_kb': 12480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2576}, {'shards': '0,2', 'objects': 3713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:14.872 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:31:14.872 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'6091", 'reported_seq': 6249, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:11.346516+0000', 'last_change': '2024-09-10T20:30:05.690055+0000', 'last_active': '2024-09-10T20:31:11.346516+0000', 'last_peered': '2024-09-10T20:31:11.346516+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:31:11.346516+0000', 'last_undegraded': '2024-09-10T20:30:05.378941+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 6091, 'log_dups_size': 0, 'ondisk_log_size': 6091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24900067, 'num_objects': 6080, 'num_object_clones': 0, 'num_object_copies': 12160, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3471, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6080, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2698, 'num_write_kb': 10792, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2612}, {'shards': '0,2', 'objects': 3468}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:14.872 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:31:15.058 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 107 20 100286 100266 3.65946 2.04688 0.0333291 0.0213416 2024-09-10T20:31:15.564 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:15.564 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39542/197130 objects degraded (20.059%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":187,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":98565,"data_bytes":403703228,"bytes_used":1301716992,"bytes_avail":288608575488,"bytes_total":289910292480,"degraded_objects":39542,"degraded_total":197130,"degraded_ratio":0.2005884441738954,"misplaced_objects":1952,"misplaced_total":197130,"misplaced_ratio":0.0099020950641708523,"recovering_objects_per_sec":31,"recovering_bytes_per_sec":127956,"recovering_keys_per_sec":0,"num_objects_recovered":375,"num_bytes_recovered":1536000,"num_keys_recovered":0,"write_bytes_sec":2332553,"read_op_per_sec":0,"write_op_per_sec":569},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:31:15.596 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:31:15.596 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:31:16.058 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 108 20 100817 100797 3.64478 2.07422 0.0384425 0.0214265 2024-09-10T20:31:16.141 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:16.141 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:31:16.169 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2024-09-10T20:31:14.852143+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":403703228,"num_objects":98565,"num_object_clones":0,"num_object_copies":197130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":1952,"num_objects_unfound":0,"num_objects_dirty":98565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62940,"num_write_kb":251734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1656,"num_bytes_recovered":6766611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":98645,"ondisk_log_size":98645,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1271208,"kb_used_data":794424,"kb_used_omap":12,"kb_used_meta":476723,"kb_avail":281844312,"statfs":{"total":289910292480,"available":288608575488,"internally_reserved":0,"allocated":813490176,"data_stored":812715368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":488164354},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,7,8,9,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28000256,"num_objects":6836,"num_object_clones":0,"num_object_copies":13672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-375,"num_objects_unfound":0,"num_objects_dirty":6836,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6836,"num_write_kb":27344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":375,"num_bytes_recovered":1536000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004119"},"pg_stats":[{"pgid":"2.f","version":"43'6130","reported_seq":6206,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:11.305981+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:31:11.305981+0000","last_peered":"2024-09-10T20:31:11.305981+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:31:11.305981+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":6130,"log_dups_size":0,"ondisk_log_size":6130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25108480,"num_objects":6130,"num_object_clones":0,"num_object_copies":12260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6130,"num_write_kb":24520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2523},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6271","reported_seq":6376,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:13.086836+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:31:13.086836+0000","last_peered":"2024-09-10T20:31:13.086836+0000","last_clean":"2024-09-10T20:31:13.086836+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:31:13.086836+0000","last_undegraded":"2024-09-10T20:31:13.086836+0000","last_fullsized":"2024-09-10T20:31:13.086836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":6271,"log_dups_size":0,"ondisk_log_size":6271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25661440,"num_objects":6265,"num_object_clones":0,"num_object_copies":12530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4457,"num_write_kb":17828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6226","reported_seq":9651,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:31:13.203938+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:31:13.203938+0000","last_peered":"2024-09-10T20:31:13.203938+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:31:13.203938+0000","last_undegraded":"2024-09-10T20:31:13.203938+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":6226,"log_dups_size":0,"ondisk_log_size":6226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25485312,"num_objects":6222,"num_object_clones":0,"num_object_copies":12444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1952,"num_objects_unfound":0,"num_objects_dirty":6222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4354,"num_write_kb":17416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1651,"num_bytes_recovered":6762496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1952},{"shards":"2","objects":4270}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6093","reported_seq":6168,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:13.116353+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:31:13.116353+0000","last_peered":"2024-09-10T20:31:13.116353+0000","last_clean":"2024-09-10T20:31:13.116353+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:31:13.116353+0000","last_undegraded":"2024-09-10T20:31:13.116353+0000","last_fullsized":"2024-09-10T20:31:13.116353+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":6093,"log_dups_size":0,"ondisk_log_size":6093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24956928,"num_objects":6093,"num_object_clones":0,"num_object_copies":12186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6093,"num_write_kb":24372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6243","reported_seq":6365,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:11.354305+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:31:11.354305+0000","last_peered":"2024-09-10T20:31:11.354305+0000","last_clean":"2024-09-10T20:31:11.354305+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:31:11.354305+0000","last_undegraded":"2024-09-10T20:31:11.354305+0000","last_fullsized":"2024-09-10T20:31:11.354305+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":6243,"log_dups_size":0,"ondisk_log_size":6243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25542656,"num_objects":6236,"num_object_clones":0,"num_object_copies":12472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3239,"num_write_kb":12956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6128","reported_seq":6259,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:11.345073+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:31:11.345073+0000","last_peered":"2024-09-10T20:31:11.345073+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:31:11.345073+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":6128,"log_dups_size":0,"ondisk_log_size":6128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25079808,"num_objects":6123,"num_object_clones":0,"num_object_copies":12246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2992,"num_write_kb":11968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2509},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6050","reported_seq":6183,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:11.159304+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:31:11.159304+0000","last_peered":"2024-09-10T20:31:11.159304+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:31:11.159304+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":6050,"log_dups_size":0,"ondisk_log_size":6050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24756224,"num_objects":6044,"num_object_clones":0,"num_object_copies":12088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2645,"num_write_kb":10580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2489},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6205","reported_seq":6325,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:11.274960+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:31:11.274960+0000","last_peered":"2024-09-10T20:31:11.274960+0000","last_clean":"2024-09-10T20:31:11.274960+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:31:11.274960+0000","last_undegraded":"2024-09-10T20:31:11.274960+0000","last_fullsized":"2024-09-10T20:31:11.274960+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":6205,"log_dups_size":0,"ondisk_log_size":6205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25407488,"num_objects":6203,"num_object_clones":0,"num_object_copies":12406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3276,"num_write_kb":13104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6055","reported_seq":6182,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:13.183587+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:31:13.183587+0000","last_peered":"2024-09-10T20:31:13.183587+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:31:13.183587+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":6055,"log_dups_size":0,"ondisk_log_size":6055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24772608,"num_objects":6048,"num_object_clones":0,"num_object_copies":12096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4048,"num_write_kb":16192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":2522}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355438+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355438+0000","last_peered":"2024-09-10T20:30:05.355438+0000","last_clean":"2024-09-10T20:30:05.355438+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355438+0000","last_undegraded":"2024-09-10T20:30:05.355438+0000","last_fullsized":"2024-09-10T20:30:05.355438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6185","reported_seq":6289,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:11.261366+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:31:11.261366+0000","last_peered":"2024-09-10T20:31:11.261366+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:31:11.261366+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":6185,"log_dups_size":0,"ondisk_log_size":6185,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25301958,"num_objects":6179,"num_object_clones":0,"num_object_copies":12358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6183,"num_write_kb":24712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2564},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6169","reported_seq":6292,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:13.116294+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:31:13.116294+0000","last_peered":"2024-09-10T20:31:13.116294+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:31:13.116294+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":6169,"log_dups_size":0,"ondisk_log_size":6169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25247744,"num_objects":6164,"num_object_clones":0,"num_object_copies":12328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3997,"num_write_kb":15988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":2531}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6235","reported_seq":6358,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:13.180719+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:31:13.180719+0000","last_peered":"2024-09-10T20:31:13.180719+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:31:13.180719+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":6235,"log_dups_size":0,"ondisk_log_size":6235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25513984,"num_objects":6229,"num_object_clones":0,"num_object_copies":12458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4021,"num_write_kb":16084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":2619}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6154","reported_seq":6284,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:11.358897+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:31:11.358897+0000","last_peered":"2024-09-10T20:31:11.358897+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:31:11.358897+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":6154,"log_dups_size":0,"ondisk_log_size":6154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25198592,"num_objects":6152,"num_object_clones":0,"num_object_copies":12304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2877,"num_write_kb":11508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2504},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6114","reported_seq":6247,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:11.316555+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:31:11.316555+0000","last_peered":"2024-09-10T20:31:11.316555+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:31:11.316555+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":6114,"log_dups_size":0,"ondisk_log_size":6114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25010176,"num_objects":6106,"num_object_clones":0,"num_object_copies":12212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2808,"num_write_kb":11232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2574},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345705+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345705+0000","last_peered":"2024-09-10T20:30:05.345705+0000","last_clean":"2024-09-10T20:30:05.345705+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345705+0000","last_undegraded":"2024-09-10T20:30:05.345705+0000","last_fullsized":"2024-09-10T20:30:05.345705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6293","reported_seq":6419,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:11.346207+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:31:11.346207+0000","last_peered":"2024-09-10T20:31:11.346207+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:31:11.346207+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":6293,"log_dups_size":0,"ondisk_log_size":6293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25759744,"num_objects":6289,"num_object_clones":0,"num_object_copies":12578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3120,"num_write_kb":12480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2576},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6091","reported_seq":6249,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:11.346516+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:31:11.346516+0000","last_peered":"2024-09-10T20:31:11.346516+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:31:11.346516+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":6091,"log_dups_size":0,"ondisk_log_size":6091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24900067,"num_objects":6080,"num_object_clones":0,"num_object_copies":12160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2698,"num_write_kb":10792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2612},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":403703209,"num_objects":98563,"num_object_clones":0,"num_object_copies":197126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":1952,"num_objects_unfound":0,"num_objects_dirty":98563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62938,"num_write_kb":251732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1654,"num_bytes_recovered":6766592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":815599616,"data_stored":815509291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":98642,"ondisk_log_size":98642,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182432,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":536092,"kb_used_data":351756,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93835748,"statfs":{"total":96636764160,"available":96087805952,"internally_reserved":0,"allocated":360198144,"data_stored":359965011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188739242},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,6,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986204,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":471348,"kb_used_data":313316,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":93900492,"statfs":{"total":96636764160,"available":96154103808,"internally_reserved":0,"allocated":320835584,"data_stored":320606528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":161803946},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659151,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":263768,"kb_used_data":129352,"kb_used_omap":4,"kb_used_meta":134395,"kb_avail":94108072,"statfs":{"total":96636764160,"available":96366665728,"internally_reserved":0,"allocated":132456448,"data_stored":132143829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":137621166},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,6,1,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":360980480,"data_stored":360976867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":133701632,"data_stored":133618533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":320917504,"data_stored":320913891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:31:17.058 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 109 20 101405 101385 3.63241 2.29688 0.0197377 0.0214996 2024-09-10T20:31:18.058 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 110 20 101991 101971 3.6202 2.28906 0.0238886 0.0215728 2024-09-10T20:31:19.058 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 111 20 102576 102556 3.60817 2.28516 0.0269866 0.0216428 2024-09-10T20:31:19.172 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:31:19.733 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:19.736 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:31:19.765 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":132,"stamp":"2024-09-10T20:31:18.853681+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":415352252,"num_objects":101409,"num_object_clones":0,"num_object_copies":202818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":1806,"num_objects_unfound":0,"num_objects_dirty":101409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65784,"num_write_kb":263110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1803,"num_bytes_recovered":7364627,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101489,"ondisk_log_size":101489,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1291380,"kb_used_data":817344,"kb_used_omap":12,"kb_used_meta":473971,"kb_avail":281824140,"statfs":{"total":289910292480,"available":288587919360,"internally_reserved":0,"allocated":836960256,"data_stored":836189061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":485346306},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,5,4,11,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27832320,"num_objects":6795,"num_object_clones":0,"num_object_copies":13590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-402,"num_objects_unfound":0,"num_objects_dirty":6795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6795,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":403,"num_bytes_recovered":1646592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004071"},"pg_stats":[{"pgid":"2.f","version":"43'6322","reported_seq":6395,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:16.396641+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:31:16.396641+0000","last_peered":"2024-09-10T20:31:16.396641+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:31:16.396641+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":6322,"log_dups_size":0,"ondisk_log_size":6322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25894912,"num_objects":6322,"num_object_clones":0,"num_object_copies":12644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6322,"num_write_kb":25288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2715},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6469","reported_seq":6573,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:18.197233+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:31:18.197233+0000","last_peered":"2024-09-10T20:31:18.197233+0000","last_clean":"2024-09-10T20:31:18.197233+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:31:18.197233+0000","last_undegraded":"2024-09-10T20:31:18.197233+0000","last_fullsized":"2024-09-10T20:31:18.197233+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":6469,"log_dups_size":0,"ondisk_log_size":6469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26472448,"num_objects":6463,"num_object_clones":0,"num_object_copies":12926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4655,"num_write_kb":18620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6388","reported_seq":10109,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:31:18.240405+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:31:18.240405+0000","last_peered":"2024-09-10T20:31:18.240405+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:31:18.240405+0000","last_undegraded":"2024-09-10T20:31:18.240405+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":6388,"log_dups_size":0,"ondisk_log_size":6388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26148864,"num_objects":6384,"num_object_clones":0,"num_object_copies":12768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1806,"num_objects_unfound":0,"num_objects_dirty":6384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4516,"num_write_kb":18064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1798,"num_bytes_recovered":7360512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1806},{"shards":"2","objects":4578}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6258","reported_seq":6333,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:18.172898+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:31:18.172898+0000","last_peered":"2024-09-10T20:31:18.172898+0000","last_clean":"2024-09-10T20:31:18.172898+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:31:18.172898+0000","last_undegraded":"2024-09-10T20:31:18.172898+0000","last_fullsized":"2024-09-10T20:31:18.172898+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":6258,"log_dups_size":0,"ondisk_log_size":6258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25632768,"num_objects":6258,"num_object_clones":0,"num_object_copies":12516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6258,"num_write_kb":25032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6429","reported_seq":6551,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:16.378917+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:31:16.378917+0000","last_peered":"2024-09-10T20:31:16.378917+0000","last_clean":"2024-09-10T20:31:16.378917+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:31:16.378917+0000","last_undegraded":"2024-09-10T20:31:16.378917+0000","last_fullsized":"2024-09-10T20:31:16.378917+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":6429,"log_dups_size":0,"ondisk_log_size":6429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26304512,"num_objects":6422,"num_object_clones":0,"num_object_copies":12844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6422,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3425,"num_write_kb":13700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6312","reported_seq":6444,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:16.376186+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:31:16.376186+0000","last_peered":"2024-09-10T20:31:16.376186+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:31:16.376186+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":6312,"log_dups_size":0,"ondisk_log_size":6312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25833472,"num_objects":6307,"num_object_clones":0,"num_object_copies":12614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3176,"num_write_kb":12704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2693},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6230","reported_seq":6363,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:16.379294+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:31:16.379294+0000","last_peered":"2024-09-10T20:31:16.379294+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:31:16.379294+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":6230,"log_dups_size":0,"ondisk_log_size":6230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25493504,"num_objects":6224,"num_object_clones":0,"num_object_copies":12448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2825,"num_write_kb":11300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2669},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6399","reported_seq":6519,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:16.378706+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:31:16.378706+0000","last_peered":"2024-09-10T20:31:16.378706+0000","last_clean":"2024-09-10T20:31:16.378706+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:31:16.378706+0000","last_undegraded":"2024-09-10T20:31:16.378706+0000","last_fullsized":"2024-09-10T20:31:16.378706+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":6399,"log_dups_size":0,"ondisk_log_size":6399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26202112,"num_objects":6397,"num_object_clones":0,"num_object_copies":12794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3470,"num_write_kb":13880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6237","reported_seq":6364,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:18.215564+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:31:18.215564+0000","last_peered":"2024-09-10T20:31:18.215564+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:31:18.215564+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":6237,"log_dups_size":0,"ondisk_log_size":6237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25518080,"num_objects":6230,"num_object_clones":0,"num_object_copies":12460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4230,"num_write_kb":16920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":2704}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355479+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355479+0000","last_peered":"2024-09-10T20:30:05.355479+0000","last_clean":"2024-09-10T20:30:05.355479+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355479+0000","last_undegraded":"2024-09-10T20:30:05.355479+0000","last_fullsized":"2024-09-10T20:30:05.355479+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6344","reported_seq":6447,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:16.401276+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:31:16.401276+0000","last_peered":"2024-09-10T20:31:16.401276+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:31:16.401276+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":6344,"log_dups_size":0,"ondisk_log_size":6344,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25953222,"num_objects":6338,"num_object_clones":0,"num_object_copies":12676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6342,"num_write_kb":25348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2723},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6337","reported_seq":6460,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:18.229834+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:31:18.229834+0000","last_peered":"2024-09-10T20:31:18.229834+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:31:18.229834+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":6337,"log_dups_size":0,"ondisk_log_size":6337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25935872,"num_objects":6332,"num_object_clones":0,"num_object_copies":12664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4165,"num_write_kb":16660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":2699}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6415","reported_seq":6539,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:18.224519+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:31:18.224519+0000","last_peered":"2024-09-10T20:31:18.224519+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:31:18.224519+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":6415,"log_dups_size":0,"ondisk_log_size":6415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26251264,"num_objects":6409,"num_object_clones":0,"num_object_copies":12818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4201,"num_write_kb":16804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":2799}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6334","reported_seq":6462,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:16.407688+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:31:16.407688+0000","last_peered":"2024-09-10T20:31:16.407688+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:31:16.407688+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":6334,"log_dups_size":0,"ondisk_log_size":6334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25935872,"num_objects":6332,"num_object_clones":0,"num_object_copies":12664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3057,"num_write_kb":12228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2684},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6313","reported_seq":6446,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:16.421200+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:31:16.421200+0000","last_peered":"2024-09-10T20:31:16.421200+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:31:16.421200+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":6313,"log_dups_size":0,"ondisk_log_size":6313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25825280,"num_objects":6305,"num_object_clones":0,"num_object_copies":12610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3007,"num_write_kb":12028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2773},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345743+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345743+0000","last_peered":"2024-09-10T20:30:05.345743+0000","last_clean":"2024-09-10T20:30:05.345743+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345743+0000","last_undegraded":"2024-09-10T20:30:05.345743+0000","last_fullsized":"2024-09-10T20:30:05.345743+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6446","reported_seq":6573,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:16.298134+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:31:16.298134+0000","last_peered":"2024-09-10T20:31:16.298134+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:31:16.298134+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":6446,"log_dups_size":0,"ondisk_log_size":6446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26386432,"num_objects":6442,"num_object_clones":0,"num_object_copies":12884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3273,"num_write_kb":13092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2729},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6253","reported_seq":6414,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:16.363816+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:31:16.363816+0000","last_peered":"2024-09-10T20:31:16.363816+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:31:16.363816+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":6253,"log_dups_size":0,"ondisk_log_size":6253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25563619,"num_objects":6242,"num_object_clones":0,"num_object_copies":12484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6242,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2860,"num_write_kb":11440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2774},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":415352233,"num_objects":101407,"num_object_clones":0,"num_object_copies":202814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":1806,"num_objects_unfound":0,"num_objects_dirty":101407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65782,"num_write_kb":263108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1801,"num_bytes_recovered":7364608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":839573504,"data_stored":839486792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101486,"ondisk_log_size":101486,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182433,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":532104,"kb_used_data":360248,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93839736,"statfs":{"total":96636764160,"available":96091889664,"internally_reserved":0,"allocated":368893952,"data_stored":368660819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":175959722},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,0,10,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986205,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499128,"kb_used_data":318884,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93872712,"statfs":{"total":96636764160,"available":96125657088,"internally_reserved":0,"allocated":326537216,"data_stored":326308160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184544938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659152,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":260148,"kb_used_data":138212,"kb_used_omap":4,"kb_used_meta":121915,"kb_avail":94111692,"statfs":{"total":96636764160,"available":96370372608,"internally_reserved":0,"allocated":141529088,"data_stored":141220082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":124841646},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,1,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":369872896,"data_stored":369869283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":142942208,"data_stored":142862722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":326758400,"data_stored":326754787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:31:19.766 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:31:20.058 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 112 20 103108 103088 3.59451 2.07812 0.0157742 0.0217256 2024-09-10T20:31:20.313 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:20.316 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:31:20.339 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":132,"stamp":"2024-09-10T20:31:18.853681+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":415352252,"num_objects":101409,"num_object_clones":0,"num_object_copies":202818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":1806,"num_objects_unfound":0,"num_objects_dirty":101409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65784,"num_write_kb":263110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1803,"num_bytes_recovered":7364627,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101489,"ondisk_log_size":101489,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1291380,"kb_used_data":817344,"kb_used_omap":12,"kb_used_meta":473971,"kb_avail":281824140,"statfs":{"total":289910292480,"available":288587919360,"internally_reserved":0,"allocated":836960256,"data_stored":836189061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":485346306},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,5,4,11,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27832320,"num_objects":6795,"num_object_clones":0,"num_object_copies":13590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-402,"num_objects_unfound":0,"num_objects_dirty":6795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6795,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":403,"num_bytes_recovered":1646592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004071"},"pg_stats":[{"pgid":"2.f","version":"43'6322","reported_seq":6395,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:16.396641+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:31:16.396641+0000","last_peered":"2024-09-10T20:31:16.396641+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:31:16.396641+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":6322,"log_dups_size":0,"ondisk_log_size":6322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25894912,"num_objects":6322,"num_object_clones":0,"num_object_copies":12644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6322,"num_write_kb":25288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2715},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6469","reported_seq":6573,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:18.197233+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:31:18.197233+0000","last_peered":"2024-09-10T20:31:18.197233+0000","last_clean":"2024-09-10T20:31:18.197233+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:31:18.197233+0000","last_undegraded":"2024-09-10T20:31:18.197233+0000","last_fullsized":"2024-09-10T20:31:18.197233+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":6469,"log_dups_size":0,"ondisk_log_size":6469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26472448,"num_objects":6463,"num_object_clones":0,"num_object_copies":12926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4655,"num_write_kb":18620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6388","reported_seq":10109,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:31:18.240405+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:31:18.240405+0000","last_peered":"2024-09-10T20:31:18.240405+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:31:18.240405+0000","last_undegraded":"2024-09-10T20:31:18.240405+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":6388,"log_dups_size":0,"ondisk_log_size":6388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26148864,"num_objects":6384,"num_object_clones":0,"num_object_copies":12768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1806,"num_objects_unfound":0,"num_objects_dirty":6384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4516,"num_write_kb":18064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1798,"num_bytes_recovered":7360512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1806},{"shards":"2","objects":4578}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6258","reported_seq":6333,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:18.172898+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:31:18.172898+0000","last_peered":"2024-09-10T20:31:18.172898+0000","last_clean":"2024-09-10T20:31:18.172898+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:31:18.172898+0000","last_undegraded":"2024-09-10T20:31:18.172898+0000","last_fullsized":"2024-09-10T20:31:18.172898+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":6258,"log_dups_size":0,"ondisk_log_size":6258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25632768,"num_objects":6258,"num_object_clones":0,"num_object_copies":12516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6258,"num_write_kb":25032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6429","reported_seq":6551,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:16.378917+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:31:16.378917+0000","last_peered":"2024-09-10T20:31:16.378917+0000","last_clean":"2024-09-10T20:31:16.378917+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:31:16.378917+0000","last_undegraded":"2024-09-10T20:31:16.378917+0000","last_fullsized":"2024-09-10T20:31:16.378917+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":6429,"log_dups_size":0,"ondisk_log_size":6429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26304512,"num_objects":6422,"num_object_clones":0,"num_object_copies":12844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6422,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3425,"num_write_kb":13700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6312","reported_seq":6444,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:16.376186+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:31:16.376186+0000","last_peered":"2024-09-10T20:31:16.376186+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:31:16.376186+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":6312,"log_dups_size":0,"ondisk_log_size":6312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25833472,"num_objects":6307,"num_object_clones":0,"num_object_copies":12614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3176,"num_write_kb":12704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2693},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6230","reported_seq":6363,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:16.379294+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:31:16.379294+0000","last_peered":"2024-09-10T20:31:16.379294+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:31:16.379294+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":6230,"log_dups_size":0,"ondisk_log_size":6230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25493504,"num_objects":6224,"num_object_clones":0,"num_object_copies":12448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2825,"num_write_kb":11300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2669},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6399","reported_seq":6519,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:16.378706+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:31:16.378706+0000","last_peered":"2024-09-10T20:31:16.378706+0000","last_clean":"2024-09-10T20:31:16.378706+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:31:16.378706+0000","last_undegraded":"2024-09-10T20:31:16.378706+0000","last_fullsized":"2024-09-10T20:31:16.378706+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":6399,"log_dups_size":0,"ondisk_log_size":6399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26202112,"num_objects":6397,"num_object_clones":0,"num_object_copies":12794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3470,"num_write_kb":13880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6237","reported_seq":6364,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:18.215564+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:31:18.215564+0000","last_peered":"2024-09-10T20:31:18.215564+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:31:18.215564+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":6237,"log_dups_size":0,"ondisk_log_size":6237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25518080,"num_objects":6230,"num_object_clones":0,"num_object_copies":12460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4230,"num_write_kb":16920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":2704}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355479+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355479+0000","last_peered":"2024-09-10T20:30:05.355479+0000","last_clean":"2024-09-10T20:30:05.355479+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355479+0000","last_undegraded":"2024-09-10T20:30:05.355479+0000","last_fullsized":"2024-09-10T20:30:05.355479+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6344","reported_seq":6447,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:16.401276+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:31:16.401276+0000","last_peered":"2024-09-10T20:31:16.401276+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:31:16.401276+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":6344,"log_dups_size":0,"ondisk_log_size":6344,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25953222,"num_objects":6338,"num_object_clones":0,"num_object_copies":12676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6342,"num_write_kb":25348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2723},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6337","reported_seq":6460,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:18.229834+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:31:18.229834+0000","last_peered":"2024-09-10T20:31:18.229834+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:31:18.229834+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":6337,"log_dups_size":0,"ondisk_log_size":6337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25935872,"num_objects":6332,"num_object_clones":0,"num_object_copies":12664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4165,"num_write_kb":16660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":2699}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6415","reported_seq":6539,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:18.224519+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:31:18.224519+0000","last_peered":"2024-09-10T20:31:18.224519+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:31:18.224519+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":6415,"log_dups_size":0,"ondisk_log_size":6415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26251264,"num_objects":6409,"num_object_clones":0,"num_object_copies":12818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4201,"num_write_kb":16804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":2799}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6334","reported_seq":6462,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:16.407688+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:31:16.407688+0000","last_peered":"2024-09-10T20:31:16.407688+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:31:16.407688+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":6334,"log_dups_size":0,"ondisk_log_size":6334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25935872,"num_objects":6332,"num_object_clones":0,"num_object_copies":12664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3057,"num_write_kb":12228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2684},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6313","reported_seq":6446,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:16.421200+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:31:16.421200+0000","last_peered":"2024-09-10T20:31:16.421200+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:31:16.421200+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":6313,"log_dups_size":0,"ondisk_log_size":6313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25825280,"num_objects":6305,"num_object_clones":0,"num_object_copies":12610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3007,"num_write_kb":12028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2773},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345743+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345743+0000","last_peered":"2024-09-10T20:30:05.345743+0000","last_clean":"2024-09-10T20:30:05.345743+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345743+0000","last_undegraded":"2024-09-10T20:30:05.345743+0000","last_fullsized":"2024-09-10T20:30:05.345743+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6446","reported_seq":6573,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:16.298134+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:31:16.298134+0000","last_peered":"2024-09-10T20:31:16.298134+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:31:16.298134+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":6446,"log_dups_size":0,"ondisk_log_size":6446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26386432,"num_objects":6442,"num_object_clones":0,"num_object_copies":12884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3273,"num_write_kb":13092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2729},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6253","reported_seq":6414,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:16.363816+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:31:16.363816+0000","last_peered":"2024-09-10T20:31:16.363816+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:31:16.363816+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":6253,"log_dups_size":0,"ondisk_log_size":6253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25563619,"num_objects":6242,"num_object_clones":0,"num_object_copies":12484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6242,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2860,"num_write_kb":11440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2774},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":415352233,"num_objects":101407,"num_object_clones":0,"num_object_copies":202814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":1806,"num_objects_unfound":0,"num_objects_dirty":101407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65782,"num_write_kb":263108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1801,"num_bytes_recovered":7364608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":839573504,"data_stored":839486792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101486,"ondisk_log_size":101486,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182433,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":532104,"kb_used_data":360248,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93839736,"statfs":{"total":96636764160,"available":96091889664,"internally_reserved":0,"allocated":368893952,"data_stored":368660819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":175959722},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,0,10,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986205,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499128,"kb_used_data":318884,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93872712,"statfs":{"total":96636764160,"available":96125657088,"internally_reserved":0,"allocated":326537216,"data_stored":326308160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184544938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659152,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":260148,"kb_used_data":138212,"kb_used_omap":4,"kb_used_meta":121915,"kb_avail":94111692,"statfs":{"total":96636764160,"available":96370372608,"internally_reserved":0,"allocated":141529088,"data_stored":141220082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":124841646},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,1,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":369872896,"data_stored":369869283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":142942208,"data_stored":142862722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":326758400,"data_stored":326754787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:31:20.341 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:31:20.341 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'6322", 'reported_seq': 6395, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:16.396641+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:31:16.396641+0000', 'last_peered': '2024-09-10T20:31:16.396641+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:31:16.396641+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 6322, 'log_dups_size': 0, 'ondisk_log_size': 6322, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25894912, 'num_objects': 6322, 'num_object_clones': 0, 'num_object_copies': 12644, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3607, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6322, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6322, 'num_write_kb': 25288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2715}, {'shards': '0,2', 'objects': 3607}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:20.341 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T20:31:20.341 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'6388", 'reported_seq': 10109, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:31:18.240405+0000', 'last_change': '2024-09-10T20:30:05.653560+0000', 'last_active': '2024-09-10T20:31:18.240405+0000', 'last_peered': '2024-09-10T20:31:18.240405+0000', 'last_clean': '2024-09-10T20:30:03.279259+0000', 'last_became_active': '2024-09-10T20:30:05.629664+0000', 'last_became_peered': '2024-09-10T20:30:05.629664+0000', 'last_unstale': '2024-09-10T20:31:18.240405+0000', 'last_undegraded': '2024-09-10T20:31:18.240405+0000', 'last_fullsized': '2024-09-10T20:30:05.312061+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1910", 'last_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'objects_scrubbed': 1868, 'log_size': 6388, 'log_dups_size': 0, 'ondisk_log_size': 6388, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 673, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26148864, 'num_objects': 6384, 'num_object_clones': 0, 'num_object_copies': 12768, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1806, 'num_objects_unfound': 0, 'num_objects_dirty': 6384, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4516, 'num_write_kb': 18064, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1798, 'num_bytes_recovered': 7360512, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1806}, {'shards': '2', 'objects': 4578}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:20.341 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:31:20.341 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'6312", 'reported_seq': 6444, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:16.376186+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:31:16.376186+0000', 'last_peered': '2024-09-10T20:31:16.376186+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:31:16.376186+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 6312, 'log_dups_size': 0, 'ondisk_log_size': 6312, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25833472, 'num_objects': 6307, 'num_object_clones': 0, 'num_object_copies': 12614, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3616, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6307, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3176, 'num_write_kb': 12704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2693}, {'shards': '0,2', 'objects': 3614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:20.341 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:31:20.341 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'6230", 'reported_seq': 6363, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:16.379294+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:31:16.379294+0000', 'last_peered': '2024-09-10T20:31:16.379294+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:31:16.379294+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 6230, 'log_dups_size': 0, 'ondisk_log_size': 6230, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25493504, 'num_objects': 6224, 'num_object_clones': 0, 'num_object_copies': 12448, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3557, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6224, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2825, 'num_write_kb': 11300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2669}, {'shards': '0,2', 'objects': 3555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:20.341 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:31:20.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'6237", 'reported_seq': 6364, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:18.215564+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:31:18.215564+0000', 'last_peered': '2024-09-10T20:31:18.215564+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:31:18.215564+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 6237, 'log_dups_size': 0, 'ondisk_log_size': 6237, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25518080, 'num_objects': 6230, 'num_object_clones': 0, 'num_object_copies': 12460, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3528, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6230, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4230, 'num_write_kb': 16920, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 2704}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:20.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:31:20.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'6344", 'reported_seq': 6447, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:16.401276+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:31:16.401276+0000', 'last_peered': '2024-09-10T20:31:16.401276+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:31:16.401276+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 6344, 'log_dups_size': 0, 'ondisk_log_size': 6344, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25953222, 'num_objects': 6338, 'num_object_clones': 0, 'num_object_copies': 12676, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3617, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6338, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6342, 'num_write_kb': 25348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2723}, {'shards': '0,2', 'objects': 3615}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:20.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:31:20.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'6337", 'reported_seq': 6460, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:18.229834+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:31:18.229834+0000', 'last_peered': '2024-09-10T20:31:18.229834+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:31:18.229834+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 6337, 'log_dups_size': 0, 'ondisk_log_size': 6337, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25935872, 'num_objects': 6332, 'num_object_clones': 0, 'num_object_copies': 12664, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3635, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6332, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4165, 'num_write_kb': 16660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3633}, {'shards': '2', 'objects': 2699}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:20.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:31:20.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'6415", 'reported_seq': 6539, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:18.224519+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:31:18.224519+0000', 'last_peered': '2024-09-10T20:31:18.224519+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:31:18.224519+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 6415, 'log_dups_size': 0, 'ondisk_log_size': 6415, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26251264, 'num_objects': 6409, 'num_object_clones': 0, 'num_object_copies': 12818, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6409, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4201, 'num_write_kb': 16804, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3610}, {'shards': '2', 'objects': 2799}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:20.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:31:20.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'6334", 'reported_seq': 6462, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:16.407688+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:31:16.407688+0000', 'last_peered': '2024-09-10T20:31:16.407688+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:31:16.407688+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 6334, 'log_dups_size': 0, 'ondisk_log_size': 6334, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25935872, 'num_objects': 6332, 'num_object_clones': 0, 'num_object_copies': 12664, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3649, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6332, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3057, 'num_write_kb': 12228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2684}, {'shards': '0,2', 'objects': 3648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:20.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:31:20.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'6313", 'reported_seq': 6446, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:16.421200+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:31:16.421200+0000', 'last_peered': '2024-09-10T20:31:16.421200+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:31:16.421200+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 6313, 'log_dups_size': 0, 'ondisk_log_size': 6313, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25825280, 'num_objects': 6305, 'num_object_clones': 0, 'num_object_copies': 12610, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3535, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6305, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3007, 'num_write_kb': 12028, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2773}, {'shards': '0,2', 'objects': 3532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:20.343 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:31:20.343 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'6446", 'reported_seq': 6573, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:16.298134+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:31:16.298134+0000', 'last_peered': '2024-09-10T20:31:16.298134+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:31:16.298134+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 6446, 'log_dups_size': 0, 'ondisk_log_size': 6446, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26386432, 'num_objects': 6442, 'num_object_clones': 0, 'num_object_copies': 12884, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3715, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6442, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3273, 'num_write_kb': 13092, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2729}, {'shards': '0,2', 'objects': 3713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:20.343 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:31:20.343 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'6253", 'reported_seq': 6414, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:16.363816+0000', 'last_change': '2024-09-10T20:30:05.690055+0000', 'last_active': '2024-09-10T20:31:16.363816+0000', 'last_peered': '2024-09-10T20:31:16.363816+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:31:16.363816+0000', 'last_undegraded': '2024-09-10T20:30:05.378941+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 6253, 'log_dups_size': 0, 'ondisk_log_size': 6253, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25563619, 'num_objects': 6242, 'num_object_clones': 0, 'num_object_copies': 12484, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3471, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6242, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2860, 'num_write_kb': 11440, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2774}, {'shards': '0,2', 'objects': 3468}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:20.343 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:31:21.049 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:21.049 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39542/202818 objects degraded (19.496%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":192,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":101409,"data_bytes":415352252,"bytes_used":1322373120,"bytes_avail":288587919360,"bytes_total":289910292480,"degraded_objects":39542,"degraded_total":202818,"degraded_ratio":0.19496297172834759,"misplaced_objects":1806,"misplaced_total":202818,"misplaced_ratio":0.0089045351004348721,"recovering_objects_per_sec":33,"recovering_bytes_per_sec":137169,"recovering_keys_per_sec":0,"num_objects_recovered":403,"num_bytes_recovered":1646592,"num_keys_recovered":0,"write_bytes_sec":2318573,"read_op_per_sec":0,"write_op_per_sec":566},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:31:21.059 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 113 20 103633 103613 3.58085 2.05078 0.0171507 0.02181 2024-09-10T20:31:21.071 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:31:21.072 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:31:21.652 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:21.653 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:31:21.677 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":133,"stamp":"2024-09-10T20:31:20.854522+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":415352252,"num_objects":101409,"num_object_clones":0,"num_object_copies":202818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":1806,"num_objects_unfound":0,"num_objects_dirty":101409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65784,"num_write_kb":263110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1803,"num_bytes_recovered":7364627,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101489,"ondisk_log_size":101489,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1291380,"kb_used_data":817344,"kb_used_omap":12,"kb_used_meta":473971,"kb_avail":281824140,"statfs":{"total":289910292480,"available":288587919360,"internally_reserved":0,"allocated":836960256,"data_stored":836189061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":485346306},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,5,4,11,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23273472,"num_objects":5682,"num_object_clones":0,"num_object_copies":11364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-270,"num_objects_unfound":0,"num_objects_dirty":5682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5682,"num_write_kb":22728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":1105920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004105"},"pg_stats":[{"pgid":"2.f","version":"43'6322","reported_seq":6395,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:16.396641+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:31:16.396641+0000","last_peered":"2024-09-10T20:31:16.396641+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:31:16.396641+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":6322,"log_dups_size":0,"ondisk_log_size":6322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25894912,"num_objects":6322,"num_object_clones":0,"num_object_copies":12644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6322,"num_write_kb":25288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2715},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6469","reported_seq":6573,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:18.197233+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:31:18.197233+0000","last_peered":"2024-09-10T20:31:18.197233+0000","last_clean":"2024-09-10T20:31:18.197233+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:31:18.197233+0000","last_undegraded":"2024-09-10T20:31:18.197233+0000","last_fullsized":"2024-09-10T20:31:18.197233+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":6469,"log_dups_size":0,"ondisk_log_size":6469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26472448,"num_objects":6463,"num_object_clones":0,"num_object_copies":12926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4655,"num_write_kb":18620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6388","reported_seq":10109,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:31:18.240405+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:31:18.240405+0000","last_peered":"2024-09-10T20:31:18.240405+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:31:18.240405+0000","last_undegraded":"2024-09-10T20:31:18.240405+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":6388,"log_dups_size":0,"ondisk_log_size":6388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26148864,"num_objects":6384,"num_object_clones":0,"num_object_copies":12768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1806,"num_objects_unfound":0,"num_objects_dirty":6384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4516,"num_write_kb":18064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1798,"num_bytes_recovered":7360512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1806},{"shards":"2","objects":4578}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6258","reported_seq":6333,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:18.172898+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:31:18.172898+0000","last_peered":"2024-09-10T20:31:18.172898+0000","last_clean":"2024-09-10T20:31:18.172898+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:31:18.172898+0000","last_undegraded":"2024-09-10T20:31:18.172898+0000","last_fullsized":"2024-09-10T20:31:18.172898+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":6258,"log_dups_size":0,"ondisk_log_size":6258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25632768,"num_objects":6258,"num_object_clones":0,"num_object_copies":12516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6258,"num_write_kb":25032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6429","reported_seq":6551,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:16.378917+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:31:16.378917+0000","last_peered":"2024-09-10T20:31:16.378917+0000","last_clean":"2024-09-10T20:31:16.378917+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:31:16.378917+0000","last_undegraded":"2024-09-10T20:31:16.378917+0000","last_fullsized":"2024-09-10T20:31:16.378917+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":6429,"log_dups_size":0,"ondisk_log_size":6429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26304512,"num_objects":6422,"num_object_clones":0,"num_object_copies":12844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6422,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3425,"num_write_kb":13700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6312","reported_seq":6444,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:16.376186+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:31:16.376186+0000","last_peered":"2024-09-10T20:31:16.376186+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:31:16.376186+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":6312,"log_dups_size":0,"ondisk_log_size":6312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25833472,"num_objects":6307,"num_object_clones":0,"num_object_copies":12614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3176,"num_write_kb":12704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2693},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6230","reported_seq":6363,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:16.379294+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:31:16.379294+0000","last_peered":"2024-09-10T20:31:16.379294+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:31:16.379294+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":6230,"log_dups_size":0,"ondisk_log_size":6230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25493504,"num_objects":6224,"num_object_clones":0,"num_object_copies":12448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2825,"num_write_kb":11300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2669},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6399","reported_seq":6519,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:16.378706+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:31:16.378706+0000","last_peered":"2024-09-10T20:31:16.378706+0000","last_clean":"2024-09-10T20:31:16.378706+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:31:16.378706+0000","last_undegraded":"2024-09-10T20:31:16.378706+0000","last_fullsized":"2024-09-10T20:31:16.378706+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":6399,"log_dups_size":0,"ondisk_log_size":6399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26202112,"num_objects":6397,"num_object_clones":0,"num_object_copies":12794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3470,"num_write_kb":13880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6237","reported_seq":6364,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:18.215564+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:31:18.215564+0000","last_peered":"2024-09-10T20:31:18.215564+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:31:18.215564+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":6237,"log_dups_size":0,"ondisk_log_size":6237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25518080,"num_objects":6230,"num_object_clones":0,"num_object_copies":12460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4230,"num_write_kb":16920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":2704}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355479+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355479+0000","last_peered":"2024-09-10T20:30:05.355479+0000","last_clean":"2024-09-10T20:30:05.355479+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355479+0000","last_undegraded":"2024-09-10T20:30:05.355479+0000","last_fullsized":"2024-09-10T20:30:05.355479+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6344","reported_seq":6447,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:16.401276+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:31:16.401276+0000","last_peered":"2024-09-10T20:31:16.401276+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:31:16.401276+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":6344,"log_dups_size":0,"ondisk_log_size":6344,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25953222,"num_objects":6338,"num_object_clones":0,"num_object_copies":12676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6342,"num_write_kb":25348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2723},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6337","reported_seq":6460,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:18.229834+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:31:18.229834+0000","last_peered":"2024-09-10T20:31:18.229834+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:31:18.229834+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":6337,"log_dups_size":0,"ondisk_log_size":6337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25935872,"num_objects":6332,"num_object_clones":0,"num_object_copies":12664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4165,"num_write_kb":16660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":2699}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6415","reported_seq":6539,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:18.224519+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:31:18.224519+0000","last_peered":"2024-09-10T20:31:18.224519+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:31:18.224519+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":6415,"log_dups_size":0,"ondisk_log_size":6415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26251264,"num_objects":6409,"num_object_clones":0,"num_object_copies":12818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4201,"num_write_kb":16804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":2799}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6334","reported_seq":6462,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:16.407688+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:31:16.407688+0000","last_peered":"2024-09-10T20:31:16.407688+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:31:16.407688+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":6334,"log_dups_size":0,"ondisk_log_size":6334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25935872,"num_objects":6332,"num_object_clones":0,"num_object_copies":12664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3057,"num_write_kb":12228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2684},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6313","reported_seq":6446,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:16.421200+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:31:16.421200+0000","last_peered":"2024-09-10T20:31:16.421200+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:31:16.421200+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":6313,"log_dups_size":0,"ondisk_log_size":6313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25825280,"num_objects":6305,"num_object_clones":0,"num_object_copies":12610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3007,"num_write_kb":12028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2773},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345743+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345743+0000","last_peered":"2024-09-10T20:30:05.345743+0000","last_clean":"2024-09-10T20:30:05.345743+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345743+0000","last_undegraded":"2024-09-10T20:30:05.345743+0000","last_fullsized":"2024-09-10T20:30:05.345743+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6446","reported_seq":6573,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:16.298134+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:31:16.298134+0000","last_peered":"2024-09-10T20:31:16.298134+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:31:16.298134+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":6446,"log_dups_size":0,"ondisk_log_size":6446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26386432,"num_objects":6442,"num_object_clones":0,"num_object_copies":12884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3273,"num_write_kb":13092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2729},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6253","reported_seq":6414,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:16.363816+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:31:16.363816+0000","last_peered":"2024-09-10T20:31:16.363816+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:31:16.363816+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":6253,"log_dups_size":0,"ondisk_log_size":6253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25563619,"num_objects":6242,"num_object_clones":0,"num_object_copies":12484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6242,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2860,"num_write_kb":11440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2774},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":415352233,"num_objects":101407,"num_object_clones":0,"num_object_copies":202814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":1806,"num_objects_unfound":0,"num_objects_dirty":101407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65782,"num_write_kb":263108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1801,"num_bytes_recovered":7364608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":839573504,"data_stored":839486792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101486,"ondisk_log_size":101486,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182433,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":532104,"kb_used_data":360248,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93839736,"statfs":{"total":96636764160,"available":96091889664,"internally_reserved":0,"allocated":368893952,"data_stored":368660819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":175959722},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,0,10,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986205,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499128,"kb_used_data":318884,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93872712,"statfs":{"total":96636764160,"available":96125657088,"internally_reserved":0,"allocated":326537216,"data_stored":326308160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184544938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659152,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":260148,"kb_used_data":138212,"kb_used_omap":4,"kb_used_meta":121915,"kb_avail":94111692,"statfs":{"total":96636764160,"available":96370372608,"internally_reserved":0,"allocated":141529088,"data_stored":141220082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":124841646},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,1,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":369872896,"data_stored":369869283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":142942208,"data_stored":142862722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":326758400,"data_stored":326754787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:31:22.058 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 114 20 104162 104142 3.56757 2.06641 0.0297646 0.0218914 2024-09-10T20:31:23.058 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 115 20 104758 104738 3.55679 2.32812 0.0170105 0.0219575 2024-09-10T20:31:24.058 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 116 20 105343 105323 3.54583 2.28516 0.0206969 0.0220251 2024-09-10T20:31:24.679 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:31:25.058 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 117 20 105907 105887 3.53435 2.20312 0.0315982 0.0220972 2024-09-10T20:31:25.207 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:25.208 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:31:25.233 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":135,"stamp":"2024-09-10T20:31:24.856008+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":426911164,"num_objects":104231,"num_object_clones":0,"num_object_copies":208462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":1683,"num_objects_unfound":0,"num_objects_dirty":104231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68606,"num_write_kb":274398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1926,"num_bytes_recovered":7868435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104311,"ondisk_log_size":104311,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1350740,"kb_used_data":842020,"kb_used_omap":12,"kb_used_meta":508659,"kb_avail":281764780,"statfs":{"total":289910292480,"available":288527134720,"internally_reserved":0,"allocated":862228480,"data_stored":861457285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":520866818},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,8,8,18,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27594752,"num_objects":6737,"num_object_clones":0,"num_object_copies":13474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-393,"num_objects_unfound":0,"num_objects_dirty":6737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6737,"num_write_kb":26948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":394,"num_bytes_recovered":1609728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004505"},"pg_stats":[{"pgid":"2.f","version":"43'6494","reported_seq":6567,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:21.478823+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:31:21.478823+0000","last_peered":"2024-09-10T20:31:21.478823+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:31:21.478823+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":6494,"log_dups_size":0,"ondisk_log_size":6494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26599424,"num_objects":6494,"num_object_clones":0,"num_object_copies":12988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6494,"num_write_kb":25976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2887},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6654","reported_seq":6759,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:23.229694+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:31:23.229694+0000","last_peered":"2024-09-10T20:31:23.229694+0000","last_clean":"2024-09-10T20:31:23.229694+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:31:23.229694+0000","last_undegraded":"2024-09-10T20:31:23.229694+0000","last_fullsized":"2024-09-10T20:31:23.229694+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":6654,"log_dups_size":0,"ondisk_log_size":6654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27230208,"num_objects":6648,"num_object_clones":0,"num_object_copies":13296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4840,"num_write_kb":19360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6574","reported_seq":10541,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:31:23.229762+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:31:23.229762+0000","last_peered":"2024-09-10T20:31:23.229762+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:31:23.229762+0000","last_undegraded":"2024-09-10T20:31:23.229762+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":6574,"log_dups_size":0,"ondisk_log_size":6574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26910720,"num_objects":6570,"num_object_clones":0,"num_object_copies":13140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1683,"num_objects_unfound":0,"num_objects_dirty":6570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4702,"num_write_kb":18808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1921,"num_bytes_recovered":7864320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1683},{"shards":"2","objects":4887}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6420","reported_seq":6495,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:23.172864+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:31:23.172864+0000","last_peered":"2024-09-10T20:31:23.172864+0000","last_clean":"2024-09-10T20:31:23.172864+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:31:23.172864+0000","last_undegraded":"2024-09-10T20:31:23.172864+0000","last_fullsized":"2024-09-10T20:31:23.172864+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":6420,"log_dups_size":0,"ondisk_log_size":6420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26296320,"num_objects":6420,"num_object_clones":0,"num_object_copies":12840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6420,"num_write_kb":25680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6611","reported_seq":6733,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:21.349949+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:31:21.349949+0000","last_peered":"2024-09-10T20:31:21.349949+0000","last_clean":"2024-09-10T20:31:21.349949+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:31:21.349949+0000","last_undegraded":"2024-09-10T20:31:21.349949+0000","last_fullsized":"2024-09-10T20:31:21.349949+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":6611,"log_dups_size":0,"ondisk_log_size":6611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27049984,"num_objects":6604,"num_object_clones":0,"num_object_copies":13208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3607,"num_write_kb":14428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6488","reported_seq":6620,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:21.378024+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:31:21.378024+0000","last_peered":"2024-09-10T20:31:21.378024+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:31:21.378024+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":6488,"log_dups_size":0,"ondisk_log_size":6488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26554368,"num_objects":6483,"num_object_clones":0,"num_object_copies":12966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3352,"num_write_kb":13408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2869},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6419","reported_seq":6552,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:21.384604+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:31:21.384604+0000","last_peered":"2024-09-10T20:31:21.384604+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:31:21.384604+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":6419,"log_dups_size":0,"ondisk_log_size":6419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26267648,"num_objects":6413,"num_object_clones":0,"num_object_copies":12826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3014,"num_write_kb":12056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2858},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6553","reported_seq":6673,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:21.449130+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:31:21.449130+0000","last_peered":"2024-09-10T20:31:21.449130+0000","last_clean":"2024-09-10T20:31:21.449130+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:31:21.449130+0000","last_undegraded":"2024-09-10T20:31:21.449130+0000","last_fullsized":"2024-09-10T20:31:21.449130+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":6553,"log_dups_size":0,"ondisk_log_size":6553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26832896,"num_objects":6551,"num_object_clones":0,"num_object_copies":13102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3624,"num_write_kb":14496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6411","reported_seq":6538,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:23.170225+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:31:23.170225+0000","last_peered":"2024-09-10T20:31:23.170225+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:31:23.170225+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":6411,"log_dups_size":0,"ondisk_log_size":6411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26230784,"num_objects":6404,"num_object_clones":0,"num_object_copies":12808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4404,"num_write_kb":17616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":2878}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355479+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355479+0000","last_peered":"2024-09-10T20:30:05.355479+0000","last_clean":"2024-09-10T20:30:05.355479+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355479+0000","last_undegraded":"2024-09-10T20:30:05.355479+0000","last_fullsized":"2024-09-10T20:30:05.355479+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6525","reported_seq":6629,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:21.409958+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:31:21.409958+0000","last_peered":"2024-09-10T20:31:21.409958+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:31:21.409958+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":6525,"log_dups_size":0,"ondisk_log_size":6525,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26694598,"num_objects":6519,"num_object_clones":0,"num_object_copies":13038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6523,"num_write_kb":26072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2904},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6497","reported_seq":6621,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:23.233328+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:31:23.233328+0000","last_peered":"2024-09-10T20:31:23.233328+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:31:23.233328+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":6497,"log_dups_size":0,"ondisk_log_size":6497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26591232,"num_objects":6492,"num_object_clones":0,"num_object_copies":12984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4325,"num_write_kb":17300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":2859}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6589","reported_seq":6713,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:23.208949+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:31:23.208949+0000","last_peered":"2024-09-10T20:31:23.208949+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:31:23.208949+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":6589,"log_dups_size":0,"ondisk_log_size":6589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26963968,"num_objects":6583,"num_object_clones":0,"num_object_copies":13166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4375,"num_write_kb":17500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":2973}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6508","reported_seq":6633,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:21.412978+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:31:21.412978+0000","last_peered":"2024-09-10T20:31:21.412978+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:31:21.412978+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":6508,"log_dups_size":0,"ondisk_log_size":6508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26648576,"num_objects":6506,"num_object_clones":0,"num_object_copies":13012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6506,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3231,"num_write_kb":12924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2858},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6502","reported_seq":6634,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:21.405168+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:31:21.405168+0000","last_peered":"2024-09-10T20:31:21.405168+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:31:21.405168+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":6502,"log_dups_size":0,"ondisk_log_size":6502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26599424,"num_objects":6494,"num_object_clones":0,"num_object_copies":12988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3196,"num_write_kb":12784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2962},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345743+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345743+0000","last_peered":"2024-09-10T20:30:05.345743+0000","last_clean":"2024-09-10T20:30:05.345743+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345743+0000","last_undegraded":"2024-09-10T20:30:05.345743+0000","last_fullsized":"2024-09-10T20:30:05.345743+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6632","reported_seq":6759,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:21.461005+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:31:21.461005+0000","last_peered":"2024-09-10T20:31:21.461005+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:31:21.461005+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":6632,"log_dups_size":0,"ondisk_log_size":6632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27148288,"num_objects":6628,"num_object_clones":0,"num_object_copies":13256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3459,"num_write_kb":13836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2915},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6431","reported_seq":6592,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:21.478411+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:31:21.478411+0000","last_peered":"2024-09-10T20:31:21.478411+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:31:21.478411+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":6431,"log_dups_size":0,"ondisk_log_size":6431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26292707,"num_objects":6420,"num_object_clones":0,"num_object_copies":12840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3038,"num_write_kb":12152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2952},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":426911145,"num_objects":104229,"num_object_clones":0,"num_object_copies":208458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":1683,"num_objects_unfound":0,"num_objects_dirty":104229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68604,"num_write_kb":274396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1924,"num_bytes_recovered":7868416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":863019008,"data_stored":862932296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104308,"ondisk_log_size":104308,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182434,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562852,"kb_used_data":368788,"kb_used_omap":4,"kb_used_meta":194043,"kb_avail":93808988,"statfs":{"total":96636764160,"available":96060403712,"internally_reserved":0,"allocated":377638912,"data_stored":377405779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":198700714},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,9],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986206,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508796,"kb_used_data":324396,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93863044,"statfs":{"total":96636764160,"available":96115757056,"internally_reserved":0,"allocated":332181504,"data_stored":331952448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188804778},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659153,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":279092,"kb_used_data":148836,"kb_used_omap":4,"kb_used_meta":130235,"kb_avail":94092748,"statfs":{"total":96636764160,"available":96350973952,"internally_reserved":0,"allocated":152408064,"data_stored":152099058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":133361326},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":378556416,"data_stored":378552803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":152084480,"data_stored":152004994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":332378112,"data_stored":332374499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:31:25.235 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:31:25.791 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:25.792 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:31:25.817 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":135,"stamp":"2024-09-10T20:31:24.856008+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":426911164,"num_objects":104231,"num_object_clones":0,"num_object_copies":208462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":1683,"num_objects_unfound":0,"num_objects_dirty":104231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68606,"num_write_kb":274398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1926,"num_bytes_recovered":7868435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104311,"ondisk_log_size":104311,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1350740,"kb_used_data":842020,"kb_used_omap":12,"kb_used_meta":508659,"kb_avail":281764780,"statfs":{"total":289910292480,"available":288527134720,"internally_reserved":0,"allocated":862228480,"data_stored":861457285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":520866818},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,8,8,18,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27594752,"num_objects":6737,"num_object_clones":0,"num_object_copies":13474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-393,"num_objects_unfound":0,"num_objects_dirty":6737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6737,"num_write_kb":26948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":394,"num_bytes_recovered":1609728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004505"},"pg_stats":[{"pgid":"2.f","version":"43'6494","reported_seq":6567,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:21.478823+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:31:21.478823+0000","last_peered":"2024-09-10T20:31:21.478823+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:31:21.478823+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":6494,"log_dups_size":0,"ondisk_log_size":6494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26599424,"num_objects":6494,"num_object_clones":0,"num_object_copies":12988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6494,"num_write_kb":25976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2887},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6654","reported_seq":6759,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:23.229694+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:31:23.229694+0000","last_peered":"2024-09-10T20:31:23.229694+0000","last_clean":"2024-09-10T20:31:23.229694+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:31:23.229694+0000","last_undegraded":"2024-09-10T20:31:23.229694+0000","last_fullsized":"2024-09-10T20:31:23.229694+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":6654,"log_dups_size":0,"ondisk_log_size":6654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27230208,"num_objects":6648,"num_object_clones":0,"num_object_copies":13296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4840,"num_write_kb":19360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6574","reported_seq":10541,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:31:23.229762+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:31:23.229762+0000","last_peered":"2024-09-10T20:31:23.229762+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:31:23.229762+0000","last_undegraded":"2024-09-10T20:31:23.229762+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":6574,"log_dups_size":0,"ondisk_log_size":6574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26910720,"num_objects":6570,"num_object_clones":0,"num_object_copies":13140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1683,"num_objects_unfound":0,"num_objects_dirty":6570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4702,"num_write_kb":18808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1921,"num_bytes_recovered":7864320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1683},{"shards":"2","objects":4887}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6420","reported_seq":6495,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:23.172864+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:31:23.172864+0000","last_peered":"2024-09-10T20:31:23.172864+0000","last_clean":"2024-09-10T20:31:23.172864+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:31:23.172864+0000","last_undegraded":"2024-09-10T20:31:23.172864+0000","last_fullsized":"2024-09-10T20:31:23.172864+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":6420,"log_dups_size":0,"ondisk_log_size":6420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26296320,"num_objects":6420,"num_object_clones":0,"num_object_copies":12840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6420,"num_write_kb":25680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6611","reported_seq":6733,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:21.349949+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:31:21.349949+0000","last_peered":"2024-09-10T20:31:21.349949+0000","last_clean":"2024-09-10T20:31:21.349949+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:31:21.349949+0000","last_undegraded":"2024-09-10T20:31:21.349949+0000","last_fullsized":"2024-09-10T20:31:21.349949+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":6611,"log_dups_size":0,"ondisk_log_size":6611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27049984,"num_objects":6604,"num_object_clones":0,"num_object_copies":13208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3607,"num_write_kb":14428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6488","reported_seq":6620,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:21.378024+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:31:21.378024+0000","last_peered":"2024-09-10T20:31:21.378024+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:31:21.378024+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":6488,"log_dups_size":0,"ondisk_log_size":6488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26554368,"num_objects":6483,"num_object_clones":0,"num_object_copies":12966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3352,"num_write_kb":13408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2869},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6419","reported_seq":6552,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:21.384604+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:31:21.384604+0000","last_peered":"2024-09-10T20:31:21.384604+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:31:21.384604+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":6419,"log_dups_size":0,"ondisk_log_size":6419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26267648,"num_objects":6413,"num_object_clones":0,"num_object_copies":12826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3014,"num_write_kb":12056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2858},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6553","reported_seq":6673,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:21.449130+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:31:21.449130+0000","last_peered":"2024-09-10T20:31:21.449130+0000","last_clean":"2024-09-10T20:31:21.449130+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:31:21.449130+0000","last_undegraded":"2024-09-10T20:31:21.449130+0000","last_fullsized":"2024-09-10T20:31:21.449130+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":6553,"log_dups_size":0,"ondisk_log_size":6553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26832896,"num_objects":6551,"num_object_clones":0,"num_object_copies":13102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3624,"num_write_kb":14496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6411","reported_seq":6538,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:23.170225+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:31:23.170225+0000","last_peered":"2024-09-10T20:31:23.170225+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:31:23.170225+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":6411,"log_dups_size":0,"ondisk_log_size":6411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26230784,"num_objects":6404,"num_object_clones":0,"num_object_copies":12808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4404,"num_write_kb":17616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":2878}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355479+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355479+0000","last_peered":"2024-09-10T20:30:05.355479+0000","last_clean":"2024-09-10T20:30:05.355479+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355479+0000","last_undegraded":"2024-09-10T20:30:05.355479+0000","last_fullsized":"2024-09-10T20:30:05.355479+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6525","reported_seq":6629,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:21.409958+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:31:21.409958+0000","last_peered":"2024-09-10T20:31:21.409958+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:31:21.409958+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":6525,"log_dups_size":0,"ondisk_log_size":6525,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26694598,"num_objects":6519,"num_object_clones":0,"num_object_copies":13038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6523,"num_write_kb":26072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2904},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6497","reported_seq":6621,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:23.233328+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:31:23.233328+0000","last_peered":"2024-09-10T20:31:23.233328+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:31:23.233328+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":6497,"log_dups_size":0,"ondisk_log_size":6497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26591232,"num_objects":6492,"num_object_clones":0,"num_object_copies":12984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4325,"num_write_kb":17300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":2859}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6589","reported_seq":6713,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:23.208949+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:31:23.208949+0000","last_peered":"2024-09-10T20:31:23.208949+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:31:23.208949+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":6589,"log_dups_size":0,"ondisk_log_size":6589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26963968,"num_objects":6583,"num_object_clones":0,"num_object_copies":13166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4375,"num_write_kb":17500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":2973}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6508","reported_seq":6633,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:21.412978+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:31:21.412978+0000","last_peered":"2024-09-10T20:31:21.412978+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:31:21.412978+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":6508,"log_dups_size":0,"ondisk_log_size":6508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26648576,"num_objects":6506,"num_object_clones":0,"num_object_copies":13012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6506,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3231,"num_write_kb":12924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2858},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6502","reported_seq":6634,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:21.405168+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:31:21.405168+0000","last_peered":"2024-09-10T20:31:21.405168+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:31:21.405168+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":6502,"log_dups_size":0,"ondisk_log_size":6502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26599424,"num_objects":6494,"num_object_clones":0,"num_object_copies":12988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3196,"num_write_kb":12784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2962},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345743+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345743+0000","last_peered":"2024-09-10T20:30:05.345743+0000","last_clean":"2024-09-10T20:30:05.345743+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345743+0000","last_undegraded":"2024-09-10T20:30:05.345743+0000","last_fullsized":"2024-09-10T20:30:05.345743+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6632","reported_seq":6759,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:21.461005+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:31:21.461005+0000","last_peered":"2024-09-10T20:31:21.461005+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:31:21.461005+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":6632,"log_dups_size":0,"ondisk_log_size":6632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27148288,"num_objects":6628,"num_object_clones":0,"num_object_copies":13256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3459,"num_write_kb":13836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2915},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6431","reported_seq":6592,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:21.478411+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:31:21.478411+0000","last_peered":"2024-09-10T20:31:21.478411+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:31:21.478411+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":6431,"log_dups_size":0,"ondisk_log_size":6431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26292707,"num_objects":6420,"num_object_clones":0,"num_object_copies":12840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3038,"num_write_kb":12152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2952},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":426911145,"num_objects":104229,"num_object_clones":0,"num_object_copies":208458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":1683,"num_objects_unfound":0,"num_objects_dirty":104229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68604,"num_write_kb":274396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1924,"num_bytes_recovered":7868416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":863019008,"data_stored":862932296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104308,"ondisk_log_size":104308,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182434,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562852,"kb_used_data":368788,"kb_used_omap":4,"kb_used_meta":194043,"kb_avail":93808988,"statfs":{"total":96636764160,"available":96060403712,"internally_reserved":0,"allocated":377638912,"data_stored":377405779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":198700714},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,9],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986206,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508796,"kb_used_data":324396,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93863044,"statfs":{"total":96636764160,"available":96115757056,"internally_reserved":0,"allocated":332181504,"data_stored":331952448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188804778},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659153,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":279092,"kb_used_data":148836,"kb_used_omap":4,"kb_used_meta":130235,"kb_avail":94092748,"statfs":{"total":96636764160,"available":96350973952,"internally_reserved":0,"allocated":152408064,"data_stored":152099058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":133361326},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":378556416,"data_stored":378552803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":152084480,"data_stored":152004994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":332378112,"data_stored":332374499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:31:25.819 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:31:25.819 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'6494", 'reported_seq': 6567, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:21.478823+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:31:21.478823+0000', 'last_peered': '2024-09-10T20:31:21.478823+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:31:21.478823+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 6494, 'log_dups_size': 0, 'ondisk_log_size': 6494, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26599424, 'num_objects': 6494, 'num_object_clones': 0, 'num_object_copies': 12988, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3607, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6494, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6494, 'num_write_kb': 25976, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2887}, {'shards': '0,2', 'objects': 3607}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:25.819 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T20:31:25.819 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'6574", 'reported_seq': 10541, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:31:23.229762+0000', 'last_change': '2024-09-10T20:30:05.653560+0000', 'last_active': '2024-09-10T20:31:23.229762+0000', 'last_peered': '2024-09-10T20:31:23.229762+0000', 'last_clean': '2024-09-10T20:30:03.279259+0000', 'last_became_active': '2024-09-10T20:30:05.629664+0000', 'last_became_peered': '2024-09-10T20:30:05.629664+0000', 'last_unstale': '2024-09-10T20:31:23.229762+0000', 'last_undegraded': '2024-09-10T20:31:23.229762+0000', 'last_fullsized': '2024-09-10T20:30:05.312061+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1910", 'last_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'objects_scrubbed': 1868, 'log_size': 6574, 'log_dups_size': 0, 'ondisk_log_size': 6574, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 673, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26910720, 'num_objects': 6570, 'num_object_clones': 0, 'num_object_copies': 13140, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1683, 'num_objects_unfound': 0, 'num_objects_dirty': 6570, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4702, 'num_write_kb': 18808, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1921, 'num_bytes_recovered': 7864320, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1683}, {'shards': '2', 'objects': 4887}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:25.819 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:31:25.819 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'6488", 'reported_seq': 6620, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:21.378024+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:31:21.378024+0000', 'last_peered': '2024-09-10T20:31:21.378024+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:31:21.378024+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 6488, 'log_dups_size': 0, 'ondisk_log_size': 6488, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26554368, 'num_objects': 6483, 'num_object_clones': 0, 'num_object_copies': 12966, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3616, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6483, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3352, 'num_write_kb': 13408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2869}, {'shards': '0,2', 'objects': 3614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:25.819 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:31:25.819 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'6419", 'reported_seq': 6552, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:21.384604+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:31:21.384604+0000', 'last_peered': '2024-09-10T20:31:21.384604+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:31:21.384604+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 6419, 'log_dups_size': 0, 'ondisk_log_size': 6419, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26267648, 'num_objects': 6413, 'num_object_clones': 0, 'num_object_copies': 12826, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3557, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6413, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3014, 'num_write_kb': 12056, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2858}, {'shards': '0,2', 'objects': 3555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:25.819 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:31:25.820 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'6411", 'reported_seq': 6538, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:23.170225+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:31:23.170225+0000', 'last_peered': '2024-09-10T20:31:23.170225+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:31:23.170225+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 6411, 'log_dups_size': 0, 'ondisk_log_size': 6411, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26230784, 'num_objects': 6404, 'num_object_clones': 0, 'num_object_copies': 12808, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3528, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6404, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4404, 'num_write_kb': 17616, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 2878}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:25.820 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:31:25.820 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'6525", 'reported_seq': 6629, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:21.409958+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:31:21.409958+0000', 'last_peered': '2024-09-10T20:31:21.409958+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:31:21.409958+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 6525, 'log_dups_size': 0, 'ondisk_log_size': 6525, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26694598, 'num_objects': 6519, 'num_object_clones': 0, 'num_object_copies': 13038, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3617, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6519, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6523, 'num_write_kb': 26072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2904}, {'shards': '0,2', 'objects': 3615}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:25.820 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:31:25.820 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'6497", 'reported_seq': 6621, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:23.233328+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:31:23.233328+0000', 'last_peered': '2024-09-10T20:31:23.233328+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:31:23.233328+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 6497, 'log_dups_size': 0, 'ondisk_log_size': 6497, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26591232, 'num_objects': 6492, 'num_object_clones': 0, 'num_object_copies': 12984, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3635, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6492, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4325, 'num_write_kb': 17300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3633}, {'shards': '2', 'objects': 2859}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:25.820 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:31:25.820 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'6589", 'reported_seq': 6713, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:23.208949+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:31:23.208949+0000', 'last_peered': '2024-09-10T20:31:23.208949+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:31:23.208949+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 6589, 'log_dups_size': 0, 'ondisk_log_size': 6589, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26963968, 'num_objects': 6583, 'num_object_clones': 0, 'num_object_copies': 13166, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6583, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4375, 'num_write_kb': 17500, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3610}, {'shards': '2', 'objects': 2973}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:25.820 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:31:25.820 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'6508", 'reported_seq': 6633, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:21.412978+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:31:21.412978+0000', 'last_peered': '2024-09-10T20:31:21.412978+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:31:21.412978+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 6508, 'log_dups_size': 0, 'ondisk_log_size': 6508, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26648576, 'num_objects': 6506, 'num_object_clones': 0, 'num_object_copies': 13012, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3649, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6506, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3231, 'num_write_kb': 12924, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2858}, {'shards': '0,2', 'objects': 3648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:25.820 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:31:25.820 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'6502", 'reported_seq': 6634, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:21.405168+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:31:21.405168+0000', 'last_peered': '2024-09-10T20:31:21.405168+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:31:21.405168+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 6502, 'log_dups_size': 0, 'ondisk_log_size': 6502, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26599424, 'num_objects': 6494, 'num_object_clones': 0, 'num_object_copies': 12988, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3535, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6494, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3196, 'num_write_kb': 12784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2962}, {'shards': '0,2', 'objects': 3532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:25.821 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:31:25.821 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'6632", 'reported_seq': 6759, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:21.461005+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:31:21.461005+0000', 'last_peered': '2024-09-10T20:31:21.461005+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:31:21.461005+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 6632, 'log_dups_size': 0, 'ondisk_log_size': 6632, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27148288, 'num_objects': 6628, 'num_object_clones': 0, 'num_object_copies': 13256, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3715, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6628, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3459, 'num_write_kb': 13836, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2915}, {'shards': '0,2', 'objects': 3713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:25.821 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:31:25.821 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'6431", 'reported_seq': 6592, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:21.478411+0000', 'last_change': '2024-09-10T20:30:05.690055+0000', 'last_active': '2024-09-10T20:31:21.478411+0000', 'last_peered': '2024-09-10T20:31:21.478411+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:31:21.478411+0000', 'last_undegraded': '2024-09-10T20:30:05.378941+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 6431, 'log_dups_size': 0, 'ondisk_log_size': 6431, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26292707, 'num_objects': 6420, 'num_object_clones': 0, 'num_object_copies': 12840, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3471, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6420, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3038, 'num_write_kb': 12152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2952}, {'shards': '0,2', 'objects': 3468}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:25.821 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:31:26.059 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 118 20 106414 106394 3.52119 1.98047 0.0366182 0.0221794 2024-09-10T20:31:26.526 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:26.526 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39542/208462 objects degraded (18.968%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":198,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":104231,"data_bytes":426911164,"bytes_used":1383157760,"bytes_avail":288527134720,"bytes_total":289910292480,"degraded_objects":39542,"degraded_total":208462,"degraded_ratio":0.1896844508831346,"misplaced_objects":1683,"misplaced_total":208462,"misplaced_ratio":0.0080734138596003108,"recovering_objects_per_sec":32,"recovering_bytes_per_sec":134093,"recovering_keys_per_sec":0,"num_objects_recovered":394,"num_bytes_recovered":1609728,"num_keys_recovered":0,"write_bytes_sec":2298699,"read_op_per_sec":0,"write_op_per_sec":561},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:31:26.549 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:31:26.549 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:31:27.059 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 119 20 106927 106907 3.50844 2.00391 0.0344439 0.0222602 2024-09-10T20:31:27.122 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:27.123 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:31:27.151 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2024-09-10T20:31:26.856506+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":433976764,"num_objects":105956,"num_object_clones":0,"num_object_copies":211912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":1683,"num_objects_unfound":0,"num_objects_dirty":105956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":70331,"num_write_kb":281298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1926,"num_bytes_recovered":7868435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106036,"ondisk_log_size":106036,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1367316,"kb_used_data":850272,"kb_used_omap":12,"kb_used_meta":516979,"kb_avail":281748204,"statfs":{"total":289910292480,"available":288510160896,"internally_reserved":0,"allocated":870678528,"data_stored":869907333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":529386498},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,6,12,12],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30273536,"num_objects":7391,"num_object_clones":0,"num_object_copies":14782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-269,"num_objects_unfound":0,"num_objects_dirty":7391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7391,"num_write_kb":29564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":1101824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004362"},"pg_stats":[{"pgid":"2.f","version":"43'6664","reported_seq":6740,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:26.488180+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:31:26.488180+0000","last_peered":"2024-09-10T20:31:26.488180+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:31:26.488180+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":6664,"log_dups_size":0,"ondisk_log_size":6664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27295744,"num_objects":6664,"num_object_clones":0,"num_object_copies":13328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6664,"num_write_kb":26656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3057},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6654","reported_seq":6759,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:23.229694+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:31:23.229694+0000","last_peered":"2024-09-10T20:31:23.229694+0000","last_clean":"2024-09-10T20:31:23.229694+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:31:23.229694+0000","last_undegraded":"2024-09-10T20:31:23.229694+0000","last_fullsized":"2024-09-10T20:31:23.229694+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":6654,"log_dups_size":0,"ondisk_log_size":6654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27230208,"num_objects":6648,"num_object_clones":0,"num_object_copies":13296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4840,"num_write_kb":19360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6574","reported_seq":10541,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:31:23.229762+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:31:23.229762+0000","last_peered":"2024-09-10T20:31:23.229762+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:31:23.229762+0000","last_undegraded":"2024-09-10T20:31:23.229762+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":6574,"log_dups_size":0,"ondisk_log_size":6574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26910720,"num_objects":6570,"num_object_clones":0,"num_object_copies":13140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1683,"num_objects_unfound":0,"num_objects_dirty":6570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4702,"num_write_kb":18808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1921,"num_bytes_recovered":7864320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1683},{"shards":"2","objects":4887}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6420","reported_seq":6495,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:23.172864+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:31:23.172864+0000","last_peered":"2024-09-10T20:31:23.172864+0000","last_clean":"2024-09-10T20:31:23.172864+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:31:23.172864+0000","last_undegraded":"2024-09-10T20:31:23.172864+0000","last_fullsized":"2024-09-10T20:31:23.172864+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":6420,"log_dups_size":0,"ondisk_log_size":6420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26296320,"num_objects":6420,"num_object_clones":0,"num_object_copies":12840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6420,"num_write_kb":25680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6786","reported_seq":6908,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:26.422506+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:31:26.422506+0000","last_peered":"2024-09-10T20:31:26.422506+0000","last_clean":"2024-09-10T20:31:26.422506+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:31:26.422506+0000","last_undegraded":"2024-09-10T20:31:26.422506+0000","last_fullsized":"2024-09-10T20:31:26.422506+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":6786,"log_dups_size":0,"ondisk_log_size":6786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27766784,"num_objects":6779,"num_object_clones":0,"num_object_copies":13558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3782,"num_write_kb":15128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6660","reported_seq":6792,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:26.501269+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:31:26.501269+0000","last_peered":"2024-09-10T20:31:26.501269+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:31:26.501269+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":6660,"log_dups_size":0,"ondisk_log_size":6660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27258880,"num_objects":6655,"num_object_clones":0,"num_object_copies":13310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3524,"num_write_kb":14096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3041},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6581","reported_seq":6714,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:26.374404+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:31:26.374404+0000","last_peered":"2024-09-10T20:31:26.374404+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:31:26.374404+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":6581,"log_dups_size":0,"ondisk_log_size":6581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26931200,"num_objects":6575,"num_object_clones":0,"num_object_copies":13150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3176,"num_write_kb":12704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3020},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6721","reported_seq":6841,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:26.451398+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:31:26.451398+0000","last_peered":"2024-09-10T20:31:26.451398+0000","last_clean":"2024-09-10T20:31:26.451398+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:31:26.451398+0000","last_undegraded":"2024-09-10T20:31:26.451398+0000","last_fullsized":"2024-09-10T20:31:26.451398+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":6721,"log_dups_size":0,"ondisk_log_size":6721,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27521024,"num_objects":6719,"num_object_clones":0,"num_object_copies":13438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3792,"num_write_kb":15168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6411","reported_seq":6538,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:23.170225+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:31:23.170225+0000","last_peered":"2024-09-10T20:31:23.170225+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:31:23.170225+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":6411,"log_dups_size":0,"ondisk_log_size":6411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26230784,"num_objects":6404,"num_object_clones":0,"num_object_copies":12808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4404,"num_write_kb":17616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":2878}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355479+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355479+0000","last_peered":"2024-09-10T20:30:05.355479+0000","last_clean":"2024-09-10T20:30:05.355479+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355479+0000","last_undegraded":"2024-09-10T20:30:05.355479+0000","last_fullsized":"2024-09-10T20:30:05.355479+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6700","reported_seq":6803,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:26.520781+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:31:26.520781+0000","last_peered":"2024-09-10T20:31:26.520781+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:31:26.520781+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":6700,"log_dups_size":0,"ondisk_log_size":6700,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27411398,"num_objects":6694,"num_object_clones":0,"num_object_copies":13388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6698,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3079},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6497","reported_seq":6621,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:23.233328+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:31:23.233328+0000","last_peered":"2024-09-10T20:31:23.233328+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:31:23.233328+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":6497,"log_dups_size":0,"ondisk_log_size":6497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26591232,"num_objects":6492,"num_object_clones":0,"num_object_copies":12984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4325,"num_write_kb":17300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":2859}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6589","reported_seq":6713,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:23.208949+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:31:23.208949+0000","last_peered":"2024-09-10T20:31:23.208949+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:31:23.208949+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":6589,"log_dups_size":0,"ondisk_log_size":6589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26963968,"num_objects":6583,"num_object_clones":0,"num_object_copies":13166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4375,"num_write_kb":17500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":2973}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6694","reported_seq":6824,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:26.487946+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:31:26.487946+0000","last_peered":"2024-09-10T20:31:26.487946+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:31:26.487946+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":6694,"log_dups_size":0,"ondisk_log_size":6694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27410432,"num_objects":6692,"num_object_clones":0,"num_object_copies":13384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3417,"num_write_kb":13668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3044},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6669","reported_seq":6803,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:26.533502+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:31:26.533502+0000","last_peered":"2024-09-10T20:31:26.533502+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:31:26.533502+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":6669,"log_dups_size":0,"ondisk_log_size":6669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27283456,"num_objects":6661,"num_object_clones":0,"num_object_copies":13322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3363,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3129},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345743+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345743+0000","last_peered":"2024-09-10T20:30:05.345743+0000","last_clean":"2024-09-10T20:30:05.345743+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345743+0000","last_undegraded":"2024-09-10T20:30:05.345743+0000","last_fullsized":"2024-09-10T20:30:05.345743+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6808","reported_seq":6935,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:26.434530+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:31:26.434530+0000","last_peered":"2024-09-10T20:31:26.434530+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:31:26.434530+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":6808,"log_dups_size":0,"ondisk_log_size":6808,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27869184,"num_objects":6804,"num_object_clones":0,"num_object_copies":13608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3635,"num_write_kb":14540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3091},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6605","reported_seq":6766,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:26.476884+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:31:26.476884+0000","last_peered":"2024-09-10T20:31:26.476884+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:31:26.476884+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":6605,"log_dups_size":0,"ondisk_log_size":6605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27005411,"num_objects":6594,"num_object_clones":0,"num_object_copies":13188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6594,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3212,"num_write_kb":12848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3126},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":433976745,"num_objects":105954,"num_object_clones":0,"num_object_copies":211908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":1683,"num_objects_unfound":0,"num_objects_dirty":105954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":70329,"num_write_kb":281296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1924,"num_bytes_recovered":7868416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":871665664,"data_stored":871578952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106033,"ondisk_log_size":106033,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182435,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":579428,"kb_used_data":377040,"kb_used_omap":4,"kb_used_meta":202363,"kb_avail":93792412,"statfs":{"total":96636764160,"available":96043429888,"internally_reserved":0,"allocated":386088960,"data_stored":385855827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":207220394},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,3,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986206,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508796,"kb_used_data":324396,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93863044,"statfs":{"total":96636764160,"available":96115757056,"internally_reserved":0,"allocated":332181504,"data_stored":331952448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188804778},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659153,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":279092,"kb_used_data":148836,"kb_used_omap":4,"kb_used_meta":130235,"kb_avail":94092748,"statfs":{"total":96636764160,"available":96350973952,"internally_reserved":0,"allocated":152408064,"data_stored":152099058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":133361326},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":387203072,"data_stored":387199459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":152084480,"data_stored":152004994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":332378112,"data_stored":332374499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:31:28.061 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout:2024-09-10T20:31:28.056148+0000 min lat: 0.00191414 max lat: 2.51074 avg lat: 0.0223291 2024-09-10T20:31:28.061 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T20:31:28.061 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 120 20 107493 107473 3.49762 2.21094 0.0514475 0.0223291 2024-09-10T20:31:29.059 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 121 20 108071 108051 3.48738 2.25781 0.0301701 0.0223943 2024-09-10T20:31:30.059 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 122 20 108647 108627 3.47724 2.25 0.011978 0.0224581 2024-09-10T20:31:30.154 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:31:30.703 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:30.704 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:31:30.731 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":137,"stamp":"2024-09-10T20:31:28.857004+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":438412732,"num_objects":107039,"num_object_clones":0,"num_object_copies":214078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":1545,"num_objects_unfound":0,"num_objects_dirty":107039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":71414,"num_write_kb":285630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2064,"num_bytes_recovered":8433683,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107119,"ondisk_log_size":107119,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1389676,"kb_used_data":862904,"kb_used_omap":12,"kb_used_meta":526707,"kb_avail":281725844,"statfs":{"total":289910292480,"available":288487264256,"internally_reserved":0,"allocated":883613696,"data_stored":882842501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":539347970},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,2,3,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27381760,"num_objects":6685,"num_object_clones":0,"num_object_copies":13370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-407,"num_objects_unfound":0,"num_objects_dirty":6685,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6685,"num_write_kb":26740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":408,"num_bytes_recovered":1667072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004096"},"pg_stats":[{"pgid":"2.f","version":"43'6664","reported_seq":6740,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:26.488180+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:31:26.488180+0000","last_peered":"2024-09-10T20:31:26.488180+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:31:26.488180+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":6664,"log_dups_size":0,"ondisk_log_size":6664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27295744,"num_objects":6664,"num_object_clones":0,"num_object_copies":13328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6664,"num_write_kb":26656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3057},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6846","reported_seq":6952,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:28.278198+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:31:28.278198+0000","last_peered":"2024-09-10T20:31:28.278198+0000","last_clean":"2024-09-10T20:31:28.278198+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:31:28.278198+0000","last_undegraded":"2024-09-10T20:31:28.278198+0000","last_fullsized":"2024-09-10T20:31:28.278198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":6846,"log_dups_size":0,"ondisk_log_size":6846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28016640,"num_objects":6840,"num_object_clones":0,"num_object_copies":13680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5032,"num_write_kb":20128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6737","reported_seq":10979,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:31:28.281487+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:31:28.281487+0000","last_peered":"2024-09-10T20:31:28.281487+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:31:28.281487+0000","last_undegraded":"2024-09-10T20:31:28.281487+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":6737,"log_dups_size":0,"ondisk_log_size":6737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27578368,"num_objects":6733,"num_object_clones":0,"num_object_copies":13466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1545,"num_objects_unfound":0,"num_objects_dirty":6733,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4865,"num_write_kb":19460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2059,"num_bytes_recovered":8429568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1545},{"shards":"2","objects":5188}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6602","reported_seq":6678,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:28.251436+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:31:28.251436+0000","last_peered":"2024-09-10T20:31:28.251436+0000","last_clean":"2024-09-10T20:31:28.251436+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:31:28.251436+0000","last_undegraded":"2024-09-10T20:31:28.251436+0000","last_fullsized":"2024-09-10T20:31:28.251436+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":6602,"log_dups_size":0,"ondisk_log_size":6602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27041792,"num_objects":6602,"num_object_clones":0,"num_object_copies":13204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6602,"num_write_kb":26408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6786","reported_seq":6908,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:26.422506+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:31:26.422506+0000","last_peered":"2024-09-10T20:31:26.422506+0000","last_clean":"2024-09-10T20:31:26.422506+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:31:26.422506+0000","last_undegraded":"2024-09-10T20:31:26.422506+0000","last_fullsized":"2024-09-10T20:31:26.422506+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":6786,"log_dups_size":0,"ondisk_log_size":6786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27766784,"num_objects":6779,"num_object_clones":0,"num_object_copies":13558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3782,"num_write_kb":15128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6660","reported_seq":6792,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:26.501269+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:31:26.501269+0000","last_peered":"2024-09-10T20:31:26.501269+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:31:26.501269+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":6660,"log_dups_size":0,"ondisk_log_size":6660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27258880,"num_objects":6655,"num_object_clones":0,"num_object_copies":13310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3524,"num_write_kb":14096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3041},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6581","reported_seq":6714,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:26.374404+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:31:26.374404+0000","last_peered":"2024-09-10T20:31:26.374404+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:31:26.374404+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":6581,"log_dups_size":0,"ondisk_log_size":6581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26931200,"num_objects":6575,"num_object_clones":0,"num_object_copies":13150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3176,"num_write_kb":12704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3020},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6721","reported_seq":6841,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:26.451398+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:31:26.451398+0000","last_peered":"2024-09-10T20:31:26.451398+0000","last_clean":"2024-09-10T20:31:26.451398+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:31:26.451398+0000","last_undegraded":"2024-09-10T20:31:26.451398+0000","last_fullsized":"2024-09-10T20:31:26.451398+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":6721,"log_dups_size":0,"ondisk_log_size":6721,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27521024,"num_objects":6719,"num_object_clones":0,"num_object_copies":13438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3792,"num_write_kb":15168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6591","reported_seq":6718,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:28.285926+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:31:28.285926+0000","last_peered":"2024-09-10T20:31:28.285926+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:31:28.285926+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":6591,"log_dups_size":0,"ondisk_log_size":6591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26968064,"num_objects":6584,"num_object_clones":0,"num_object_copies":13168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4584,"num_write_kb":18336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":3058}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355479+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355479+0000","last_peered":"2024-09-10T20:30:05.355479+0000","last_clean":"2024-09-10T20:30:05.355479+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355479+0000","last_undegraded":"2024-09-10T20:30:05.355479+0000","last_fullsized":"2024-09-10T20:30:05.355479+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6700","reported_seq":6803,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:26.520781+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:31:26.520781+0000","last_peered":"2024-09-10T20:31:26.520781+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:31:26.520781+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":6700,"log_dups_size":0,"ondisk_log_size":6700,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27411398,"num_objects":6694,"num_object_clones":0,"num_object_copies":13388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6698,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3079},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6677","reported_seq":6801,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:28.288389+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:31:28.288389+0000","last_peered":"2024-09-10T20:31:28.288389+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:31:28.288389+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":6677,"log_dups_size":0,"ondisk_log_size":6677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27328512,"num_objects":6672,"num_object_clones":0,"num_object_copies":13344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4505,"num_write_kb":18020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":3039}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6775","reported_seq":6899,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:28.231406+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:31:28.231406+0000","last_peered":"2024-09-10T20:31:28.231406+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:31:28.231406+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":6775,"log_dups_size":0,"ondisk_log_size":6775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27725824,"num_objects":6769,"num_object_clones":0,"num_object_copies":13538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4561,"num_write_kb":18244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":3159}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6694","reported_seq":6824,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:26.487946+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:31:26.487946+0000","last_peered":"2024-09-10T20:31:26.487946+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:31:26.487946+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":6694,"log_dups_size":0,"ondisk_log_size":6694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27410432,"num_objects":6692,"num_object_clones":0,"num_object_copies":13384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3417,"num_write_kb":13668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3044},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6669","reported_seq":6803,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:26.533502+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:31:26.533502+0000","last_peered":"2024-09-10T20:31:26.533502+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:31:26.533502+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":6669,"log_dups_size":0,"ondisk_log_size":6669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27283456,"num_objects":6661,"num_object_clones":0,"num_object_copies":13322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3363,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3129},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345743+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345743+0000","last_peered":"2024-09-10T20:30:05.345743+0000","last_clean":"2024-09-10T20:30:05.345743+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345743+0000","last_undegraded":"2024-09-10T20:30:05.345743+0000","last_fullsized":"2024-09-10T20:30:05.345743+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6808","reported_seq":6935,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:26.434530+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:31:26.434530+0000","last_peered":"2024-09-10T20:31:26.434530+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:31:26.434530+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":6808,"log_dups_size":0,"ondisk_log_size":6808,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27869184,"num_objects":6804,"num_object_clones":0,"num_object_copies":13608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3635,"num_write_kb":14540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3091},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6605","reported_seq":6766,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:26.476884+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:31:26.476884+0000","last_peered":"2024-09-10T20:31:26.476884+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:31:26.476884+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":6605,"log_dups_size":0,"ondisk_log_size":6605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27005411,"num_objects":6594,"num_object_clones":0,"num_object_copies":13188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6594,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3212,"num_write_kb":12848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3126},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":438412713,"num_objects":107037,"num_object_clones":0,"num_object_copies":214074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":1545,"num_objects_unfound":0,"num_objects_dirty":107037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":71412,"num_write_kb":285628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2062,"num_bytes_recovered":8433664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":886419456,"data_stored":886332744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107116,"ondisk_log_size":107116,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182435,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":579428,"kb_used_data":377040,"kb_used_omap":4,"kb_used_meta":202363,"kb_avail":93792412,"statfs":{"total":96636764160,"available":96043429888,"internally_reserved":0,"allocated":386088960,"data_stored":385855827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":207220394},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,3,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986207,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501928,"kb_used_data":330008,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93869912,"statfs":{"total":96636764160,"available":96122789888,"internally_reserved":0,"allocated":337928192,"data_stored":337699136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":176025258},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659154,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308320,"kb_used_data":155856,"kb_used_omap":4,"kb_used_meta":152443,"kb_avail":94063520,"statfs":{"total":96636764160,"available":96321044480,"internally_reserved":0,"allocated":159596544,"data_stored":159287538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":156102318},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":387203072,"data_stored":387199459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":161005568,"data_stored":160926082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":338210816,"data_stored":338207203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:31:30.732 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:31:31.059 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 123 20 109154 109134 3.46507 1.98047 0.0293234 0.0225383 2024-09-10T20:31:31.257 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:31.257 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:31:31.292 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":138,"stamp":"2024-09-10T20:31:30.857675+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":438412732,"num_objects":107039,"num_object_clones":0,"num_object_copies":214078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":1545,"num_objects_unfound":0,"num_objects_dirty":107039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":71414,"num_write_kb":285630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2064,"num_bytes_recovered":8433683,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107119,"ondisk_log_size":107119,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1389676,"kb_used_data":862904,"kb_used_omap":12,"kb_used_meta":526707,"kb_avail":281725844,"statfs":{"total":289910292480,"available":288487264256,"internally_reserved":0,"allocated":883613696,"data_stored":882842501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":539347970},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,2,3,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23060480,"num_objects":5630,"num_object_clones":0,"num_object_copies":11260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-261,"num_objects_unfound":0,"num_objects_dirty":5630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5630,"num_write_kb":22520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":1069056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003993"},"pg_stats":[{"pgid":"2.f","version":"43'6664","reported_seq":6740,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:26.488180+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:31:26.488180+0000","last_peered":"2024-09-10T20:31:26.488180+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:31:26.488180+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":6664,"log_dups_size":0,"ondisk_log_size":6664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27295744,"num_objects":6664,"num_object_clones":0,"num_object_copies":13328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6664,"num_write_kb":26656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3057},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6846","reported_seq":6952,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:28.278198+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:31:28.278198+0000","last_peered":"2024-09-10T20:31:28.278198+0000","last_clean":"2024-09-10T20:31:28.278198+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:31:28.278198+0000","last_undegraded":"2024-09-10T20:31:28.278198+0000","last_fullsized":"2024-09-10T20:31:28.278198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":6846,"log_dups_size":0,"ondisk_log_size":6846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28016640,"num_objects":6840,"num_object_clones":0,"num_object_copies":13680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5032,"num_write_kb":20128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6737","reported_seq":10979,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:31:28.281487+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:31:28.281487+0000","last_peered":"2024-09-10T20:31:28.281487+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:31:28.281487+0000","last_undegraded":"2024-09-10T20:31:28.281487+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":6737,"log_dups_size":0,"ondisk_log_size":6737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27578368,"num_objects":6733,"num_object_clones":0,"num_object_copies":13466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1545,"num_objects_unfound":0,"num_objects_dirty":6733,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4865,"num_write_kb":19460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2059,"num_bytes_recovered":8429568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1545},{"shards":"2","objects":5188}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6602","reported_seq":6678,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:28.251436+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:31:28.251436+0000","last_peered":"2024-09-10T20:31:28.251436+0000","last_clean":"2024-09-10T20:31:28.251436+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:31:28.251436+0000","last_undegraded":"2024-09-10T20:31:28.251436+0000","last_fullsized":"2024-09-10T20:31:28.251436+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":6602,"log_dups_size":0,"ondisk_log_size":6602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27041792,"num_objects":6602,"num_object_clones":0,"num_object_copies":13204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6602,"num_write_kb":26408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6786","reported_seq":6908,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:26.422506+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:31:26.422506+0000","last_peered":"2024-09-10T20:31:26.422506+0000","last_clean":"2024-09-10T20:31:26.422506+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:31:26.422506+0000","last_undegraded":"2024-09-10T20:31:26.422506+0000","last_fullsized":"2024-09-10T20:31:26.422506+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":6786,"log_dups_size":0,"ondisk_log_size":6786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27766784,"num_objects":6779,"num_object_clones":0,"num_object_copies":13558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3782,"num_write_kb":15128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6660","reported_seq":6792,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:26.501269+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:31:26.501269+0000","last_peered":"2024-09-10T20:31:26.501269+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:31:26.501269+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":6660,"log_dups_size":0,"ondisk_log_size":6660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27258880,"num_objects":6655,"num_object_clones":0,"num_object_copies":13310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3524,"num_write_kb":14096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3041},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6581","reported_seq":6714,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:26.374404+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:31:26.374404+0000","last_peered":"2024-09-10T20:31:26.374404+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:31:26.374404+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":6581,"log_dups_size":0,"ondisk_log_size":6581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26931200,"num_objects":6575,"num_object_clones":0,"num_object_copies":13150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3176,"num_write_kb":12704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3020},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6721","reported_seq":6841,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:26.451398+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:31:26.451398+0000","last_peered":"2024-09-10T20:31:26.451398+0000","last_clean":"2024-09-10T20:31:26.451398+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:31:26.451398+0000","last_undegraded":"2024-09-10T20:31:26.451398+0000","last_fullsized":"2024-09-10T20:31:26.451398+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":6721,"log_dups_size":0,"ondisk_log_size":6721,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27521024,"num_objects":6719,"num_object_clones":0,"num_object_copies":13438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3792,"num_write_kb":15168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6591","reported_seq":6718,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:28.285926+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:31:28.285926+0000","last_peered":"2024-09-10T20:31:28.285926+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:31:28.285926+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":6591,"log_dups_size":0,"ondisk_log_size":6591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26968064,"num_objects":6584,"num_object_clones":0,"num_object_copies":13168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4584,"num_write_kb":18336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":3058}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355479+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355479+0000","last_peered":"2024-09-10T20:30:05.355479+0000","last_clean":"2024-09-10T20:30:05.355479+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355479+0000","last_undegraded":"2024-09-10T20:30:05.355479+0000","last_fullsized":"2024-09-10T20:30:05.355479+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6700","reported_seq":6803,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:26.520781+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:31:26.520781+0000","last_peered":"2024-09-10T20:31:26.520781+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:31:26.520781+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":6700,"log_dups_size":0,"ondisk_log_size":6700,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27411398,"num_objects":6694,"num_object_clones":0,"num_object_copies":13388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6698,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3079},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6677","reported_seq":6801,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:28.288389+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:31:28.288389+0000","last_peered":"2024-09-10T20:31:28.288389+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:31:28.288389+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":6677,"log_dups_size":0,"ondisk_log_size":6677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27328512,"num_objects":6672,"num_object_clones":0,"num_object_copies":13344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4505,"num_write_kb":18020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":3039}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6775","reported_seq":6899,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:28.231406+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:31:28.231406+0000","last_peered":"2024-09-10T20:31:28.231406+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:31:28.231406+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":6775,"log_dups_size":0,"ondisk_log_size":6775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27725824,"num_objects":6769,"num_object_clones":0,"num_object_copies":13538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4561,"num_write_kb":18244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":3159}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6694","reported_seq":6824,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:26.487946+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:31:26.487946+0000","last_peered":"2024-09-10T20:31:26.487946+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:31:26.487946+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":6694,"log_dups_size":0,"ondisk_log_size":6694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27410432,"num_objects":6692,"num_object_clones":0,"num_object_copies":13384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3417,"num_write_kb":13668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3044},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6669","reported_seq":6803,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:26.533502+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:31:26.533502+0000","last_peered":"2024-09-10T20:31:26.533502+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:31:26.533502+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":6669,"log_dups_size":0,"ondisk_log_size":6669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27283456,"num_objects":6661,"num_object_clones":0,"num_object_copies":13322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3363,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3129},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345743+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345743+0000","last_peered":"2024-09-10T20:30:05.345743+0000","last_clean":"2024-09-10T20:30:05.345743+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345743+0000","last_undegraded":"2024-09-10T20:30:05.345743+0000","last_fullsized":"2024-09-10T20:30:05.345743+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6808","reported_seq":6935,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:26.434530+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:31:26.434530+0000","last_peered":"2024-09-10T20:31:26.434530+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:31:26.434530+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":6808,"log_dups_size":0,"ondisk_log_size":6808,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27869184,"num_objects":6804,"num_object_clones":0,"num_object_copies":13608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3635,"num_write_kb":14540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3091},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6605","reported_seq":6766,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:26.476884+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:31:26.476884+0000","last_peered":"2024-09-10T20:31:26.476884+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:31:26.476884+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":6605,"log_dups_size":0,"ondisk_log_size":6605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27005411,"num_objects":6594,"num_object_clones":0,"num_object_copies":13188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6594,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3212,"num_write_kb":12848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3126},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":438412713,"num_objects":107037,"num_object_clones":0,"num_object_copies":214074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":1545,"num_objects_unfound":0,"num_objects_dirty":107037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":71412,"num_write_kb":285628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2062,"num_bytes_recovered":8433664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":886419456,"data_stored":886332744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107116,"ondisk_log_size":107116,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182435,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":579428,"kb_used_data":377040,"kb_used_omap":4,"kb_used_meta":202363,"kb_avail":93792412,"statfs":{"total":96636764160,"available":96043429888,"internally_reserved":0,"allocated":386088960,"data_stored":385855827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":207220394},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,3,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986207,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501928,"kb_used_data":330008,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93869912,"statfs":{"total":96636764160,"available":96122789888,"internally_reserved":0,"allocated":337928192,"data_stored":337699136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":176025258},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659154,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308320,"kb_used_data":155856,"kb_used_omap":4,"kb_used_meta":152443,"kb_avail":94063520,"statfs":{"total":96636764160,"available":96321044480,"internally_reserved":0,"allocated":159596544,"data_stored":159287538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":156102318},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":387203072,"data_stored":387199459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":161005568,"data_stored":160926082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":338210816,"data_stored":338207203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:31:31.293 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:31:31.293 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'6664", 'reported_seq': 6740, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:26.488180+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:31:26.488180+0000', 'last_peered': '2024-09-10T20:31:26.488180+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:31:26.488180+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 6664, 'log_dups_size': 0, 'ondisk_log_size': 6664, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27295744, 'num_objects': 6664, 'num_object_clones': 0, 'num_object_copies': 13328, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3607, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6664, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6664, 'num_write_kb': 26656, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3057}, {'shards': '0,2', 'objects': 3607}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:31.293 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T20:31:31.293 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'6737", 'reported_seq': 10979, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:31:28.281487+0000', 'last_change': '2024-09-10T20:30:05.653560+0000', 'last_active': '2024-09-10T20:31:28.281487+0000', 'last_peered': '2024-09-10T20:31:28.281487+0000', 'last_clean': '2024-09-10T20:30:03.279259+0000', 'last_became_active': '2024-09-10T20:30:05.629664+0000', 'last_became_peered': '2024-09-10T20:30:05.629664+0000', 'last_unstale': '2024-09-10T20:31:28.281487+0000', 'last_undegraded': '2024-09-10T20:31:28.281487+0000', 'last_fullsized': '2024-09-10T20:30:05.312061+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1910", 'last_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'objects_scrubbed': 1868, 'log_size': 6737, 'log_dups_size': 0, 'ondisk_log_size': 6737, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 673, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27578368, 'num_objects': 6733, 'num_object_clones': 0, 'num_object_copies': 13466, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1545, 'num_objects_unfound': 0, 'num_objects_dirty': 6733, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4865, 'num_write_kb': 19460, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2059, 'num_bytes_recovered': 8429568, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1545}, {'shards': '2', 'objects': 5188}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:31.293 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:31:31.293 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'6660", 'reported_seq': 6792, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:26.501269+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:31:26.501269+0000', 'last_peered': '2024-09-10T20:31:26.501269+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:31:26.501269+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 6660, 'log_dups_size': 0, 'ondisk_log_size': 6660, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27258880, 'num_objects': 6655, 'num_object_clones': 0, 'num_object_copies': 13310, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3616, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6655, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3524, 'num_write_kb': 14096, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3041}, {'shards': '0,2', 'objects': 3614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:31.294 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:31:31.294 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'6581", 'reported_seq': 6714, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:26.374404+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:31:26.374404+0000', 'last_peered': '2024-09-10T20:31:26.374404+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:31:26.374404+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 6581, 'log_dups_size': 0, 'ondisk_log_size': 6581, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26931200, 'num_objects': 6575, 'num_object_clones': 0, 'num_object_copies': 13150, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3557, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6575, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3176, 'num_write_kb': 12704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3020}, {'shards': '0,2', 'objects': 3555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:31.294 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:31:31.294 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'6591", 'reported_seq': 6718, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:28.285926+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:31:28.285926+0000', 'last_peered': '2024-09-10T20:31:28.285926+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:31:28.285926+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 6591, 'log_dups_size': 0, 'ondisk_log_size': 6591, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26968064, 'num_objects': 6584, 'num_object_clones': 0, 'num_object_copies': 13168, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3528, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6584, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4584, 'num_write_kb': 18336, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 3058}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:31.294 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:31:31.294 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'6700", 'reported_seq': 6803, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:26.520781+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:31:26.520781+0000', 'last_peered': '2024-09-10T20:31:26.520781+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:31:26.520781+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 6700, 'log_dups_size': 0, 'ondisk_log_size': 6700, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27411398, 'num_objects': 6694, 'num_object_clones': 0, 'num_object_copies': 13388, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3617, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6694, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6698, 'num_write_kb': 26772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3079}, {'shards': '0,2', 'objects': 3615}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:31.294 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:31:31.294 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'6677", 'reported_seq': 6801, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:28.288389+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:31:28.288389+0000', 'last_peered': '2024-09-10T20:31:28.288389+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:31:28.288389+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 6677, 'log_dups_size': 0, 'ondisk_log_size': 6677, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27328512, 'num_objects': 6672, 'num_object_clones': 0, 'num_object_copies': 13344, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3635, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6672, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4505, 'num_write_kb': 18020, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3633}, {'shards': '2', 'objects': 3039}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:31.294 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:31:31.294 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'6775", 'reported_seq': 6899, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:28.231406+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:31:28.231406+0000', 'last_peered': '2024-09-10T20:31:28.231406+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:31:28.231406+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 6775, 'log_dups_size': 0, 'ondisk_log_size': 6775, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27725824, 'num_objects': 6769, 'num_object_clones': 0, 'num_object_copies': 13538, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6769, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4561, 'num_write_kb': 18244, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3610}, {'shards': '2', 'objects': 3159}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:31.294 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:31:31.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'6694", 'reported_seq': 6824, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:26.487946+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:31:26.487946+0000', 'last_peered': '2024-09-10T20:31:26.487946+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:31:26.487946+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 6694, 'log_dups_size': 0, 'ondisk_log_size': 6694, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27410432, 'num_objects': 6692, 'num_object_clones': 0, 'num_object_copies': 13384, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3649, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6692, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3417, 'num_write_kb': 13668, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3044}, {'shards': '0,2', 'objects': 3648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:31.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:31:31.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'6669", 'reported_seq': 6803, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:26.533502+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:31:26.533502+0000', 'last_peered': '2024-09-10T20:31:26.533502+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:31:26.533502+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 6669, 'log_dups_size': 0, 'ondisk_log_size': 6669, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27283456, 'num_objects': 6661, 'num_object_clones': 0, 'num_object_copies': 13322, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3535, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6661, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3363, 'num_write_kb': 13452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3129}, {'shards': '0,2', 'objects': 3532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:31.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:31:31.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'6808", 'reported_seq': 6935, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:26.434530+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:31:26.434530+0000', 'last_peered': '2024-09-10T20:31:26.434530+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:31:26.434530+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 6808, 'log_dups_size': 0, 'ondisk_log_size': 6808, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27869184, 'num_objects': 6804, 'num_object_clones': 0, 'num_object_copies': 13608, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3715, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6804, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3635, 'num_write_kb': 14540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3091}, {'shards': '0,2', 'objects': 3713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:31.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:31:31.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'6605", 'reported_seq': 6766, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:26.476884+0000', 'last_change': '2024-09-10T20:30:05.690055+0000', 'last_active': '2024-09-10T20:31:26.476884+0000', 'last_peered': '2024-09-10T20:31:26.476884+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:31:26.476884+0000', 'last_undegraded': '2024-09-10T20:30:05.378941+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 6605, 'log_dups_size': 0, 'ondisk_log_size': 6605, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27005411, 'num_objects': 6594, 'num_object_clones': 0, 'num_object_copies': 13188, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3471, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6594, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3212, 'num_write_kb': 12848, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3126}, {'shards': '0,2', 'objects': 3468}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:31.296 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:31:31.932 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:31.932 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39542/214078 objects degraded (18.471%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":203,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":107039,"data_bytes":438412732,"bytes_used":1423028224,"bytes_avail":288487264256,"bytes_total":289910292480,"degraded_objects":39542,"degraded_total":214078,"degraded_ratio":0.18470837732041592,"misplaced_objects":1545,"misplaced_total":214078,"misplaced_ratio":0.0072169956744737897,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":89058,"recovering_keys_per_sec":0,"num_objects_recovered":261,"num_bytes_recovered":1069056,"num_keys_recovered":0,"write_bytes_sec":1921067,"read_op_per_sec":0,"write_op_per_sec":469},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:31:31.957 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:31:31.958 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:31:32.059 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 124 20 109607 109587 3.45139 1.76953 0.0328665 0.0226263 2024-09-10T20:31:32.530 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:32.530 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:31:32.552 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":138,"stamp":"2024-09-10T20:31:30.857675+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":438412732,"num_objects":107039,"num_object_clones":0,"num_object_copies":214078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":1545,"num_objects_unfound":0,"num_objects_dirty":107039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":71414,"num_write_kb":285630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2064,"num_bytes_recovered":8433683,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107119,"ondisk_log_size":107119,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1389676,"kb_used_data":862904,"kb_used_omap":12,"kb_used_meta":526707,"kb_avail":281725844,"statfs":{"total":289910292480,"available":288487264256,"internally_reserved":0,"allocated":883613696,"data_stored":882842501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":539347970},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,2,3,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23060480,"num_objects":5630,"num_object_clones":0,"num_object_copies":11260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-261,"num_objects_unfound":0,"num_objects_dirty":5630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5630,"num_write_kb":22520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":1069056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003993"},"pg_stats":[{"pgid":"2.f","version":"43'6664","reported_seq":6740,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:26.488180+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:31:26.488180+0000","last_peered":"2024-09-10T20:31:26.488180+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:31:26.488180+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":6664,"log_dups_size":0,"ondisk_log_size":6664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27295744,"num_objects":6664,"num_object_clones":0,"num_object_copies":13328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6664,"num_write_kb":26656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3057},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6846","reported_seq":6952,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:28.278198+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:31:28.278198+0000","last_peered":"2024-09-10T20:31:28.278198+0000","last_clean":"2024-09-10T20:31:28.278198+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:31:28.278198+0000","last_undegraded":"2024-09-10T20:31:28.278198+0000","last_fullsized":"2024-09-10T20:31:28.278198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":6846,"log_dups_size":0,"ondisk_log_size":6846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28016640,"num_objects":6840,"num_object_clones":0,"num_object_copies":13680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5032,"num_write_kb":20128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6737","reported_seq":10979,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:31:28.281487+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:31:28.281487+0000","last_peered":"2024-09-10T20:31:28.281487+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:31:28.281487+0000","last_undegraded":"2024-09-10T20:31:28.281487+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":6737,"log_dups_size":0,"ondisk_log_size":6737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27578368,"num_objects":6733,"num_object_clones":0,"num_object_copies":13466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1545,"num_objects_unfound":0,"num_objects_dirty":6733,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4865,"num_write_kb":19460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2059,"num_bytes_recovered":8429568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1545},{"shards":"2","objects":5188}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6602","reported_seq":6678,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:28.251436+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:31:28.251436+0000","last_peered":"2024-09-10T20:31:28.251436+0000","last_clean":"2024-09-10T20:31:28.251436+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:31:28.251436+0000","last_undegraded":"2024-09-10T20:31:28.251436+0000","last_fullsized":"2024-09-10T20:31:28.251436+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":6602,"log_dups_size":0,"ondisk_log_size":6602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27041792,"num_objects":6602,"num_object_clones":0,"num_object_copies":13204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6602,"num_write_kb":26408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6786","reported_seq":6908,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:26.422506+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:31:26.422506+0000","last_peered":"2024-09-10T20:31:26.422506+0000","last_clean":"2024-09-10T20:31:26.422506+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:31:26.422506+0000","last_undegraded":"2024-09-10T20:31:26.422506+0000","last_fullsized":"2024-09-10T20:31:26.422506+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":6786,"log_dups_size":0,"ondisk_log_size":6786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27766784,"num_objects":6779,"num_object_clones":0,"num_object_copies":13558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3782,"num_write_kb":15128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6660","reported_seq":6792,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:26.501269+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:31:26.501269+0000","last_peered":"2024-09-10T20:31:26.501269+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:31:26.501269+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":6660,"log_dups_size":0,"ondisk_log_size":6660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27258880,"num_objects":6655,"num_object_clones":0,"num_object_copies":13310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3524,"num_write_kb":14096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3041},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6581","reported_seq":6714,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:26.374404+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:31:26.374404+0000","last_peered":"2024-09-10T20:31:26.374404+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:31:26.374404+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":6581,"log_dups_size":0,"ondisk_log_size":6581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26931200,"num_objects":6575,"num_object_clones":0,"num_object_copies":13150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3176,"num_write_kb":12704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3020},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6721","reported_seq":6841,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:26.451398+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:31:26.451398+0000","last_peered":"2024-09-10T20:31:26.451398+0000","last_clean":"2024-09-10T20:31:26.451398+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:31:26.451398+0000","last_undegraded":"2024-09-10T20:31:26.451398+0000","last_fullsized":"2024-09-10T20:31:26.451398+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":6721,"log_dups_size":0,"ondisk_log_size":6721,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27521024,"num_objects":6719,"num_object_clones":0,"num_object_copies":13438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3792,"num_write_kb":15168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6591","reported_seq":6718,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:28.285926+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:31:28.285926+0000","last_peered":"2024-09-10T20:31:28.285926+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:31:28.285926+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":6591,"log_dups_size":0,"ondisk_log_size":6591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26968064,"num_objects":6584,"num_object_clones":0,"num_object_copies":13168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4584,"num_write_kb":18336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":3058}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355479+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355479+0000","last_peered":"2024-09-10T20:30:05.355479+0000","last_clean":"2024-09-10T20:30:05.355479+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355479+0000","last_undegraded":"2024-09-10T20:30:05.355479+0000","last_fullsized":"2024-09-10T20:30:05.355479+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6700","reported_seq":6803,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:26.520781+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:31:26.520781+0000","last_peered":"2024-09-10T20:31:26.520781+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:31:26.520781+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":6700,"log_dups_size":0,"ondisk_log_size":6700,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27411398,"num_objects":6694,"num_object_clones":0,"num_object_copies":13388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6698,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3079},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6677","reported_seq":6801,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:28.288389+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:31:28.288389+0000","last_peered":"2024-09-10T20:31:28.288389+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:31:28.288389+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":6677,"log_dups_size":0,"ondisk_log_size":6677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27328512,"num_objects":6672,"num_object_clones":0,"num_object_copies":13344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4505,"num_write_kb":18020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":3039}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6775","reported_seq":6899,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:28.231406+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:31:28.231406+0000","last_peered":"2024-09-10T20:31:28.231406+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:31:28.231406+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":6775,"log_dups_size":0,"ondisk_log_size":6775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27725824,"num_objects":6769,"num_object_clones":0,"num_object_copies":13538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4561,"num_write_kb":18244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":3159}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6694","reported_seq":6824,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:26.487946+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:31:26.487946+0000","last_peered":"2024-09-10T20:31:26.487946+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:31:26.487946+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":6694,"log_dups_size":0,"ondisk_log_size":6694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27410432,"num_objects":6692,"num_object_clones":0,"num_object_copies":13384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3417,"num_write_kb":13668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3044},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6669","reported_seq":6803,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:26.533502+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:31:26.533502+0000","last_peered":"2024-09-10T20:31:26.533502+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:31:26.533502+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":6669,"log_dups_size":0,"ondisk_log_size":6669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27283456,"num_objects":6661,"num_object_clones":0,"num_object_copies":13322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3363,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3129},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345743+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345743+0000","last_peered":"2024-09-10T20:30:05.345743+0000","last_clean":"2024-09-10T20:30:05.345743+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345743+0000","last_undegraded":"2024-09-10T20:30:05.345743+0000","last_fullsized":"2024-09-10T20:30:05.345743+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6808","reported_seq":6935,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:26.434530+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:31:26.434530+0000","last_peered":"2024-09-10T20:31:26.434530+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:31:26.434530+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":6808,"log_dups_size":0,"ondisk_log_size":6808,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27869184,"num_objects":6804,"num_object_clones":0,"num_object_copies":13608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3635,"num_write_kb":14540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3091},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6605","reported_seq":6766,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:26.476884+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:31:26.476884+0000","last_peered":"2024-09-10T20:31:26.476884+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:31:26.476884+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":6605,"log_dups_size":0,"ondisk_log_size":6605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27005411,"num_objects":6594,"num_object_clones":0,"num_object_copies":13188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6594,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3212,"num_write_kb":12848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3126},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":438412713,"num_objects":107037,"num_object_clones":0,"num_object_copies":214074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":1545,"num_objects_unfound":0,"num_objects_dirty":107037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":71412,"num_write_kb":285628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2062,"num_bytes_recovered":8433664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":886419456,"data_stored":886332744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107116,"ondisk_log_size":107116,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182435,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":579428,"kb_used_data":377040,"kb_used_omap":4,"kb_used_meta":202363,"kb_avail":93792412,"statfs":{"total":96636764160,"available":96043429888,"internally_reserved":0,"allocated":386088960,"data_stored":385855827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":207220394},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,3,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986207,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501928,"kb_used_data":330008,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93869912,"statfs":{"total":96636764160,"available":96122789888,"internally_reserved":0,"allocated":337928192,"data_stored":337699136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":176025258},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659154,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308320,"kb_used_data":155856,"kb_used_omap":4,"kb_used_meta":152443,"kb_avail":94063520,"statfs":{"total":96636764160,"available":96321044480,"internally_reserved":0,"allocated":159596544,"data_stored":159287538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":156102318},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":387203072,"data_stored":387199459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":161005568,"data_stored":160926082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":338210816,"data_stored":338207203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:31:33.059 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 125 20 110150 110130 3.44075 2.12109 0.0854824 0.0226962 2024-09-10T20:31:34.059 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 126 20 110732 110712 3.43149 2.27344 0.0312998 0.0227596 2024-09-10T20:31:35.059 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 127 20 111305 111285 3.42209 2.23828 0.0216485 0.022823 2024-09-10T20:31:35.555 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:31:36.059 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 128 20 111840 111820 3.41168 2.08984 0.0195442 0.0228899 2024-09-10T20:31:36.099 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:36.100 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:31:36.122 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2024-09-10T20:31:34.860566+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":449369532,"num_objects":109714,"num_object_clones":0,"num_object_copies":219428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":1418,"num_objects_unfound":0,"num_objects_dirty":109714,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74089,"num_write_kb":296330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2191,"num_bytes_recovered":8953875,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109794,"ondisk_log_size":109794,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1410696,"kb_used_data":886680,"kb_used_omap":12,"kb_used_meta":523955,"kb_avail":281704824,"statfs":{"total":289910292480,"available":288465739776,"internally_reserved":0,"allocated":907960320,"data_stored":907189125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":536529922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,10,4,2,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26722304,"num_objects":6524,"num_object_clones":0,"num_object_copies":13048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-388,"num_objects_unfound":0,"num_objects_dirty":6524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6524,"num_write_kb":26096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":388,"num_bytes_recovered":1589248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005208"},"pg_stats":[{"pgid":"2.f","version":"43'6842","reported_seq":6918,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:31.331842+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:31:31.331842+0000","last_peered":"2024-09-10T20:31:31.331842+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:31:31.331842+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":6842,"log_dups_size":0,"ondisk_log_size":6842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28024832,"num_objects":6842,"num_object_clones":0,"num_object_copies":13684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6842,"num_write_kb":27368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3235},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7010","reported_seq":7116,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:33.267964+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:31:33.267964+0000","last_peered":"2024-09-10T20:31:33.267964+0000","last_clean":"2024-09-10T20:31:33.267964+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:31:33.267964+0000","last_undegraded":"2024-09-10T20:31:33.267964+0000","last_fullsized":"2024-09-10T20:31:33.267964+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":7010,"log_dups_size":0,"ondisk_log_size":7010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28688384,"num_objects":7004,"num_object_clones":0,"num_object_copies":14008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5196,"num_write_kb":20784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6930","reported_seq":11426,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:31:33.270664+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:31:33.270664+0000","last_peered":"2024-09-10T20:31:33.270664+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:31:33.270664+0000","last_undegraded":"2024-09-10T20:31:33.270664+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":6930,"log_dups_size":0,"ondisk_log_size":6930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28368896,"num_objects":6926,"num_object_clones":0,"num_object_copies":13852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1418,"num_objects_unfound":0,"num_objects_dirty":6926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5058,"num_write_kb":20232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2186,"num_bytes_recovered":8949760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1418},{"shards":"2","objects":5508}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6763","reported_seq":6839,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:33.220433+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:31:33.220433+0000","last_peered":"2024-09-10T20:31:33.220433+0000","last_clean":"2024-09-10T20:31:33.220433+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:31:33.220433+0000","last_undegraded":"2024-09-10T20:31:33.220433+0000","last_fullsized":"2024-09-10T20:31:33.220433+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":6763,"log_dups_size":0,"ondisk_log_size":6763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27701248,"num_objects":6763,"num_object_clones":0,"num_object_copies":13526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6763,"num_write_kb":27052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6947","reported_seq":7069,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:31.514895+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:31:31.514895+0000","last_peered":"2024-09-10T20:31:31.514895+0000","last_clean":"2024-09-10T20:31:31.514895+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:31:31.514895+0000","last_undegraded":"2024-09-10T20:31:31.514895+0000","last_fullsized":"2024-09-10T20:31:31.514895+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":6947,"log_dups_size":0,"ondisk_log_size":6947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28426240,"num_objects":6940,"num_object_clones":0,"num_object_copies":13880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3943,"num_write_kb":15772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6823","reported_seq":6955,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:31.486488+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:31:31.486488+0000","last_peered":"2024-09-10T20:31:31.486488+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:31:31.486488+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":6823,"log_dups_size":0,"ondisk_log_size":6823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27926528,"num_objects":6818,"num_object_clones":0,"num_object_copies":13636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6818,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3687,"num_write_kb":14748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3204},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6748","reported_seq":6881,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:31.481027+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:31:31.481027+0000","last_peered":"2024-09-10T20:31:31.481027+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:31:31.481027+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":6748,"log_dups_size":0,"ondisk_log_size":6748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27615232,"num_objects":6742,"num_object_clones":0,"num_object_copies":13484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3343,"num_write_kb":13372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3187},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6889","reported_seq":7009,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:31.576771+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:31:31.576771+0000","last_peered":"2024-09-10T20:31:31.576771+0000","last_clean":"2024-09-10T20:31:31.576771+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:31:31.576771+0000","last_undegraded":"2024-09-10T20:31:31.576771+0000","last_fullsized":"2024-09-10T20:31:31.576771+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":6889,"log_dups_size":0,"ondisk_log_size":6889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28209152,"num_objects":6887,"num_object_clones":0,"num_object_copies":13774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3960,"num_write_kb":15840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6768","reported_seq":6895,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:33.302755+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:31:33.302755+0000","last_peered":"2024-09-10T20:31:33.302755+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:31:33.302755+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":6768,"log_dups_size":0,"ondisk_log_size":6768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27693056,"num_objects":6761,"num_object_clones":0,"num_object_copies":13522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4761,"num_write_kb":19044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":3235}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355479+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355479+0000","last_peered":"2024-09-10T20:30:05.355479+0000","last_clean":"2024-09-10T20:30:05.355479+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355479+0000","last_undegraded":"2024-09-10T20:30:05.355479+0000","last_fullsized":"2024-09-10T20:30:05.355479+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6868","reported_seq":6970,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:31.591904+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:31:31.591904+0000","last_peered":"2024-09-10T20:31:31.591904+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:31:31.591904+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":6868,"log_dups_size":0,"ondisk_log_size":6868,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28099526,"num_objects":6862,"num_object_clones":0,"num_object_copies":13724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6866,"num_write_kb":27444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3247},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6820","reported_seq":6944,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:33.243961+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:31:33.243961+0000","last_peered":"2024-09-10T20:31:33.243961+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:31:33.243961+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":6820,"log_dups_size":0,"ondisk_log_size":6820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27914240,"num_objects":6815,"num_object_clones":0,"num_object_copies":13630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4648,"num_write_kb":18592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":3182}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6940","reported_seq":7064,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:33.214303+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:31:33.214303+0000","last_peered":"2024-09-10T20:31:33.214303+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:31:33.214303+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":6940,"log_dups_size":0,"ondisk_log_size":6940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28401664,"num_objects":6934,"num_object_clones":0,"num_object_copies":13868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4726,"num_write_kb":18904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":3324}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6875","reported_seq":7005,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:31.551132+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:31:31.551132+0000","last_peered":"2024-09-10T20:31:31.551132+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:31:31.551132+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":6875,"log_dups_size":0,"ondisk_log_size":6875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28151808,"num_objects":6873,"num_object_clones":0,"num_object_copies":13746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3598,"num_write_kb":14392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3225},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6837","reported_seq":6971,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:31.601879+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:31:31.601879+0000","last_peered":"2024-09-10T20:31:31.601879+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:31:31.601879+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":6837,"log_dups_size":0,"ondisk_log_size":6837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27971584,"num_objects":6829,"num_object_clones":0,"num_object_copies":13658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3531,"num_write_kb":14124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3297},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345743+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345743+0000","last_peered":"2024-09-10T20:30:05.345743+0000","last_clean":"2024-09-10T20:30:05.345743+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345743+0000","last_undegraded":"2024-09-10T20:30:05.345743+0000","last_fullsized":"2024-09-10T20:30:05.345743+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6979","reported_seq":7102,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:31.523115+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:31:31.523115+0000","last_peered":"2024-09-10T20:31:31.523115+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:31:31.523115+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":6979,"log_dups_size":0,"ondisk_log_size":6979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28569600,"num_objects":6975,"num_object_clones":0,"num_object_copies":13950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3806,"num_write_kb":15224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3262},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6752","reported_seq":6913,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:31.571930+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:31:31.571930+0000","last_peered":"2024-09-10T20:31:31.571930+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:31:31.571930+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":6752,"log_dups_size":0,"ondisk_log_size":6752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27607523,"num_objects":6741,"num_object_clones":0,"num_object_copies":13482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3359,"num_write_kb":13436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3273},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":449369513,"num_objects":109712,"num_object_clones":0,"num_object_copies":219424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":1418,"num_objects_unfound":0,"num_objects_dirty":109712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74087,"num_write_kb":296328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2189,"num_bytes_recovered":8953856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":908967936,"data_stored":908881224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109791,"ondisk_log_size":109791,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182436,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":593172,"kb_used_data":385220,"kb_used_omap":4,"kb_used_meta":207931,"kb_avail":93778668,"statfs":{"total":96636764160,"available":96029356032,"internally_reserved":0,"allocated":394465280,"data_stored":394232147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":212922026},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,8,2,0,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986208,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511356,"kb_used_data":335276,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93860484,"statfs":{"total":96636764160,"available":96113135616,"internally_reserved":0,"allocated":343322624,"data_stored":343093568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180285098},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659155,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306168,"kb_used_data":166184,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":94065672,"statfs":{"total":96636764160,"available":96323248128,"internally_reserved":0,"allocated":170172416,"data_stored":169863410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":143322798},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":395436032,"data_stored":395432419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":169852928,"data_stored":169773442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":343678976,"data_stored":343675363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:31:36.123 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:31:36.679 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:36.679 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:31:36.704 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2024-09-10T20:31:34.860566+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":449369532,"num_objects":109714,"num_object_clones":0,"num_object_copies":219428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":1418,"num_objects_unfound":0,"num_objects_dirty":109714,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74089,"num_write_kb":296330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2191,"num_bytes_recovered":8953875,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109794,"ondisk_log_size":109794,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1410696,"kb_used_data":886680,"kb_used_omap":12,"kb_used_meta":523955,"kb_avail":281704824,"statfs":{"total":289910292480,"available":288465739776,"internally_reserved":0,"allocated":907960320,"data_stored":907189125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":536529922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,10,4,2,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26722304,"num_objects":6524,"num_object_clones":0,"num_object_copies":13048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-388,"num_objects_unfound":0,"num_objects_dirty":6524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6524,"num_write_kb":26096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":388,"num_bytes_recovered":1589248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005208"},"pg_stats":[{"pgid":"2.f","version":"43'6842","reported_seq":6918,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:31.331842+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:31:31.331842+0000","last_peered":"2024-09-10T20:31:31.331842+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:31:31.331842+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":6842,"log_dups_size":0,"ondisk_log_size":6842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28024832,"num_objects":6842,"num_object_clones":0,"num_object_copies":13684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6842,"num_write_kb":27368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3235},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7010","reported_seq":7116,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:33.267964+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:31:33.267964+0000","last_peered":"2024-09-10T20:31:33.267964+0000","last_clean":"2024-09-10T20:31:33.267964+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:31:33.267964+0000","last_undegraded":"2024-09-10T20:31:33.267964+0000","last_fullsized":"2024-09-10T20:31:33.267964+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":7010,"log_dups_size":0,"ondisk_log_size":7010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28688384,"num_objects":7004,"num_object_clones":0,"num_object_copies":14008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5196,"num_write_kb":20784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6930","reported_seq":11426,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:31:33.270664+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:31:33.270664+0000","last_peered":"2024-09-10T20:31:33.270664+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:31:33.270664+0000","last_undegraded":"2024-09-10T20:31:33.270664+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":6930,"log_dups_size":0,"ondisk_log_size":6930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28368896,"num_objects":6926,"num_object_clones":0,"num_object_copies":13852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1418,"num_objects_unfound":0,"num_objects_dirty":6926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5058,"num_write_kb":20232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2186,"num_bytes_recovered":8949760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1418},{"shards":"2","objects":5508}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6763","reported_seq":6839,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:33.220433+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:31:33.220433+0000","last_peered":"2024-09-10T20:31:33.220433+0000","last_clean":"2024-09-10T20:31:33.220433+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:31:33.220433+0000","last_undegraded":"2024-09-10T20:31:33.220433+0000","last_fullsized":"2024-09-10T20:31:33.220433+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":6763,"log_dups_size":0,"ondisk_log_size":6763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27701248,"num_objects":6763,"num_object_clones":0,"num_object_copies":13526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6763,"num_write_kb":27052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6947","reported_seq":7069,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:31.514895+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:31:31.514895+0000","last_peered":"2024-09-10T20:31:31.514895+0000","last_clean":"2024-09-10T20:31:31.514895+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:31:31.514895+0000","last_undegraded":"2024-09-10T20:31:31.514895+0000","last_fullsized":"2024-09-10T20:31:31.514895+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":6947,"log_dups_size":0,"ondisk_log_size":6947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28426240,"num_objects":6940,"num_object_clones":0,"num_object_copies":13880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3943,"num_write_kb":15772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6823","reported_seq":6955,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:31.486488+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:31:31.486488+0000","last_peered":"2024-09-10T20:31:31.486488+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:31:31.486488+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":6823,"log_dups_size":0,"ondisk_log_size":6823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27926528,"num_objects":6818,"num_object_clones":0,"num_object_copies":13636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6818,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3687,"num_write_kb":14748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3204},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6748","reported_seq":6881,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:31.481027+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:31:31.481027+0000","last_peered":"2024-09-10T20:31:31.481027+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:31:31.481027+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":6748,"log_dups_size":0,"ondisk_log_size":6748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27615232,"num_objects":6742,"num_object_clones":0,"num_object_copies":13484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3343,"num_write_kb":13372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3187},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6889","reported_seq":7009,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:31.576771+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:31:31.576771+0000","last_peered":"2024-09-10T20:31:31.576771+0000","last_clean":"2024-09-10T20:31:31.576771+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:31:31.576771+0000","last_undegraded":"2024-09-10T20:31:31.576771+0000","last_fullsized":"2024-09-10T20:31:31.576771+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":6889,"log_dups_size":0,"ondisk_log_size":6889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28209152,"num_objects":6887,"num_object_clones":0,"num_object_copies":13774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3960,"num_write_kb":15840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6768","reported_seq":6895,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:33.302755+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:31:33.302755+0000","last_peered":"2024-09-10T20:31:33.302755+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:31:33.302755+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":6768,"log_dups_size":0,"ondisk_log_size":6768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27693056,"num_objects":6761,"num_object_clones":0,"num_object_copies":13522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4761,"num_write_kb":19044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":3235}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355479+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355479+0000","last_peered":"2024-09-10T20:30:05.355479+0000","last_clean":"2024-09-10T20:30:05.355479+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355479+0000","last_undegraded":"2024-09-10T20:30:05.355479+0000","last_fullsized":"2024-09-10T20:30:05.355479+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6868","reported_seq":6970,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:31.591904+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:31:31.591904+0000","last_peered":"2024-09-10T20:31:31.591904+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:31:31.591904+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":6868,"log_dups_size":0,"ondisk_log_size":6868,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28099526,"num_objects":6862,"num_object_clones":0,"num_object_copies":13724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6866,"num_write_kb":27444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3247},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6820","reported_seq":6944,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:33.243961+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:31:33.243961+0000","last_peered":"2024-09-10T20:31:33.243961+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:31:33.243961+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":6820,"log_dups_size":0,"ondisk_log_size":6820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27914240,"num_objects":6815,"num_object_clones":0,"num_object_copies":13630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4648,"num_write_kb":18592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":3182}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6940","reported_seq":7064,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:33.214303+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:31:33.214303+0000","last_peered":"2024-09-10T20:31:33.214303+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:31:33.214303+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":6940,"log_dups_size":0,"ondisk_log_size":6940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28401664,"num_objects":6934,"num_object_clones":0,"num_object_copies":13868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4726,"num_write_kb":18904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":3324}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6875","reported_seq":7005,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:31.551132+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:31:31.551132+0000","last_peered":"2024-09-10T20:31:31.551132+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:31:31.551132+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":6875,"log_dups_size":0,"ondisk_log_size":6875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28151808,"num_objects":6873,"num_object_clones":0,"num_object_copies":13746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3598,"num_write_kb":14392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3225},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6837","reported_seq":6971,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:31.601879+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:31:31.601879+0000","last_peered":"2024-09-10T20:31:31.601879+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:31:31.601879+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":6837,"log_dups_size":0,"ondisk_log_size":6837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27971584,"num_objects":6829,"num_object_clones":0,"num_object_copies":13658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3531,"num_write_kb":14124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3297},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345743+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345743+0000","last_peered":"2024-09-10T20:30:05.345743+0000","last_clean":"2024-09-10T20:30:05.345743+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345743+0000","last_undegraded":"2024-09-10T20:30:05.345743+0000","last_fullsized":"2024-09-10T20:30:05.345743+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6979","reported_seq":7102,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:31.523115+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:31:31.523115+0000","last_peered":"2024-09-10T20:31:31.523115+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:31:31.523115+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":6979,"log_dups_size":0,"ondisk_log_size":6979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28569600,"num_objects":6975,"num_object_clones":0,"num_object_copies":13950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3806,"num_write_kb":15224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3262},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6752","reported_seq":6913,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:31.571930+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:31:31.571930+0000","last_peered":"2024-09-10T20:31:31.571930+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:31:31.571930+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":6752,"log_dups_size":0,"ondisk_log_size":6752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27607523,"num_objects":6741,"num_object_clones":0,"num_object_copies":13482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3359,"num_write_kb":13436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3273},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":449369513,"num_objects":109712,"num_object_clones":0,"num_object_copies":219424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":1418,"num_objects_unfound":0,"num_objects_dirty":109712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74087,"num_write_kb":296328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2189,"num_bytes_recovered":8953856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":908967936,"data_stored":908881224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109791,"ondisk_log_size":109791,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182436,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":593172,"kb_used_data":385220,"kb_used_omap":4,"kb_used_meta":207931,"kb_avail":93778668,"statfs":{"total":96636764160,"available":96029356032,"internally_reserved":0,"allocated":394465280,"data_stored":394232147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":212922026},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,8,2,0,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986208,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511356,"kb_used_data":335276,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93860484,"statfs":{"total":96636764160,"available":96113135616,"internally_reserved":0,"allocated":343322624,"data_stored":343093568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180285098},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659155,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306168,"kb_used_data":166184,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":94065672,"statfs":{"total":96636764160,"available":96323248128,"internally_reserved":0,"allocated":170172416,"data_stored":169863410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":143322798},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":395436032,"data_stored":395432419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":169852928,"data_stored":169773442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":343678976,"data_stored":343675363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:31:36.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:31:36.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'6842", 'reported_seq': 6918, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:31.331842+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:31:31.331842+0000', 'last_peered': '2024-09-10T20:31:31.331842+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:31:31.331842+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 6842, 'log_dups_size': 0, 'ondisk_log_size': 6842, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28024832, 'num_objects': 6842, 'num_object_clones': 0, 'num_object_copies': 13684, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3607, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6842, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6842, 'num_write_kb': 27368, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3235}, {'shards': '0,2', 'objects': 3607}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:36.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T20:31:36.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'6930", 'reported_seq': 11426, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:31:33.270664+0000', 'last_change': '2024-09-10T20:30:05.653560+0000', 'last_active': '2024-09-10T20:31:33.270664+0000', 'last_peered': '2024-09-10T20:31:33.270664+0000', 'last_clean': '2024-09-10T20:30:03.279259+0000', 'last_became_active': '2024-09-10T20:30:05.629664+0000', 'last_became_peered': '2024-09-10T20:30:05.629664+0000', 'last_unstale': '2024-09-10T20:31:33.270664+0000', 'last_undegraded': '2024-09-10T20:31:33.270664+0000', 'last_fullsized': '2024-09-10T20:30:05.312061+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1910", 'last_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'objects_scrubbed': 1868, 'log_size': 6930, 'log_dups_size': 0, 'ondisk_log_size': 6930, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 673, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28368896, 'num_objects': 6926, 'num_object_clones': 0, 'num_object_copies': 13852, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1418, 'num_objects_unfound': 0, 'num_objects_dirty': 6926, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5058, 'num_write_kb': 20232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2186, 'num_bytes_recovered': 8949760, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1418}, {'shards': '2', 'objects': 5508}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:36.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:31:36.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'6823", 'reported_seq': 6955, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:31.486488+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:31:31.486488+0000', 'last_peered': '2024-09-10T20:31:31.486488+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:31:31.486488+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 6823, 'log_dups_size': 0, 'ondisk_log_size': 6823, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27926528, 'num_objects': 6818, 'num_object_clones': 0, 'num_object_copies': 13636, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3616, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6818, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3687, 'num_write_kb': 14748, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3204}, {'shards': '0,2', 'objects': 3614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:36.706 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:31:36.706 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'6748", 'reported_seq': 6881, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:31.481027+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:31:31.481027+0000', 'last_peered': '2024-09-10T20:31:31.481027+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:31:31.481027+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 6748, 'log_dups_size': 0, 'ondisk_log_size': 6748, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27615232, 'num_objects': 6742, 'num_object_clones': 0, 'num_object_copies': 13484, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3557, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6742, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3343, 'num_write_kb': 13372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3187}, {'shards': '0,2', 'objects': 3555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:36.706 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:31:36.706 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'6768", 'reported_seq': 6895, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:33.302755+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:31:33.302755+0000', 'last_peered': '2024-09-10T20:31:33.302755+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:31:33.302755+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 6768, 'log_dups_size': 0, 'ondisk_log_size': 6768, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27693056, 'num_objects': 6761, 'num_object_clones': 0, 'num_object_copies': 13522, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3528, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6761, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4761, 'num_write_kb': 19044, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 3235}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:36.706 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:31:36.706 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'6868", 'reported_seq': 6970, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:31.591904+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:31:31.591904+0000', 'last_peered': '2024-09-10T20:31:31.591904+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:31:31.591904+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 6868, 'log_dups_size': 0, 'ondisk_log_size': 6868, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28099526, 'num_objects': 6862, 'num_object_clones': 0, 'num_object_copies': 13724, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3617, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6862, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6866, 'num_write_kb': 27444, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3247}, {'shards': '0,2', 'objects': 3615}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:36.706 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:31:36.707 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'6820", 'reported_seq': 6944, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:33.243961+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:31:33.243961+0000', 'last_peered': '2024-09-10T20:31:33.243961+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:31:33.243961+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 6820, 'log_dups_size': 0, 'ondisk_log_size': 6820, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27914240, 'num_objects': 6815, 'num_object_clones': 0, 'num_object_copies': 13630, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3635, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6815, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4648, 'num_write_kb': 18592, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3633}, {'shards': '2', 'objects': 3182}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:36.707 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:31:36.707 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'6940", 'reported_seq': 7064, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:33.214303+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:31:33.214303+0000', 'last_peered': '2024-09-10T20:31:33.214303+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:31:33.214303+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 6940, 'log_dups_size': 0, 'ondisk_log_size': 6940, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28401664, 'num_objects': 6934, 'num_object_clones': 0, 'num_object_copies': 13868, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6934, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4726, 'num_write_kb': 18904, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3610}, {'shards': '2', 'objects': 3324}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:36.707 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:31:36.707 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'6875", 'reported_seq': 7005, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:31.551132+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:31:31.551132+0000', 'last_peered': '2024-09-10T20:31:31.551132+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:31:31.551132+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 6875, 'log_dups_size': 0, 'ondisk_log_size': 6875, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28151808, 'num_objects': 6873, 'num_object_clones': 0, 'num_object_copies': 13746, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3649, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6873, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3598, 'num_write_kb': 14392, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3225}, {'shards': '0,2', 'objects': 3648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:36.707 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:31:36.707 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'6837", 'reported_seq': 6971, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:31.601879+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:31:31.601879+0000', 'last_peered': '2024-09-10T20:31:31.601879+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:31:31.601879+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 6837, 'log_dups_size': 0, 'ondisk_log_size': 6837, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27971584, 'num_objects': 6829, 'num_object_clones': 0, 'num_object_copies': 13658, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3535, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6829, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3531, 'num_write_kb': 14124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3297}, {'shards': '0,2', 'objects': 3532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:36.707 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:31:36.707 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'6979", 'reported_seq': 7102, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:31.523115+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:31:31.523115+0000', 'last_peered': '2024-09-10T20:31:31.523115+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:31:31.523115+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 6979, 'log_dups_size': 0, 'ondisk_log_size': 6979, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28569600, 'num_objects': 6975, 'num_object_clones': 0, 'num_object_copies': 13950, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3715, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6975, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3806, 'num_write_kb': 15224, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3262}, {'shards': '0,2', 'objects': 3713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:36.707 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:31:36.707 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'6752", 'reported_seq': 6913, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:31.571930+0000', 'last_change': '2024-09-10T20:30:05.690055+0000', 'last_active': '2024-09-10T20:31:31.571930+0000', 'last_peered': '2024-09-10T20:31:31.571930+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:31:31.571930+0000', 'last_undegraded': '2024-09-10T20:30:05.378941+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 6752, 'log_dups_size': 0, 'ondisk_log_size': 6752, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27607523, 'num_objects': 6741, 'num_object_clones': 0, 'num_object_copies': 13482, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3471, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6741, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3359, 'num_write_kb': 13436, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3273}, {'shards': '0,2', 'objects': 3468}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:36.708 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:31:37.059 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 129 20 112334 112314 3.4002 1.92969 0.0361704 0.0229684 2024-09-10T20:31:37.407 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:37.407 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39542/222902 objects degraded (17.740%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":209,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":111451,"data_bytes":456484284,"bytes_used":1513959424,"bytes_avail":288396333056,"bytes_total":289910292480,"degraded_objects":39542,"degraded_total":222902,"degraded_ratio":0.17739634458192388,"misplaced_objects":1418,"misplaced_total":222902,"misplaced_ratio":0.0063615400489901389,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":90413,"recovering_keys_per_sec":0,"num_objects_recovered":265,"num_bytes_recovered":1085440,"num_keys_recovered":0,"write_bytes_sec":2463344,"read_op_per_sec":0,"write_op_per_sec":601},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:31:37.435 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:31:37.436 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:31:37.983 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:37.984 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:31:38.006 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2024-09-10T20:31:36.861280+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":456484284,"num_objects":111451,"num_object_clones":0,"num_object_copies":222902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":1418,"num_objects_unfound":0,"num_objects_dirty":111451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":75826,"num_write_kb":303278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2191,"num_bytes_recovered":8953875,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111531,"ondisk_log_size":111531,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1478476,"kb_used_data":896156,"kb_used_omap":12,"kb_used_meta":582259,"kb_avail":281637044,"statfs":{"total":289910292480,"available":288396333056,"internally_reserved":0,"allocated":917663744,"data_stored":916892549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":596233218},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,6,8,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29573120,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-265,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7220,"num_write_kb":28880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":265,"num_bytes_recovered":1085440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005271"},"pg_stats":[{"pgid":"2.f","version":"43'7012","reported_seq":7088,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:36.601963+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:31:36.601963+0000","last_peered":"2024-09-10T20:31:36.601963+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:31:36.601963+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":7012,"log_dups_size":0,"ondisk_log_size":7012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28721152,"num_objects":7012,"num_object_clones":0,"num_object_copies":14024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7012,"num_write_kb":28048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3405},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7010","reported_seq":7116,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:33.267964+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:31:33.267964+0000","last_peered":"2024-09-10T20:31:33.267964+0000","last_clean":"2024-09-10T20:31:33.267964+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:31:33.267964+0000","last_undegraded":"2024-09-10T20:31:33.267964+0000","last_fullsized":"2024-09-10T20:31:33.267964+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":7010,"log_dups_size":0,"ondisk_log_size":7010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28688384,"num_objects":7004,"num_object_clones":0,"num_object_copies":14008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5196,"num_write_kb":20784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6930","reported_seq":11426,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:31:33.270664+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:31:33.270664+0000","last_peered":"2024-09-10T20:31:33.270664+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:31:33.270664+0000","last_undegraded":"2024-09-10T20:31:33.270664+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":6930,"log_dups_size":0,"ondisk_log_size":6930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28368896,"num_objects":6926,"num_object_clones":0,"num_object_copies":13852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1418,"num_objects_unfound":0,"num_objects_dirty":6926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5058,"num_write_kb":20232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2186,"num_bytes_recovered":8949760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1418},{"shards":"2","objects":5508}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6763","reported_seq":6839,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:33.220433+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:31:33.220433+0000","last_peered":"2024-09-10T20:31:33.220433+0000","last_clean":"2024-09-10T20:31:33.220433+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:31:33.220433+0000","last_undegraded":"2024-09-10T20:31:33.220433+0000","last_fullsized":"2024-09-10T20:31:33.220433+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":6763,"log_dups_size":0,"ondisk_log_size":6763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27701248,"num_objects":6763,"num_object_clones":0,"num_object_copies":13526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6763,"num_write_kb":27052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7121","reported_seq":7243,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:36.603843+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:31:36.603843+0000","last_peered":"2024-09-10T20:31:36.603843+0000","last_clean":"2024-09-10T20:31:36.603843+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:31:36.603843+0000","last_undegraded":"2024-09-10T20:31:36.603843+0000","last_fullsized":"2024-09-10T20:31:36.603843+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":7121,"log_dups_size":0,"ondisk_log_size":7121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29138944,"num_objects":7114,"num_object_clones":0,"num_object_copies":14228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4117,"num_write_kb":16468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6971","reported_seq":7103,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:36.658028+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:31:36.658028+0000","last_peered":"2024-09-10T20:31:36.658028+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:31:36.658028+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":6971,"log_dups_size":0,"ondisk_log_size":6971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28532736,"num_objects":6966,"num_object_clones":0,"num_object_copies":13932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3835,"num_write_kb":15340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3352},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6918","reported_seq":7050,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:36.663988+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:31:36.663988+0000","last_peered":"2024-09-10T20:31:36.663988+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:31:36.663988+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":6918,"log_dups_size":0,"ondisk_log_size":6918,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28311552,"num_objects":6912,"num_object_clones":0,"num_object_copies":13824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3513,"num_write_kb":14052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3357},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7083","reported_seq":7203,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:36.631361+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:31:36.631361+0000","last_peered":"2024-09-10T20:31:36.631361+0000","last_clean":"2024-09-10T20:31:36.631361+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:31:36.631361+0000","last_undegraded":"2024-09-10T20:31:36.631361+0000","last_fullsized":"2024-09-10T20:31:36.631361+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":7083,"log_dups_size":0,"ondisk_log_size":7083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29003776,"num_objects":7081,"num_object_clones":0,"num_object_copies":14162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4154,"num_write_kb":16616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6768","reported_seq":6895,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:33.302755+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:31:33.302755+0000","last_peered":"2024-09-10T20:31:33.302755+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:31:33.302755+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":6768,"log_dups_size":0,"ondisk_log_size":6768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27693056,"num_objects":6761,"num_object_clones":0,"num_object_copies":13522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4761,"num_write_kb":19044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":3235}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355479+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355479+0000","last_peered":"2024-09-10T20:30:05.355479+0000","last_clean":"2024-09-10T20:30:05.355479+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355479+0000","last_undegraded":"2024-09-10T20:30:05.355479+0000","last_fullsized":"2024-09-10T20:30:05.355479+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7060","reported_seq":7163,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:36.615261+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:31:36.615261+0000","last_peered":"2024-09-10T20:31:36.615261+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:31:36.615261+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":7060,"log_dups_size":0,"ondisk_log_size":7060,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28885958,"num_objects":7054,"num_object_clones":0,"num_object_copies":14108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7058,"num_write_kb":28212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3439},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6820","reported_seq":6944,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:33.243961+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:31:33.243961+0000","last_peered":"2024-09-10T20:31:33.243961+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:31:33.243961+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":6820,"log_dups_size":0,"ondisk_log_size":6820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27914240,"num_objects":6815,"num_object_clones":0,"num_object_copies":13630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4648,"num_write_kb":18592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":3182}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6940","reported_seq":7064,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:33.214303+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:31:33.214303+0000","last_peered":"2024-09-10T20:31:33.214303+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:31:33.214303+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":6940,"log_dups_size":0,"ondisk_log_size":6940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28401664,"num_objects":6934,"num_object_clones":0,"num_object_copies":13868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4726,"num_write_kb":18904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":3324}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7046","reported_seq":7175,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:36.686594+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:31:36.686594+0000","last_peered":"2024-09-10T20:31:36.686594+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:31:36.686594+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":7046,"log_dups_size":0,"ondisk_log_size":7046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28852224,"num_objects":7044,"num_object_clones":0,"num_object_copies":14088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3769,"num_write_kb":15076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3396},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7012","reported_seq":7146,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:36.662296+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:31:36.662296+0000","last_peered":"2024-09-10T20:31:36.662296+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:31:36.662296+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":7012,"log_dups_size":0,"ondisk_log_size":7012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28688384,"num_objects":7004,"num_object_clones":0,"num_object_copies":14008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3706,"num_write_kb":14824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3472},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345743+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345743+0000","last_peered":"2024-09-10T20:30:05.345743+0000","last_clean":"2024-09-10T20:30:05.345743+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345743+0000","last_undegraded":"2024-09-10T20:30:05.345743+0000","last_fullsized":"2024-09-10T20:30:05.345743+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7139","reported_seq":7265,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:36.552378+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:31:36.552378+0000","last_peered":"2024-09-10T20:31:36.552378+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:31:36.552378+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":7139,"log_dups_size":0,"ondisk_log_size":7139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29224960,"num_objects":7135,"num_object_clones":0,"num_object_copies":14270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3966,"num_write_kb":15864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3422},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6935","reported_seq":7096,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:36.614487+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:31:36.614487+0000","last_peered":"2024-09-10T20:31:36.614487+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:31:36.614487+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":6935,"log_dups_size":0,"ondisk_log_size":6935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28357091,"num_objects":6924,"num_object_clones":0,"num_object_copies":13848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3542,"num_write_kb":14168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3456},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":456484265,"num_objects":111449,"num_object_clones":0,"num_object_copies":222898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":1418,"num_objects_unfound":0,"num_objects_dirty":111449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":75824,"num_write_kb":303276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2189,"num_bytes_recovered":8953856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":917536768,"data_stored":917450056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111528,"ondisk_log_size":111528,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182437,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":660952,"kb_used_data":394696,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93710888,"statfs":{"total":96636764160,"available":95959949312,"internally_reserved":0,"allocated":404168704,"data_stored":403935571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986208,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511356,"kb_used_data":335276,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93860484,"statfs":{"total":96636764160,"available":96113135616,"internally_reserved":0,"allocated":343322624,"data_stored":343093568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180285098},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659155,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306168,"kb_used_data":166184,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":94065672,"statfs":{"total":96636764160,"available":96323248128,"internally_reserved":0,"allocated":170172416,"data_stored":169863410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":143322798},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":404004864,"data_stored":404001251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":169852928,"data_stored":169773442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":343678976,"data_stored":343675363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:31:38.060 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 130 20 112829 112809 3.38891 1.93359 0.0176987 0.0230458 2024-09-10T20:31:39.061 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 131 20 113392 113372 3.37979 2.19922 0.0172334 0.0231075 2024-09-10T20:31:40.061 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 132 20 113960 113940 3.371 2.21875 0.0620211 0.0231677 2024-09-10T20:31:41.009 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:31:41.061 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 133 20 114514 114494 3.36192 2.16406 0.0958001 0.0232282 2024-09-10T20:31:41.555 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:41.556 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:31:41.583 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2024-09-10T20:31:40.862675+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":460506556,"num_objects":112433,"num_object_clones":0,"num_object_copies":224866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":1279,"num_objects_unfound":0,"num_objects_dirty":112433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":76808,"num_write_kb":307206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2330,"num_bytes_recovered":9523219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112513,"ondisk_log_size":112513,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1522960,"kb_used_data":910112,"kb_used_omap":12,"kb_used_meta":612787,"kb_avail":281592560,"statfs":{"total":289910292480,"available":288350781440,"internally_reserved":0,"allocated":931954688,"data_stored":931183493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":627493890},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,9,9,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22093824,"num_objects":5394,"num_object_clones":0,"num_object_copies":10788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-266,"num_objects_unfound":0,"num_objects_dirty":5394,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5394,"num_write_kb":21576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":1089536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005671"},"pg_stats":[{"pgid":"2.f","version":"43'7012","reported_seq":7088,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:36.601963+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:31:36.601963+0000","last_peered":"2024-09-10T20:31:36.601963+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:31:36.601963+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":7012,"log_dups_size":0,"ondisk_log_size":7012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28721152,"num_objects":7012,"num_object_clones":0,"num_object_copies":14024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7012,"num_write_kb":28048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3405},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7169","reported_seq":7275,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:38.278787+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:31:38.278787+0000","last_peered":"2024-09-10T20:31:38.278787+0000","last_clean":"2024-09-10T20:31:38.278787+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:31:38.278787+0000","last_undegraded":"2024-09-10T20:31:38.278787+0000","last_fullsized":"2024-09-10T20:31:38.278787+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":7169,"log_dups_size":0,"ondisk_log_size":7169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29339648,"num_objects":7163,"num_object_clones":0,"num_object_copies":14326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5355,"num_write_kb":21420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7095","reported_seq":11870,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:31:38.311903+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:31:38.311903+0000","last_peered":"2024-09-10T20:31:38.311903+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:31:38.311903+0000","last_undegraded":"2024-09-10T20:31:38.311903+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":7095,"log_dups_size":0,"ondisk_log_size":7095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29044736,"num_objects":7091,"num_object_clones":0,"num_object_copies":14182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1279,"num_objects_unfound":0,"num_objects_dirty":7091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5223,"num_write_kb":20892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2325,"num_bytes_recovered":9519104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1279},{"shards":"2","objects":5812}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6930","reported_seq":7006,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:38.261845+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:31:38.261845+0000","last_peered":"2024-09-10T20:31:38.261845+0000","last_clean":"2024-09-10T20:31:38.261845+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:31:38.261845+0000","last_undegraded":"2024-09-10T20:31:38.261845+0000","last_fullsized":"2024-09-10T20:31:38.261845+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":6930,"log_dups_size":0,"ondisk_log_size":6930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28385280,"num_objects":6930,"num_object_clones":0,"num_object_copies":13860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6930,"num_write_kb":27720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7121","reported_seq":7243,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:36.603843+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:31:36.603843+0000","last_peered":"2024-09-10T20:31:36.603843+0000","last_clean":"2024-09-10T20:31:36.603843+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:31:36.603843+0000","last_undegraded":"2024-09-10T20:31:36.603843+0000","last_fullsized":"2024-09-10T20:31:36.603843+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":7121,"log_dups_size":0,"ondisk_log_size":7121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29138944,"num_objects":7114,"num_object_clones":0,"num_object_copies":14228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4117,"num_write_kb":16468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6971","reported_seq":7103,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:36.658028+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:31:36.658028+0000","last_peered":"2024-09-10T20:31:36.658028+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:31:36.658028+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":6971,"log_dups_size":0,"ondisk_log_size":6971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28532736,"num_objects":6966,"num_object_clones":0,"num_object_copies":13932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3835,"num_write_kb":15340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3352},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6918","reported_seq":7050,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:36.663988+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:31:36.663988+0000","last_peered":"2024-09-10T20:31:36.663988+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:31:36.663988+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":6918,"log_dups_size":0,"ondisk_log_size":6918,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28311552,"num_objects":6912,"num_object_clones":0,"num_object_copies":13824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3513,"num_write_kb":14052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3357},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7083","reported_seq":7203,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:36.631361+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:31:36.631361+0000","last_peered":"2024-09-10T20:31:36.631361+0000","last_clean":"2024-09-10T20:31:36.631361+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:31:36.631361+0000","last_undegraded":"2024-09-10T20:31:36.631361+0000","last_fullsized":"2024-09-10T20:31:36.631361+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":7083,"log_dups_size":0,"ondisk_log_size":7083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29003776,"num_objects":7081,"num_object_clones":0,"num_object_copies":14162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4154,"num_write_kb":16616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6921","reported_seq":7048,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:38.261679+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:31:38.261679+0000","last_peered":"2024-09-10T20:31:38.261679+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:31:38.261679+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":6921,"log_dups_size":0,"ondisk_log_size":6921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28319744,"num_objects":6914,"num_object_clones":0,"num_object_copies":13828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4914,"num_write_kb":19656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":3388}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355479+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355479+0000","last_peered":"2024-09-10T20:30:05.355479+0000","last_clean":"2024-09-10T20:30:05.355479+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355479+0000","last_undegraded":"2024-09-10T20:30:05.355479+0000","last_fullsized":"2024-09-10T20:30:05.355479+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7060","reported_seq":7163,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:36.615261+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:31:36.615261+0000","last_peered":"2024-09-10T20:31:36.615261+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:31:36.615261+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":7060,"log_dups_size":0,"ondisk_log_size":7060,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28885958,"num_objects":7054,"num_object_clones":0,"num_object_copies":14108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7058,"num_write_kb":28212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3439},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7005","reported_seq":7128,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:38.270445+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:31:38.270445+0000","last_peered":"2024-09-10T20:31:38.270445+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:31:38.270445+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":7005,"log_dups_size":0,"ondisk_log_size":7005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28672000,"num_objects":7000,"num_object_clones":0,"num_object_copies":14000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4833,"num_write_kb":19332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":3367}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7093","reported_seq":7217,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:38.301786+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:31:38.301786+0000","last_peered":"2024-09-10T20:31:38.301786+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:31:38.301786+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":7093,"log_dups_size":0,"ondisk_log_size":7093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29028352,"num_objects":7087,"num_object_clones":0,"num_object_copies":14174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7087,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4879,"num_write_kb":19516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":3477}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7046","reported_seq":7175,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:36.686594+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:31:36.686594+0000","last_peered":"2024-09-10T20:31:36.686594+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:31:36.686594+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":7046,"log_dups_size":0,"ondisk_log_size":7046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28852224,"num_objects":7044,"num_object_clones":0,"num_object_copies":14088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3769,"num_write_kb":15076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3396},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7012","reported_seq":7146,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:36.662296+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:31:36.662296+0000","last_peered":"2024-09-10T20:31:36.662296+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:31:36.662296+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":7012,"log_dups_size":0,"ondisk_log_size":7012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28688384,"num_objects":7004,"num_object_clones":0,"num_object_copies":14008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3706,"num_write_kb":14824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3472},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345743+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345743+0000","last_peered":"2024-09-10T20:30:05.345743+0000","last_clean":"2024-09-10T20:30:05.345743+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345743+0000","last_undegraded":"2024-09-10T20:30:05.345743+0000","last_fullsized":"2024-09-10T20:30:05.345743+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7139","reported_seq":7265,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:36.552378+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:31:36.552378+0000","last_peered":"2024-09-10T20:31:36.552378+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:31:36.552378+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":7139,"log_dups_size":0,"ondisk_log_size":7139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29224960,"num_objects":7135,"num_object_clones":0,"num_object_copies":14270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3966,"num_write_kb":15864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3422},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6935","reported_seq":7096,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:36.614487+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:31:36.614487+0000","last_peered":"2024-09-10T20:31:36.614487+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:31:36.614487+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":6935,"log_dups_size":0,"ondisk_log_size":6935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28357091,"num_objects":6924,"num_object_clones":0,"num_object_copies":13848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3542,"num_write_kb":14168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3456},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":460506537,"num_objects":112431,"num_object_clones":0,"num_object_copies":224862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":1279,"num_objects_unfound":0,"num_objects_dirty":112431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":76806,"num_write_kb":307204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2328,"num_bytes_recovered":9523200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":931831808,"data_stored":931745096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112510,"ondisk_log_size":112510,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182437,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":660952,"kb_used_data":394696,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93710888,"statfs":{"total":96636764160,"available":95959949312,"internally_reserved":0,"allocated":404168704,"data_stored":403935571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986209,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520944,"kb_used_data":340704,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93850896,"statfs":{"total":96636764160,"available":96103317504,"internally_reserved":0,"allocated":348880896,"data_stored":348651840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184544938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659156,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":341064,"kb_used_data":174712,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":94030776,"statfs":{"total":96636764160,"available":96287514624,"internally_reserved":0,"allocated":178905088,"data_stored":178596082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":170323630},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,2,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":404004864,"data_stored":404001251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":178688000,"data_stored":178608514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":349138944,"data_stored":349135331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:31:41.584 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:31:42.062 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 134 20 115006 114986 3.35117 1.92188 0.0231783 0.023304 2024-09-10T20:31:42.130 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:42.131 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:31:42.172 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2024-09-10T20:31:40.862675+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":460506556,"num_objects":112433,"num_object_clones":0,"num_object_copies":224866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":1279,"num_objects_unfound":0,"num_objects_dirty":112433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":76808,"num_write_kb":307206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2330,"num_bytes_recovered":9523219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112513,"ondisk_log_size":112513,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1522960,"kb_used_data":910112,"kb_used_omap":12,"kb_used_meta":612787,"kb_avail":281592560,"statfs":{"total":289910292480,"available":288350781440,"internally_reserved":0,"allocated":931954688,"data_stored":931183493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":627493890},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,9,9,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22093824,"num_objects":5394,"num_object_clones":0,"num_object_copies":10788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-266,"num_objects_unfound":0,"num_objects_dirty":5394,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5394,"num_write_kb":21576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":1089536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005671"},"pg_stats":[{"pgid":"2.f","version":"43'7012","reported_seq":7088,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:36.601963+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:31:36.601963+0000","last_peered":"2024-09-10T20:31:36.601963+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:31:36.601963+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":7012,"log_dups_size":0,"ondisk_log_size":7012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28721152,"num_objects":7012,"num_object_clones":0,"num_object_copies":14024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7012,"num_write_kb":28048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3405},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7169","reported_seq":7275,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:38.278787+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:31:38.278787+0000","last_peered":"2024-09-10T20:31:38.278787+0000","last_clean":"2024-09-10T20:31:38.278787+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:31:38.278787+0000","last_undegraded":"2024-09-10T20:31:38.278787+0000","last_fullsized":"2024-09-10T20:31:38.278787+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":7169,"log_dups_size":0,"ondisk_log_size":7169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29339648,"num_objects":7163,"num_object_clones":0,"num_object_copies":14326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5355,"num_write_kb":21420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7095","reported_seq":11870,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:31:38.311903+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:31:38.311903+0000","last_peered":"2024-09-10T20:31:38.311903+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:31:38.311903+0000","last_undegraded":"2024-09-10T20:31:38.311903+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":7095,"log_dups_size":0,"ondisk_log_size":7095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29044736,"num_objects":7091,"num_object_clones":0,"num_object_copies":14182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1279,"num_objects_unfound":0,"num_objects_dirty":7091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5223,"num_write_kb":20892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2325,"num_bytes_recovered":9519104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1279},{"shards":"2","objects":5812}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6930","reported_seq":7006,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:38.261845+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:31:38.261845+0000","last_peered":"2024-09-10T20:31:38.261845+0000","last_clean":"2024-09-10T20:31:38.261845+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:31:38.261845+0000","last_undegraded":"2024-09-10T20:31:38.261845+0000","last_fullsized":"2024-09-10T20:31:38.261845+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":6930,"log_dups_size":0,"ondisk_log_size":6930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28385280,"num_objects":6930,"num_object_clones":0,"num_object_copies":13860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6930,"num_write_kb":27720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7121","reported_seq":7243,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:36.603843+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:31:36.603843+0000","last_peered":"2024-09-10T20:31:36.603843+0000","last_clean":"2024-09-10T20:31:36.603843+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:31:36.603843+0000","last_undegraded":"2024-09-10T20:31:36.603843+0000","last_fullsized":"2024-09-10T20:31:36.603843+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":7121,"log_dups_size":0,"ondisk_log_size":7121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29138944,"num_objects":7114,"num_object_clones":0,"num_object_copies":14228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4117,"num_write_kb":16468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6971","reported_seq":7103,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:36.658028+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:31:36.658028+0000","last_peered":"2024-09-10T20:31:36.658028+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:31:36.658028+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":6971,"log_dups_size":0,"ondisk_log_size":6971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28532736,"num_objects":6966,"num_object_clones":0,"num_object_copies":13932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3835,"num_write_kb":15340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3352},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6918","reported_seq":7050,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:36.663988+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:31:36.663988+0000","last_peered":"2024-09-10T20:31:36.663988+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:31:36.663988+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":6918,"log_dups_size":0,"ondisk_log_size":6918,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28311552,"num_objects":6912,"num_object_clones":0,"num_object_copies":13824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3513,"num_write_kb":14052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3357},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7083","reported_seq":7203,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:36.631361+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:31:36.631361+0000","last_peered":"2024-09-10T20:31:36.631361+0000","last_clean":"2024-09-10T20:31:36.631361+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:31:36.631361+0000","last_undegraded":"2024-09-10T20:31:36.631361+0000","last_fullsized":"2024-09-10T20:31:36.631361+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":7083,"log_dups_size":0,"ondisk_log_size":7083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29003776,"num_objects":7081,"num_object_clones":0,"num_object_copies":14162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4154,"num_write_kb":16616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6921","reported_seq":7048,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:38.261679+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:31:38.261679+0000","last_peered":"2024-09-10T20:31:38.261679+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:31:38.261679+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":6921,"log_dups_size":0,"ondisk_log_size":6921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28319744,"num_objects":6914,"num_object_clones":0,"num_object_copies":13828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4914,"num_write_kb":19656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":3388}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355479+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355479+0000","last_peered":"2024-09-10T20:30:05.355479+0000","last_clean":"2024-09-10T20:30:05.355479+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355479+0000","last_undegraded":"2024-09-10T20:30:05.355479+0000","last_fullsized":"2024-09-10T20:30:05.355479+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7060","reported_seq":7163,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:36.615261+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:31:36.615261+0000","last_peered":"2024-09-10T20:31:36.615261+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:31:36.615261+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":7060,"log_dups_size":0,"ondisk_log_size":7060,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28885958,"num_objects":7054,"num_object_clones":0,"num_object_copies":14108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7058,"num_write_kb":28212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3439},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7005","reported_seq":7128,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:38.270445+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:31:38.270445+0000","last_peered":"2024-09-10T20:31:38.270445+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:31:38.270445+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":7005,"log_dups_size":0,"ondisk_log_size":7005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28672000,"num_objects":7000,"num_object_clones":0,"num_object_copies":14000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4833,"num_write_kb":19332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":3367}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7093","reported_seq":7217,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:38.301786+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:31:38.301786+0000","last_peered":"2024-09-10T20:31:38.301786+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:31:38.301786+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":7093,"log_dups_size":0,"ondisk_log_size":7093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29028352,"num_objects":7087,"num_object_clones":0,"num_object_copies":14174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7087,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4879,"num_write_kb":19516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":3477}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7046","reported_seq":7175,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:36.686594+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:31:36.686594+0000","last_peered":"2024-09-10T20:31:36.686594+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:31:36.686594+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":7046,"log_dups_size":0,"ondisk_log_size":7046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28852224,"num_objects":7044,"num_object_clones":0,"num_object_copies":14088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3769,"num_write_kb":15076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3396},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7012","reported_seq":7146,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:36.662296+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:31:36.662296+0000","last_peered":"2024-09-10T20:31:36.662296+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:31:36.662296+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":7012,"log_dups_size":0,"ondisk_log_size":7012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28688384,"num_objects":7004,"num_object_clones":0,"num_object_copies":14008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3706,"num_write_kb":14824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3472},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345743+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345743+0000","last_peered":"2024-09-10T20:30:05.345743+0000","last_clean":"2024-09-10T20:30:05.345743+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345743+0000","last_undegraded":"2024-09-10T20:30:05.345743+0000","last_fullsized":"2024-09-10T20:30:05.345743+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7139","reported_seq":7265,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:36.552378+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:31:36.552378+0000","last_peered":"2024-09-10T20:31:36.552378+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:31:36.552378+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":7139,"log_dups_size":0,"ondisk_log_size":7139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29224960,"num_objects":7135,"num_object_clones":0,"num_object_copies":14270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3966,"num_write_kb":15864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3422},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6935","reported_seq":7096,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:36.614487+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:31:36.614487+0000","last_peered":"2024-09-10T20:31:36.614487+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:31:36.614487+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":6935,"log_dups_size":0,"ondisk_log_size":6935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28357091,"num_objects":6924,"num_object_clones":0,"num_object_copies":13848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3542,"num_write_kb":14168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3456},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":460506537,"num_objects":112431,"num_object_clones":0,"num_object_copies":224862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":1279,"num_objects_unfound":0,"num_objects_dirty":112431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":76806,"num_write_kb":307204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2328,"num_bytes_recovered":9523200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":931831808,"data_stored":931745096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112510,"ondisk_log_size":112510,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182437,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":660952,"kb_used_data":394696,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93710888,"statfs":{"total":96636764160,"available":95959949312,"internally_reserved":0,"allocated":404168704,"data_stored":403935571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986209,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520944,"kb_used_data":340704,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93850896,"statfs":{"total":96636764160,"available":96103317504,"internally_reserved":0,"allocated":348880896,"data_stored":348651840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184544938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659156,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":341064,"kb_used_data":174712,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":94030776,"statfs":{"total":96636764160,"available":96287514624,"internally_reserved":0,"allocated":178905088,"data_stored":178596082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":170323630},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,2,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":404004864,"data_stored":404001251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":178688000,"data_stored":178608514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":349138944,"data_stored":349135331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:31:42.174 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:31:42.174 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'7012", 'reported_seq': 7088, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:36.601963+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:31:36.601963+0000', 'last_peered': '2024-09-10T20:31:36.601963+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:31:36.601963+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 7012, 'log_dups_size': 0, 'ondisk_log_size': 7012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28721152, 'num_objects': 7012, 'num_object_clones': 0, 'num_object_copies': 14024, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3607, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7012, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7012, 'num_write_kb': 28048, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3405}, {'shards': '0,2', 'objects': 3607}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:42.174 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T20:31:42.174 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'7095", 'reported_seq': 11870, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:31:38.311903+0000', 'last_change': '2024-09-10T20:30:05.653560+0000', 'last_active': '2024-09-10T20:31:38.311903+0000', 'last_peered': '2024-09-10T20:31:38.311903+0000', 'last_clean': '2024-09-10T20:30:03.279259+0000', 'last_became_active': '2024-09-10T20:30:05.629664+0000', 'last_became_peered': '2024-09-10T20:30:05.629664+0000', 'last_unstale': '2024-09-10T20:31:38.311903+0000', 'last_undegraded': '2024-09-10T20:31:38.311903+0000', 'last_fullsized': '2024-09-10T20:30:05.312061+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1910", 'last_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'objects_scrubbed': 1868, 'log_size': 7095, 'log_dups_size': 0, 'ondisk_log_size': 7095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 673, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29044736, 'num_objects': 7091, 'num_object_clones': 0, 'num_object_copies': 14182, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1279, 'num_objects_unfound': 0, 'num_objects_dirty': 7091, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5223, 'num_write_kb': 20892, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2325, 'num_bytes_recovered': 9519104, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1279}, {'shards': '2', 'objects': 5812}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:42.174 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:31:42.174 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'6971", 'reported_seq': 7103, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:36.658028+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:31:36.658028+0000', 'last_peered': '2024-09-10T20:31:36.658028+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:31:36.658028+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 6971, 'log_dups_size': 0, 'ondisk_log_size': 6971, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28532736, 'num_objects': 6966, 'num_object_clones': 0, 'num_object_copies': 13932, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3616, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6966, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3835, 'num_write_kb': 15340, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3352}, {'shards': '0,2', 'objects': 3614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:42.174 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:31:42.175 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'6918", 'reported_seq': 7050, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:36.663988+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:31:36.663988+0000', 'last_peered': '2024-09-10T20:31:36.663988+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:31:36.663988+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 6918, 'log_dups_size': 0, 'ondisk_log_size': 6918, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28311552, 'num_objects': 6912, 'num_object_clones': 0, 'num_object_copies': 13824, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3557, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6912, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3513, 'num_write_kb': 14052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3357}, {'shards': '0,2', 'objects': 3555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:42.175 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:31:42.175 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'6921", 'reported_seq': 7048, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:38.261679+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:31:38.261679+0000', 'last_peered': '2024-09-10T20:31:38.261679+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:31:38.261679+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 6921, 'log_dups_size': 0, 'ondisk_log_size': 6921, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28319744, 'num_objects': 6914, 'num_object_clones': 0, 'num_object_copies': 13828, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3528, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6914, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4914, 'num_write_kb': 19656, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 3388}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:42.175 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:31:42.175 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'7060", 'reported_seq': 7163, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:36.615261+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:31:36.615261+0000', 'last_peered': '2024-09-10T20:31:36.615261+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:31:36.615261+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 7060, 'log_dups_size': 0, 'ondisk_log_size': 7060, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28885958, 'num_objects': 7054, 'num_object_clones': 0, 'num_object_copies': 14108, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3617, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7054, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7058, 'num_write_kb': 28212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3439}, {'shards': '0,2', 'objects': 3615}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:42.175 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:31:42.175 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'7005", 'reported_seq': 7128, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:38.270445+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:31:38.270445+0000', 'last_peered': '2024-09-10T20:31:38.270445+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:31:38.270445+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 7005, 'log_dups_size': 0, 'ondisk_log_size': 7005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28672000, 'num_objects': 7000, 'num_object_clones': 0, 'num_object_copies': 14000, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3635, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7000, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4833, 'num_write_kb': 19332, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3633}, {'shards': '2', 'objects': 3367}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:42.175 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:31:42.175 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'7093", 'reported_seq': 7217, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:38.301786+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:31:38.301786+0000', 'last_peered': '2024-09-10T20:31:38.301786+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:31:38.301786+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 7093, 'log_dups_size': 0, 'ondisk_log_size': 7093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29028352, 'num_objects': 7087, 'num_object_clones': 0, 'num_object_copies': 14174, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7087, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4879, 'num_write_kb': 19516, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3610}, {'shards': '2', 'objects': 3477}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:42.175 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:31:42.176 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'7046", 'reported_seq': 7175, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:36.686594+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:31:36.686594+0000', 'last_peered': '2024-09-10T20:31:36.686594+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:31:36.686594+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 7046, 'log_dups_size': 0, 'ondisk_log_size': 7046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28852224, 'num_objects': 7044, 'num_object_clones': 0, 'num_object_copies': 14088, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3649, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7044, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3769, 'num_write_kb': 15076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3396}, {'shards': '0,2', 'objects': 3648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:42.176 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:31:42.176 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'7012", 'reported_seq': 7146, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:36.662296+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:31:36.662296+0000', 'last_peered': '2024-09-10T20:31:36.662296+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:31:36.662296+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 7012, 'log_dups_size': 0, 'ondisk_log_size': 7012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28688384, 'num_objects': 7004, 'num_object_clones': 0, 'num_object_copies': 14008, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3535, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7004, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3706, 'num_write_kb': 14824, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3472}, {'shards': '0,2', 'objects': 3532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:42.176 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:31:42.176 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'7139", 'reported_seq': 7265, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:36.552378+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:31:36.552378+0000', 'last_peered': '2024-09-10T20:31:36.552378+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:31:36.552378+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 7139, 'log_dups_size': 0, 'ondisk_log_size': 7139, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29224960, 'num_objects': 7135, 'num_object_clones': 0, 'num_object_copies': 14270, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3715, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7135, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3966, 'num_write_kb': 15864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3422}, {'shards': '0,2', 'objects': 3713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:42.176 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:31:42.176 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'6935", 'reported_seq': 7096, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:36.614487+0000', 'last_change': '2024-09-10T20:30:05.690055+0000', 'last_active': '2024-09-10T20:31:36.614487+0000', 'last_peered': '2024-09-10T20:31:36.614487+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:31:36.614487+0000', 'last_undegraded': '2024-09-10T20:30:05.378941+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 6935, 'log_dups_size': 0, 'ondisk_log_size': 6935, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28357091, 'num_objects': 6924, 'num_object_clones': 0, 'num_object_copies': 13848, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3471, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6924, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3542, 'num_write_kb': 14168, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3456}, {'shards': '0,2', 'objects': 3468}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:42.177 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:31:42.872 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:42.873 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39542/224866 objects degraded (17.585%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":214,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":112433,"data_bytes":460506556,"bytes_used":1559511040,"bytes_avail":288350781440,"bytes_total":289910292480,"degraded_objects":39542,"degraded_total":224866,"degraded_ratio":0.17584694884953705,"misplaced_objects":1279,"misplaced_total":224866,"misplaced_ratio":0.0056878318643102995,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":90751,"recovering_keys_per_sec":0,"num_objects_recovered":266,"num_bytes_recovered":1089536,"num_keys_recovered":0,"write_bytes_sec":1840282,"read_op_per_sec":0,"write_op_per_sec":449},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:31:42.897 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:31:42.898 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:31:43.062 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 135 20 115480 115460 3.34007 1.85156 0.0550471 0.0233818 2024-09-10T20:31:43.445 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:43.447 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:31:43.470 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2024-09-10T20:31:42.863489+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":467289532,"num_objects":114089,"num_object_clones":0,"num_object_copies":228178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":1279,"num_objects_unfound":0,"num_objects_dirty":114089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78464,"num_write_kb":313830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2330,"num_bytes_recovered":9523219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114169,"ondisk_log_size":114169,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1539228,"kb_used_data":918060,"kb_used_omap":12,"kb_used_meta":621107,"kb_avail":281576292,"statfs":{"total":289910292480,"available":288334123008,"internally_reserved":0,"allocated":940093440,"data_stored":939322245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":636013570},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,8,12,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28876800,"num_objects":7050,"num_object_clones":0,"num_object_copies":14100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-266,"num_objects_unfound":0,"num_objects_dirty":7050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7050,"num_write_kb":28200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":1089536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005814"},"pg_stats":[{"pgid":"2.f","version":"43'7176","reported_seq":7252,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:41.618849+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:31:41.618849+0000","last_peered":"2024-09-10T20:31:41.618849+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:31:41.618849+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":7176,"log_dups_size":0,"ondisk_log_size":7176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29392896,"num_objects":7176,"num_object_clones":0,"num_object_copies":14352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7176,"num_write_kb":28704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3569},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7169","reported_seq":7275,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:38.278787+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:31:38.278787+0000","last_peered":"2024-09-10T20:31:38.278787+0000","last_clean":"2024-09-10T20:31:38.278787+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:31:38.278787+0000","last_undegraded":"2024-09-10T20:31:38.278787+0000","last_fullsized":"2024-09-10T20:31:38.278787+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":7169,"log_dups_size":0,"ondisk_log_size":7169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29339648,"num_objects":7163,"num_object_clones":0,"num_object_copies":14326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5355,"num_write_kb":21420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7095","reported_seq":11870,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:31:38.311903+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:31:38.311903+0000","last_peered":"2024-09-10T20:31:38.311903+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:31:38.311903+0000","last_undegraded":"2024-09-10T20:31:38.311903+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":7095,"log_dups_size":0,"ondisk_log_size":7095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29044736,"num_objects":7091,"num_object_clones":0,"num_object_copies":14182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1279,"num_objects_unfound":0,"num_objects_dirty":7091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5223,"num_write_kb":20892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2325,"num_bytes_recovered":9519104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1279},{"shards":"2","objects":5812}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6930","reported_seq":7006,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:38.261845+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:31:38.261845+0000","last_peered":"2024-09-10T20:31:38.261845+0000","last_clean":"2024-09-10T20:31:38.261845+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:31:38.261845+0000","last_undegraded":"2024-09-10T20:31:38.261845+0000","last_fullsized":"2024-09-10T20:31:38.261845+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":6930,"log_dups_size":0,"ondisk_log_size":6930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28385280,"num_objects":6930,"num_object_clones":0,"num_object_copies":13860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6930,"num_write_kb":27720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7279","reported_seq":7401,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:41.705908+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:31:41.705908+0000","last_peered":"2024-09-10T20:31:41.705908+0000","last_clean":"2024-09-10T20:31:41.705908+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:31:41.705908+0000","last_undegraded":"2024-09-10T20:31:41.705908+0000","last_fullsized":"2024-09-10T20:31:41.705908+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":7279,"log_dups_size":0,"ondisk_log_size":7279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29786112,"num_objects":7272,"num_object_clones":0,"num_object_copies":14544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4275,"num_write_kb":17100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7130","reported_seq":7262,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:41.664124+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:31:41.664124+0000","last_peered":"2024-09-10T20:31:41.664124+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:31:41.664124+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":7130,"log_dups_size":0,"ondisk_log_size":7130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29184000,"num_objects":7125,"num_object_clones":0,"num_object_copies":14250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3994,"num_write_kb":15976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3511},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7085","reported_seq":7217,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:41.679372+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:31:41.679372+0000","last_peered":"2024-09-10T20:31:41.679372+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:31:41.679372+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":7085,"log_dups_size":0,"ondisk_log_size":7085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28995584,"num_objects":7079,"num_object_clones":0,"num_object_copies":14158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3680,"num_write_kb":14720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3524},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7241","reported_seq":7361,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:41.670854+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:31:41.670854+0000","last_peered":"2024-09-10T20:31:41.670854+0000","last_clean":"2024-09-10T20:31:41.670854+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:31:41.670854+0000","last_undegraded":"2024-09-10T20:31:41.670854+0000","last_fullsized":"2024-09-10T20:31:41.670854+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":7241,"log_dups_size":0,"ondisk_log_size":7241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29650944,"num_objects":7239,"num_object_clones":0,"num_object_copies":14478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4312,"num_write_kb":17248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6921","reported_seq":7048,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:38.261679+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:31:38.261679+0000","last_peered":"2024-09-10T20:31:38.261679+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:31:38.261679+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":6921,"log_dups_size":0,"ondisk_log_size":6921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28319744,"num_objects":6914,"num_object_clones":0,"num_object_copies":13828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4914,"num_write_kb":19656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":3388}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355479+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355479+0000","last_peered":"2024-09-10T20:30:05.355479+0000","last_clean":"2024-09-10T20:30:05.355479+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355479+0000","last_undegraded":"2024-09-10T20:30:05.355479+0000","last_fullsized":"2024-09-10T20:30:05.355479+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7219","reported_seq":7322,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:41.667305+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:31:41.667305+0000","last_peered":"2024-09-10T20:31:41.667305+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:31:41.667305+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":7219,"log_dups_size":0,"ondisk_log_size":7219,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29537222,"num_objects":7213,"num_object_clones":0,"num_object_copies":14426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7213,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7217,"num_write_kb":28848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3598},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7005","reported_seq":7128,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:38.270445+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:31:38.270445+0000","last_peered":"2024-09-10T20:31:38.270445+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:31:38.270445+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":7005,"log_dups_size":0,"ondisk_log_size":7005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28672000,"num_objects":7000,"num_object_clones":0,"num_object_copies":14000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4833,"num_write_kb":19332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":3367}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7093","reported_seq":7217,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:38.301786+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:31:38.301786+0000","last_peered":"2024-09-10T20:31:38.301786+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:31:38.301786+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":7093,"log_dups_size":0,"ondisk_log_size":7093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29028352,"num_objects":7087,"num_object_clones":0,"num_object_copies":14174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7087,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4879,"num_write_kb":19516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":3477}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7226","reported_seq":7356,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:41.673883+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:31:41.673883+0000","last_peered":"2024-09-10T20:31:41.673883+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:31:41.673883+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":7226,"log_dups_size":0,"ondisk_log_size":7226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29589504,"num_objects":7224,"num_object_clones":0,"num_object_copies":14448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3949,"num_write_kb":15796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3576},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7184","reported_seq":7314,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:41.722040+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:31:41.722040+0000","last_peered":"2024-09-10T20:31:41.722040+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:31:41.722040+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":7184,"log_dups_size":0,"ondisk_log_size":7184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29392896,"num_objects":7176,"num_object_clones":0,"num_object_copies":14352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3878,"num_write_kb":15512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3644},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345743+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345743+0000","last_peered":"2024-09-10T20:30:05.345743+0000","last_clean":"2024-09-10T20:30:05.345743+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345743+0000","last_undegraded":"2024-09-10T20:30:05.345743+0000","last_fullsized":"2024-09-10T20:30:05.345743+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7313","reported_seq":7440,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:41.667289+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:31:41.667289+0000","last_peered":"2024-09-10T20:31:41.667289+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:31:41.667289+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":7313,"log_dups_size":0,"ondisk_log_size":7313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29937664,"num_objects":7309,"num_object_clones":0,"num_object_copies":14618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4140,"num_write_kb":16560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3596},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7100","reported_seq":7261,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:41.656362+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:31:41.656362+0000","last_peered":"2024-09-10T20:31:41.656362+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:31:41.656362+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":7100,"log_dups_size":0,"ondisk_log_size":7100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29032931,"num_objects":7089,"num_object_clones":0,"num_object_copies":14178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3707,"num_write_kb":14828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3621},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":467289513,"num_objects":114087,"num_object_clones":0,"num_object_copies":228174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":1279,"num_objects_unfound":0,"num_objects_dirty":114087,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78462,"num_write_kb":313828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2328,"num_bytes_recovered":9523200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":939999232,"data_stored":939912520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114166,"ondisk_log_size":114166,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182438,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":677220,"kb_used_data":402644,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93694620,"statfs":{"total":96636764160,"available":95943290880,"internally_reserved":0,"allocated":412307456,"data_stored":412074323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,9,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986209,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520944,"kb_used_data":340704,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93850896,"statfs":{"total":96636764160,"available":96103317504,"internally_reserved":0,"allocated":348880896,"data_stored":348651840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184544938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659156,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":341064,"kb_used_data":174712,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":94030776,"statfs":{"total":96636764160,"available":96287514624,"internally_reserved":0,"allocated":178905088,"data_stored":178596082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":170323630},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,2,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":412172288,"data_stored":412168675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":178688000,"data_stored":178608514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":349138944,"data_stored":349135331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:31:44.062 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 136 20 116019 115999 3.33099 2.10547 0.0366299 0.0234476 2024-09-10T20:31:45.062 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 137 20 116576 116556 3.32256 2.17578 0.0237406 0.0235053 2024-09-10T20:31:46.062 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 138 20 117141 117121 3.31446 2.20703 0.0338212 0.0235643 2024-09-10T20:31:46.472 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:31:47.026 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:47.026 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:31:47.056 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":146,"stamp":"2024-09-10T20:31:46.864655+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":478373308,"num_objects":116795,"num_object_clones":0,"num_object_copies":233590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":1121,"num_objects_unfound":0,"num_objects_dirty":116795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81170,"num_write_kb":324654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2488,"num_bytes_recovered":10170387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":116875,"ondisk_log_size":116875,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1540544,"kb_used_data":940176,"kb_used_omap":12,"kb_used_meta":600307,"kb_avail":281574976,"statfs":{"total":289910292480,"available":288332775424,"internally_reserved":0,"allocated":962740224,"data_stored":961969029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":614714370},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,8,4,14,12,13,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29003776,"num_objects":7081,"num_object_clones":0,"num_object_copies":14162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-297,"num_objects_unfound":0,"num_objects_dirty":7081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7081,"num_write_kb":28324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":297,"num_bytes_recovered":1216512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004089"},"pg_stats":[{"pgid":"2.f","version":"43'7313","reported_seq":7389,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:46.581572+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:31:46.581572+0000","last_peered":"2024-09-10T20:31:46.581572+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:31:46.581572+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":7313,"log_dups_size":0,"ondisk_log_size":7313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29954048,"num_objects":7313,"num_object_clones":0,"num_object_copies":14626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7313,"num_write_kb":29252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3706},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7341","reported_seq":7445,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:43.206428+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:31:43.206428+0000","last_peered":"2024-09-10T20:31:43.206428+0000","last_clean":"2024-09-10T20:31:43.206428+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:31:43.206428+0000","last_undegraded":"2024-09-10T20:31:43.206428+0000","last_fullsized":"2024-09-10T20:31:43.206428+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":7341,"log_dups_size":0,"ondisk_log_size":7341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30044160,"num_objects":7335,"num_object_clones":0,"num_object_copies":14670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5527,"num_write_kb":22108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7258","reported_seq":12350,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:31:43.315325+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:31:43.315325+0000","last_peered":"2024-09-10T20:31:43.315325+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:31:43.315325+0000","last_undegraded":"2024-09-10T20:31:43.315325+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":7258,"log_dups_size":0,"ondisk_log_size":7258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29712384,"num_objects":7254,"num_object_clones":0,"num_object_copies":14508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1121,"num_objects_unfound":0,"num_objects_dirty":7254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5386,"num_write_kb":21544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2483,"num_bytes_recovered":10166272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1121},{"shards":"2","objects":6133}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7100","reported_seq":7176,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:43.225643+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:31:43.225643+0000","last_peered":"2024-09-10T20:31:43.225643+0000","last_clean":"2024-09-10T20:31:43.225643+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:31:43.225643+0000","last_undegraded":"2024-09-10T20:31:43.225643+0000","last_fullsized":"2024-09-10T20:31:43.225643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":7100,"log_dups_size":0,"ondisk_log_size":7100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29081600,"num_objects":7100,"num_object_clones":0,"num_object_copies":14200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7100,"num_write_kb":28400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7461","reported_seq":7583,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:46.710376+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:31:46.710376+0000","last_peered":"2024-09-10T20:31:46.710376+0000","last_clean":"2024-09-10T20:31:46.710376+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:31:46.710376+0000","last_undegraded":"2024-09-10T20:31:46.710376+0000","last_fullsized":"2024-09-10T20:31:46.710376+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":7461,"log_dups_size":0,"ondisk_log_size":7461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30531584,"num_objects":7454,"num_object_clones":0,"num_object_copies":14908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4457,"num_write_kb":17828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7281","reported_seq":7413,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:46.685451+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:31:46.685451+0000","last_peered":"2024-09-10T20:31:46.685451+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:31:46.685451+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":7281,"log_dups_size":0,"ondisk_log_size":7281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29802496,"num_objects":7276,"num_object_clones":0,"num_object_copies":14552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4145,"num_write_kb":16580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3662},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7254","reported_seq":7387,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:46.675027+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:31:46.675027+0000","last_peered":"2024-09-10T20:31:46.675027+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:31:46.675027+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":7254,"log_dups_size":0,"ondisk_log_size":7254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29687808,"num_objects":7248,"num_object_clones":0,"num_object_copies":14496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3849,"num_write_kb":15396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3693},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7403","reported_seq":7523,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:46.621462+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:31:46.621462+0000","last_peered":"2024-09-10T20:31:46.621462+0000","last_clean":"2024-09-10T20:31:46.621462+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:31:46.621462+0000","last_undegraded":"2024-09-10T20:31:46.621462+0000","last_fullsized":"2024-09-10T20:31:46.621462+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":7403,"log_dups_size":0,"ondisk_log_size":7403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30314496,"num_objects":7401,"num_object_clones":0,"num_object_copies":14802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4474,"num_write_kb":17896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7097","reported_seq":7224,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:43.294639+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:31:43.294639+0000","last_peered":"2024-09-10T20:31:43.294639+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:31:43.294639+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":7097,"log_dups_size":0,"ondisk_log_size":7097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29040640,"num_objects":7090,"num_object_clones":0,"num_object_copies":14180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5090,"num_write_kb":20360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":3564}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355479+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355479+0000","last_peered":"2024-09-10T20:30:05.355479+0000","last_clean":"2024-09-10T20:30:05.355479+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355479+0000","last_undegraded":"2024-09-10T20:30:05.355479+0000","last_fullsized":"2024-09-10T20:30:05.355479+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7405","reported_seq":7509,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:46.695624+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:31:46.695624+0000","last_peered":"2024-09-10T20:31:46.695624+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:31:46.695624+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":7405,"log_dups_size":0,"ondisk_log_size":7405,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30299078,"num_objects":7399,"num_object_clones":0,"num_object_copies":14798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7403,"num_write_kb":29592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3784},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7177","reported_seq":7301,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:43.287789+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:31:43.287789+0000","last_peered":"2024-09-10T20:31:43.287789+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:31:43.287789+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":7177,"log_dups_size":0,"ondisk_log_size":7177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29376512,"num_objects":7172,"num_object_clones":0,"num_object_copies":14344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5005,"num_write_kb":20020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":3539}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7266","reported_seq":7390,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:43.265395+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:31:43.265395+0000","last_peered":"2024-09-10T20:31:43.265395+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:31:43.265395+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":7266,"log_dups_size":0,"ondisk_log_size":7266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29736960,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5052,"num_write_kb":20208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":3650}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7387","reported_seq":7516,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:46.631013+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:31:46.631013+0000","last_peered":"2024-09-10T20:31:46.631013+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:31:46.631013+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":7387,"log_dups_size":0,"ondisk_log_size":7387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30248960,"num_objects":7385,"num_object_clones":0,"num_object_copies":14770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4110,"num_write_kb":16440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3737},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7361","reported_seq":7492,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:46.720802+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:31:46.720802+0000","last_peered":"2024-09-10T20:31:46.720802+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:31:46.720802+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":7361,"log_dups_size":0,"ondisk_log_size":7361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30117888,"num_objects":7353,"num_object_clones":0,"num_object_copies":14706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4055,"num_write_kb":16220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3821},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345743+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345743+0000","last_peered":"2024-09-10T20:30:05.345743+0000","last_clean":"2024-09-10T20:30:05.345743+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345743+0000","last_undegraded":"2024-09-10T20:30:05.345743+0000","last_fullsized":"2024-09-10T20:30:05.345743+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7476","reported_seq":7602,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:46.676684+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:31:46.676684+0000","last_peered":"2024-09-10T20:31:46.676684+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:31:46.676684+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":7476,"log_dups_size":0,"ondisk_log_size":7476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30605312,"num_objects":7472,"num_object_clones":0,"num_object_copies":14944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4303,"num_write_kb":17212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3759},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7292","reported_seq":7451,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:46.574603+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:31:46.574603+0000","last_peered":"2024-09-10T20:31:46.574603+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:31:46.574603+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":7292,"log_dups_size":0,"ondisk_log_size":7292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29819363,"num_objects":7281,"num_object_clones":0,"num_object_copies":14562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3899,"num_write_kb":15596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3813},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":478373289,"num_objects":116793,"num_object_clones":0,"num_object_copies":233586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":1121,"num_objects_unfound":0,"num_objects_dirty":116793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81168,"num_write_kb":324652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2486,"num_bytes_recovered":10170368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":962498560,"data_stored":962411848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":116872,"ondisk_log_size":116872,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182439,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":672784,"kb_used_data":410688,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93699056,"statfs":{"total":96636764160,"available":95947833344,"internally_reserved":0,"allocated":420544512,"data_stored":420311379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268365482},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,5,0,7,4,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986210,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":530528,"kb_used_data":346128,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93841312,"statfs":{"total":96636764160,"available":96093503488,"internally_reserved":0,"allocated":354435072,"data_stored":354206016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188804778},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,2,4,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659157,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":337232,"kb_used_data":183360,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":94034608,"statfs":{"total":96636764160,"available":96291438592,"internally_reserved":0,"allocated":187760640,"data_stored":187451634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":157544110},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,5,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":420347904,"data_stored":420344291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":187572224,"data_stored":187492738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":354578432,"data_stored":354574819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:31:47.058 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:31:47.062 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 139 20 117658 117638 3.30514 2.01953 0.0666594 0.0236296 2024-09-10T20:31:47.642 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:47.642 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:31:47.671 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":146,"stamp":"2024-09-10T20:31:46.864655+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":478373308,"num_objects":116795,"num_object_clones":0,"num_object_copies":233590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":1121,"num_objects_unfound":0,"num_objects_dirty":116795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81170,"num_write_kb":324654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2488,"num_bytes_recovered":10170387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":116875,"ondisk_log_size":116875,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1540544,"kb_used_data":940176,"kb_used_omap":12,"kb_used_meta":600307,"kb_avail":281574976,"statfs":{"total":289910292480,"available":288332775424,"internally_reserved":0,"allocated":962740224,"data_stored":961969029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":614714370},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,8,4,14,12,13,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29003776,"num_objects":7081,"num_object_clones":0,"num_object_copies":14162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-297,"num_objects_unfound":0,"num_objects_dirty":7081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7081,"num_write_kb":28324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":297,"num_bytes_recovered":1216512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004089"},"pg_stats":[{"pgid":"2.f","version":"43'7313","reported_seq":7389,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:46.581572+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:31:46.581572+0000","last_peered":"2024-09-10T20:31:46.581572+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:31:46.581572+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":7313,"log_dups_size":0,"ondisk_log_size":7313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29954048,"num_objects":7313,"num_object_clones":0,"num_object_copies":14626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7313,"num_write_kb":29252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3706},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7341","reported_seq":7445,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:43.206428+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:31:43.206428+0000","last_peered":"2024-09-10T20:31:43.206428+0000","last_clean":"2024-09-10T20:31:43.206428+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:31:43.206428+0000","last_undegraded":"2024-09-10T20:31:43.206428+0000","last_fullsized":"2024-09-10T20:31:43.206428+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":7341,"log_dups_size":0,"ondisk_log_size":7341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30044160,"num_objects":7335,"num_object_clones":0,"num_object_copies":14670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5527,"num_write_kb":22108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7258","reported_seq":12350,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:31:43.315325+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:31:43.315325+0000","last_peered":"2024-09-10T20:31:43.315325+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:31:43.315325+0000","last_undegraded":"2024-09-10T20:31:43.315325+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":7258,"log_dups_size":0,"ondisk_log_size":7258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29712384,"num_objects":7254,"num_object_clones":0,"num_object_copies":14508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1121,"num_objects_unfound":0,"num_objects_dirty":7254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5386,"num_write_kb":21544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2483,"num_bytes_recovered":10166272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1121},{"shards":"2","objects":6133}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7100","reported_seq":7176,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:43.225643+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:31:43.225643+0000","last_peered":"2024-09-10T20:31:43.225643+0000","last_clean":"2024-09-10T20:31:43.225643+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:31:43.225643+0000","last_undegraded":"2024-09-10T20:31:43.225643+0000","last_fullsized":"2024-09-10T20:31:43.225643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":7100,"log_dups_size":0,"ondisk_log_size":7100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29081600,"num_objects":7100,"num_object_clones":0,"num_object_copies":14200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7100,"num_write_kb":28400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7461","reported_seq":7583,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:46.710376+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:31:46.710376+0000","last_peered":"2024-09-10T20:31:46.710376+0000","last_clean":"2024-09-10T20:31:46.710376+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:31:46.710376+0000","last_undegraded":"2024-09-10T20:31:46.710376+0000","last_fullsized":"2024-09-10T20:31:46.710376+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":7461,"log_dups_size":0,"ondisk_log_size":7461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30531584,"num_objects":7454,"num_object_clones":0,"num_object_copies":14908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4457,"num_write_kb":17828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7281","reported_seq":7413,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:46.685451+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:31:46.685451+0000","last_peered":"2024-09-10T20:31:46.685451+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:31:46.685451+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":7281,"log_dups_size":0,"ondisk_log_size":7281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29802496,"num_objects":7276,"num_object_clones":0,"num_object_copies":14552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4145,"num_write_kb":16580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3662},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7254","reported_seq":7387,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:46.675027+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:31:46.675027+0000","last_peered":"2024-09-10T20:31:46.675027+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:31:46.675027+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":7254,"log_dups_size":0,"ondisk_log_size":7254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29687808,"num_objects":7248,"num_object_clones":0,"num_object_copies":14496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3849,"num_write_kb":15396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3693},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7403","reported_seq":7523,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:46.621462+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:31:46.621462+0000","last_peered":"2024-09-10T20:31:46.621462+0000","last_clean":"2024-09-10T20:31:46.621462+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:31:46.621462+0000","last_undegraded":"2024-09-10T20:31:46.621462+0000","last_fullsized":"2024-09-10T20:31:46.621462+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":7403,"log_dups_size":0,"ondisk_log_size":7403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30314496,"num_objects":7401,"num_object_clones":0,"num_object_copies":14802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4474,"num_write_kb":17896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7097","reported_seq":7224,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:43.294639+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:31:43.294639+0000","last_peered":"2024-09-10T20:31:43.294639+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:31:43.294639+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":7097,"log_dups_size":0,"ondisk_log_size":7097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29040640,"num_objects":7090,"num_object_clones":0,"num_object_copies":14180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5090,"num_write_kb":20360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":3564}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355479+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355479+0000","last_peered":"2024-09-10T20:30:05.355479+0000","last_clean":"2024-09-10T20:30:05.355479+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355479+0000","last_undegraded":"2024-09-10T20:30:05.355479+0000","last_fullsized":"2024-09-10T20:30:05.355479+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7405","reported_seq":7509,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:46.695624+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:31:46.695624+0000","last_peered":"2024-09-10T20:31:46.695624+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:31:46.695624+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":7405,"log_dups_size":0,"ondisk_log_size":7405,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30299078,"num_objects":7399,"num_object_clones":0,"num_object_copies":14798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7403,"num_write_kb":29592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3784},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7177","reported_seq":7301,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:43.287789+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:31:43.287789+0000","last_peered":"2024-09-10T20:31:43.287789+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:31:43.287789+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":7177,"log_dups_size":0,"ondisk_log_size":7177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29376512,"num_objects":7172,"num_object_clones":0,"num_object_copies":14344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5005,"num_write_kb":20020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":3539}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7266","reported_seq":7390,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:43.265395+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:31:43.265395+0000","last_peered":"2024-09-10T20:31:43.265395+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:31:43.265395+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":7266,"log_dups_size":0,"ondisk_log_size":7266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29736960,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5052,"num_write_kb":20208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":3650}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7387","reported_seq":7516,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:46.631013+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:31:46.631013+0000","last_peered":"2024-09-10T20:31:46.631013+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:31:46.631013+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":7387,"log_dups_size":0,"ondisk_log_size":7387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30248960,"num_objects":7385,"num_object_clones":0,"num_object_copies":14770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4110,"num_write_kb":16440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3737},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7361","reported_seq":7492,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:46.720802+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:31:46.720802+0000","last_peered":"2024-09-10T20:31:46.720802+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:31:46.720802+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":7361,"log_dups_size":0,"ondisk_log_size":7361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30117888,"num_objects":7353,"num_object_clones":0,"num_object_copies":14706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4055,"num_write_kb":16220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3821},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345743+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345743+0000","last_peered":"2024-09-10T20:30:05.345743+0000","last_clean":"2024-09-10T20:30:05.345743+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345743+0000","last_undegraded":"2024-09-10T20:30:05.345743+0000","last_fullsized":"2024-09-10T20:30:05.345743+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7476","reported_seq":7602,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:46.676684+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:31:46.676684+0000","last_peered":"2024-09-10T20:31:46.676684+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:31:46.676684+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":7476,"log_dups_size":0,"ondisk_log_size":7476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30605312,"num_objects":7472,"num_object_clones":0,"num_object_copies":14944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4303,"num_write_kb":17212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3759},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7292","reported_seq":7451,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:46.574603+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:31:46.574603+0000","last_peered":"2024-09-10T20:31:46.574603+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:31:46.574603+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":7292,"log_dups_size":0,"ondisk_log_size":7292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29819363,"num_objects":7281,"num_object_clones":0,"num_object_copies":14562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3899,"num_write_kb":15596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3813},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":478373289,"num_objects":116793,"num_object_clones":0,"num_object_copies":233586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":1121,"num_objects_unfound":0,"num_objects_dirty":116793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81168,"num_write_kb":324652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2486,"num_bytes_recovered":10170368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":962498560,"data_stored":962411848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":116872,"ondisk_log_size":116872,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182439,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":672784,"kb_used_data":410688,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93699056,"statfs":{"total":96636764160,"available":95947833344,"internally_reserved":0,"allocated":420544512,"data_stored":420311379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268365482},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,5,0,7,4,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986210,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":530528,"kb_used_data":346128,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93841312,"statfs":{"total":96636764160,"available":96093503488,"internally_reserved":0,"allocated":354435072,"data_stored":354206016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188804778},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,2,4,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659157,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":337232,"kb_used_data":183360,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":94034608,"statfs":{"total":96636764160,"available":96291438592,"internally_reserved":0,"allocated":187760640,"data_stored":187451634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":157544110},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,5,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":420347904,"data_stored":420344291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":187572224,"data_stored":187492738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":354578432,"data_stored":354574819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:31:47.672 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:31:47.672 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'7313", 'reported_seq': 7389, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:46.581572+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:31:46.581572+0000', 'last_peered': '2024-09-10T20:31:46.581572+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:31:46.581572+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 7313, 'log_dups_size': 0, 'ondisk_log_size': 7313, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29954048, 'num_objects': 7313, 'num_object_clones': 0, 'num_object_copies': 14626, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3607, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7313, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7313, 'num_write_kb': 29252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3706}, {'shards': '0,2', 'objects': 3607}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:47.673 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T20:31:47.673 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'7258", 'reported_seq': 12350, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:31:43.315325+0000', 'last_change': '2024-09-10T20:30:05.653560+0000', 'last_active': '2024-09-10T20:31:43.315325+0000', 'last_peered': '2024-09-10T20:31:43.315325+0000', 'last_clean': '2024-09-10T20:30:03.279259+0000', 'last_became_active': '2024-09-10T20:30:05.629664+0000', 'last_became_peered': '2024-09-10T20:30:05.629664+0000', 'last_unstale': '2024-09-10T20:31:43.315325+0000', 'last_undegraded': '2024-09-10T20:31:43.315325+0000', 'last_fullsized': '2024-09-10T20:30:05.312061+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1910", 'last_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'objects_scrubbed': 1868, 'log_size': 7258, 'log_dups_size': 0, 'ondisk_log_size': 7258, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 673, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29712384, 'num_objects': 7254, 'num_object_clones': 0, 'num_object_copies': 14508, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1121, 'num_objects_unfound': 0, 'num_objects_dirty': 7254, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5386, 'num_write_kb': 21544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2483, 'num_bytes_recovered': 10166272, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1121}, {'shards': '2', 'objects': 6133}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:47.673 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:31:47.673 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'7281", 'reported_seq': 7413, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:46.685451+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:31:46.685451+0000', 'last_peered': '2024-09-10T20:31:46.685451+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:31:46.685451+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 7281, 'log_dups_size': 0, 'ondisk_log_size': 7281, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29802496, 'num_objects': 7276, 'num_object_clones': 0, 'num_object_copies': 14552, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3616, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7276, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4145, 'num_write_kb': 16580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3662}, {'shards': '0,2', 'objects': 3614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:47.673 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:31:47.673 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'7254", 'reported_seq': 7387, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:46.675027+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:31:46.675027+0000', 'last_peered': '2024-09-10T20:31:46.675027+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:31:46.675027+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 7254, 'log_dups_size': 0, 'ondisk_log_size': 7254, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29687808, 'num_objects': 7248, 'num_object_clones': 0, 'num_object_copies': 14496, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3557, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7248, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3849, 'num_write_kb': 15396, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3693}, {'shards': '0,2', 'objects': 3555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:47.673 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:31:47.673 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'7097", 'reported_seq': 7224, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:43.294639+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:31:43.294639+0000', 'last_peered': '2024-09-10T20:31:43.294639+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:31:43.294639+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 7097, 'log_dups_size': 0, 'ondisk_log_size': 7097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29040640, 'num_objects': 7090, 'num_object_clones': 0, 'num_object_copies': 14180, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3528, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7090, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5090, 'num_write_kb': 20360, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 3564}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:47.673 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:31:47.673 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'7405", 'reported_seq': 7509, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:46.695624+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:31:46.695624+0000', 'last_peered': '2024-09-10T20:31:46.695624+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:31:46.695624+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 7405, 'log_dups_size': 0, 'ondisk_log_size': 7405, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30299078, 'num_objects': 7399, 'num_object_clones': 0, 'num_object_copies': 14798, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3617, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7399, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7403, 'num_write_kb': 29592, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3784}, {'shards': '0,2', 'objects': 3615}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:47.674 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:31:47.674 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'7177", 'reported_seq': 7301, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:43.287789+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:31:43.287789+0000', 'last_peered': '2024-09-10T20:31:43.287789+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:31:43.287789+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 7177, 'log_dups_size': 0, 'ondisk_log_size': 7177, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29376512, 'num_objects': 7172, 'num_object_clones': 0, 'num_object_copies': 14344, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3635, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7172, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5005, 'num_write_kb': 20020, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3633}, {'shards': '2', 'objects': 3539}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:47.674 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:31:47.674 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'7266", 'reported_seq': 7390, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:43.265395+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:31:43.265395+0000', 'last_peered': '2024-09-10T20:31:43.265395+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:31:43.265395+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 7266, 'log_dups_size': 0, 'ondisk_log_size': 7266, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29736960, 'num_objects': 7260, 'num_object_clones': 0, 'num_object_copies': 14520, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7260, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5052, 'num_write_kb': 20208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3610}, {'shards': '2', 'objects': 3650}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:47.674 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:31:47.674 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'7387", 'reported_seq': 7516, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:46.631013+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:31:46.631013+0000', 'last_peered': '2024-09-10T20:31:46.631013+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:31:46.631013+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 7387, 'log_dups_size': 0, 'ondisk_log_size': 7387, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30248960, 'num_objects': 7385, 'num_object_clones': 0, 'num_object_copies': 14770, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3649, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7385, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4110, 'num_write_kb': 16440, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3737}, {'shards': '0,2', 'objects': 3648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:47.674 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:31:47.674 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'7361", 'reported_seq': 7492, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:46.720802+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:31:46.720802+0000', 'last_peered': '2024-09-10T20:31:46.720802+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:31:46.720802+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 7361, 'log_dups_size': 0, 'ondisk_log_size': 7361, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30117888, 'num_objects': 7353, 'num_object_clones': 0, 'num_object_copies': 14706, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3535, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7353, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4055, 'num_write_kb': 16220, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3821}, {'shards': '0,2', 'objects': 3532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:47.674 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:31:47.674 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'7476", 'reported_seq': 7602, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:46.676684+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:31:46.676684+0000', 'last_peered': '2024-09-10T20:31:46.676684+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:31:46.676684+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 7476, 'log_dups_size': 0, 'ondisk_log_size': 7476, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30605312, 'num_objects': 7472, 'num_object_clones': 0, 'num_object_copies': 14944, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3715, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7472, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4303, 'num_write_kb': 17212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3759}, {'shards': '0,2', 'objects': 3713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:47.675 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:31:47.675 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'7292", 'reported_seq': 7451, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:46.574603+0000', 'last_change': '2024-09-10T20:30:05.690055+0000', 'last_active': '2024-09-10T20:31:46.574603+0000', 'last_peered': '2024-09-10T20:31:46.574603+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:31:46.574603+0000', 'last_undegraded': '2024-09-10T20:30:05.378941+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 7292, 'log_dups_size': 0, 'ondisk_log_size': 7292, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29819363, 'num_objects': 7281, 'num_object_clones': 0, 'num_object_copies': 14562, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3471, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7281, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3899, 'num_write_kb': 15596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3813}, {'shards': '0,2', 'objects': 3468}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:47.675 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:31:48.063 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout:2024-09-10T20:31:48.060007+0000 min lat: 0.00191414 max lat: 2.51074 avg lat: 0.0236997 2024-09-10T20:31:48.063 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T20:31:48.063 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 140 20 118153 118133 3.29535 1.93359 0.0382051 0.0236997 2024-09-10T20:31:48.397 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:48.397 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39542/233590 objects degraded (16.928%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":220,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":116795,"data_bytes":478373308,"bytes_used":1577517056,"bytes_avail":288332775424,"bytes_total":289910292480,"degraded_objects":39542,"degraded_total":233590,"degraded_ratio":0.16927950682820325,"misplaced_objects":1121,"misplaced_total":233590,"misplaced_ratio":0.0047990068067982365,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":101341,"recovering_keys_per_sec":0,"num_objects_recovered":297,"num_bytes_recovered":1216512,"num_keys_recovered":0,"write_bytes_sec":2416157,"read_op_per_sec":0,"write_op_per_sec":589},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:31:48.421 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:31:48.421 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:31:48.995 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:48.998 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:31:49.019 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2024-09-10T20:31:48.865194+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":482411964,"num_objects":117781,"num_object_clones":0,"num_object_copies":235562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":971,"num_objects_unfound":0,"num_objects_dirty":117781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82156,"num_write_kb":328598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2638,"num_bytes_recovered":10784787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117861,"ondisk_log_size":117861,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1546344,"kb_used_data":954296,"kb_used_omap":12,"kb_used_meta":591987,"kb_avail":281569176,"statfs":{"total":289910292480,"available":288326836224,"internally_reserved":0,"allocated":977199104,"data_stored":976427909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":606194690},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,7,2,16,12,13],"upper_bound":128},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25927680,"num_objects":6330,"num_object_clones":0,"num_object_copies":12660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-447,"num_objects_unfound":0,"num_objects_dirty":6330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6330,"num_write_kb":25320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":447,"num_bytes_recovered":1830912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003914"},"pg_stats":[{"pgid":"2.f","version":"43'7313","reported_seq":7389,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:46.581572+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:31:46.581572+0000","last_peered":"2024-09-10T20:31:46.581572+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:31:46.581572+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":7313,"log_dups_size":0,"ondisk_log_size":7313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29954048,"num_objects":7313,"num_object_clones":0,"num_object_copies":14626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7313,"num_write_kb":29252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3706},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7519","reported_seq":7625,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:48.302884+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:31:48.302884+0000","last_peered":"2024-09-10T20:31:48.302884+0000","last_clean":"2024-09-10T20:31:48.302884+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:31:48.302884+0000","last_undegraded":"2024-09-10T20:31:48.302884+0000","last_fullsized":"2024-09-10T20:31:48.302884+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":7519,"log_dups_size":0,"ondisk_log_size":7519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30773248,"num_objects":7513,"num_object_clones":0,"num_object_copies":15026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7513,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5705,"num_write_kb":22820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7417","reported_seq":12808,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:31:48.329974+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:31:48.329974+0000","last_peered":"2024-09-10T20:31:48.329974+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:31:48.329974+0000","last_undegraded":"2024-09-10T20:31:48.329974+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":7417,"log_dups_size":0,"ondisk_log_size":7417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30363648,"num_objects":7413,"num_object_clones":0,"num_object_copies":14826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":971,"num_objects_unfound":0,"num_objects_dirty":7413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5545,"num_write_kb":22180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2633,"num_bytes_recovered":10780672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":971},{"shards":"2","objects":6442}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7252","reported_seq":7328,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:48.269942+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:31:48.269942+0000","last_peered":"2024-09-10T20:31:48.269942+0000","last_clean":"2024-09-10T20:31:48.269942+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:31:48.269942+0000","last_undegraded":"2024-09-10T20:31:48.269942+0000","last_fullsized":"2024-09-10T20:31:48.269942+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":7252,"log_dups_size":0,"ondisk_log_size":7252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29704192,"num_objects":7252,"num_object_clones":0,"num_object_copies":14504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7252,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7252,"num_write_kb":29008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7461","reported_seq":7583,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:46.710376+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:31:46.710376+0000","last_peered":"2024-09-10T20:31:46.710376+0000","last_clean":"2024-09-10T20:31:46.710376+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:31:46.710376+0000","last_undegraded":"2024-09-10T20:31:46.710376+0000","last_fullsized":"2024-09-10T20:31:46.710376+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":7461,"log_dups_size":0,"ondisk_log_size":7461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30531584,"num_objects":7454,"num_object_clones":0,"num_object_copies":14908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4457,"num_write_kb":17828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7281","reported_seq":7413,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:46.685451+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:31:46.685451+0000","last_peered":"2024-09-10T20:31:46.685451+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:31:46.685451+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":7281,"log_dups_size":0,"ondisk_log_size":7281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29802496,"num_objects":7276,"num_object_clones":0,"num_object_copies":14552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4145,"num_write_kb":16580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3662},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7254","reported_seq":7387,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:46.675027+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:31:46.675027+0000","last_peered":"2024-09-10T20:31:46.675027+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:31:46.675027+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":7254,"log_dups_size":0,"ondisk_log_size":7254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29687808,"num_objects":7248,"num_object_clones":0,"num_object_copies":14496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3849,"num_write_kb":15396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3693},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7403","reported_seq":7523,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:46.621462+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:31:46.621462+0000","last_peered":"2024-09-10T20:31:46.621462+0000","last_clean":"2024-09-10T20:31:46.621462+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:31:46.621462+0000","last_undegraded":"2024-09-10T20:31:46.621462+0000","last_fullsized":"2024-09-10T20:31:46.621462+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":7403,"log_dups_size":0,"ondisk_log_size":7403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30314496,"num_objects":7401,"num_object_clones":0,"num_object_copies":14802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4474,"num_write_kb":17896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7254","reported_seq":7382,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:48.315307+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:31:48.315307+0000","last_peered":"2024-09-10T20:31:48.315307+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:31:48.315307+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":7254,"log_dups_size":0,"ondisk_log_size":7254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29683712,"num_objects":7247,"num_object_clones":0,"num_object_copies":14494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5247,"num_write_kb":20988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":3721}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355479+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355479+0000","last_peered":"2024-09-10T20:30:05.355479+0000","last_clean":"2024-09-10T20:30:05.355479+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355479+0000","last_undegraded":"2024-09-10T20:30:05.355479+0000","last_fullsized":"2024-09-10T20:30:05.355479+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7405","reported_seq":7509,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:46.695624+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:31:46.695624+0000","last_peered":"2024-09-10T20:31:46.695624+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:31:46.695624+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":7405,"log_dups_size":0,"ondisk_log_size":7405,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30299078,"num_objects":7399,"num_object_clones":0,"num_object_copies":14798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7403,"num_write_kb":29592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3784},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7344","reported_seq":7468,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:48.286118+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:31:48.286118+0000","last_peered":"2024-09-10T20:31:48.286118+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:31:48.286118+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":7344,"log_dups_size":0,"ondisk_log_size":7344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30060544,"num_objects":7339,"num_object_clones":0,"num_object_copies":14678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5172,"num_write_kb":20688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":3706}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7439","reported_seq":7563,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:48.274527+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:31:48.274527+0000","last_peered":"2024-09-10T20:31:48.274527+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:31:48.274527+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":7439,"log_dups_size":0,"ondisk_log_size":7439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30445568,"num_objects":7433,"num_object_clones":0,"num_object_copies":14866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5225,"num_write_kb":20900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":3823}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7387","reported_seq":7516,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:46.631013+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:31:46.631013+0000","last_peered":"2024-09-10T20:31:46.631013+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:31:46.631013+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":7387,"log_dups_size":0,"ondisk_log_size":7387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30248960,"num_objects":7385,"num_object_clones":0,"num_object_copies":14770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4110,"num_write_kb":16440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3737},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7361","reported_seq":7492,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:46.720802+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:31:46.720802+0000","last_peered":"2024-09-10T20:31:46.720802+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:31:46.720802+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":7361,"log_dups_size":0,"ondisk_log_size":7361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30117888,"num_objects":7353,"num_object_clones":0,"num_object_copies":14706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4055,"num_write_kb":16220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3821},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345743+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345743+0000","last_peered":"2024-09-10T20:30:05.345743+0000","last_clean":"2024-09-10T20:30:05.345743+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345743+0000","last_undegraded":"2024-09-10T20:30:05.345743+0000","last_fullsized":"2024-09-10T20:30:05.345743+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7476","reported_seq":7602,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:46.676684+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:31:46.676684+0000","last_peered":"2024-09-10T20:31:46.676684+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:31:46.676684+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":7476,"log_dups_size":0,"ondisk_log_size":7476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30605312,"num_objects":7472,"num_object_clones":0,"num_object_copies":14944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4303,"num_write_kb":17212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3759},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7292","reported_seq":7451,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:46.574603+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:31:46.574603+0000","last_peered":"2024-09-10T20:31:46.574603+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:31:46.574603+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":7292,"log_dups_size":0,"ondisk_log_size":7292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29819363,"num_objects":7281,"num_object_clones":0,"num_object_copies":14562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3899,"num_write_kb":15596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3813},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":482411945,"num_objects":117779,"num_object_clones":0,"num_object_copies":235558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":971,"num_objects_unfound":0,"num_objects_dirty":117779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82154,"num_write_kb":328596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2636,"num_bytes_recovered":10784768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":976920576,"data_stored":976833864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117858,"ondisk_log_size":117858,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182439,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":672784,"kb_used_data":410688,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93699056,"statfs":{"total":96636764160,"available":95947833344,"internally_reserved":0,"allocated":420544512,"data_stored":420311379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268365482},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,5,0,7,4,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986211,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523336,"kb_used_data":351416,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93848504,"statfs":{"total":96636764160,"available":96100868096,"internally_reserved":0,"allocated":359849984,"data_stored":359620928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":176025258},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,3,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659158,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":350224,"kb_used_data":192192,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":94021616,"statfs":{"total":96636764160,"available":96278134784,"internally_reserved":0,"allocated":196804608,"data_stored":196495602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":161803950},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,6,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":420347904,"data_stored":420344291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":196493312,"data_stored":196413826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360079360,"data_stored":360075747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:31:49.063 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 141 20 118648 118628 3.28569 1.93359 0.0304941 0.0237692 2024-09-10T20:31:50.063 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 142 20 119215 119195 3.27815 2.21484 0.0450624 0.0238254 2024-09-10T20:31:51.063 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 143 20 119749 119729 3.26981 2.08594 0.0248109 0.0238853 2024-09-10T20:31:52.021 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:31:52.065 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 144 20 120312 120292 3.26232 2.19922 0.0443257 0.0239417 2024-09-10T20:31:52.585 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:52.586 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:31:52.610 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":148,"stamp":"2024-09-10T20:31:50.866002+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":482411964,"num_objects":117781,"num_object_clones":0,"num_object_copies":235562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":971,"num_objects_unfound":0,"num_objects_dirty":117781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82156,"num_write_kb":328598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2638,"num_bytes_recovered":10784787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117861,"ondisk_log_size":117861,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1546344,"kb_used_data":954296,"kb_used_omap":12,"kb_used_meta":591987,"kb_avail":281569176,"statfs":{"total":289910292480,"available":288326836224,"internally_reserved":0,"allocated":977199104,"data_stored":976427909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":606194690},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,7,2,16,12,13],"upper_bound":128},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21905408,"num_objects":5348,"num_object_clones":0,"num_object_copies":10696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-308,"num_objects_unfound":0,"num_objects_dirty":5348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5348,"num_write_kb":21392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":1261568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003939"},"pg_stats":[{"pgid":"2.f","version":"43'7313","reported_seq":7389,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:46.581572+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:31:46.581572+0000","last_peered":"2024-09-10T20:31:46.581572+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:31:46.581572+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":7313,"log_dups_size":0,"ondisk_log_size":7313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29954048,"num_objects":7313,"num_object_clones":0,"num_object_copies":14626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7313,"num_write_kb":29252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3706},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7519","reported_seq":7625,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:48.302884+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:31:48.302884+0000","last_peered":"2024-09-10T20:31:48.302884+0000","last_clean":"2024-09-10T20:31:48.302884+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:31:48.302884+0000","last_undegraded":"2024-09-10T20:31:48.302884+0000","last_fullsized":"2024-09-10T20:31:48.302884+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":7519,"log_dups_size":0,"ondisk_log_size":7519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30773248,"num_objects":7513,"num_object_clones":0,"num_object_copies":15026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7513,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5705,"num_write_kb":22820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7417","reported_seq":12808,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:31:48.329974+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:31:48.329974+0000","last_peered":"2024-09-10T20:31:48.329974+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:31:48.329974+0000","last_undegraded":"2024-09-10T20:31:48.329974+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":7417,"log_dups_size":0,"ondisk_log_size":7417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30363648,"num_objects":7413,"num_object_clones":0,"num_object_copies":14826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":971,"num_objects_unfound":0,"num_objects_dirty":7413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5545,"num_write_kb":22180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2633,"num_bytes_recovered":10780672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":971},{"shards":"2","objects":6442}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7252","reported_seq":7328,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:48.269942+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:31:48.269942+0000","last_peered":"2024-09-10T20:31:48.269942+0000","last_clean":"2024-09-10T20:31:48.269942+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:31:48.269942+0000","last_undegraded":"2024-09-10T20:31:48.269942+0000","last_fullsized":"2024-09-10T20:31:48.269942+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":7252,"log_dups_size":0,"ondisk_log_size":7252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29704192,"num_objects":7252,"num_object_clones":0,"num_object_copies":14504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7252,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7252,"num_write_kb":29008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7461","reported_seq":7583,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:46.710376+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:31:46.710376+0000","last_peered":"2024-09-10T20:31:46.710376+0000","last_clean":"2024-09-10T20:31:46.710376+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:31:46.710376+0000","last_undegraded":"2024-09-10T20:31:46.710376+0000","last_fullsized":"2024-09-10T20:31:46.710376+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":7461,"log_dups_size":0,"ondisk_log_size":7461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30531584,"num_objects":7454,"num_object_clones":0,"num_object_copies":14908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4457,"num_write_kb":17828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7281","reported_seq":7413,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:46.685451+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:31:46.685451+0000","last_peered":"2024-09-10T20:31:46.685451+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:31:46.685451+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":7281,"log_dups_size":0,"ondisk_log_size":7281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29802496,"num_objects":7276,"num_object_clones":0,"num_object_copies":14552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4145,"num_write_kb":16580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3662},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7254","reported_seq":7387,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:46.675027+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:31:46.675027+0000","last_peered":"2024-09-10T20:31:46.675027+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:31:46.675027+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":7254,"log_dups_size":0,"ondisk_log_size":7254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29687808,"num_objects":7248,"num_object_clones":0,"num_object_copies":14496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3849,"num_write_kb":15396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3693},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7403","reported_seq":7523,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:46.621462+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:31:46.621462+0000","last_peered":"2024-09-10T20:31:46.621462+0000","last_clean":"2024-09-10T20:31:46.621462+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:31:46.621462+0000","last_undegraded":"2024-09-10T20:31:46.621462+0000","last_fullsized":"2024-09-10T20:31:46.621462+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":7403,"log_dups_size":0,"ondisk_log_size":7403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30314496,"num_objects":7401,"num_object_clones":0,"num_object_copies":14802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4474,"num_write_kb":17896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7254","reported_seq":7382,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:48.315307+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:31:48.315307+0000","last_peered":"2024-09-10T20:31:48.315307+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:31:48.315307+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":7254,"log_dups_size":0,"ondisk_log_size":7254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29683712,"num_objects":7247,"num_object_clones":0,"num_object_copies":14494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5247,"num_write_kb":20988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":3721}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355479+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355479+0000","last_peered":"2024-09-10T20:30:05.355479+0000","last_clean":"2024-09-10T20:30:05.355479+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355479+0000","last_undegraded":"2024-09-10T20:30:05.355479+0000","last_fullsized":"2024-09-10T20:30:05.355479+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7405","reported_seq":7509,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:46.695624+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:31:46.695624+0000","last_peered":"2024-09-10T20:31:46.695624+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:31:46.695624+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":7405,"log_dups_size":0,"ondisk_log_size":7405,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30299078,"num_objects":7399,"num_object_clones":0,"num_object_copies":14798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7403,"num_write_kb":29592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3784},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7344","reported_seq":7468,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:48.286118+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:31:48.286118+0000","last_peered":"2024-09-10T20:31:48.286118+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:31:48.286118+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":7344,"log_dups_size":0,"ondisk_log_size":7344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30060544,"num_objects":7339,"num_object_clones":0,"num_object_copies":14678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5172,"num_write_kb":20688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":3706}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7439","reported_seq":7563,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:48.274527+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:31:48.274527+0000","last_peered":"2024-09-10T20:31:48.274527+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:31:48.274527+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":7439,"log_dups_size":0,"ondisk_log_size":7439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30445568,"num_objects":7433,"num_object_clones":0,"num_object_copies":14866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5225,"num_write_kb":20900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":3823}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7387","reported_seq":7516,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:46.631013+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:31:46.631013+0000","last_peered":"2024-09-10T20:31:46.631013+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:31:46.631013+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":7387,"log_dups_size":0,"ondisk_log_size":7387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30248960,"num_objects":7385,"num_object_clones":0,"num_object_copies":14770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4110,"num_write_kb":16440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3737},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7361","reported_seq":7492,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:46.720802+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:31:46.720802+0000","last_peered":"2024-09-10T20:31:46.720802+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:31:46.720802+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":7361,"log_dups_size":0,"ondisk_log_size":7361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30117888,"num_objects":7353,"num_object_clones":0,"num_object_copies":14706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4055,"num_write_kb":16220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3821},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345743+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345743+0000","last_peered":"2024-09-10T20:30:05.345743+0000","last_clean":"2024-09-10T20:30:05.345743+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345743+0000","last_undegraded":"2024-09-10T20:30:05.345743+0000","last_fullsized":"2024-09-10T20:30:05.345743+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7476","reported_seq":7602,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:46.676684+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:31:46.676684+0000","last_peered":"2024-09-10T20:31:46.676684+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:31:46.676684+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":7476,"log_dups_size":0,"ondisk_log_size":7476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30605312,"num_objects":7472,"num_object_clones":0,"num_object_copies":14944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4303,"num_write_kb":17212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3759},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7292","reported_seq":7451,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:46.574603+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:31:46.574603+0000","last_peered":"2024-09-10T20:31:46.574603+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:31:46.574603+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":7292,"log_dups_size":0,"ondisk_log_size":7292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29819363,"num_objects":7281,"num_object_clones":0,"num_object_copies":14562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3899,"num_write_kb":15596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3813},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":482411945,"num_objects":117779,"num_object_clones":0,"num_object_copies":235558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":971,"num_objects_unfound":0,"num_objects_dirty":117779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82154,"num_write_kb":328596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2636,"num_bytes_recovered":10784768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":976920576,"data_stored":976833864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117858,"ondisk_log_size":117858,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182439,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":672784,"kb_used_data":410688,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93699056,"statfs":{"total":96636764160,"available":95947833344,"internally_reserved":0,"allocated":420544512,"data_stored":420311379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268365482},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,5,0,7,4,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986211,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523336,"kb_used_data":351416,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93848504,"statfs":{"total":96636764160,"available":96100868096,"internally_reserved":0,"allocated":359849984,"data_stored":359620928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":176025258},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,3,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659158,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":350224,"kb_used_data":192192,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":94021616,"statfs":{"total":96636764160,"available":96278134784,"internally_reserved":0,"allocated":196804608,"data_stored":196495602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":161803950},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,6,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":420347904,"data_stored":420344291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":196493312,"data_stored":196413826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360079360,"data_stored":360075747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:31:52.612 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:31:53.065 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 145 20 120781 120761 3.25246 1.83203 0.0789612 0.0240124 2024-09-10T20:31:53.223 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:53.224 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:31:53.257 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":149,"stamp":"2024-09-10T20:31:52.866410+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":489186748,"num_objects":119435,"num_object_clones":0,"num_object_copies":238870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":971,"num_objects_unfound":0,"num_objects_dirty":119435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83810,"num_write_kb":335214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2638,"num_bytes_recovered":10784787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119515,"ondisk_log_size":119515,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1558596,"kb_used_data":962388,"kb_used_omap":12,"kb_used_meta":596147,"kb_avail":281556924,"statfs":{"total":289910292480,"available":288314290176,"internally_reserved":0,"allocated":985485312,"data_stored":984714117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":610454530},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,3,5,9,11,10,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28680192,"num_objects":7002,"num_object_clones":0,"num_object_copies":14004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-308,"num_objects_unfound":0,"num_objects_dirty":7002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7002,"num_write_kb":28008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":1261568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003734"},"pg_stats":[{"pgid":"2.f","version":"43'7485","reported_seq":7560,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:51.723614+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:31:51.723614+0000","last_peered":"2024-09-10T20:31:51.723614+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:31:51.723614+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":7485,"log_dups_size":0,"ondisk_log_size":7485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30658560,"num_objects":7485,"num_object_clones":0,"num_object_copies":14970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7485,"num_write_kb":29940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3878},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7519","reported_seq":7625,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:48.302884+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:31:48.302884+0000","last_peered":"2024-09-10T20:31:48.302884+0000","last_clean":"2024-09-10T20:31:48.302884+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:31:48.302884+0000","last_undegraded":"2024-09-10T20:31:48.302884+0000","last_fullsized":"2024-09-10T20:31:48.302884+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":7519,"log_dups_size":0,"ondisk_log_size":7519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30773248,"num_objects":7513,"num_object_clones":0,"num_object_copies":15026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7513,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5705,"num_write_kb":22820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7417","reported_seq":12808,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:31:48.329974+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:31:48.329974+0000","last_peered":"2024-09-10T20:31:48.329974+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:31:48.329974+0000","last_undegraded":"2024-09-10T20:31:48.329974+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":7417,"log_dups_size":0,"ondisk_log_size":7417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30363648,"num_objects":7413,"num_object_clones":0,"num_object_copies":14826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":971,"num_objects_unfound":0,"num_objects_dirty":7413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5545,"num_write_kb":22180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2633,"num_bytes_recovered":10780672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":971},{"shards":"2","objects":6442}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7252","reported_seq":7328,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:48.269942+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:31:48.269942+0000","last_peered":"2024-09-10T20:31:48.269942+0000","last_clean":"2024-09-10T20:31:48.269942+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:31:48.269942+0000","last_undegraded":"2024-09-10T20:31:48.269942+0000","last_fullsized":"2024-09-10T20:31:48.269942+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":7252,"log_dups_size":0,"ondisk_log_size":7252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29704192,"num_objects":7252,"num_object_clones":0,"num_object_copies":14504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7252,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7252,"num_write_kb":29008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7644","reported_seq":7765,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:51.732156+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:31:51.732156+0000","last_peered":"2024-09-10T20:31:51.732156+0000","last_clean":"2024-09-10T20:31:51.732156+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:31:51.732156+0000","last_undegraded":"2024-09-10T20:31:51.732156+0000","last_fullsized":"2024-09-10T20:31:51.732156+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":7644,"log_dups_size":0,"ondisk_log_size":7644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31281152,"num_objects":7637,"num_object_clones":0,"num_object_copies":15274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4640,"num_write_kb":18560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7429","reported_seq":7561,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:51.589346+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:31:51.589346+0000","last_peered":"2024-09-10T20:31:51.589346+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:31:51.589346+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":7429,"log_dups_size":0,"ondisk_log_size":7429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30408704,"num_objects":7424,"num_object_clones":0,"num_object_copies":14848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7424,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4293,"num_write_kb":17172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3810},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7427","reported_seq":7560,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:51.725962+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:31:51.725962+0000","last_peered":"2024-09-10T20:31:51.725962+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:31:51.725962+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":7427,"log_dups_size":0,"ondisk_log_size":7427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30396416,"num_objects":7421,"num_object_clones":0,"num_object_copies":14842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4022,"num_write_kb":16088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3866},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7566","reported_seq":7686,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:51.660995+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:31:51.660995+0000","last_peered":"2024-09-10T20:31:51.660995+0000","last_clean":"2024-09-10T20:31:51.660995+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:31:51.660995+0000","last_undegraded":"2024-09-10T20:31:51.660995+0000","last_fullsized":"2024-09-10T20:31:51.660995+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":7566,"log_dups_size":0,"ondisk_log_size":7566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30982144,"num_objects":7564,"num_object_clones":0,"num_object_copies":15128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7564,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4637,"num_write_kb":18548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7254","reported_seq":7382,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:48.315307+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:31:48.315307+0000","last_peered":"2024-09-10T20:31:48.315307+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:31:48.315307+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":7254,"log_dups_size":0,"ondisk_log_size":7254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29683712,"num_objects":7247,"num_object_clones":0,"num_object_copies":14494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5247,"num_write_kb":20988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":3721}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355479+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355479+0000","last_peered":"2024-09-10T20:30:05.355479+0000","last_clean":"2024-09-10T20:30:05.355479+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355479+0000","last_undegraded":"2024-09-10T20:30:05.355479+0000","last_fullsized":"2024-09-10T20:30:05.355479+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7578","reported_seq":7682,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:51.698452+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:31:51.698452+0000","last_peered":"2024-09-10T20:31:51.698452+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:31:51.698452+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":7578,"log_dups_size":0,"ondisk_log_size":7578,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31007686,"num_objects":7572,"num_object_clones":0,"num_object_copies":15144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7576,"num_write_kb":30284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3957},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7344","reported_seq":7468,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:48.286118+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:31:48.286118+0000","last_peered":"2024-09-10T20:31:48.286118+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:31:48.286118+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":7344,"log_dups_size":0,"ondisk_log_size":7344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30060544,"num_objects":7339,"num_object_clones":0,"num_object_copies":14678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5172,"num_write_kb":20688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":3706}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7439","reported_seq":7563,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:48.274527+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:31:48.274527+0000","last_peered":"2024-09-10T20:31:48.274527+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:31:48.274527+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":7439,"log_dups_size":0,"ondisk_log_size":7439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30445568,"num_objects":7433,"num_object_clones":0,"num_object_copies":14866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5225,"num_write_kb":20900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":3823}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7530","reported_seq":7660,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:51.708009+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:31:51.708009+0000","last_peered":"2024-09-10T20:31:51.708009+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:31:51.708009+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":7530,"log_dups_size":0,"ondisk_log_size":7530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30834688,"num_objects":7528,"num_object_clones":0,"num_object_copies":15056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4253,"num_write_kb":17012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3880},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7536","reported_seq":7670,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:51.741716+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:31:51.741716+0000","last_peered":"2024-09-10T20:31:51.741716+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:31:51.741716+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":7536,"log_dups_size":0,"ondisk_log_size":7536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30834688,"num_objects":7528,"num_object_clones":0,"num_object_copies":15056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4230,"num_write_kb":16920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3996},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345743+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345743+0000","last_peered":"2024-09-10T20:30:05.345743+0000","last_clean":"2024-09-10T20:30:05.345743+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345743+0000","last_undegraded":"2024-09-10T20:30:05.345743+0000","last_fullsized":"2024-09-10T20:30:05.345743+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7638","reported_seq":7765,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:51.737566+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:31:51.737566+0000","last_peered":"2024-09-10T20:31:51.737566+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:31:51.737566+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":7638,"log_dups_size":0,"ondisk_log_size":7638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31268864,"num_objects":7634,"num_object_clones":0,"num_object_copies":15268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4465,"num_write_kb":17860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3921},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7454","reported_seq":7615,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:51.654718+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:31:51.654718+0000","last_peered":"2024-09-10T20:31:51.654718+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:31:51.654718+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":7454,"log_dups_size":0,"ondisk_log_size":7454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30482915,"num_objects":7443,"num_object_clones":0,"num_object_copies":14886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4061,"num_write_kb":16244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3975},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":489186729,"num_objects":119433,"num_object_clones":0,"num_object_copies":238866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":971,"num_objects_unfound":0,"num_objects_dirty":119433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83808,"num_write_kb":335212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2636,"num_bytes_recovered":10784768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":985063424,"data_stored":984976712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119512,"ondisk_log_size":119512,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182440,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":685036,"kb_used_data":418780,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93686804,"statfs":{"total":96636764160,"available":95935287296,"internally_reserved":0,"allocated":428830720,"data_stored":428597587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,0,3,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986211,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523336,"kb_used_data":351416,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93848504,"statfs":{"total":96636764160,"available":96100868096,"internally_reserved":0,"allocated":359849984,"data_stored":359620928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":176025258},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,3,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659158,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":350224,"kb_used_data":192192,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":94021616,"statfs":{"total":96636764160,"available":96278134784,"internally_reserved":0,"allocated":196804608,"data_stored":196495602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":161803950},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,6,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":428490752,"data_stored":428487139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":196493312,"data_stored":196413826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360079360,"data_stored":360075747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:31:53.258 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:31:53.258 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'7485", 'reported_seq': 7560, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:51.723614+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:31:51.723614+0000', 'last_peered': '2024-09-10T20:31:51.723614+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:31:51.723614+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 7485, 'log_dups_size': 0, 'ondisk_log_size': 7485, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30658560, 'num_objects': 7485, 'num_object_clones': 0, 'num_object_copies': 14970, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3607, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7485, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7485, 'num_write_kb': 29940, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3878}, {'shards': '0,2', 'objects': 3607}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:53.258 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T20:31:53.258 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'7417", 'reported_seq': 12808, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:31:48.329974+0000', 'last_change': '2024-09-10T20:30:05.653560+0000', 'last_active': '2024-09-10T20:31:48.329974+0000', 'last_peered': '2024-09-10T20:31:48.329974+0000', 'last_clean': '2024-09-10T20:30:03.279259+0000', 'last_became_active': '2024-09-10T20:30:05.629664+0000', 'last_became_peered': '2024-09-10T20:30:05.629664+0000', 'last_unstale': '2024-09-10T20:31:48.329974+0000', 'last_undegraded': '2024-09-10T20:31:48.329974+0000', 'last_fullsized': '2024-09-10T20:30:05.312061+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1910", 'last_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'objects_scrubbed': 1868, 'log_size': 7417, 'log_dups_size': 0, 'ondisk_log_size': 7417, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 673, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30363648, 'num_objects': 7413, 'num_object_clones': 0, 'num_object_copies': 14826, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 971, 'num_objects_unfound': 0, 'num_objects_dirty': 7413, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5545, 'num_write_kb': 22180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2633, 'num_bytes_recovered': 10780672, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 971}, {'shards': '2', 'objects': 6442}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:53.258 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:31:53.258 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'7429", 'reported_seq': 7561, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:51.589346+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:31:51.589346+0000', 'last_peered': '2024-09-10T20:31:51.589346+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:31:51.589346+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 7429, 'log_dups_size': 0, 'ondisk_log_size': 7429, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30408704, 'num_objects': 7424, 'num_object_clones': 0, 'num_object_copies': 14848, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3616, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7424, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4293, 'num_write_kb': 17172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3810}, {'shards': '0,2', 'objects': 3614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:53.259 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:31:53.259 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'7427", 'reported_seq': 7560, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:51.725962+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:31:51.725962+0000', 'last_peered': '2024-09-10T20:31:51.725962+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:31:51.725962+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 7427, 'log_dups_size': 0, 'ondisk_log_size': 7427, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30396416, 'num_objects': 7421, 'num_object_clones': 0, 'num_object_copies': 14842, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3557, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7421, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4022, 'num_write_kb': 16088, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3866}, {'shards': '0,2', 'objects': 3555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:53.259 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:31:53.259 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'7254", 'reported_seq': 7382, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:48.315307+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:31:48.315307+0000', 'last_peered': '2024-09-10T20:31:48.315307+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:31:48.315307+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 7254, 'log_dups_size': 0, 'ondisk_log_size': 7254, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29683712, 'num_objects': 7247, 'num_object_clones': 0, 'num_object_copies': 14494, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3528, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7247, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5247, 'num_write_kb': 20988, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 3721}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:53.259 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:31:53.259 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'7578", 'reported_seq': 7682, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:51.698452+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:31:51.698452+0000', 'last_peered': '2024-09-10T20:31:51.698452+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:31:51.698452+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 7578, 'log_dups_size': 0, 'ondisk_log_size': 7578, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31007686, 'num_objects': 7572, 'num_object_clones': 0, 'num_object_copies': 15144, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3617, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7572, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7576, 'num_write_kb': 30284, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3957}, {'shards': '0,2', 'objects': 3615}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:53.259 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:31:53.259 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'7344", 'reported_seq': 7468, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:48.286118+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:31:48.286118+0000', 'last_peered': '2024-09-10T20:31:48.286118+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:31:48.286118+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 7344, 'log_dups_size': 0, 'ondisk_log_size': 7344, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30060544, 'num_objects': 7339, 'num_object_clones': 0, 'num_object_copies': 14678, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3635, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7339, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5172, 'num_write_kb': 20688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3633}, {'shards': '2', 'objects': 3706}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:53.259 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:31:53.259 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'7439", 'reported_seq': 7563, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:48.274527+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:31:48.274527+0000', 'last_peered': '2024-09-10T20:31:48.274527+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:31:48.274527+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 7439, 'log_dups_size': 0, 'ondisk_log_size': 7439, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30445568, 'num_objects': 7433, 'num_object_clones': 0, 'num_object_copies': 14866, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7433, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5225, 'num_write_kb': 20900, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3610}, {'shards': '2', 'objects': 3823}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:53.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:31:53.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'7530", 'reported_seq': 7660, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:51.708009+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:31:51.708009+0000', 'last_peered': '2024-09-10T20:31:51.708009+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:31:51.708009+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 7530, 'log_dups_size': 0, 'ondisk_log_size': 7530, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30834688, 'num_objects': 7528, 'num_object_clones': 0, 'num_object_copies': 15056, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3649, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7528, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4253, 'num_write_kb': 17012, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3880}, {'shards': '0,2', 'objects': 3648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:53.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:31:53.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'7536", 'reported_seq': 7670, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:51.741716+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:31:51.741716+0000', 'last_peered': '2024-09-10T20:31:51.741716+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:31:51.741716+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 7536, 'log_dups_size': 0, 'ondisk_log_size': 7536, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30834688, 'num_objects': 7528, 'num_object_clones': 0, 'num_object_copies': 15056, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3535, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7528, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4230, 'num_write_kb': 16920, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3996}, {'shards': '0,2', 'objects': 3532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:53.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:31:53.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'7638", 'reported_seq': 7765, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:51.737566+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:31:51.737566+0000', 'last_peered': '2024-09-10T20:31:51.737566+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:31:51.737566+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 7638, 'log_dups_size': 0, 'ondisk_log_size': 7638, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31268864, 'num_objects': 7634, 'num_object_clones': 0, 'num_object_copies': 15268, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3715, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7634, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4465, 'num_write_kb': 17860, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3921}, {'shards': '0,2', 'objects': 3713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:53.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:31:53.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'7454", 'reported_seq': 7615, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:51.654718+0000', 'last_change': '2024-09-10T20:30:05.690055+0000', 'last_active': '2024-09-10T20:31:51.654718+0000', 'last_peered': '2024-09-10T20:31:51.654718+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:31:51.654718+0000', 'last_undegraded': '2024-09-10T20:30:05.378941+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 7454, 'log_dups_size': 0, 'ondisk_log_size': 7454, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30482915, 'num_objects': 7443, 'num_object_clones': 0, 'num_object_copies': 14886, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3471, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7443, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4061, 'num_write_kb': 16244, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3975}, {'shards': '0,2', 'objects': 3468}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:53.260 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:31:53.937 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:53.937 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39542/238870 objects degraded (16.554%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":225,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":119435,"data_bytes":489186748,"bytes_used":1596002304,"bytes_avail":288314290176,"bytes_total":289910292480,"degraded_objects":39542,"degraded_total":238870,"degraded_ratio":0.16553774019341064,"misplaced_objects":971,"misplaced_total":238870,"misplaced_ratio":0.0040649725792271949,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":105097,"recovering_keys_per_sec":0,"num_objects_recovered":308,"num_bytes_recovered":1261568,"num_keys_recovered":0,"write_bytes_sec":2389272,"read_op_per_sec":0,"write_op_per_sec":583},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:31:53.964 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:31:53.965 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:31:54.066 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 146 20 121271 121251 3.24329 1.91406 0.0125722 0.0240811 2024-09-10T20:31:54.502 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:54.505 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:31:54.526 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":149,"stamp":"2024-09-10T20:31:52.866410+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":489186748,"num_objects":119435,"num_object_clones":0,"num_object_copies":238870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":971,"num_objects_unfound":0,"num_objects_dirty":119435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83810,"num_write_kb":335214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2638,"num_bytes_recovered":10784787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119515,"ondisk_log_size":119515,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1558596,"kb_used_data":962388,"kb_used_omap":12,"kb_used_meta":596147,"kb_avail":281556924,"statfs":{"total":289910292480,"available":288314290176,"internally_reserved":0,"allocated":985485312,"data_stored":984714117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":610454530},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,3,5,9,11,10,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28680192,"num_objects":7002,"num_object_clones":0,"num_object_copies":14004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-308,"num_objects_unfound":0,"num_objects_dirty":7002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7002,"num_write_kb":28008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":1261568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003734"},"pg_stats":[{"pgid":"2.f","version":"43'7485","reported_seq":7560,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:51.723614+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:31:51.723614+0000","last_peered":"2024-09-10T20:31:51.723614+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:31:51.723614+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":7485,"log_dups_size":0,"ondisk_log_size":7485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30658560,"num_objects":7485,"num_object_clones":0,"num_object_copies":14970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7485,"num_write_kb":29940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3878},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7519","reported_seq":7625,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:48.302884+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:31:48.302884+0000","last_peered":"2024-09-10T20:31:48.302884+0000","last_clean":"2024-09-10T20:31:48.302884+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:31:48.302884+0000","last_undegraded":"2024-09-10T20:31:48.302884+0000","last_fullsized":"2024-09-10T20:31:48.302884+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":7519,"log_dups_size":0,"ondisk_log_size":7519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30773248,"num_objects":7513,"num_object_clones":0,"num_object_copies":15026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7513,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5705,"num_write_kb":22820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7417","reported_seq":12808,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:31:48.329974+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:31:48.329974+0000","last_peered":"2024-09-10T20:31:48.329974+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:31:48.329974+0000","last_undegraded":"2024-09-10T20:31:48.329974+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":7417,"log_dups_size":0,"ondisk_log_size":7417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30363648,"num_objects":7413,"num_object_clones":0,"num_object_copies":14826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":971,"num_objects_unfound":0,"num_objects_dirty":7413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5545,"num_write_kb":22180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2633,"num_bytes_recovered":10780672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":971},{"shards":"2","objects":6442}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7252","reported_seq":7328,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:48.269942+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:31:48.269942+0000","last_peered":"2024-09-10T20:31:48.269942+0000","last_clean":"2024-09-10T20:31:48.269942+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:31:48.269942+0000","last_undegraded":"2024-09-10T20:31:48.269942+0000","last_fullsized":"2024-09-10T20:31:48.269942+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":7252,"log_dups_size":0,"ondisk_log_size":7252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29704192,"num_objects":7252,"num_object_clones":0,"num_object_copies":14504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7252,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7252,"num_write_kb":29008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7644","reported_seq":7765,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:51.732156+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:31:51.732156+0000","last_peered":"2024-09-10T20:31:51.732156+0000","last_clean":"2024-09-10T20:31:51.732156+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:31:51.732156+0000","last_undegraded":"2024-09-10T20:31:51.732156+0000","last_fullsized":"2024-09-10T20:31:51.732156+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":7644,"log_dups_size":0,"ondisk_log_size":7644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31281152,"num_objects":7637,"num_object_clones":0,"num_object_copies":15274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4640,"num_write_kb":18560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7429","reported_seq":7561,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:51.589346+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:31:51.589346+0000","last_peered":"2024-09-10T20:31:51.589346+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:31:51.589346+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":7429,"log_dups_size":0,"ondisk_log_size":7429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30408704,"num_objects":7424,"num_object_clones":0,"num_object_copies":14848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7424,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4293,"num_write_kb":17172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3810},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7427","reported_seq":7560,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:51.725962+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:31:51.725962+0000","last_peered":"2024-09-10T20:31:51.725962+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:31:51.725962+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":7427,"log_dups_size":0,"ondisk_log_size":7427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30396416,"num_objects":7421,"num_object_clones":0,"num_object_copies":14842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4022,"num_write_kb":16088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3866},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7566","reported_seq":7686,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:51.660995+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:31:51.660995+0000","last_peered":"2024-09-10T20:31:51.660995+0000","last_clean":"2024-09-10T20:31:51.660995+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:31:51.660995+0000","last_undegraded":"2024-09-10T20:31:51.660995+0000","last_fullsized":"2024-09-10T20:31:51.660995+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":7566,"log_dups_size":0,"ondisk_log_size":7566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30982144,"num_objects":7564,"num_object_clones":0,"num_object_copies":15128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7564,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4637,"num_write_kb":18548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7254","reported_seq":7382,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:48.315307+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:31:48.315307+0000","last_peered":"2024-09-10T20:31:48.315307+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:31:48.315307+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":7254,"log_dups_size":0,"ondisk_log_size":7254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29683712,"num_objects":7247,"num_object_clones":0,"num_object_copies":14494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5247,"num_write_kb":20988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":3721}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355479+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355479+0000","last_peered":"2024-09-10T20:30:05.355479+0000","last_clean":"2024-09-10T20:30:05.355479+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355479+0000","last_undegraded":"2024-09-10T20:30:05.355479+0000","last_fullsized":"2024-09-10T20:30:05.355479+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7578","reported_seq":7682,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:51.698452+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:31:51.698452+0000","last_peered":"2024-09-10T20:31:51.698452+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:31:51.698452+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":7578,"log_dups_size":0,"ondisk_log_size":7578,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31007686,"num_objects":7572,"num_object_clones":0,"num_object_copies":15144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7576,"num_write_kb":30284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3957},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7344","reported_seq":7468,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:48.286118+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:31:48.286118+0000","last_peered":"2024-09-10T20:31:48.286118+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:31:48.286118+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":7344,"log_dups_size":0,"ondisk_log_size":7344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30060544,"num_objects":7339,"num_object_clones":0,"num_object_copies":14678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5172,"num_write_kb":20688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":3706}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7439","reported_seq":7563,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:48.274527+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:31:48.274527+0000","last_peered":"2024-09-10T20:31:48.274527+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:31:48.274527+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":7439,"log_dups_size":0,"ondisk_log_size":7439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30445568,"num_objects":7433,"num_object_clones":0,"num_object_copies":14866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5225,"num_write_kb":20900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":3823}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7530","reported_seq":7660,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:51.708009+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:31:51.708009+0000","last_peered":"2024-09-10T20:31:51.708009+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:31:51.708009+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":7530,"log_dups_size":0,"ondisk_log_size":7530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30834688,"num_objects":7528,"num_object_clones":0,"num_object_copies":15056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4253,"num_write_kb":17012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3880},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7536","reported_seq":7670,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:51.741716+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:31:51.741716+0000","last_peered":"2024-09-10T20:31:51.741716+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:31:51.741716+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":7536,"log_dups_size":0,"ondisk_log_size":7536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30834688,"num_objects":7528,"num_object_clones":0,"num_object_copies":15056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4230,"num_write_kb":16920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3996},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345743+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345743+0000","last_peered":"2024-09-10T20:30:05.345743+0000","last_clean":"2024-09-10T20:30:05.345743+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345743+0000","last_undegraded":"2024-09-10T20:30:05.345743+0000","last_fullsized":"2024-09-10T20:30:05.345743+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7638","reported_seq":7765,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:51.737566+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:31:51.737566+0000","last_peered":"2024-09-10T20:31:51.737566+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:31:51.737566+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":7638,"log_dups_size":0,"ondisk_log_size":7638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31268864,"num_objects":7634,"num_object_clones":0,"num_object_copies":15268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4465,"num_write_kb":17860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3921},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7454","reported_seq":7615,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:51.654718+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:31:51.654718+0000","last_peered":"2024-09-10T20:31:51.654718+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:31:51.654718+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":7454,"log_dups_size":0,"ondisk_log_size":7454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30482915,"num_objects":7443,"num_object_clones":0,"num_object_copies":14886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4061,"num_write_kb":16244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3975},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":489186729,"num_objects":119433,"num_object_clones":0,"num_object_copies":238866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":971,"num_objects_unfound":0,"num_objects_dirty":119433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83808,"num_write_kb":335212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2636,"num_bytes_recovered":10784768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":985063424,"data_stored":984976712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119512,"ondisk_log_size":119512,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182440,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":685036,"kb_used_data":418780,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93686804,"statfs":{"total":96636764160,"available":95935287296,"internally_reserved":0,"allocated":428830720,"data_stored":428597587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,0,3,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986211,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523336,"kb_used_data":351416,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93848504,"statfs":{"total":96636764160,"available":96100868096,"internally_reserved":0,"allocated":359849984,"data_stored":359620928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":176025258},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,3,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659158,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":350224,"kb_used_data":192192,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":94021616,"statfs":{"total":96636764160,"available":96278134784,"internally_reserved":0,"allocated":196804608,"data_stored":196495602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":161803950},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,6,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":428490752,"data_stored":428487139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":196493312,"data_stored":196413826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360079360,"data_stored":360075747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:31:55.065 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 147 20 121792 121772 3.23507 2.03516 0.02025 0.024143 2024-09-10T20:31:56.067 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 148 20 122340 122320 3.22764 2.14062 0.0136546 0.0241974 2024-09-10T20:31:57.067 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 149 20 122884 122864 3.22024 2.125 0.0699361 0.0242521 2024-09-10T20:31:57.528 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:31:58.068 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 150 20 123396 123376 3.21211 2 0.0809931 0.0243134 2024-09-10T20:31:58.080 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:58.082 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:31:58.102 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":151,"stamp":"2024-09-10T20:31:56.867731+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":499877308,"num_objects":122045,"num_object_clones":0,"num_object_copies":244090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":804,"num_objects_unfound":0,"num_objects_dirty":122045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86420,"num_write_kb":345654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2805,"num_bytes_recovered":11468819,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":122125,"ondisk_log_size":122125,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1595412,"kb_used_data":982564,"kb_used_omap":12,"kb_used_meta":612787,"kb_avail":281520108,"statfs":{"total":289910292480,"available":288276590592,"internally_reserved":0,"allocated":1006145536,"data_stored":1005374341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":627493890},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,7,10,5,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28385280,"num_objects":6930,"num_object_clones":0,"num_object_copies":13860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-317,"num_objects_unfound":0,"num_objects_dirty":6930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6930,"num_write_kb":27720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":317,"num_bytes_recovered":1298432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003589"},"pg_stats":[{"pgid":"2.f","version":"43'7658","reported_seq":7734,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:56.762173+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:31:56.762173+0000","last_peered":"2024-09-10T20:31:56.762173+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:31:56.762173+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":7658,"log_dups_size":0,"ondisk_log_size":7658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31367168,"num_objects":7658,"num_object_clones":0,"num_object_copies":15316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7658,"num_write_kb":30632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4051},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7682","reported_seq":7788,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:53.255939+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:31:53.255939+0000","last_peered":"2024-09-10T20:31:53.255939+0000","last_clean":"2024-09-10T20:31:53.255939+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:31:53.255939+0000","last_undegraded":"2024-09-10T20:31:53.255939+0000","last_fullsized":"2024-09-10T20:31:53.255939+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":7682,"log_dups_size":0,"ondisk_log_size":7682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31440896,"num_objects":7676,"num_object_clones":0,"num_object_copies":15352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7676,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5868,"num_write_kb":23472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7568","reported_seq":13293,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:31:53.328302+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:31:53.328302+0000","last_peered":"2024-09-10T20:31:53.328302+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:31:53.328302+0000","last_undegraded":"2024-09-10T20:31:53.328302+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":7568,"log_dups_size":0,"ondisk_log_size":7568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30982144,"num_objects":7564,"num_object_clones":0,"num_object_copies":15128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":804,"num_objects_unfound":0,"num_objects_dirty":7564,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5696,"num_write_kb":22784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2800,"num_bytes_recovered":11464704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":804},{"shards":"2","objects":6760}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7418","reported_seq":7494,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:53.309914+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:31:53.309914+0000","last_peered":"2024-09-10T20:31:53.309914+0000","last_clean":"2024-09-10T20:31:53.309914+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:31:53.309914+0000","last_undegraded":"2024-09-10T20:31:53.309914+0000","last_fullsized":"2024-09-10T20:31:53.309914+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":7418,"log_dups_size":0,"ondisk_log_size":7418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30384128,"num_objects":7418,"num_object_clones":0,"num_object_copies":14836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7418,"num_write_kb":29672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7821","reported_seq":7943,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:56.785977+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:31:56.785977+0000","last_peered":"2024-09-10T20:31:56.785977+0000","last_clean":"2024-09-10T20:31:56.785977+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:31:56.785977+0000","last_undegraded":"2024-09-10T20:31:56.785977+0000","last_fullsized":"2024-09-10T20:31:56.785977+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":7821,"log_dups_size":0,"ondisk_log_size":7821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32006144,"num_objects":7814,"num_object_clones":0,"num_object_copies":15628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4817,"num_write_kb":19268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7583","reported_seq":7715,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:56.738943+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:31:56.738943+0000","last_peered":"2024-09-10T20:31:56.738943+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:31:56.738943+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":7583,"log_dups_size":0,"ondisk_log_size":7583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31039488,"num_objects":7578,"num_object_clones":0,"num_object_copies":15156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4447,"num_write_kb":17788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3964},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7591","reported_seq":7723,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:56.734578+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:31:56.734578+0000","last_peered":"2024-09-10T20:31:56.734578+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:31:56.734578+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":7591,"log_dups_size":0,"ondisk_log_size":7591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31068160,"num_objects":7585,"num_object_clones":0,"num_object_copies":15170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4186,"num_write_kb":16744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4030},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7719","reported_seq":7839,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:56.784643+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:31:56.784643+0000","last_peered":"2024-09-10T20:31:56.784643+0000","last_clean":"2024-09-10T20:31:56.784643+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:31:56.784643+0000","last_undegraded":"2024-09-10T20:31:56.784643+0000","last_fullsized":"2024-09-10T20:31:56.784643+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":7719,"log_dups_size":0,"ondisk_log_size":7719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31608832,"num_objects":7717,"num_object_clones":0,"num_object_copies":15434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4790,"num_write_kb":19160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7425","reported_seq":7553,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:53.319015+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:31:53.319015+0000","last_peered":"2024-09-10T20:31:53.319015+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:31:53.319015+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":7425,"log_dups_size":0,"ondisk_log_size":7425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30384128,"num_objects":7418,"num_object_clones":0,"num_object_copies":14836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5418,"num_write_kb":21672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":3892}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355479+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355479+0000","last_peered":"2024-09-10T20:30:05.355479+0000","last_clean":"2024-09-10T20:30:05.355479+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355479+0000","last_undegraded":"2024-09-10T20:30:05.355479+0000","last_fullsized":"2024-09-10T20:30:05.355479+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7732","reported_seq":7836,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:56.675230+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:31:56.675230+0000","last_peered":"2024-09-10T20:31:56.675230+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:31:56.675230+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":7732,"log_dups_size":0,"ondisk_log_size":7732,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31638470,"num_objects":7726,"num_object_clones":0,"num_object_copies":15452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7730,"num_write_kb":30900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4111},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7505","reported_seq":7629,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:53.282182+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:31:53.282182+0000","last_peered":"2024-09-10T20:31:53.282182+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:31:53.282182+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":7505,"log_dups_size":0,"ondisk_log_size":7505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30720000,"num_objects":7500,"num_object_clones":0,"num_object_copies":15000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5333,"num_write_kb":21332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":3867}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7603","reported_seq":7727,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:53.297759+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:31:53.297759+0000","last_peered":"2024-09-10T20:31:53.297759+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:31:53.297759+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":7603,"log_dups_size":0,"ondisk_log_size":7603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31117312,"num_objects":7597,"num_object_clones":0,"num_object_copies":15194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5389,"num_write_kb":21556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":3987}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7704","reported_seq":7834,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:56.764136+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:31:56.764136+0000","last_peered":"2024-09-10T20:31:56.764136+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:31:56.764136+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":7704,"log_dups_size":0,"ondisk_log_size":7704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31547392,"num_objects":7702,"num_object_clones":0,"num_object_copies":15404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4427,"num_write_kb":17708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4054},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7700","reported_seq":7833,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:56.753608+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:31:56.753608+0000","last_peered":"2024-09-10T20:31:56.753608+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:31:56.753608+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":7700,"log_dups_size":0,"ondisk_log_size":7700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31506432,"num_objects":7692,"num_object_clones":0,"num_object_copies":15384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4394,"num_write_kb":17576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4160},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345743+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345743+0000","last_peered":"2024-09-10T20:30:05.345743+0000","last_clean":"2024-09-10T20:30:05.345743+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345743+0000","last_undegraded":"2024-09-10T20:30:05.345743+0000","last_fullsized":"2024-09-10T20:30:05.345743+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7808","reported_seq":7935,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:56.760372+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:31:56.760372+0000","last_peered":"2024-09-10T20:31:56.760372+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:31:56.760372+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":7808,"log_dups_size":0,"ondisk_log_size":7808,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31965184,"num_objects":7804,"num_object_clones":0,"num_object_copies":15608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4635,"num_write_kb":18540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4091},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7605","reported_seq":7766,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:56.755199+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:31:56.755199+0000","last_peered":"2024-09-10T20:31:56.755199+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:31:56.755199+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":7605,"log_dups_size":0,"ondisk_log_size":7605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31101411,"num_objects":7594,"num_object_clones":0,"num_object_copies":15188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7594,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4212,"num_write_kb":16848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4126},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":499877289,"num_objects":122043,"num_object_clones":0,"num_object_copies":244086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":804,"num_objects_unfound":0,"num_objects_dirty":122043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86418,"num_write_kb":345652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2803,"num_bytes_recovered":11468800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1007325184,"data_stored":1007238472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":122122,"ondisk_log_size":122122,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182441,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":695520,"kb_used_data":425104,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93676320,"statfs":{"total":96636764160,"available":95924551680,"internally_reserved":0,"allocated":435306496,"data_stored":435073363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,3,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986212,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":532744,"kb_used_data":356664,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93839096,"statfs":{"total":96636764160,"available":96091234304,"internally_reserved":0,"allocated":365223936,"data_stored":364994880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180285098},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659159,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":367148,"kb_used_data":200796,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":94004692,"statfs":{"total":96636764160,"available":96260804608,"internally_reserved":0,"allocated":205615104,"data_stored":205306098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":170323630},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,5,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":436547584,"data_stored":436543971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":205303808,"data_stored":205224322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":365473792,"data_stored":365470179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:31:58.104 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:31:58.670 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:58.674 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:31:58.694 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":151,"stamp":"2024-09-10T20:31:56.867731+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":499877308,"num_objects":122045,"num_object_clones":0,"num_object_copies":244090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":804,"num_objects_unfound":0,"num_objects_dirty":122045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86420,"num_write_kb":345654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2805,"num_bytes_recovered":11468819,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":122125,"ondisk_log_size":122125,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1595412,"kb_used_data":982564,"kb_used_omap":12,"kb_used_meta":612787,"kb_avail":281520108,"statfs":{"total":289910292480,"available":288276590592,"internally_reserved":0,"allocated":1006145536,"data_stored":1005374341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":627493890},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,7,10,5,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28385280,"num_objects":6930,"num_object_clones":0,"num_object_copies":13860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-317,"num_objects_unfound":0,"num_objects_dirty":6930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6930,"num_write_kb":27720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":317,"num_bytes_recovered":1298432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003589"},"pg_stats":[{"pgid":"2.f","version":"43'7658","reported_seq":7734,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:56.762173+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:31:56.762173+0000","last_peered":"2024-09-10T20:31:56.762173+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:31:56.762173+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":7658,"log_dups_size":0,"ondisk_log_size":7658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31367168,"num_objects":7658,"num_object_clones":0,"num_object_copies":15316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7658,"num_write_kb":30632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4051},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7682","reported_seq":7788,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:53.255939+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:31:53.255939+0000","last_peered":"2024-09-10T20:31:53.255939+0000","last_clean":"2024-09-10T20:31:53.255939+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:31:53.255939+0000","last_undegraded":"2024-09-10T20:31:53.255939+0000","last_fullsized":"2024-09-10T20:31:53.255939+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":7682,"log_dups_size":0,"ondisk_log_size":7682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31440896,"num_objects":7676,"num_object_clones":0,"num_object_copies":15352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7676,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5868,"num_write_kb":23472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7568","reported_seq":13293,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:31:53.328302+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:31:53.328302+0000","last_peered":"2024-09-10T20:31:53.328302+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:31:53.328302+0000","last_undegraded":"2024-09-10T20:31:53.328302+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":7568,"log_dups_size":0,"ondisk_log_size":7568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30982144,"num_objects":7564,"num_object_clones":0,"num_object_copies":15128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":804,"num_objects_unfound":0,"num_objects_dirty":7564,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5696,"num_write_kb":22784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2800,"num_bytes_recovered":11464704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":804},{"shards":"2","objects":6760}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7418","reported_seq":7494,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:53.309914+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:31:53.309914+0000","last_peered":"2024-09-10T20:31:53.309914+0000","last_clean":"2024-09-10T20:31:53.309914+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:31:53.309914+0000","last_undegraded":"2024-09-10T20:31:53.309914+0000","last_fullsized":"2024-09-10T20:31:53.309914+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":7418,"log_dups_size":0,"ondisk_log_size":7418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30384128,"num_objects":7418,"num_object_clones":0,"num_object_copies":14836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7418,"num_write_kb":29672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7821","reported_seq":7943,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:56.785977+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:31:56.785977+0000","last_peered":"2024-09-10T20:31:56.785977+0000","last_clean":"2024-09-10T20:31:56.785977+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:31:56.785977+0000","last_undegraded":"2024-09-10T20:31:56.785977+0000","last_fullsized":"2024-09-10T20:31:56.785977+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":7821,"log_dups_size":0,"ondisk_log_size":7821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32006144,"num_objects":7814,"num_object_clones":0,"num_object_copies":15628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4817,"num_write_kb":19268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7583","reported_seq":7715,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:56.738943+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:31:56.738943+0000","last_peered":"2024-09-10T20:31:56.738943+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:31:56.738943+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":7583,"log_dups_size":0,"ondisk_log_size":7583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31039488,"num_objects":7578,"num_object_clones":0,"num_object_copies":15156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4447,"num_write_kb":17788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3964},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7591","reported_seq":7723,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:56.734578+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:31:56.734578+0000","last_peered":"2024-09-10T20:31:56.734578+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:31:56.734578+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":7591,"log_dups_size":0,"ondisk_log_size":7591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31068160,"num_objects":7585,"num_object_clones":0,"num_object_copies":15170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4186,"num_write_kb":16744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4030},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7719","reported_seq":7839,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:56.784643+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:31:56.784643+0000","last_peered":"2024-09-10T20:31:56.784643+0000","last_clean":"2024-09-10T20:31:56.784643+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:31:56.784643+0000","last_undegraded":"2024-09-10T20:31:56.784643+0000","last_fullsized":"2024-09-10T20:31:56.784643+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":7719,"log_dups_size":0,"ondisk_log_size":7719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31608832,"num_objects":7717,"num_object_clones":0,"num_object_copies":15434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4790,"num_write_kb":19160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7425","reported_seq":7553,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:53.319015+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:31:53.319015+0000","last_peered":"2024-09-10T20:31:53.319015+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:31:53.319015+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":7425,"log_dups_size":0,"ondisk_log_size":7425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30384128,"num_objects":7418,"num_object_clones":0,"num_object_copies":14836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5418,"num_write_kb":21672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":3892}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355479+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355479+0000","last_peered":"2024-09-10T20:30:05.355479+0000","last_clean":"2024-09-10T20:30:05.355479+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355479+0000","last_undegraded":"2024-09-10T20:30:05.355479+0000","last_fullsized":"2024-09-10T20:30:05.355479+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7732","reported_seq":7836,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:56.675230+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:31:56.675230+0000","last_peered":"2024-09-10T20:31:56.675230+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:31:56.675230+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":7732,"log_dups_size":0,"ondisk_log_size":7732,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31638470,"num_objects":7726,"num_object_clones":0,"num_object_copies":15452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7730,"num_write_kb":30900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4111},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7505","reported_seq":7629,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:53.282182+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:31:53.282182+0000","last_peered":"2024-09-10T20:31:53.282182+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:31:53.282182+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":7505,"log_dups_size":0,"ondisk_log_size":7505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30720000,"num_objects":7500,"num_object_clones":0,"num_object_copies":15000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5333,"num_write_kb":21332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":3867}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7603","reported_seq":7727,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:53.297759+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:31:53.297759+0000","last_peered":"2024-09-10T20:31:53.297759+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:31:53.297759+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":7603,"log_dups_size":0,"ondisk_log_size":7603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31117312,"num_objects":7597,"num_object_clones":0,"num_object_copies":15194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5389,"num_write_kb":21556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":3987}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7704","reported_seq":7834,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:56.764136+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:31:56.764136+0000","last_peered":"2024-09-10T20:31:56.764136+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:31:56.764136+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":7704,"log_dups_size":0,"ondisk_log_size":7704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31547392,"num_objects":7702,"num_object_clones":0,"num_object_copies":15404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4427,"num_write_kb":17708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4054},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7700","reported_seq":7833,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:56.753608+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:31:56.753608+0000","last_peered":"2024-09-10T20:31:56.753608+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:31:56.753608+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":7700,"log_dups_size":0,"ondisk_log_size":7700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31506432,"num_objects":7692,"num_object_clones":0,"num_object_copies":15384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4394,"num_write_kb":17576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4160},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345743+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345743+0000","last_peered":"2024-09-10T20:30:05.345743+0000","last_clean":"2024-09-10T20:30:05.345743+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345743+0000","last_undegraded":"2024-09-10T20:30:05.345743+0000","last_fullsized":"2024-09-10T20:30:05.345743+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7808","reported_seq":7935,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:56.760372+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:31:56.760372+0000","last_peered":"2024-09-10T20:31:56.760372+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:31:56.760372+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":7808,"log_dups_size":0,"ondisk_log_size":7808,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31965184,"num_objects":7804,"num_object_clones":0,"num_object_copies":15608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4635,"num_write_kb":18540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4091},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7605","reported_seq":7766,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:56.755199+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:31:56.755199+0000","last_peered":"2024-09-10T20:31:56.755199+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:31:56.755199+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":7605,"log_dups_size":0,"ondisk_log_size":7605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31101411,"num_objects":7594,"num_object_clones":0,"num_object_copies":15188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7594,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4212,"num_write_kb":16848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4126},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":499877289,"num_objects":122043,"num_object_clones":0,"num_object_copies":244086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":804,"num_objects_unfound":0,"num_objects_dirty":122043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86418,"num_write_kb":345652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2803,"num_bytes_recovered":11468800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1007325184,"data_stored":1007238472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":122122,"ondisk_log_size":122122,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182441,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":695520,"kb_used_data":425104,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93676320,"statfs":{"total":96636764160,"available":95924551680,"internally_reserved":0,"allocated":435306496,"data_stored":435073363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,3,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986212,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":532744,"kb_used_data":356664,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93839096,"statfs":{"total":96636764160,"available":96091234304,"internally_reserved":0,"allocated":365223936,"data_stored":364994880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180285098},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659159,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":367148,"kb_used_data":200796,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":94004692,"statfs":{"total":96636764160,"available":96260804608,"internally_reserved":0,"allocated":205615104,"data_stored":205306098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":170323630},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,5,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":436547584,"data_stored":436543971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":205303808,"data_stored":205224322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":365473792,"data_stored":365470179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:31:58.696 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:31:58.696 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'7658", 'reported_seq': 7734, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:56.762173+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:31:56.762173+0000', 'last_peered': '2024-09-10T20:31:56.762173+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:31:56.762173+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 7658, 'log_dups_size': 0, 'ondisk_log_size': 7658, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31367168, 'num_objects': 7658, 'num_object_clones': 0, 'num_object_copies': 15316, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3607, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7658, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7658, 'num_write_kb': 30632, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4051}, {'shards': '0,2', 'objects': 3607}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:58.696 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T20:31:58.696 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'7568", 'reported_seq': 13293, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:31:53.328302+0000', 'last_change': '2024-09-10T20:30:05.653560+0000', 'last_active': '2024-09-10T20:31:53.328302+0000', 'last_peered': '2024-09-10T20:31:53.328302+0000', 'last_clean': '2024-09-10T20:30:03.279259+0000', 'last_became_active': '2024-09-10T20:30:05.629664+0000', 'last_became_peered': '2024-09-10T20:30:05.629664+0000', 'last_unstale': '2024-09-10T20:31:53.328302+0000', 'last_undegraded': '2024-09-10T20:31:53.328302+0000', 'last_fullsized': '2024-09-10T20:30:05.312061+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1910", 'last_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'objects_scrubbed': 1868, 'log_size': 7568, 'log_dups_size': 0, 'ondisk_log_size': 7568, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 673, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30982144, 'num_objects': 7564, 'num_object_clones': 0, 'num_object_copies': 15128, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 804, 'num_objects_unfound': 0, 'num_objects_dirty': 7564, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5696, 'num_write_kb': 22784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2800, 'num_bytes_recovered': 11464704, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 804}, {'shards': '2', 'objects': 6760}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:58.696 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:31:58.696 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'7583", 'reported_seq': 7715, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:56.738943+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:31:56.738943+0000', 'last_peered': '2024-09-10T20:31:56.738943+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:31:56.738943+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 7583, 'log_dups_size': 0, 'ondisk_log_size': 7583, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31039488, 'num_objects': 7578, 'num_object_clones': 0, 'num_object_copies': 15156, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3616, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7578, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4447, 'num_write_kb': 17788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3964}, {'shards': '0,2', 'objects': 3614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:58.696 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:31:58.696 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'7591", 'reported_seq': 7723, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:56.734578+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:31:56.734578+0000', 'last_peered': '2024-09-10T20:31:56.734578+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:31:56.734578+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 7591, 'log_dups_size': 0, 'ondisk_log_size': 7591, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31068160, 'num_objects': 7585, 'num_object_clones': 0, 'num_object_copies': 15170, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3557, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7585, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4186, 'num_write_kb': 16744, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4030}, {'shards': '0,2', 'objects': 3555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:58.697 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:31:58.697 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'7425", 'reported_seq': 7553, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:53.319015+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:31:53.319015+0000', 'last_peered': '2024-09-10T20:31:53.319015+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:31:53.319015+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 7425, 'log_dups_size': 0, 'ondisk_log_size': 7425, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30384128, 'num_objects': 7418, 'num_object_clones': 0, 'num_object_copies': 14836, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3528, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7418, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5418, 'num_write_kb': 21672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 3892}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:58.697 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:31:58.697 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'7732", 'reported_seq': 7836, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:56.675230+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:31:56.675230+0000', 'last_peered': '2024-09-10T20:31:56.675230+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:31:56.675230+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 7732, 'log_dups_size': 0, 'ondisk_log_size': 7732, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31638470, 'num_objects': 7726, 'num_object_clones': 0, 'num_object_copies': 15452, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3617, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7726, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7730, 'num_write_kb': 30900, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4111}, {'shards': '0,2', 'objects': 3615}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:58.697 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:31:58.697 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'7505", 'reported_seq': 7629, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:53.282182+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:31:53.282182+0000', 'last_peered': '2024-09-10T20:31:53.282182+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:31:53.282182+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 7505, 'log_dups_size': 0, 'ondisk_log_size': 7505, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30720000, 'num_objects': 7500, 'num_object_clones': 0, 'num_object_copies': 15000, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3635, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7500, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5333, 'num_write_kb': 21332, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3633}, {'shards': '2', 'objects': 3867}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:58.697 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:31:58.697 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'7603", 'reported_seq': 7727, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:53.297759+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:31:53.297759+0000', 'last_peered': '2024-09-10T20:31:53.297759+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:31:53.297759+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 7603, 'log_dups_size': 0, 'ondisk_log_size': 7603, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31117312, 'num_objects': 7597, 'num_object_clones': 0, 'num_object_copies': 15194, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7597, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5389, 'num_write_kb': 21556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3610}, {'shards': '2', 'objects': 3987}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:31:58.697 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:31:58.697 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'7704", 'reported_seq': 7834, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:56.764136+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:31:56.764136+0000', 'last_peered': '2024-09-10T20:31:56.764136+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:31:56.764136+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 7704, 'log_dups_size': 0, 'ondisk_log_size': 7704, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31547392, 'num_objects': 7702, 'num_object_clones': 0, 'num_object_copies': 15404, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3649, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7702, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4427, 'num_write_kb': 17708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4054}, {'shards': '0,2', 'objects': 3648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:58.698 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:31:58.698 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'7700", 'reported_seq': 7833, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:56.753608+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:31:56.753608+0000', 'last_peered': '2024-09-10T20:31:56.753608+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:31:56.753608+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 7700, 'log_dups_size': 0, 'ondisk_log_size': 7700, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31506432, 'num_objects': 7692, 'num_object_clones': 0, 'num_object_copies': 15384, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3535, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7692, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4394, 'num_write_kb': 17576, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4160}, {'shards': '0,2', 'objects': 3532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:58.698 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:31:58.698 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'7808", 'reported_seq': 7935, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:56.760372+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:31:56.760372+0000', 'last_peered': '2024-09-10T20:31:56.760372+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:31:56.760372+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 7808, 'log_dups_size': 0, 'ondisk_log_size': 7808, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31965184, 'num_objects': 7804, 'num_object_clones': 0, 'num_object_copies': 15608, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3715, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7804, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4635, 'num_write_kb': 18540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4091}, {'shards': '0,2', 'objects': 3713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:58.698 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:31:58.698 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'7605", 'reported_seq': 7766, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:56.755199+0000', 'last_change': '2024-09-10T20:30:05.690055+0000', 'last_active': '2024-09-10T20:31:56.755199+0000', 'last_peered': '2024-09-10T20:31:56.755199+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:31:56.755199+0000', 'last_undegraded': '2024-09-10T20:30:05.378941+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 7605, 'log_dups_size': 0, 'ondisk_log_size': 7605, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31101411, 'num_objects': 7594, 'num_object_clones': 0, 'num_object_copies': 15188, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3471, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7594, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4212, 'num_write_kb': 16848, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4126}, {'shards': '0,2', 'objects': 3468}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:31:58.698 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:31:59.068 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 151 20 123884 123864 3.20346 1.90625 0.0474042 0.0243809 2024-09-10T20:31:59.400 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:59.401 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39542/244090 objects degraded (16.200%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":231,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":122045,"data_bytes":499877308,"bytes_used":1633701888,"bytes_avail":288276590592,"bytes_total":289910292480,"degraded_objects":39542,"degraded_total":244090,"degraded_ratio":0.16199762382727684,"misplaced_objects":804,"misplaced_total":244090,"misplaced_ratio":0.003293867016264493,"recovering_objects_per_sec":26,"recovering_bytes_per_sec":108170,"recovering_keys_per_sec":0,"num_objects_recovered":317,"num_bytes_recovered":1298432,"num_keys_recovered":0,"write_bytes_sec":2364732,"read_op_per_sec":0,"write_op_per_sec":577},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:31:59.428 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:31:59.429 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:31:59.983 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:31:59.986 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:32:00.013 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":152,"stamp":"2024-09-10T20:31:58.868248+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":503854524,"num_objects":123016,"num_object_clones":0,"num_object_copies":246032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":657,"num_objects_unfound":0,"num_objects_dirty":123016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87391,"num_write_kb":349538,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2952,"num_bytes_recovered":12070931,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123096,"ondisk_log_size":123096,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1600756,"kb_used_data":996228,"kb_used_omap":12,"kb_used_meta":604467,"kb_avail":281514764,"statfs":{"total":289910292480,"available":288271118336,"internally_reserved":0,"allocated":1020137472,"data_stored":1019366277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":618974210},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,5,7,7,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25481216,"num_objects":6221,"num_object_clones":0,"num_object_copies":12442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-464,"num_objects_unfound":0,"num_objects_dirty":6221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6221,"num_write_kb":24884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":464,"num_bytes_recovered":1900544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003592"},"pg_stats":[{"pgid":"2.f","version":"43'7658","reported_seq":7734,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:56.762173+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:31:56.762173+0000","last_peered":"2024-09-10T20:31:56.762173+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:31:56.762173+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":7658,"log_dups_size":0,"ondisk_log_size":7658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31367168,"num_objects":7658,"num_object_clones":0,"num_object_copies":15316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7658,"num_write_kb":30632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4051},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7840","reported_seq":7946,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:58.322931+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:31:58.322931+0000","last_peered":"2024-09-10T20:31:58.322931+0000","last_clean":"2024-09-10T20:31:58.322931+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:31:58.322931+0000","last_undegraded":"2024-09-10T20:31:58.322931+0000","last_fullsized":"2024-09-10T20:31:58.322931+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":7840,"log_dups_size":0,"ondisk_log_size":7840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32088064,"num_objects":7834,"num_object_clones":0,"num_object_copies":15668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7834,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6026,"num_write_kb":24104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7726","reported_seq":13745,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:31:58.352012+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:31:58.352012+0000","last_peered":"2024-09-10T20:31:58.352012+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:31:58.352012+0000","last_undegraded":"2024-09-10T20:31:58.352012+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":7726,"log_dups_size":0,"ondisk_log_size":7726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31629312,"num_objects":7722,"num_object_clones":0,"num_object_copies":15444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":657,"num_objects_unfound":0,"num_objects_dirty":7722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5854,"num_write_kb":23416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2947,"num_bytes_recovered":12066816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":657},{"shards":"2","objects":7065}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7579","reported_seq":7652,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:58.327247+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:31:58.327247+0000","last_peered":"2024-09-10T20:31:58.327247+0000","last_clean":"2024-09-10T20:31:58.327247+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:31:58.327247+0000","last_undegraded":"2024-09-10T20:31:58.327247+0000","last_fullsized":"2024-09-10T20:31:58.327247+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":7579,"log_dups_size":0,"ondisk_log_size":7579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31043584,"num_objects":7579,"num_object_clones":0,"num_object_copies":15158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7579,"num_write_kb":30316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7821","reported_seq":7943,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:56.785977+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:31:56.785977+0000","last_peered":"2024-09-10T20:31:56.785977+0000","last_clean":"2024-09-10T20:31:56.785977+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:31:56.785977+0000","last_undegraded":"2024-09-10T20:31:56.785977+0000","last_fullsized":"2024-09-10T20:31:56.785977+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":7821,"log_dups_size":0,"ondisk_log_size":7821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32006144,"num_objects":7814,"num_object_clones":0,"num_object_copies":15628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4817,"num_write_kb":19268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7583","reported_seq":7715,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:56.738943+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:31:56.738943+0000","last_peered":"2024-09-10T20:31:56.738943+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:31:56.738943+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":7583,"log_dups_size":0,"ondisk_log_size":7583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31039488,"num_objects":7578,"num_object_clones":0,"num_object_copies":15156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4447,"num_write_kb":17788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3964},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7591","reported_seq":7723,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:56.734578+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:31:56.734578+0000","last_peered":"2024-09-10T20:31:56.734578+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:31:56.734578+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":7591,"log_dups_size":0,"ondisk_log_size":7591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31068160,"num_objects":7585,"num_object_clones":0,"num_object_copies":15170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4186,"num_write_kb":16744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4030},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7719","reported_seq":7839,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:56.784643+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:31:56.784643+0000","last_peered":"2024-09-10T20:31:56.784643+0000","last_clean":"2024-09-10T20:31:56.784643+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:31:56.784643+0000","last_undegraded":"2024-09-10T20:31:56.784643+0000","last_fullsized":"2024-09-10T20:31:56.784643+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":7719,"log_dups_size":0,"ondisk_log_size":7719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31608832,"num_objects":7717,"num_object_clones":0,"num_object_copies":15434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4790,"num_write_kb":19160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7590","reported_seq":7718,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:58.129719+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:31:58.129719+0000","last_peered":"2024-09-10T20:31:58.129719+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:31:58.129719+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":7590,"log_dups_size":0,"ondisk_log_size":7590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31059968,"num_objects":7583,"num_object_clones":0,"num_object_copies":15166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5583,"num_write_kb":22332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":4057}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355479+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355479+0000","last_peered":"2024-09-10T20:30:05.355479+0000","last_clean":"2024-09-10T20:30:05.355479+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355479+0000","last_undegraded":"2024-09-10T20:30:05.355479+0000","last_fullsized":"2024-09-10T20:30:05.355479+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7732","reported_seq":7836,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:56.675230+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:31:56.675230+0000","last_peered":"2024-09-10T20:31:56.675230+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:31:56.675230+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":7732,"log_dups_size":0,"ondisk_log_size":7732,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31638470,"num_objects":7726,"num_object_clones":0,"num_object_copies":15452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7730,"num_write_kb":30900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4111},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7667","reported_seq":7791,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:58.293604+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:31:58.293604+0000","last_peered":"2024-09-10T20:31:58.293604+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:31:58.293604+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":7667,"log_dups_size":0,"ondisk_log_size":7667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31383552,"num_objects":7662,"num_object_clones":0,"num_object_copies":15324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5495,"num_write_kb":21980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":4029}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7770","reported_seq":7894,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:58.287942+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:31:58.287942+0000","last_peered":"2024-09-10T20:31:58.287942+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:31:58.287942+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":7770,"log_dups_size":0,"ondisk_log_size":7770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31801344,"num_objects":7764,"num_object_clones":0,"num_object_copies":15528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5556,"num_write_kb":22224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":4154}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7704","reported_seq":7834,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:56.764136+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:31:56.764136+0000","last_peered":"2024-09-10T20:31:56.764136+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:31:56.764136+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":7704,"log_dups_size":0,"ondisk_log_size":7704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31547392,"num_objects":7702,"num_object_clones":0,"num_object_copies":15404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4427,"num_write_kb":17708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4054},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7700","reported_seq":7833,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:56.753608+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:31:56.753608+0000","last_peered":"2024-09-10T20:31:56.753608+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:31:56.753608+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":7700,"log_dups_size":0,"ondisk_log_size":7700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31506432,"num_objects":7692,"num_object_clones":0,"num_object_copies":15384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4394,"num_write_kb":17576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4160},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345743+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345743+0000","last_peered":"2024-09-10T20:30:05.345743+0000","last_clean":"2024-09-10T20:30:05.345743+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345743+0000","last_undegraded":"2024-09-10T20:30:05.345743+0000","last_fullsized":"2024-09-10T20:30:05.345743+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7808","reported_seq":7935,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:56.760372+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:31:56.760372+0000","last_peered":"2024-09-10T20:31:56.760372+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:31:56.760372+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":7808,"log_dups_size":0,"ondisk_log_size":7808,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31965184,"num_objects":7804,"num_object_clones":0,"num_object_copies":15608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4635,"num_write_kb":18540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4091},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7605","reported_seq":7766,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:56.755199+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:31:56.755199+0000","last_peered":"2024-09-10T20:31:56.755199+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:31:56.755199+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":7605,"log_dups_size":0,"ondisk_log_size":7605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31101411,"num_objects":7594,"num_object_clones":0,"num_object_copies":15188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7594,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4212,"num_write_kb":16848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4126},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":503854505,"num_objects":123014,"num_object_clones":0,"num_object_copies":246028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":657,"num_objects_unfound":0,"num_objects_dirty":123014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87389,"num_write_kb":349536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2950,"num_bytes_recovered":12070912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1021317120,"data_stored":1021230408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123093,"ondisk_log_size":123093,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182441,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":695520,"kb_used_data":425104,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93676320,"statfs":{"total":96636764160,"available":95924551680,"internally_reserved":0,"allocated":435306496,"data_stored":435073363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,3,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986213,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":542188,"kb_used_data":361948,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93829652,"statfs":{"total":96636764160,"available":96081563648,"internally_reserved":0,"allocated":370634752,"data_stored":370405696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184544938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659160,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":363048,"kb_used_data":209176,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":94008792,"statfs":{"total":96636764160,"available":96265003008,"internally_reserved":0,"allocated":214196224,"data_stored":213887218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":157544110},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":436547584,"data_stored":436543971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":213942272,"data_stored":213862786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":370827264,"data_stored":370823651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:32:00.068 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 152 20 124370 124350 3.19488 1.89844 0.03028 0.0244451 2024-09-10T20:32:01.068 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 153 20 124911 124891 3.18781 2.11328 0.0212502 0.0244996 2024-09-10T20:32:02.068 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 154 20 125454 125434 3.18088 2.12109 0.0666466 0.0245535 2024-09-10T20:32:03.015 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:32:03.068 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 155 20 126003 125983 3.17419 2.14453 0.0468279 0.0246044 2024-09-10T20:32:03.585 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:03.588 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:32:03.609 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":154,"stamp":"2024-09-10T20:32:02.869361+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":510436796,"num_objects":124623,"num_object_clones":0,"num_object_copies":249246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":657,"num_objects_unfound":0,"num_objects_dirty":124623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88998,"num_write_kb":355966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2952,"num_bytes_recovered":12070931,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124703,"ondisk_log_size":124703,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1612612,"kb_used_data":1003924,"kb_used_omap":12,"kb_used_meta":608627,"kb_avail":281502908,"statfs":{"total":289910292480,"available":288258977792,"internally_reserved":0,"allocated":1028018176,"data_stored":1027246981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":623234050},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,4,4,6,11,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28024832,"num_objects":6842,"num_object_clones":0,"num_object_copies":13684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-314,"num_objects_unfound":0,"num_objects_dirty":6842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6842,"num_write_kb":27368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":314,"num_bytes_recovered":1286144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003358"},"pg_stats":[{"pgid":"2.f","version":"43'7827","reported_seq":7903,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:01.747663+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:32:01.747663+0000","last_peered":"2024-09-10T20:32:01.747663+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:32:01.747663+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":7827,"log_dups_size":0,"ondisk_log_size":7827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32059392,"num_objects":7827,"num_object_clones":0,"num_object_copies":15654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7827,"num_write_kb":31308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4220},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7840","reported_seq":7946,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:58.322931+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:31:58.322931+0000","last_peered":"2024-09-10T20:31:58.322931+0000","last_clean":"2024-09-10T20:31:58.322931+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:31:58.322931+0000","last_undegraded":"2024-09-10T20:31:58.322931+0000","last_fullsized":"2024-09-10T20:31:58.322931+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":7840,"log_dups_size":0,"ondisk_log_size":7840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32088064,"num_objects":7834,"num_object_clones":0,"num_object_copies":15668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7834,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6026,"num_write_kb":24104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7726","reported_seq":13745,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:31:58.352012+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:31:58.352012+0000","last_peered":"2024-09-10T20:31:58.352012+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:31:58.352012+0000","last_undegraded":"2024-09-10T20:31:58.352012+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":7726,"log_dups_size":0,"ondisk_log_size":7726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31629312,"num_objects":7722,"num_object_clones":0,"num_object_copies":15444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":657,"num_objects_unfound":0,"num_objects_dirty":7722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5854,"num_write_kb":23416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2947,"num_bytes_recovered":12066816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":657},{"shards":"2","objects":7065}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7579","reported_seq":7652,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:58.327247+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:31:58.327247+0000","last_peered":"2024-09-10T20:31:58.327247+0000","last_clean":"2024-09-10T20:31:58.327247+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:31:58.327247+0000","last_undegraded":"2024-09-10T20:31:58.327247+0000","last_fullsized":"2024-09-10T20:31:58.327247+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":7579,"log_dups_size":0,"ondisk_log_size":7579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31043584,"num_objects":7579,"num_object_clones":0,"num_object_copies":15158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7579,"num_write_kb":30316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7983","reported_seq":8105,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:32:01.790594+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:32:01.790594+0000","last_peered":"2024-09-10T20:32:01.790594+0000","last_clean":"2024-09-10T20:32:01.790594+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:32:01.790594+0000","last_undegraded":"2024-09-10T20:32:01.790594+0000","last_fullsized":"2024-09-10T20:32:01.790594+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":7983,"log_dups_size":0,"ondisk_log_size":7983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32669696,"num_objects":7976,"num_object_clones":0,"num_object_copies":15952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4979,"num_write_kb":19916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7746","reported_seq":7878,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:01.822240+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:32:01.822240+0000","last_peered":"2024-09-10T20:32:01.822240+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:32:01.822240+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":7746,"log_dups_size":0,"ondisk_log_size":7746,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31707136,"num_objects":7741,"num_object_clones":0,"num_object_copies":15482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4610,"num_write_kb":18440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4127},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7754","reported_seq":7887,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:01.778467+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:32:01.778467+0000","last_peered":"2024-09-10T20:32:01.778467+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:32:01.778467+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":7754,"log_dups_size":0,"ondisk_log_size":7754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31735808,"num_objects":7748,"num_object_clones":0,"num_object_copies":15496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4349,"num_write_kb":17396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4193},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7883","reported_seq":8003,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:32:01.822826+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:32:01.822826+0000","last_peered":"2024-09-10T20:32:01.822826+0000","last_clean":"2024-09-10T20:32:01.822826+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:32:01.822826+0000","last_undegraded":"2024-09-10T20:32:01.822826+0000","last_fullsized":"2024-09-10T20:32:01.822826+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":7883,"log_dups_size":0,"ondisk_log_size":7883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32280576,"num_objects":7881,"num_object_clones":0,"num_object_copies":15762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4954,"num_write_kb":19816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7590","reported_seq":7718,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:58.129719+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:31:58.129719+0000","last_peered":"2024-09-10T20:31:58.129719+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:31:58.129719+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":7590,"log_dups_size":0,"ondisk_log_size":7590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31059968,"num_objects":7583,"num_object_clones":0,"num_object_copies":15166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5583,"num_write_kb":22332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":4057}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355479+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355479+0000","last_peered":"2024-09-10T20:30:05.355479+0000","last_clean":"2024-09-10T20:30:05.355479+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355479+0000","last_undegraded":"2024-09-10T20:30:05.355479+0000","last_fullsized":"2024-09-10T20:30:05.355479+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7881","reported_seq":7985,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:01.792329+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:32:01.792329+0000","last_peered":"2024-09-10T20:32:01.792329+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:32:01.792329+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":7881,"log_dups_size":0,"ondisk_log_size":7881,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32248774,"num_objects":7875,"num_object_clones":0,"num_object_copies":15750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7879,"num_write_kb":31496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4260},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7667","reported_seq":7791,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:58.293604+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:31:58.293604+0000","last_peered":"2024-09-10T20:31:58.293604+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:31:58.293604+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":7667,"log_dups_size":0,"ondisk_log_size":7667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31383552,"num_objects":7662,"num_object_clones":0,"num_object_copies":15324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5495,"num_write_kb":21980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":4029}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7770","reported_seq":7894,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:58.287942+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:31:58.287942+0000","last_peered":"2024-09-10T20:31:58.287942+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:31:58.287942+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":7770,"log_dups_size":0,"ondisk_log_size":7770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31801344,"num_objects":7764,"num_object_clones":0,"num_object_copies":15528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5556,"num_write_kb":22224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":4154}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7869","reported_seq":7998,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:01.830215+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:32:01.830215+0000","last_peered":"2024-09-10T20:32:01.830215+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:32:01.830215+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":7869,"log_dups_size":0,"ondisk_log_size":7869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32223232,"num_objects":7867,"num_object_clones":0,"num_object_copies":15734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4592,"num_write_kb":18368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4219},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7857","reported_seq":7991,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:01.723421+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:32:01.723421+0000","last_peered":"2024-09-10T20:32:01.723421+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:32:01.723421+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":7857,"log_dups_size":0,"ondisk_log_size":7857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32149504,"num_objects":7849,"num_object_clones":0,"num_object_copies":15698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4551,"num_write_kb":18204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4317},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345743+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345743+0000","last_peered":"2024-09-10T20:30:05.345743+0000","last_clean":"2024-09-10T20:30:05.345743+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345743+0000","last_undegraded":"2024-09-10T20:30:05.345743+0000","last_fullsized":"2024-09-10T20:30:05.345743+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7980","reported_seq":8107,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:01.793748+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:32:01.793748+0000","last_peered":"2024-09-10T20:32:01.793748+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:32:01.793748+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":7980,"log_dups_size":0,"ondisk_log_size":7980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32669696,"num_objects":7976,"num_object_clones":0,"num_object_copies":15952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4807,"num_write_kb":19228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4263},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7748","reported_seq":7909,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:01.787440+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:32:01.787440+0000","last_peered":"2024-09-10T20:32:01.787440+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:32:01.787440+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":7748,"log_dups_size":0,"ondisk_log_size":7748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31687139,"num_objects":7737,"num_object_clones":0,"num_object_copies":15474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4355,"num_write_kb":17420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4269},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":510436777,"num_objects":124621,"num_object_clones":0,"num_object_copies":249242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":657,"num_objects_unfound":0,"num_objects_dirty":124621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88996,"num_write_kb":355964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2950,"num_bytes_recovered":12070912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1029177344,"data_stored":1029090632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124700,"ondisk_log_size":124700,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182442,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":707376,"kb_used_data":432800,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93664464,"statfs":{"total":96636764160,"available":95912411136,"internally_reserved":0,"allocated":443187200,"data_stored":442954067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,2,2,5,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986213,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":542188,"kb_used_data":361948,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93829652,"statfs":{"total":96636764160,"available":96081563648,"internally_reserved":0,"allocated":370634752,"data_stored":370405696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184544938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659160,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":363048,"kb_used_data":209176,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":94008792,"statfs":{"total":96636764160,"available":96265003008,"internally_reserved":0,"allocated":214196224,"data_stored":213887218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":157544110},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":444407808,"data_stored":444404195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":213942272,"data_stored":213862786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":370827264,"data_stored":370823651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:32:03.610 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:32:04.068 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 156 20 126482 126462 3.16584 1.87109 0.0459381 0.0246691 2024-09-10T20:32:04.191 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:04.192 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:32:04.213 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":154,"stamp":"2024-09-10T20:32:02.869361+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":510436796,"num_objects":124623,"num_object_clones":0,"num_object_copies":249246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":657,"num_objects_unfound":0,"num_objects_dirty":124623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88998,"num_write_kb":355966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2952,"num_bytes_recovered":12070931,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124703,"ondisk_log_size":124703,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1612612,"kb_used_data":1003924,"kb_used_omap":12,"kb_used_meta":608627,"kb_avail":281502908,"statfs":{"total":289910292480,"available":288258977792,"internally_reserved":0,"allocated":1028018176,"data_stored":1027246981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":623234050},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,4,4,6,11,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28024832,"num_objects":6842,"num_object_clones":0,"num_object_copies":13684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-314,"num_objects_unfound":0,"num_objects_dirty":6842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6842,"num_write_kb":27368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":314,"num_bytes_recovered":1286144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003358"},"pg_stats":[{"pgid":"2.f","version":"43'7827","reported_seq":7903,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:01.747663+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:32:01.747663+0000","last_peered":"2024-09-10T20:32:01.747663+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:32:01.747663+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":7827,"log_dups_size":0,"ondisk_log_size":7827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32059392,"num_objects":7827,"num_object_clones":0,"num_object_copies":15654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7827,"num_write_kb":31308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4220},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7840","reported_seq":7946,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:58.322931+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:31:58.322931+0000","last_peered":"2024-09-10T20:31:58.322931+0000","last_clean":"2024-09-10T20:31:58.322931+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:31:58.322931+0000","last_undegraded":"2024-09-10T20:31:58.322931+0000","last_fullsized":"2024-09-10T20:31:58.322931+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":7840,"log_dups_size":0,"ondisk_log_size":7840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32088064,"num_objects":7834,"num_object_clones":0,"num_object_copies":15668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7834,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6026,"num_write_kb":24104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7726","reported_seq":13745,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:31:58.352012+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:31:58.352012+0000","last_peered":"2024-09-10T20:31:58.352012+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:31:58.352012+0000","last_undegraded":"2024-09-10T20:31:58.352012+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":7726,"log_dups_size":0,"ondisk_log_size":7726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31629312,"num_objects":7722,"num_object_clones":0,"num_object_copies":15444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":657,"num_objects_unfound":0,"num_objects_dirty":7722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5854,"num_write_kb":23416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2947,"num_bytes_recovered":12066816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":657},{"shards":"2","objects":7065}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7579","reported_seq":7652,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:31:58.327247+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:31:58.327247+0000","last_peered":"2024-09-10T20:31:58.327247+0000","last_clean":"2024-09-10T20:31:58.327247+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:31:58.327247+0000","last_undegraded":"2024-09-10T20:31:58.327247+0000","last_fullsized":"2024-09-10T20:31:58.327247+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":7579,"log_dups_size":0,"ondisk_log_size":7579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31043584,"num_objects":7579,"num_object_clones":0,"num_object_copies":15158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7579,"num_write_kb":30316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7983","reported_seq":8105,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:32:01.790594+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:32:01.790594+0000","last_peered":"2024-09-10T20:32:01.790594+0000","last_clean":"2024-09-10T20:32:01.790594+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:32:01.790594+0000","last_undegraded":"2024-09-10T20:32:01.790594+0000","last_fullsized":"2024-09-10T20:32:01.790594+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":7983,"log_dups_size":0,"ondisk_log_size":7983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32669696,"num_objects":7976,"num_object_clones":0,"num_object_copies":15952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4979,"num_write_kb":19916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7746","reported_seq":7878,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:01.822240+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:32:01.822240+0000","last_peered":"2024-09-10T20:32:01.822240+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:32:01.822240+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":7746,"log_dups_size":0,"ondisk_log_size":7746,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31707136,"num_objects":7741,"num_object_clones":0,"num_object_copies":15482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4610,"num_write_kb":18440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4127},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7754","reported_seq":7887,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:01.778467+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:32:01.778467+0000","last_peered":"2024-09-10T20:32:01.778467+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:32:01.778467+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":7754,"log_dups_size":0,"ondisk_log_size":7754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31735808,"num_objects":7748,"num_object_clones":0,"num_object_copies":15496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4349,"num_write_kb":17396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4193},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7883","reported_seq":8003,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:32:01.822826+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:32:01.822826+0000","last_peered":"2024-09-10T20:32:01.822826+0000","last_clean":"2024-09-10T20:32:01.822826+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:32:01.822826+0000","last_undegraded":"2024-09-10T20:32:01.822826+0000","last_fullsized":"2024-09-10T20:32:01.822826+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":7883,"log_dups_size":0,"ondisk_log_size":7883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32280576,"num_objects":7881,"num_object_clones":0,"num_object_copies":15762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4954,"num_write_kb":19816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316429+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.316429+0000","last_peered":"2024-09-10T20:30:05.316429+0000","last_clean":"2024-09-10T20:30:05.316429+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.316429+0000","last_undegraded":"2024-09-10T20:30:05.316429+0000","last_fullsized":"2024-09-10T20:30:05.316429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7590","reported_seq":7718,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:58.129719+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:31:58.129719+0000","last_peered":"2024-09-10T20:31:58.129719+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:31:58.129719+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":7590,"log_dups_size":0,"ondisk_log_size":7590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31059968,"num_objects":7583,"num_object_clones":0,"num_object_copies":15166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5583,"num_write_kb":22332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":4057}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355479+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355479+0000","last_peered":"2024-09-10T20:30:05.355479+0000","last_clean":"2024-09-10T20:30:05.355479+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355479+0000","last_undegraded":"2024-09-10T20:30:05.355479+0000","last_fullsized":"2024-09-10T20:30:05.355479+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7881","reported_seq":7985,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:01.792329+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:32:01.792329+0000","last_peered":"2024-09-10T20:32:01.792329+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:32:01.792329+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":7881,"log_dups_size":0,"ondisk_log_size":7881,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32248774,"num_objects":7875,"num_object_clones":0,"num_object_copies":15750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7879,"num_write_kb":31496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4260},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7667","reported_seq":7791,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:58.293604+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:31:58.293604+0000","last_peered":"2024-09-10T20:31:58.293604+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:31:58.293604+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":7667,"log_dups_size":0,"ondisk_log_size":7667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31383552,"num_objects":7662,"num_object_clones":0,"num_object_copies":15324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5495,"num_write_kb":21980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":4029}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7770","reported_seq":7894,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:31:58.287942+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:31:58.287942+0000","last_peered":"2024-09-10T20:31:58.287942+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:31:58.287942+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":7770,"log_dups_size":0,"ondisk_log_size":7770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31801344,"num_objects":7764,"num_object_clones":0,"num_object_copies":15528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5556,"num_write_kb":22224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":4154}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7869","reported_seq":7998,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:01.830215+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:32:01.830215+0000","last_peered":"2024-09-10T20:32:01.830215+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:32:01.830215+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":7869,"log_dups_size":0,"ondisk_log_size":7869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32223232,"num_objects":7867,"num_object_clones":0,"num_object_copies":15734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4592,"num_write_kb":18368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4219},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7857","reported_seq":7991,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:01.723421+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:32:01.723421+0000","last_peered":"2024-09-10T20:32:01.723421+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:32:01.723421+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":7857,"log_dups_size":0,"ondisk_log_size":7857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32149504,"num_objects":7849,"num_object_clones":0,"num_object_copies":15698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4551,"num_write_kb":18204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4317},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345743+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345743+0000","last_peered":"2024-09-10T20:30:05.345743+0000","last_clean":"2024-09-10T20:30:05.345743+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345743+0000","last_undegraded":"2024-09-10T20:30:05.345743+0000","last_fullsized":"2024-09-10T20:30:05.345743+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7980","reported_seq":8107,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:01.793748+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:32:01.793748+0000","last_peered":"2024-09-10T20:32:01.793748+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:32:01.793748+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":7980,"log_dups_size":0,"ondisk_log_size":7980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32669696,"num_objects":7976,"num_object_clones":0,"num_object_copies":15952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4807,"num_write_kb":19228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4263},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7748","reported_seq":7909,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:01.787440+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:32:01.787440+0000","last_peered":"2024-09-10T20:32:01.787440+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:32:01.787440+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":7748,"log_dups_size":0,"ondisk_log_size":7748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31687139,"num_objects":7737,"num_object_clones":0,"num_object_copies":15474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4355,"num_write_kb":17420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4269},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":510436777,"num_objects":124621,"num_object_clones":0,"num_object_copies":249242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":657,"num_objects_unfound":0,"num_objects_dirty":124621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88996,"num_write_kb":355964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2950,"num_bytes_recovered":12070912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1029177344,"data_stored":1029090632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124700,"ondisk_log_size":124700,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182442,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":707376,"kb_used_data":432800,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93664464,"statfs":{"total":96636764160,"available":95912411136,"internally_reserved":0,"allocated":443187200,"data_stored":442954067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,2,2,5,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986213,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":542188,"kb_used_data":361948,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93829652,"statfs":{"total":96636764160,"available":96081563648,"internally_reserved":0,"allocated":370634752,"data_stored":370405696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184544938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659160,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":363048,"kb_used_data":209176,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":94008792,"statfs":{"total":96636764160,"available":96265003008,"internally_reserved":0,"allocated":214196224,"data_stored":213887218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":157544110},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":444407808,"data_stored":444404195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":213942272,"data_stored":213862786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":370827264,"data_stored":370823651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:32:04.215 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:32:04.215 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'7827", 'reported_seq': 7903, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:01.747663+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:32:01.747663+0000', 'last_peered': '2024-09-10T20:32:01.747663+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:32:01.747663+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 7827, 'log_dups_size': 0, 'ondisk_log_size': 7827, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32059392, 'num_objects': 7827, 'num_object_clones': 0, 'num_object_copies': 15654, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3607, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7827, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7827, 'num_write_kb': 31308, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4220}, {'shards': '0,2', 'objects': 3607}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:04.215 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T20:32:04.215 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'7726", 'reported_seq': 13745, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:31:58.352012+0000', 'last_change': '2024-09-10T20:30:05.653560+0000', 'last_active': '2024-09-10T20:31:58.352012+0000', 'last_peered': '2024-09-10T20:31:58.352012+0000', 'last_clean': '2024-09-10T20:30:03.279259+0000', 'last_became_active': '2024-09-10T20:30:05.629664+0000', 'last_became_peered': '2024-09-10T20:30:05.629664+0000', 'last_unstale': '2024-09-10T20:31:58.352012+0000', 'last_undegraded': '2024-09-10T20:31:58.352012+0000', 'last_fullsized': '2024-09-10T20:30:05.312061+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1910", 'last_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'objects_scrubbed': 1868, 'log_size': 7726, 'log_dups_size': 0, 'ondisk_log_size': 7726, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 673, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31629312, 'num_objects': 7722, 'num_object_clones': 0, 'num_object_copies': 15444, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 657, 'num_objects_unfound': 0, 'num_objects_dirty': 7722, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5854, 'num_write_kb': 23416, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2947, 'num_bytes_recovered': 12066816, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 657}, {'shards': '2', 'objects': 7065}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:32:04.215 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:32:04.215 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'7746", 'reported_seq': 7878, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:01.822240+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:32:01.822240+0000', 'last_peered': '2024-09-10T20:32:01.822240+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:32:01.822240+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 7746, 'log_dups_size': 0, 'ondisk_log_size': 7746, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31707136, 'num_objects': 7741, 'num_object_clones': 0, 'num_object_copies': 15482, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3616, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7741, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4610, 'num_write_kb': 18440, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4127}, {'shards': '0,2', 'objects': 3614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:04.215 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:32:04.215 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'7754", 'reported_seq': 7887, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:01.778467+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:32:01.778467+0000', 'last_peered': '2024-09-10T20:32:01.778467+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:32:01.778467+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 7754, 'log_dups_size': 0, 'ondisk_log_size': 7754, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31735808, 'num_objects': 7748, 'num_object_clones': 0, 'num_object_copies': 15496, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3557, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7748, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4349, 'num_write_kb': 17396, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4193}, {'shards': '0,2', 'objects': 3555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:04.215 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:32:04.216 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'7590", 'reported_seq': 7718, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:58.129719+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:31:58.129719+0000', 'last_peered': '2024-09-10T20:31:58.129719+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:31:58.129719+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 7590, 'log_dups_size': 0, 'ondisk_log_size': 7590, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31059968, 'num_objects': 7583, 'num_object_clones': 0, 'num_object_copies': 15166, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3528, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7583, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5583, 'num_write_kb': 22332, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 4057}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:32:04.216 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:32:04.216 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'7881", 'reported_seq': 7985, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:01.792329+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:32:01.792329+0000', 'last_peered': '2024-09-10T20:32:01.792329+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:32:01.792329+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 7881, 'log_dups_size': 0, 'ondisk_log_size': 7881, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32248774, 'num_objects': 7875, 'num_object_clones': 0, 'num_object_copies': 15750, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3617, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7875, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7879, 'num_write_kb': 31496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4260}, {'shards': '0,2', 'objects': 3615}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:04.216 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:32:04.216 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'7667", 'reported_seq': 7791, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:58.293604+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:31:58.293604+0000', 'last_peered': '2024-09-10T20:31:58.293604+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:31:58.293604+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 7667, 'log_dups_size': 0, 'ondisk_log_size': 7667, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31383552, 'num_objects': 7662, 'num_object_clones': 0, 'num_object_copies': 15324, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3635, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7662, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5495, 'num_write_kb': 21980, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3633}, {'shards': '2', 'objects': 4029}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:32:04.216 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:32:04.216 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'7770", 'reported_seq': 7894, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:31:58.287942+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:31:58.287942+0000', 'last_peered': '2024-09-10T20:31:58.287942+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:31:58.287942+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 7770, 'log_dups_size': 0, 'ondisk_log_size': 7770, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31801344, 'num_objects': 7764, 'num_object_clones': 0, 'num_object_copies': 15528, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7764, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5556, 'num_write_kb': 22224, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3610}, {'shards': '2', 'objects': 4154}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:32:04.216 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:32:04.216 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'7869", 'reported_seq': 7998, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:01.830215+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:32:01.830215+0000', 'last_peered': '2024-09-10T20:32:01.830215+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:32:01.830215+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 7869, 'log_dups_size': 0, 'ondisk_log_size': 7869, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32223232, 'num_objects': 7867, 'num_object_clones': 0, 'num_object_copies': 15734, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3649, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7867, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4592, 'num_write_kb': 18368, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4219}, {'shards': '0,2', 'objects': 3648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:04.217 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:32:04.217 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'7857", 'reported_seq': 7991, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:01.723421+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:32:01.723421+0000', 'last_peered': '2024-09-10T20:32:01.723421+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:32:01.723421+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 7857, 'log_dups_size': 0, 'ondisk_log_size': 7857, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32149504, 'num_objects': 7849, 'num_object_clones': 0, 'num_object_copies': 15698, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3535, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7849, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4551, 'num_write_kb': 18204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4317}, {'shards': '0,2', 'objects': 3532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:04.217 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:32:04.217 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'7980", 'reported_seq': 8107, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:01.793748+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:32:01.793748+0000', 'last_peered': '2024-09-10T20:32:01.793748+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:32:01.793748+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 7980, 'log_dups_size': 0, 'ondisk_log_size': 7980, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32669696, 'num_objects': 7976, 'num_object_clones': 0, 'num_object_copies': 15952, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3715, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7976, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4807, 'num_write_kb': 19228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4263}, {'shards': '0,2', 'objects': 3713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:04.217 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:32:04.217 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'7748", 'reported_seq': 7909, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:01.787440+0000', 'last_change': '2024-09-10T20:30:05.690055+0000', 'last_active': '2024-09-10T20:32:01.787440+0000', 'last_peered': '2024-09-10T20:32:01.787440+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:32:01.787440+0000', 'last_undegraded': '2024-09-10T20:30:05.378941+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 7748, 'log_dups_size': 0, 'ondisk_log_size': 7748, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31687139, 'num_objects': 7737, 'num_object_clones': 0, 'num_object_copies': 15474, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3471, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7737, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4355, 'num_write_kb': 17420, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4269}, {'shards': '0,2', 'objects': 3468}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:04.217 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:32:04.905 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:04.905 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39542/249246 objects degraded (15.865%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":236,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":124623,"data_bytes":510436796,"bytes_used":1651314688,"bytes_avail":288258977792,"bytes_total":289910292480,"degraded_objects":39542,"degraded_total":249246,"degraded_ratio":0.15864647777697535,"misplaced_objects":657,"misplaced_total":249246,"misplaced_ratio":0.0026359500252762331,"recovering_objects_per_sec":26,"recovering_bytes_per_sec":107148,"recovering_keys_per_sec":0,"num_objects_recovered":314,"num_bytes_recovered":1286144,"num_keys_recovered":0,"write_bytes_sec":2334749,"read_op_per_sec":0,"write_op_per_sec":570},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:32:04.931 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:32:04.931 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:32:05.068 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 157 20 126954 126934 3.15742 1.84375 0.0241834 0.0247361 2024-09-10T20:32:05.485 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:05.485 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:32:05.513 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2024-09-10T20:32:04.869901+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":514618812,"num_objects":125644,"num_object_clones":0,"num_object_copies":251288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":515,"num_objects_unfound":0,"num_objects_dirty":125644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90019,"num_write_kb":360050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3094,"num_bytes_recovered":12652563,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":125724,"ondisk_log_size":125724,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1652700,"kb_used_data":1017644,"kb_used_omap":12,"kb_used_meta":634995,"kb_avail":281462820,"statfs":{"total":289910292480,"available":288217927680,"internally_reserved":0,"allocated":1042067456,"data_stored":1041296261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":650234882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,3,2,8,13,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25432064,"num_objects":6209,"num_object_clones":0,"num_object_copies":12418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-456,"num_objects_unfound":0,"num_objects_dirty":6209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6209,"num_write_kb":24836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":456,"num_bytes_recovered":1867776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003491"},"pg_stats":[{"pgid":"2.f","version":"43'7827","reported_seq":7903,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:01.747663+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:32:01.747663+0000","last_peered":"2024-09-10T20:32:01.747663+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:32:01.747663+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":7827,"log_dups_size":0,"ondisk_log_size":7827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32059392,"num_objects":7827,"num_object_clones":0,"num_object_copies":15654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7827,"num_write_kb":31308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4220},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8016","reported_seq":8122,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:32:03.369198+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:32:03.369198+0000","last_peered":"2024-09-10T20:32:03.369198+0000","last_clean":"2024-09-10T20:32:03.369198+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:32:03.369198+0000","last_undegraded":"2024-09-10T20:32:03.369198+0000","last_fullsized":"2024-09-10T20:32:03.369198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":8016,"log_dups_size":0,"ondisk_log_size":8016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32808960,"num_objects":8010,"num_object_clones":0,"num_object_copies":16020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6202,"num_write_kb":24808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7913","reported_seq":14214,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:32:03.355845+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:32:03.355845+0000","last_peered":"2024-09-10T20:32:03.355845+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:32:03.355845+0000","last_undegraded":"2024-09-10T20:32:03.355845+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":7913,"log_dups_size":0,"ondisk_log_size":7913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32395264,"num_objects":7909,"num_object_clones":0,"num_object_copies":15818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":515,"num_objects_unfound":0,"num_objects_dirty":7909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6041,"num_write_kb":24164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3089,"num_bytes_recovered":12648448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":515},{"shards":"2","objects":7394}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7746","reported_seq":7821,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:32:03.345506+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:32:03.345506+0000","last_peered":"2024-09-10T20:32:03.345506+0000","last_clean":"2024-09-10T20:32:03.345506+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:32:03.345506+0000","last_undegraded":"2024-09-10T20:32:03.345506+0000","last_fullsized":"2024-09-10T20:32:03.345506+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":7746,"log_dups_size":0,"ondisk_log_size":7746,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31727616,"num_objects":7746,"num_object_clones":0,"num_object_copies":15492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7746,"num_write_kb":30984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7983","reported_seq":8105,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:32:01.790594+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:32:01.790594+0000","last_peered":"2024-09-10T20:32:01.790594+0000","last_clean":"2024-09-10T20:32:01.790594+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:32:01.790594+0000","last_undegraded":"2024-09-10T20:32:01.790594+0000","last_fullsized":"2024-09-10T20:32:01.790594+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":7983,"log_dups_size":0,"ondisk_log_size":7983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32669696,"num_objects":7976,"num_object_clones":0,"num_object_copies":15952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4979,"num_write_kb":19916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7746","reported_seq":7878,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:01.822240+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:32:01.822240+0000","last_peered":"2024-09-10T20:32:01.822240+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:32:01.822240+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":7746,"log_dups_size":0,"ondisk_log_size":7746,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31707136,"num_objects":7741,"num_object_clones":0,"num_object_copies":15482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4610,"num_write_kb":18440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4127},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7754","reported_seq":7887,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:01.778467+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:32:01.778467+0000","last_peered":"2024-09-10T20:32:01.778467+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:32:01.778467+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":7754,"log_dups_size":0,"ondisk_log_size":7754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31735808,"num_objects":7748,"num_object_clones":0,"num_object_copies":15496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4349,"num_write_kb":17396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4193},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7883","reported_seq":8003,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:32:01.822826+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:32:01.822826+0000","last_peered":"2024-09-10T20:32:01.822826+0000","last_clean":"2024-09-10T20:32:01.822826+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:32:01.822826+0000","last_undegraded":"2024-09-10T20:32:01.822826+0000","last_fullsized":"2024-09-10T20:32:01.822826+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":7883,"log_dups_size":0,"ondisk_log_size":7883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32280576,"num_objects":7881,"num_object_clones":0,"num_object_copies":15762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4954,"num_write_kb":19816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.319278+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.319278+0000","last_peered":"2024-09-10T20:30:05.319278+0000","last_clean":"2024-09-10T20:30:05.319278+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.319278+0000","last_undegraded":"2024-09-10T20:30:05.319278+0000","last_fullsized":"2024-09-10T20:30:05.319278+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7750","reported_seq":7878,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:03.310429+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:32:03.310429+0000","last_peered":"2024-09-10T20:32:03.310429+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:32:03.310429+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":7750,"log_dups_size":0,"ondisk_log_size":7750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31715328,"num_objects":7743,"num_object_clones":0,"num_object_copies":15486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5743,"num_write_kb":22972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":4217}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355479+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355479+0000","last_peered":"2024-09-10T20:30:05.355479+0000","last_clean":"2024-09-10T20:30:05.355479+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355479+0000","last_undegraded":"2024-09-10T20:30:05.355479+0000","last_fullsized":"2024-09-10T20:30:05.355479+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7881","reported_seq":7985,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:01.792329+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:32:01.792329+0000","last_peered":"2024-09-10T20:32:01.792329+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:32:01.792329+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":7881,"log_dups_size":0,"ondisk_log_size":7881,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32248774,"num_objects":7875,"num_object_clones":0,"num_object_copies":15750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7879,"num_write_kb":31496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4260},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7844","reported_seq":7967,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:03.368670+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:32:03.368670+0000","last_peered":"2024-09-10T20:32:03.368670+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:32:03.368670+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":7844,"log_dups_size":0,"ondisk_log_size":7844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32108544,"num_objects":7839,"num_object_clones":0,"num_object_copies":15678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5672,"num_write_kb":22688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":4206}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.385096+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.385096+0000","last_peered":"2024-09-10T20:30:05.385096+0000","last_clean":"2024-09-10T20:30:05.385096+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.385096+0000","last_undegraded":"2024-09-10T20:30:05.385096+0000","last_fullsized":"2024-09-10T20:30:05.385096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7924","reported_seq":8048,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:03.314658+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:32:03.314658+0000","last_peered":"2024-09-10T20:32:03.314658+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:32:03.314658+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":7924,"log_dups_size":0,"ondisk_log_size":7924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32432128,"num_objects":7918,"num_object_clones":0,"num_object_copies":15836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5710,"num_write_kb":22840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":4308}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7869","reported_seq":7998,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:01.830215+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:32:01.830215+0000","last_peered":"2024-09-10T20:32:01.830215+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:32:01.830215+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":7869,"log_dups_size":0,"ondisk_log_size":7869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32223232,"num_objects":7867,"num_object_clones":0,"num_object_copies":15734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4592,"num_write_kb":18368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4219},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.463405+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.463405+0000","last_peered":"2024-09-10T20:30:05.463405+0000","last_clean":"2024-09-10T20:30:05.463405+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.463405+0000","last_undegraded":"2024-09-10T20:30:05.463405+0000","last_fullsized":"2024-09-10T20:30:05.463405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7857","reported_seq":7991,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:01.723421+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:32:01.723421+0000","last_peered":"2024-09-10T20:32:01.723421+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:32:01.723421+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":7857,"log_dups_size":0,"ondisk_log_size":7857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32149504,"num_objects":7849,"num_object_clones":0,"num_object_copies":15698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4551,"num_write_kb":18204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4317},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345743+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345743+0000","last_peered":"2024-09-10T20:30:05.345743+0000","last_clean":"2024-09-10T20:30:05.345743+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345743+0000","last_undegraded":"2024-09-10T20:30:05.345743+0000","last_fullsized":"2024-09-10T20:30:05.345743+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7980","reported_seq":8107,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:01.793748+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:32:01.793748+0000","last_peered":"2024-09-10T20:32:01.793748+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:32:01.793748+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":7980,"log_dups_size":0,"ondisk_log_size":7980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32669696,"num_objects":7976,"num_object_clones":0,"num_object_copies":15952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4807,"num_write_kb":19228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4263},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7748","reported_seq":7909,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:01.787440+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:32:01.787440+0000","last_peered":"2024-09-10T20:32:01.787440+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:32:01.787440+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":7748,"log_dups_size":0,"ondisk_log_size":7748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31687139,"num_objects":7737,"num_object_clones":0,"num_object_copies":15474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4355,"num_write_kb":17420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4269},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":514618793,"num_objects":125642,"num_object_clones":0,"num_object_copies":251284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":515,"num_objects_unfound":0,"num_objects_dirty":125642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90017,"num_write_kb":360048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3092,"num_bytes_recovered":12652544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1043292160,"data_stored":1043205448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":125721,"ondisk_log_size":125721,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182442,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":707376,"kb_used_data":432800,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93664464,"statfs":{"total":96636764160,"available":95912411136,"internally_reserved":0,"allocated":443187200,"data_stored":442954067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,2,2,5,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986214,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":551640,"kb_used_data":367240,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93820200,"statfs":{"total":96636764160,"available":96071884800,"internally_reserved":0,"allocated":376053760,"data_stored":375824704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188804778},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659161,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":393684,"kb_used_data":217604,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93978156,"statfs":{"total":96636764160,"available":96233631744,"internally_reserved":0,"allocated":222826496,"data_stored":222517490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":180285102},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,4,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":444407808,"data_stored":444404195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":222498816,"data_stored":222419330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":376385536,"data_stored":376381923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:32:06.068 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 158 20 127464 127444 3.15004 1.99219 0.0606979 0.0247929 2024-09-10T20:32:07.068 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 159 20 128004 127984 3.1435 2.10938 0.0308875 0.024846 2024-09-10T20:32:08.068 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout:2024-09-10T20:32:08.065776+0000 min lat: 0.00191414 max lat: 2.51074 avg lat: 0.0248968 2024-09-10T20:32:08.069 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T20:32:08.069 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 160 20 128539 128519 3.13691 2.08984 0.0139235 0.0248968 2024-09-10T20:32:08.514 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:32:09.066 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:09.067 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:32:09.068 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 161 20 129050 129030 3.12983 1.99609 0.0477586 0.0249546 2024-09-10T20:32:09.091 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2024-09-10T20:32:08.873384+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":525145532,"num_objects":128214,"num_object_clones":0,"num_object_copies":256428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":357,"num_objects_unfound":0,"num_objects_dirty":128214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":92589,"num_write_kb":370330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3252,"num_bytes_recovered":13299731,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128294,"ondisk_log_size":128294,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1665660,"kb_used_data":1038924,"kb_used_omap":12,"kb_used_meta":626675,"kb_avail":281449860,"statfs":{"total":289910292480,"available":288204656640,"internally_reserved":0,"allocated":1063858176,"data_stored":1063086981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":641715202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,4,12,14,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25268224,"num_objects":6169,"num_object_clones":0,"num_object_copies":12338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-447,"num_objects_unfound":0,"num_objects_dirty":6169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6169,"num_write_kb":24676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":447,"num_bytes_recovered":1830912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005653"},"pg_stats":[{"pgid":"2.f","version":"43'7971","reported_seq":8047,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:06.806010+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:32:06.806010+0000","last_peered":"2024-09-10T20:32:06.806010+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:32:06.806010+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":7971,"log_dups_size":0,"ondisk_log_size":7971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32649216,"num_objects":7971,"num_object_clones":0,"num_object_copies":15942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7971,"num_write_kb":31884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4364},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8184","reported_seq":8290,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:32:08.381691+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:32:08.381691+0000","last_peered":"2024-09-10T20:32:08.381691+0000","last_clean":"2024-09-10T20:32:08.381691+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:32:08.381691+0000","last_undegraded":"2024-09-10T20:32:08.381691+0000","last_fullsized":"2024-09-10T20:32:08.381691+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":8184,"log_dups_size":0,"ondisk_log_size":8184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33497088,"num_objects":8178,"num_object_clones":0,"num_object_copies":16356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6370,"num_write_kb":25480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8086","reported_seq":14706,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:32:08.386052+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:32:08.386052+0000","last_peered":"2024-09-10T20:32:08.386052+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:32:08.386052+0000","last_undegraded":"2024-09-10T20:32:08.386052+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":8086,"log_dups_size":0,"ondisk_log_size":8086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33103872,"num_objects":8082,"num_object_clones":0,"num_object_copies":16164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":357,"num_objects_unfound":0,"num_objects_dirty":8082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6214,"num_write_kb":24856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3247,"num_bytes_recovered":13295616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":357},{"shards":"2","objects":7725}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7919","reported_seq":7995,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:32:08.341744+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:32:08.341744+0000","last_peered":"2024-09-10T20:32:08.341744+0000","last_clean":"2024-09-10T20:32:08.341744+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:32:08.341744+0000","last_undegraded":"2024-09-10T20:32:08.341744+0000","last_fullsized":"2024-09-10T20:32:08.341744+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":7919,"log_dups_size":0,"ondisk_log_size":7919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32436224,"num_objects":7919,"num_object_clones":0,"num_object_copies":15838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7919,"num_write_kb":31676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8148","reported_seq":8270,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:32:06.813705+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:32:06.813705+0000","last_peered":"2024-09-10T20:32:06.813705+0000","last_clean":"2024-09-10T20:32:06.813705+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:32:06.813705+0000","last_undegraded":"2024-09-10T20:32:06.813705+0000","last_fullsized":"2024-09-10T20:32:06.813705+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":8148,"log_dups_size":0,"ondisk_log_size":8148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33345536,"num_objects":8141,"num_object_clones":0,"num_object_copies":16282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8141,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5144,"num_write_kb":20576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7900","reported_seq":8030,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:06.858970+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:32:06.858970+0000","last_peered":"2024-09-10T20:32:06.858970+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:32:06.858970+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":7900,"log_dups_size":0,"ondisk_log_size":7900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32337920,"num_objects":7895,"num_object_clones":0,"num_object_copies":15790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4764,"num_write_kb":19056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4281},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7902","reported_seq":8035,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:06.834527+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:32:06.834527+0000","last_peered":"2024-09-10T20:32:06.834527+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:32:06.834527+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":7902,"log_dups_size":0,"ondisk_log_size":7902,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32342016,"num_objects":7896,"num_object_clones":0,"num_object_copies":15792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4497,"num_write_kb":17988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4341},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8029","reported_seq":8149,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:32:06.786816+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:32:06.786816+0000","last_peered":"2024-09-10T20:32:06.786816+0000","last_clean":"2024-09-10T20:32:06.786816+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:32:06.786816+0000","last_undegraded":"2024-09-10T20:32:06.786816+0000","last_fullsized":"2024-09-10T20:32:06.786816+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":8029,"log_dups_size":0,"ondisk_log_size":8029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32878592,"num_objects":8027,"num_object_clones":0,"num_object_copies":16054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8027,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5100,"num_write_kb":20400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.319278+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.319278+0000","last_peered":"2024-09-10T20:30:05.319278+0000","last_clean":"2024-09-10T20:30:05.319278+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.319278+0000","last_undegraded":"2024-09-10T20:30:05.319278+0000","last_fullsized":"2024-09-10T20:30:05.319278+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7925","reported_seq":8053,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:08.310905+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:32:08.310905+0000","last_peered":"2024-09-10T20:32:08.310905+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:32:08.310905+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":7925,"log_dups_size":0,"ondisk_log_size":7925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32432128,"num_objects":7918,"num_object_clones":0,"num_object_copies":15836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5918,"num_write_kb":23672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":4392}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355479+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355479+0000","last_peered":"2024-09-10T20:30:05.355479+0000","last_clean":"2024-09-10T20:30:05.355479+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355479+0000","last_undegraded":"2024-09-10T20:30:05.355479+0000","last_fullsized":"2024-09-10T20:30:05.355479+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8047","reported_seq":8147,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:06.792514+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:32:06.792514+0000","last_peered":"2024-09-10T20:32:06.792514+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:32:06.792514+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":8047,"log_dups_size":0,"ondisk_log_size":8047,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32928710,"num_objects":8041,"num_object_clones":0,"num_object_copies":16082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8045,"num_write_kb":32160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4426},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7996","reported_seq":8120,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:08.277468+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:32:08.277468+0000","last_peered":"2024-09-10T20:32:08.277468+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:32:08.277468+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":7996,"log_dups_size":0,"ondisk_log_size":7996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32731136,"num_objects":7991,"num_object_clones":0,"num_object_copies":15982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5824,"num_write_kb":23296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":4358}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.703564+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.703564+0000","last_peered":"2024-09-10T20:30:05.703564+0000","last_clean":"2024-09-10T20:30:05.703564+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.703564+0000","last_undegraded":"2024-09-10T20:30:05.703564+0000","last_fullsized":"2024-09-10T20:30:05.703564+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8077","reported_seq":8201,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:08.356668+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:32:08.356668+0000","last_peered":"2024-09-10T20:32:08.356668+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:32:08.356668+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":8077,"log_dups_size":0,"ondisk_log_size":8077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33058816,"num_objects":8071,"num_object_clones":0,"num_object_copies":16142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5863,"num_write_kb":23452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":4461}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8010","reported_seq":8139,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:06.844804+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:32:06.844804+0000","last_peered":"2024-09-10T20:32:06.844804+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:32:06.844804+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":8010,"log_dups_size":0,"ondisk_log_size":8010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32800768,"num_objects":8008,"num_object_clones":0,"num_object_copies":16016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4733,"num_write_kb":18932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4360},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.473122+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.473122+0000","last_peered":"2024-09-10T20:30:05.473122+0000","last_clean":"2024-09-10T20:30:05.473122+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.473122+0000","last_undegraded":"2024-09-10T20:30:05.473122+0000","last_fullsized":"2024-09-10T20:30:05.473122+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8013","reported_seq":8147,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:06.758342+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:32:06.758342+0000","last_peered":"2024-09-10T20:32:06.758342+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:32:06.758342+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":8013,"log_dups_size":0,"ondisk_log_size":8013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32788480,"num_objects":8005,"num_object_clones":0,"num_object_copies":16010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4707,"num_write_kb":18828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4473},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345743+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345743+0000","last_peered":"2024-09-10T20:30:05.345743+0000","last_clean":"2024-09-10T20:30:05.345743+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345743+0000","last_undegraded":"2024-09-10T20:30:05.345743+0000","last_fullsized":"2024-09-10T20:30:05.345743+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8150","reported_seq":8277,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:06.833480+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:32:06.833480+0000","last_peered":"2024-09-10T20:32:06.833480+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:32:06.833480+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":8150,"log_dups_size":0,"ondisk_log_size":8150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33366016,"num_objects":8146,"num_object_clones":0,"num_object_copies":16292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4977,"num_write_kb":19908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4433},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7934","reported_seq":8095,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:06.847922+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:32:06.847922+0000","last_peered":"2024-09-10T20:32:06.847922+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:32:06.847922+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":7934,"log_dups_size":0,"ondisk_log_size":7934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32448995,"num_objects":7923,"num_object_clones":0,"num_object_copies":15846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4541,"num_write_kb":18164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4455},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":525145513,"num_objects":128212,"num_object_clones":0,"num_object_copies":256424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":357,"num_objects_unfound":0,"num_objects_dirty":128212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":92587,"num_write_kb":370328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3250,"num_bytes_recovered":13299712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1065091072,"data_stored":1065004360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128291,"ondisk_log_size":128291,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182443,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":706796,"kb_used_data":440540,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93665044,"statfs":{"total":96636764160,"available":95913005056,"internally_reserved":0,"allocated":451112960,"data_stored":450879827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,9,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986215,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548452,"kb_used_data":372372,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93823388,"statfs":{"total":96636764160,"available":96075149312,"internally_reserved":0,"allocated":381308928,"data_stored":381079872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180285098},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659162,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":410412,"kb_used_data":226012,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93961428,"statfs":{"total":96636764160,"available":96216502272,"internally_reserved":0,"allocated":231436288,"data_stored":231127282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":188804782},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,2,4,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":452304896,"data_stored":452301283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":231084032,"data_stored":231004546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":381702144,"data_stored":381698531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:32:09.092 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:32:09.645 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:09.645 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:32:09.673 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2024-09-10T20:32:08.873384+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":525145532,"num_objects":128214,"num_object_clones":0,"num_object_copies":256428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":357,"num_objects_unfound":0,"num_objects_dirty":128214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":92589,"num_write_kb":370330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3252,"num_bytes_recovered":13299731,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128294,"ondisk_log_size":128294,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1665660,"kb_used_data":1038924,"kb_used_omap":12,"kb_used_meta":626675,"kb_avail":281449860,"statfs":{"total":289910292480,"available":288204656640,"internally_reserved":0,"allocated":1063858176,"data_stored":1063086981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":641715202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,4,12,14,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25268224,"num_objects":6169,"num_object_clones":0,"num_object_copies":12338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-447,"num_objects_unfound":0,"num_objects_dirty":6169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6169,"num_write_kb":24676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":447,"num_bytes_recovered":1830912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005653"},"pg_stats":[{"pgid":"2.f","version":"43'7971","reported_seq":8047,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:06.806010+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:32:06.806010+0000","last_peered":"2024-09-10T20:32:06.806010+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:32:06.806010+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":7971,"log_dups_size":0,"ondisk_log_size":7971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32649216,"num_objects":7971,"num_object_clones":0,"num_object_copies":15942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7971,"num_write_kb":31884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4364},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8184","reported_seq":8290,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:32:08.381691+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:32:08.381691+0000","last_peered":"2024-09-10T20:32:08.381691+0000","last_clean":"2024-09-10T20:32:08.381691+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:32:08.381691+0000","last_undegraded":"2024-09-10T20:32:08.381691+0000","last_fullsized":"2024-09-10T20:32:08.381691+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":8184,"log_dups_size":0,"ondisk_log_size":8184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33497088,"num_objects":8178,"num_object_clones":0,"num_object_copies":16356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6370,"num_write_kb":25480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8086","reported_seq":14706,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:32:08.386052+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:32:08.386052+0000","last_peered":"2024-09-10T20:32:08.386052+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:32:08.386052+0000","last_undegraded":"2024-09-10T20:32:08.386052+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":8086,"log_dups_size":0,"ondisk_log_size":8086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33103872,"num_objects":8082,"num_object_clones":0,"num_object_copies":16164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":357,"num_objects_unfound":0,"num_objects_dirty":8082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6214,"num_write_kb":24856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3247,"num_bytes_recovered":13295616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":357},{"shards":"2","objects":7725}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7919","reported_seq":7995,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:32:08.341744+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:32:08.341744+0000","last_peered":"2024-09-10T20:32:08.341744+0000","last_clean":"2024-09-10T20:32:08.341744+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:32:08.341744+0000","last_undegraded":"2024-09-10T20:32:08.341744+0000","last_fullsized":"2024-09-10T20:32:08.341744+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":7919,"log_dups_size":0,"ondisk_log_size":7919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32436224,"num_objects":7919,"num_object_clones":0,"num_object_copies":15838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7919,"num_write_kb":31676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8148","reported_seq":8270,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:32:06.813705+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:32:06.813705+0000","last_peered":"2024-09-10T20:32:06.813705+0000","last_clean":"2024-09-10T20:32:06.813705+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:32:06.813705+0000","last_undegraded":"2024-09-10T20:32:06.813705+0000","last_fullsized":"2024-09-10T20:32:06.813705+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":8148,"log_dups_size":0,"ondisk_log_size":8148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33345536,"num_objects":8141,"num_object_clones":0,"num_object_copies":16282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8141,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5144,"num_write_kb":20576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7900","reported_seq":8030,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:06.858970+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:32:06.858970+0000","last_peered":"2024-09-10T20:32:06.858970+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:32:06.858970+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":7900,"log_dups_size":0,"ondisk_log_size":7900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32337920,"num_objects":7895,"num_object_clones":0,"num_object_copies":15790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4764,"num_write_kb":19056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4281},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7902","reported_seq":8035,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:06.834527+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:32:06.834527+0000","last_peered":"2024-09-10T20:32:06.834527+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:32:06.834527+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":7902,"log_dups_size":0,"ondisk_log_size":7902,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32342016,"num_objects":7896,"num_object_clones":0,"num_object_copies":15792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4497,"num_write_kb":17988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4341},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8029","reported_seq":8149,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:32:06.786816+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:32:06.786816+0000","last_peered":"2024-09-10T20:32:06.786816+0000","last_clean":"2024-09-10T20:32:06.786816+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:32:06.786816+0000","last_undegraded":"2024-09-10T20:32:06.786816+0000","last_fullsized":"2024-09-10T20:32:06.786816+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":8029,"log_dups_size":0,"ondisk_log_size":8029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32878592,"num_objects":8027,"num_object_clones":0,"num_object_copies":16054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8027,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5100,"num_write_kb":20400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.319278+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.319278+0000","last_peered":"2024-09-10T20:30:05.319278+0000","last_clean":"2024-09-10T20:30:05.319278+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.319278+0000","last_undegraded":"2024-09-10T20:30:05.319278+0000","last_fullsized":"2024-09-10T20:30:05.319278+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7925","reported_seq":8053,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:08.310905+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:32:08.310905+0000","last_peered":"2024-09-10T20:32:08.310905+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:32:08.310905+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":7925,"log_dups_size":0,"ondisk_log_size":7925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32432128,"num_objects":7918,"num_object_clones":0,"num_object_copies":15836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5918,"num_write_kb":23672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":4392}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355479+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355479+0000","last_peered":"2024-09-10T20:30:05.355479+0000","last_clean":"2024-09-10T20:30:05.355479+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355479+0000","last_undegraded":"2024-09-10T20:30:05.355479+0000","last_fullsized":"2024-09-10T20:30:05.355479+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8047","reported_seq":8147,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:06.792514+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:32:06.792514+0000","last_peered":"2024-09-10T20:32:06.792514+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:32:06.792514+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":8047,"log_dups_size":0,"ondisk_log_size":8047,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32928710,"num_objects":8041,"num_object_clones":0,"num_object_copies":16082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8045,"num_write_kb":32160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4426},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7996","reported_seq":8120,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:08.277468+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:32:08.277468+0000","last_peered":"2024-09-10T20:32:08.277468+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:32:08.277468+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":7996,"log_dups_size":0,"ondisk_log_size":7996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32731136,"num_objects":7991,"num_object_clones":0,"num_object_copies":15982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5824,"num_write_kb":23296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":4358}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.703564+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.703564+0000","last_peered":"2024-09-10T20:30:05.703564+0000","last_clean":"2024-09-10T20:30:05.703564+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.703564+0000","last_undegraded":"2024-09-10T20:30:05.703564+0000","last_fullsized":"2024-09-10T20:30:05.703564+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8077","reported_seq":8201,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:08.356668+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:32:08.356668+0000","last_peered":"2024-09-10T20:32:08.356668+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:32:08.356668+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":8077,"log_dups_size":0,"ondisk_log_size":8077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33058816,"num_objects":8071,"num_object_clones":0,"num_object_copies":16142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5863,"num_write_kb":23452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":4461}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8010","reported_seq":8139,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:06.844804+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:32:06.844804+0000","last_peered":"2024-09-10T20:32:06.844804+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:32:06.844804+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":8010,"log_dups_size":0,"ondisk_log_size":8010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32800768,"num_objects":8008,"num_object_clones":0,"num_object_copies":16016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4733,"num_write_kb":18932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4360},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.473122+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.473122+0000","last_peered":"2024-09-10T20:30:05.473122+0000","last_clean":"2024-09-10T20:30:05.473122+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.473122+0000","last_undegraded":"2024-09-10T20:30:05.473122+0000","last_fullsized":"2024-09-10T20:30:05.473122+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8013","reported_seq":8147,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:06.758342+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:32:06.758342+0000","last_peered":"2024-09-10T20:32:06.758342+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:32:06.758342+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":8013,"log_dups_size":0,"ondisk_log_size":8013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32788480,"num_objects":8005,"num_object_clones":0,"num_object_copies":16010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4707,"num_write_kb":18828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4473},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345743+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345743+0000","last_peered":"2024-09-10T20:30:05.345743+0000","last_clean":"2024-09-10T20:30:05.345743+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345743+0000","last_undegraded":"2024-09-10T20:30:05.345743+0000","last_fullsized":"2024-09-10T20:30:05.345743+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8150","reported_seq":8277,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:06.833480+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:32:06.833480+0000","last_peered":"2024-09-10T20:32:06.833480+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:32:06.833480+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":8150,"log_dups_size":0,"ondisk_log_size":8150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33366016,"num_objects":8146,"num_object_clones":0,"num_object_copies":16292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4977,"num_write_kb":19908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4433},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7934","reported_seq":8095,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:06.847922+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:32:06.847922+0000","last_peered":"2024-09-10T20:32:06.847922+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:32:06.847922+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":7934,"log_dups_size":0,"ondisk_log_size":7934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32448995,"num_objects":7923,"num_object_clones":0,"num_object_copies":15846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4541,"num_write_kb":18164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4455},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":525145513,"num_objects":128212,"num_object_clones":0,"num_object_copies":256424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":357,"num_objects_unfound":0,"num_objects_dirty":128212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":92587,"num_write_kb":370328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3250,"num_bytes_recovered":13299712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1065091072,"data_stored":1065004360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128291,"ondisk_log_size":128291,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182443,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":706796,"kb_used_data":440540,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93665044,"statfs":{"total":96636764160,"available":95913005056,"internally_reserved":0,"allocated":451112960,"data_stored":450879827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,9,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986215,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548452,"kb_used_data":372372,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93823388,"statfs":{"total":96636764160,"available":96075149312,"internally_reserved":0,"allocated":381308928,"data_stored":381079872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180285098},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659162,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":410412,"kb_used_data":226012,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93961428,"statfs":{"total":96636764160,"available":96216502272,"internally_reserved":0,"allocated":231436288,"data_stored":231127282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":188804782},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,2,4,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":452304896,"data_stored":452301283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":231084032,"data_stored":231004546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":381702144,"data_stored":381698531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:32:09.675 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:32:09.675 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'7971", 'reported_seq': 8047, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:06.806010+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:32:06.806010+0000', 'last_peered': '2024-09-10T20:32:06.806010+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:32:06.806010+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 7971, 'log_dups_size': 0, 'ondisk_log_size': 7971, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32649216, 'num_objects': 7971, 'num_object_clones': 0, 'num_object_copies': 15942, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3607, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7971, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7971, 'num_write_kb': 31884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4364}, {'shards': '0,2', 'objects': 3607}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:09.675 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T20:32:09.675 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'8086", 'reported_seq': 14706, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:32:08.386052+0000', 'last_change': '2024-09-10T20:30:05.653560+0000', 'last_active': '2024-09-10T20:32:08.386052+0000', 'last_peered': '2024-09-10T20:32:08.386052+0000', 'last_clean': '2024-09-10T20:30:03.279259+0000', 'last_became_active': '2024-09-10T20:30:05.629664+0000', 'last_became_peered': '2024-09-10T20:30:05.629664+0000', 'last_unstale': '2024-09-10T20:32:08.386052+0000', 'last_undegraded': '2024-09-10T20:32:08.386052+0000', 'last_fullsized': '2024-09-10T20:30:05.312061+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1910", 'last_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'objects_scrubbed': 1868, 'log_size': 8086, 'log_dups_size': 0, 'ondisk_log_size': 8086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 673, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33103872, 'num_objects': 8082, 'num_object_clones': 0, 'num_object_copies': 16164, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 357, 'num_objects_unfound': 0, 'num_objects_dirty': 8082, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6214, 'num_write_kb': 24856, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3247, 'num_bytes_recovered': 13295616, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 357}, {'shards': '2', 'objects': 7725}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:32:09.675 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:32:09.675 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'7900", 'reported_seq': 8030, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:06.858970+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:32:06.858970+0000', 'last_peered': '2024-09-10T20:32:06.858970+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:32:06.858970+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 7900, 'log_dups_size': 0, 'ondisk_log_size': 7900, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32337920, 'num_objects': 7895, 'num_object_clones': 0, 'num_object_copies': 15790, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3616, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7895, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4764, 'num_write_kb': 19056, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4281}, {'shards': '0,2', 'objects': 3614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:09.675 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:32:09.675 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'7902", 'reported_seq': 8035, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:06.834527+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:32:06.834527+0000', 'last_peered': '2024-09-10T20:32:06.834527+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:32:06.834527+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 7902, 'log_dups_size': 0, 'ondisk_log_size': 7902, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32342016, 'num_objects': 7896, 'num_object_clones': 0, 'num_object_copies': 15792, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3557, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7896, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4497, 'num_write_kb': 17988, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4341}, {'shards': '0,2', 'objects': 3555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:09.675 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:32:09.675 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'7925", 'reported_seq': 8053, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:08.310905+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:32:08.310905+0000', 'last_peered': '2024-09-10T20:32:08.310905+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:32:08.310905+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 7925, 'log_dups_size': 0, 'ondisk_log_size': 7925, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32432128, 'num_objects': 7918, 'num_object_clones': 0, 'num_object_copies': 15836, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3528, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7918, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5918, 'num_write_kb': 23672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 4392}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:32:09.676 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:32:09.676 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'8047", 'reported_seq': 8147, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:06.792514+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:32:06.792514+0000', 'last_peered': '2024-09-10T20:32:06.792514+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:32:06.792514+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 8047, 'log_dups_size': 0, 'ondisk_log_size': 8047, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32928710, 'num_objects': 8041, 'num_object_clones': 0, 'num_object_copies': 16082, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3617, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8041, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8045, 'num_write_kb': 32160, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4426}, {'shards': '0,2', 'objects': 3615}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:09.676 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:32:09.676 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'7996", 'reported_seq': 8120, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:08.277468+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:32:08.277468+0000', 'last_peered': '2024-09-10T20:32:08.277468+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:32:08.277468+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 7996, 'log_dups_size': 0, 'ondisk_log_size': 7996, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32731136, 'num_objects': 7991, 'num_object_clones': 0, 'num_object_copies': 15982, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3635, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7991, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5824, 'num_write_kb': 23296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3633}, {'shards': '2', 'objects': 4358}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:32:09.676 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:32:09.676 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'8077", 'reported_seq': 8201, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:08.356668+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:32:08.356668+0000', 'last_peered': '2024-09-10T20:32:08.356668+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:32:08.356668+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 8077, 'log_dups_size': 0, 'ondisk_log_size': 8077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33058816, 'num_objects': 8071, 'num_object_clones': 0, 'num_object_copies': 16142, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8071, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5863, 'num_write_kb': 23452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3610}, {'shards': '2', 'objects': 4461}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:32:09.676 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:32:09.676 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'8010", 'reported_seq': 8139, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:06.844804+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:32:06.844804+0000', 'last_peered': '2024-09-10T20:32:06.844804+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:32:06.844804+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 8010, 'log_dups_size': 0, 'ondisk_log_size': 8010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32800768, 'num_objects': 8008, 'num_object_clones': 0, 'num_object_copies': 16016, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3649, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8008, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4733, 'num_write_kb': 18932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4360}, {'shards': '0,2', 'objects': 3648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:09.676 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:32:09.677 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'8013", 'reported_seq': 8147, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:06.758342+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:32:06.758342+0000', 'last_peered': '2024-09-10T20:32:06.758342+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:32:06.758342+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 8013, 'log_dups_size': 0, 'ondisk_log_size': 8013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32788480, 'num_objects': 8005, 'num_object_clones': 0, 'num_object_copies': 16010, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3535, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8005, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4707, 'num_write_kb': 18828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4473}, {'shards': '0,2', 'objects': 3532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:09.677 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:32:09.677 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'8150", 'reported_seq': 8277, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:06.833480+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:32:06.833480+0000', 'last_peered': '2024-09-10T20:32:06.833480+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:32:06.833480+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 8150, 'log_dups_size': 0, 'ondisk_log_size': 8150, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33366016, 'num_objects': 8146, 'num_object_clones': 0, 'num_object_copies': 16292, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3715, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8146, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4977, 'num_write_kb': 19908, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4433}, {'shards': '0,2', 'objects': 3713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:09.677 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:32:09.677 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'7934", 'reported_seq': 8095, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:06.847922+0000', 'last_change': '2024-09-10T20:30:05.690055+0000', 'last_active': '2024-09-10T20:32:06.847922+0000', 'last_peered': '2024-09-10T20:32:06.847922+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:32:06.847922+0000', 'last_undegraded': '2024-09-10T20:30:05.378941+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 7934, 'log_dups_size': 0, 'ondisk_log_size': 7934, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32448995, 'num_objects': 7923, 'num_object_clones': 0, 'num_object_copies': 15846, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3471, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7923, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4541, 'num_write_kb': 18164, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4455}, {'shards': '0,2', 'objects': 3468}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:09.677 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:32:10.069 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 162 20 129522 129502 3.12189 1.84375 0.0400957 0.025018 2024-09-10T20:32:10.351 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:10.352 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39542/256428 objects degraded (15.420%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":241,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":128214,"data_bytes":525145532,"bytes_used":1705635840,"bytes_avail":288204656640,"bytes_total":289910292480,"degraded_objects":39542,"degraded_total":256428,"degraded_ratio":0.15420312914346326,"misplaced_objects":357,"misplaced_total":256428,"misplaced_ratio":0.0013922036595067622,"recovering_objects_per_sec":37,"recovering_bytes_per_sec":152504,"recovering_keys_per_sec":0,"num_objects_recovered":447,"num_bytes_recovered":1830912,"num_keys_recovered":0,"write_bytes_sec":2104693,"read_op_per_sec":0,"write_op_per_sec":513},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:32:10.375 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:32:10.375 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:32:10.940 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:10.941 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:32:10.964 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2024-09-10T20:32:10.874405+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":525145532,"num_objects":128214,"num_object_clones":0,"num_object_copies":256428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":357,"num_objects_unfound":0,"num_objects_dirty":128214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":92589,"num_write_kb":370330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3252,"num_bytes_recovered":13299731,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128294,"ondisk_log_size":128294,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1665660,"kb_used_data":1038924,"kb_used_omap":12,"kb_used_meta":626675,"kb_avail":281449860,"statfs":{"total":289910292480,"available":288204656640,"internally_reserved":0,"allocated":1063858176,"data_stored":1063086981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":641715202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,4,12,14,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21291008,"num_objects":5198,"num_object_clones":0,"num_object_copies":10396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-300,"num_objects_unfound":0,"num_objects_dirty":5198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5198,"num_write_kb":20792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":1228800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006156"},"pg_stats":[{"pgid":"2.f","version":"43'7971","reported_seq":8047,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:06.806010+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:32:06.806010+0000","last_peered":"2024-09-10T20:32:06.806010+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:32:06.806010+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":7971,"log_dups_size":0,"ondisk_log_size":7971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32649216,"num_objects":7971,"num_object_clones":0,"num_object_copies":15942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7971,"num_write_kb":31884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4364},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8184","reported_seq":8290,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:32:08.381691+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:32:08.381691+0000","last_peered":"2024-09-10T20:32:08.381691+0000","last_clean":"2024-09-10T20:32:08.381691+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:32:08.381691+0000","last_undegraded":"2024-09-10T20:32:08.381691+0000","last_fullsized":"2024-09-10T20:32:08.381691+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":8184,"log_dups_size":0,"ondisk_log_size":8184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33497088,"num_objects":8178,"num_object_clones":0,"num_object_copies":16356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6370,"num_write_kb":25480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8086","reported_seq":14706,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:32:08.386052+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:32:08.386052+0000","last_peered":"2024-09-10T20:32:08.386052+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:32:08.386052+0000","last_undegraded":"2024-09-10T20:32:08.386052+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":8086,"log_dups_size":0,"ondisk_log_size":8086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33103872,"num_objects":8082,"num_object_clones":0,"num_object_copies":16164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":357,"num_objects_unfound":0,"num_objects_dirty":8082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6214,"num_write_kb":24856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3247,"num_bytes_recovered":13295616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":357},{"shards":"2","objects":7725}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7919","reported_seq":7995,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:32:08.341744+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:32:08.341744+0000","last_peered":"2024-09-10T20:32:08.341744+0000","last_clean":"2024-09-10T20:32:08.341744+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:32:08.341744+0000","last_undegraded":"2024-09-10T20:32:08.341744+0000","last_fullsized":"2024-09-10T20:32:08.341744+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":7919,"log_dups_size":0,"ondisk_log_size":7919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32436224,"num_objects":7919,"num_object_clones":0,"num_object_copies":15838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7919,"num_write_kb":31676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8148","reported_seq":8270,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:32:06.813705+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:32:06.813705+0000","last_peered":"2024-09-10T20:32:06.813705+0000","last_clean":"2024-09-10T20:32:06.813705+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:32:06.813705+0000","last_undegraded":"2024-09-10T20:32:06.813705+0000","last_fullsized":"2024-09-10T20:32:06.813705+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":8148,"log_dups_size":0,"ondisk_log_size":8148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33345536,"num_objects":8141,"num_object_clones":0,"num_object_copies":16282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8141,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5144,"num_write_kb":20576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7900","reported_seq":8030,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:06.858970+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:32:06.858970+0000","last_peered":"2024-09-10T20:32:06.858970+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:32:06.858970+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":7900,"log_dups_size":0,"ondisk_log_size":7900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32337920,"num_objects":7895,"num_object_clones":0,"num_object_copies":15790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4764,"num_write_kb":19056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4281},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7902","reported_seq":8035,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:06.834527+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:32:06.834527+0000","last_peered":"2024-09-10T20:32:06.834527+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:32:06.834527+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":7902,"log_dups_size":0,"ondisk_log_size":7902,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32342016,"num_objects":7896,"num_object_clones":0,"num_object_copies":15792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4497,"num_write_kb":17988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4341},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8029","reported_seq":8149,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:32:06.786816+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:32:06.786816+0000","last_peered":"2024-09-10T20:32:06.786816+0000","last_clean":"2024-09-10T20:32:06.786816+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:32:06.786816+0000","last_undegraded":"2024-09-10T20:32:06.786816+0000","last_fullsized":"2024-09-10T20:32:06.786816+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":8029,"log_dups_size":0,"ondisk_log_size":8029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32878592,"num_objects":8027,"num_object_clones":0,"num_object_copies":16054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8027,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5100,"num_write_kb":20400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.319278+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.319278+0000","last_peered":"2024-09-10T20:30:05.319278+0000","last_clean":"2024-09-10T20:30:05.319278+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.319278+0000","last_undegraded":"2024-09-10T20:30:05.319278+0000","last_fullsized":"2024-09-10T20:30:05.319278+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7925","reported_seq":8053,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:08.310905+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:32:08.310905+0000","last_peered":"2024-09-10T20:32:08.310905+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:32:08.310905+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":7925,"log_dups_size":0,"ondisk_log_size":7925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32432128,"num_objects":7918,"num_object_clones":0,"num_object_copies":15836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5918,"num_write_kb":23672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":4392}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355479+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355479+0000","last_peered":"2024-09-10T20:30:05.355479+0000","last_clean":"2024-09-10T20:30:05.355479+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355479+0000","last_undegraded":"2024-09-10T20:30:05.355479+0000","last_fullsized":"2024-09-10T20:30:05.355479+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8047","reported_seq":8147,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:06.792514+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:32:06.792514+0000","last_peered":"2024-09-10T20:32:06.792514+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:32:06.792514+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":8047,"log_dups_size":0,"ondisk_log_size":8047,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32928710,"num_objects":8041,"num_object_clones":0,"num_object_copies":16082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8045,"num_write_kb":32160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4426},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7996","reported_seq":8120,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:08.277468+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:32:08.277468+0000","last_peered":"2024-09-10T20:32:08.277468+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:32:08.277468+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":7996,"log_dups_size":0,"ondisk_log_size":7996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32731136,"num_objects":7991,"num_object_clones":0,"num_object_copies":15982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5824,"num_write_kb":23296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":4358}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.703564+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.703564+0000","last_peered":"2024-09-10T20:30:05.703564+0000","last_clean":"2024-09-10T20:30:05.703564+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.703564+0000","last_undegraded":"2024-09-10T20:30:05.703564+0000","last_fullsized":"2024-09-10T20:30:05.703564+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8077","reported_seq":8201,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:08.356668+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:32:08.356668+0000","last_peered":"2024-09-10T20:32:08.356668+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:32:08.356668+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":8077,"log_dups_size":0,"ondisk_log_size":8077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33058816,"num_objects":8071,"num_object_clones":0,"num_object_copies":16142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5863,"num_write_kb":23452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":4461}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8010","reported_seq":8139,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:06.844804+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:32:06.844804+0000","last_peered":"2024-09-10T20:32:06.844804+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:32:06.844804+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":8010,"log_dups_size":0,"ondisk_log_size":8010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32800768,"num_objects":8008,"num_object_clones":0,"num_object_copies":16016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4733,"num_write_kb":18932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4360},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.473122+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.473122+0000","last_peered":"2024-09-10T20:30:05.473122+0000","last_clean":"2024-09-10T20:30:05.473122+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.473122+0000","last_undegraded":"2024-09-10T20:30:05.473122+0000","last_fullsized":"2024-09-10T20:30:05.473122+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8013","reported_seq":8147,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:06.758342+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:32:06.758342+0000","last_peered":"2024-09-10T20:32:06.758342+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:32:06.758342+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":8013,"log_dups_size":0,"ondisk_log_size":8013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32788480,"num_objects":8005,"num_object_clones":0,"num_object_copies":16010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4707,"num_write_kb":18828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4473},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345743+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345743+0000","last_peered":"2024-09-10T20:30:05.345743+0000","last_clean":"2024-09-10T20:30:05.345743+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345743+0000","last_undegraded":"2024-09-10T20:30:05.345743+0000","last_fullsized":"2024-09-10T20:30:05.345743+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8150","reported_seq":8277,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:06.833480+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:32:06.833480+0000","last_peered":"2024-09-10T20:32:06.833480+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:32:06.833480+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":8150,"log_dups_size":0,"ondisk_log_size":8150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33366016,"num_objects":8146,"num_object_clones":0,"num_object_copies":16292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4977,"num_write_kb":19908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4433},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7934","reported_seq":8095,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:06.847922+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:32:06.847922+0000","last_peered":"2024-09-10T20:32:06.847922+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:32:06.847922+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":7934,"log_dups_size":0,"ondisk_log_size":7934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32448995,"num_objects":7923,"num_object_clones":0,"num_object_copies":15846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4541,"num_write_kb":18164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4455},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":525145513,"num_objects":128212,"num_object_clones":0,"num_object_copies":256424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":357,"num_objects_unfound":0,"num_objects_dirty":128212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":92587,"num_write_kb":370328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3250,"num_bytes_recovered":13299712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1065091072,"data_stored":1065004360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128291,"ondisk_log_size":128291,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182443,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":706796,"kb_used_data":440540,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93665044,"statfs":{"total":96636764160,"available":95913005056,"internally_reserved":0,"allocated":451112960,"data_stored":450879827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,9,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986215,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548452,"kb_used_data":372372,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93823388,"statfs":{"total":96636764160,"available":96075149312,"internally_reserved":0,"allocated":381308928,"data_stored":381079872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180285098},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659162,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":410412,"kb_used_data":226012,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93961428,"statfs":{"total":96636764160,"available":96216502272,"internally_reserved":0,"allocated":231436288,"data_stored":231127282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":188804782},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,2,4,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":452304896,"data_stored":452301283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":231084032,"data_stored":231004546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":381702144,"data_stored":381698531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:32:11.069 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 163 20 129985 129965 3.11383 1.80859 0.048238 0.0250826 2024-09-10T20:32:12.069 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 164 20 130517 130497 3.10751 2.07812 0.0304928 0.0251336 2024-09-10T20:32:13.069 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 165 20 131052 131032 3.10135 2.08984 0.0226336 0.0251839 2024-09-10T20:32:13.966 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:32:14.069 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 166 20 131577 131557 3.09502 2.05078 0.0522429 0.0252358 2024-09-10T20:32:14.544 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:14.545 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:32:14.578 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":159,"stamp":"2024-09-10T20:32:12.875364+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":531555772,"num_objects":129779,"num_object_clones":0,"num_object_copies":259558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":357,"num_objects_unfound":0,"num_objects_dirty":129779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94154,"num_write_kb":376590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3252,"num_bytes_recovered":13299731,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129859,"ondisk_log_size":129859,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1696832,"kb_used_data":1047888,"kb_used_omap":12,"kb_used_meta":648883,"kb_avail":281418688,"statfs":{"total":289910292480,"available":288172736512,"internally_reserved":0,"allocated":1073037312,"data_stored":1072266117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":664456194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,4,12,14,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27701248,"num_objects":6763,"num_object_clones":0,"num_object_copies":13526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-300,"num_objects_unfound":0,"num_objects_dirty":6763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6763,"num_write_kb":27052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":1228800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006460"},"pg_stats":[{"pgid":"2.f","version":"43'8131","reported_seq":8207,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:11.838082+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:32:11.838082+0000","last_peered":"2024-09-10T20:32:11.838082+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:32:11.838082+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":8131,"log_dups_size":0,"ondisk_log_size":8131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33304576,"num_objects":8131,"num_object_clones":0,"num_object_copies":16262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8131,"num_write_kb":32524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4524},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8184","reported_seq":8290,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:32:08.381691+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:32:08.381691+0000","last_peered":"2024-09-10T20:32:08.381691+0000","last_clean":"2024-09-10T20:32:08.381691+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:32:08.381691+0000","last_undegraded":"2024-09-10T20:32:08.381691+0000","last_fullsized":"2024-09-10T20:32:08.381691+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":8184,"log_dups_size":0,"ondisk_log_size":8184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33497088,"num_objects":8178,"num_object_clones":0,"num_object_copies":16356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6370,"num_write_kb":25480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8086","reported_seq":14706,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:32:08.386052+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:32:08.386052+0000","last_peered":"2024-09-10T20:32:08.386052+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:32:08.386052+0000","last_undegraded":"2024-09-10T20:32:08.386052+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":8086,"log_dups_size":0,"ondisk_log_size":8086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33103872,"num_objects":8082,"num_object_clones":0,"num_object_copies":16164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":357,"num_objects_unfound":0,"num_objects_dirty":8082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6214,"num_write_kb":24856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3247,"num_bytes_recovered":13295616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":357},{"shards":"2","objects":7725}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7919","reported_seq":7995,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:32:08.341744+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:32:08.341744+0000","last_peered":"2024-09-10T20:32:08.341744+0000","last_clean":"2024-09-10T20:32:08.341744+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:32:08.341744+0000","last_undegraded":"2024-09-10T20:32:08.341744+0000","last_fullsized":"2024-09-10T20:32:08.341744+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":7919,"log_dups_size":0,"ondisk_log_size":7919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32436224,"num_objects":7919,"num_object_clones":0,"num_object_copies":15838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7919,"num_write_kb":31676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8309","reported_seq":8430,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:32:11.864940+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:32:11.864940+0000","last_peered":"2024-09-10T20:32:11.864940+0000","last_clean":"2024-09-10T20:32:11.864940+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:32:11.864940+0000","last_undegraded":"2024-09-10T20:32:11.864940+0000","last_fullsized":"2024-09-10T20:32:11.864940+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":8309,"log_dups_size":0,"ondisk_log_size":8309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34004992,"num_objects":8302,"num_object_clones":0,"num_object_copies":16604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5305,"num_write_kb":21220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8048","reported_seq":8180,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:11.858939+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:32:11.858939+0000","last_peered":"2024-09-10T20:32:11.858939+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:32:11.858939+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":8048,"log_dups_size":0,"ondisk_log_size":8048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32944128,"num_objects":8043,"num_object_clones":0,"num_object_copies":16086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4912,"num_write_kb":19648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4429},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8047","reported_seq":8180,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:11.888726+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:32:11.888726+0000","last_peered":"2024-09-10T20:32:11.888726+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:32:11.888726+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":8047,"log_dups_size":0,"ondisk_log_size":8047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32935936,"num_objects":8041,"num_object_clones":0,"num_object_copies":16082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4642,"num_write_kb":18568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4486},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8190","reported_seq":8310,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:32:11.806233+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:32:11.806233+0000","last_peered":"2024-09-10T20:32:11.806233+0000","last_clean":"2024-09-10T20:32:11.806233+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:32:11.806233+0000","last_undegraded":"2024-09-10T20:32:11.806233+0000","last_fullsized":"2024-09-10T20:32:11.806233+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":8190,"log_dups_size":0,"ondisk_log_size":8190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33538048,"num_objects":8188,"num_object_clones":0,"num_object_copies":16376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5261,"num_write_kb":21044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.319278+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.319278+0000","last_peered":"2024-09-10T20:30:05.319278+0000","last_clean":"2024-09-10T20:30:05.319278+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.319278+0000","last_undegraded":"2024-09-10T20:30:05.319278+0000","last_fullsized":"2024-09-10T20:30:05.319278+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7925","reported_seq":8053,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:08.310905+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:32:08.310905+0000","last_peered":"2024-09-10T20:32:08.310905+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:32:08.310905+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":7925,"log_dups_size":0,"ondisk_log_size":7925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32432128,"num_objects":7918,"num_object_clones":0,"num_object_copies":15836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5918,"num_write_kb":23672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":4392}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355479+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355479+0000","last_peered":"2024-09-10T20:30:05.355479+0000","last_clean":"2024-09-10T20:30:05.355479+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355479+0000","last_undegraded":"2024-09-10T20:30:05.355479+0000","last_fullsized":"2024-09-10T20:30:05.355479+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8205","reported_seq":8309,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:11.860288+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:32:11.860288+0000","last_peered":"2024-09-10T20:32:11.860288+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:32:11.860288+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":8205,"log_dups_size":0,"ondisk_log_size":8205,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33575878,"num_objects":8199,"num_object_clones":0,"num_object_copies":16398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8203,"num_write_kb":32792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4584},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7996","reported_seq":8120,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:08.277468+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:32:08.277468+0000","last_peered":"2024-09-10T20:32:08.277468+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:32:08.277468+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":7996,"log_dups_size":0,"ondisk_log_size":7996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32731136,"num_objects":7991,"num_object_clones":0,"num_object_copies":15982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5824,"num_write_kb":23296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":4358}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.703564+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.703564+0000","last_peered":"2024-09-10T20:30:05.703564+0000","last_clean":"2024-09-10T20:30:05.703564+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.703564+0000","last_undegraded":"2024-09-10T20:30:05.703564+0000","last_fullsized":"2024-09-10T20:30:05.703564+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8077","reported_seq":8201,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:08.356668+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:32:08.356668+0000","last_peered":"2024-09-10T20:32:08.356668+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:32:08.356668+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":8077,"log_dups_size":0,"ondisk_log_size":8077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33058816,"num_objects":8071,"num_object_clones":0,"num_object_copies":16142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5863,"num_write_kb":23452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":4461}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8167","reported_seq":8298,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:11.869930+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:32:11.869930+0000","last_peered":"2024-09-10T20:32:11.869930+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:32:11.869930+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":8167,"log_dups_size":0,"ondisk_log_size":8167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33443840,"num_objects":8165,"num_object_clones":0,"num_object_copies":16330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4890,"num_write_kb":19560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4517},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.473122+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.473122+0000","last_peered":"2024-09-10T20:30:05.473122+0000","last_clean":"2024-09-10T20:30:05.473122+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.473122+0000","last_undegraded":"2024-09-10T20:30:05.473122+0000","last_fullsized":"2024-09-10T20:30:05.473122+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8167","reported_seq":8301,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:11.867146+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:32:11.867146+0000","last_peered":"2024-09-10T20:32:11.867146+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:32:11.867146+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":8167,"log_dups_size":0,"ondisk_log_size":8167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33419264,"num_objects":8159,"num_object_clones":0,"num_object_copies":16318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4861,"num_write_kb":19444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4627},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345743+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345743+0000","last_peered":"2024-09-10T20:30:05.345743+0000","last_clean":"2024-09-10T20:30:05.345743+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345743+0000","last_undegraded":"2024-09-10T20:30:05.345743+0000","last_fullsized":"2024-09-10T20:30:05.345743+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8314","reported_seq":8439,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:11.908590+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:32:11.908590+0000","last_peered":"2024-09-10T20:32:11.908590+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:32:11.908590+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":8314,"log_dups_size":0,"ondisk_log_size":8314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34037760,"num_objects":8310,"num_object_clones":0,"num_object_copies":16620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5141,"num_write_kb":20564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4597},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.445399+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.445399+0000","last_peered":"2024-09-10T20:30:05.445399+0000","last_clean":"2024-09-10T20:30:05.445399+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.445399+0000","last_undegraded":"2024-09-10T20:30:05.445399+0000","last_fullsized":"2024-09-10T20:30:05.445399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.314596+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.314596+0000","last_peered":"2024-09-10T20:30:05.314596+0000","last_clean":"2024-09-10T20:30:05.314596+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.314596+0000","last_undegraded":"2024-09-10T20:30:05.314596+0000","last_fullsized":"2024-09-10T20:30:05.314596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8091","reported_seq":8251,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:11.895378+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:32:11.895378+0000","last_peered":"2024-09-10T20:32:11.895378+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:32:11.895378+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":8091,"log_dups_size":0,"ondisk_log_size":8091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33092067,"num_objects":8080,"num_object_clones":0,"num_object_copies":16160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4698,"num_write_kb":18792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4612},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":531555753,"num_objects":129777,"num_object_clones":0,"num_object_copies":259554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":357,"num_objects_unfound":0,"num_objects_dirty":129777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94152,"num_write_kb":376588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3250,"num_bytes_recovered":13299712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1072852992,"data_stored":1072766280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129856,"ondisk_log_size":129856,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182444,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":737968,"kb_used_data":449504,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93633872,"statfs":{"total":96636764160,"available":95881084928,"internally_reserved":0,"allocated":460292096,"data_stored":460058963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":295366314},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,9,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986215,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548452,"kb_used_data":372372,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93823388,"statfs":{"total":96636764160,"available":96075149312,"internally_reserved":0,"allocated":381308928,"data_stored":381079872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180285098},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659162,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":410412,"kb_used_data":226012,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93961428,"statfs":{"total":96636764160,"available":96216502272,"internally_reserved":0,"allocated":231436288,"data_stored":231127282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":188804782},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,2,4,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":460066816,"data_stored":460063203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":231084032,"data_stored":231004546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":381702144,"data_stored":381698531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:32:14.579 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:32:15.070 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 167 20 132044 132024 3.08741 1.82422 0.0374822 0.025296 2024-09-10T20:32:15.211 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:15.212 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:32:15.241 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-09-10T20:32:14.876048+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":535414204,"num_objects":130721,"num_object_clones":0,"num_object_copies":261442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":160,"num_objects_unfound":0,"num_objects_dirty":130721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95096,"num_write_kb":380358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3449,"num_bytes_recovered":14106643,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130801,"ondisk_log_size":130801,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1737972,"kb_used_data":1061252,"kb_used_omap":12,"kb_used_meta":676659,"kb_avail":281377548,"statfs":{"total":289910292480,"available":288130609152,"internally_reserved":0,"allocated":1086722048,"data_stored":1085950853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":692898818},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,4,9,13,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24977408,"num_objects":6098,"num_object_clones":0,"num_object_copies":12196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-497,"num_objects_unfound":0,"num_objects_dirty":6098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6098,"num_write_kb":24392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":2035712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006687"},"pg_stats":[{"pgid":"2.f","version":"43'8131","reported_seq":8207,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:11.838082+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:32:11.838082+0000","last_peered":"2024-09-10T20:32:11.838082+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:32:11.838082+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":8131,"log_dups_size":0,"ondisk_log_size":8131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33304576,"num_objects":8131,"num_object_clones":0,"num_object_copies":16262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8131,"num_write_kb":32524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4524},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8346","reported_seq":8452,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:32:13.345776+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:32:13.345776+0000","last_peered":"2024-09-10T20:32:13.345776+0000","last_clean":"2024-09-10T20:32:13.345776+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:32:13.345776+0000","last_undegraded":"2024-09-10T20:32:13.345776+0000","last_fullsized":"2024-09-10T20:32:13.345776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":8346,"log_dups_size":0,"ondisk_log_size":8346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34160640,"num_objects":8340,"num_object_clones":0,"num_object_copies":16680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6532,"num_write_kb":26128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8227","reported_seq":15241,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:32:13.425030+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:32:13.425030+0000","last_peered":"2024-09-10T20:32:13.425030+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:32:13.425030+0000","last_undegraded":"2024-09-10T20:32:13.425030+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":8227,"log_dups_size":0,"ondisk_log_size":8227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33681408,"num_objects":8223,"num_object_clones":0,"num_object_copies":16446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":160,"num_objects_unfound":0,"num_objects_dirty":8223,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6355,"num_write_kb":25420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":14102528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":160},{"shards":"2","objects":8063}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8091","reported_seq":8167,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:32:13.401843+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:32:13.401843+0000","last_peered":"2024-09-10T20:32:13.401843+0000","last_clean":"2024-09-10T20:32:13.401843+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:32:13.401843+0000","last_undegraded":"2024-09-10T20:32:13.401843+0000","last_fullsized":"2024-09-10T20:32:13.401843+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":8091,"log_dups_size":0,"ondisk_log_size":8091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33140736,"num_objects":8091,"num_object_clones":0,"num_object_copies":16182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8091,"num_write_kb":32364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8309","reported_seq":8430,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:32:11.864940+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:32:11.864940+0000","last_peered":"2024-09-10T20:32:11.864940+0000","last_clean":"2024-09-10T20:32:11.864940+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:32:11.864940+0000","last_undegraded":"2024-09-10T20:32:11.864940+0000","last_fullsized":"2024-09-10T20:32:11.864940+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":8309,"log_dups_size":0,"ondisk_log_size":8309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34004992,"num_objects":8302,"num_object_clones":0,"num_object_copies":16604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5305,"num_write_kb":21220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8048","reported_seq":8180,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:11.858939+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:32:11.858939+0000","last_peered":"2024-09-10T20:32:11.858939+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:32:11.858939+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":8048,"log_dups_size":0,"ondisk_log_size":8048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32944128,"num_objects":8043,"num_object_clones":0,"num_object_copies":16086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4912,"num_write_kb":19648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4429},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8047","reported_seq":8180,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:11.888726+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:32:11.888726+0000","last_peered":"2024-09-10T20:32:11.888726+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:32:11.888726+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":8047,"log_dups_size":0,"ondisk_log_size":8047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32935936,"num_objects":8041,"num_object_clones":0,"num_object_copies":16082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4642,"num_write_kb":18568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4486},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8190","reported_seq":8310,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:32:11.806233+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:32:11.806233+0000","last_peered":"2024-09-10T20:32:11.806233+0000","last_clean":"2024-09-10T20:32:11.806233+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:32:11.806233+0000","last_undegraded":"2024-09-10T20:32:11.806233+0000","last_fullsized":"2024-09-10T20:32:11.806233+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":8190,"log_dups_size":0,"ondisk_log_size":8190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33538048,"num_objects":8188,"num_object_clones":0,"num_object_copies":16376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5261,"num_write_kb":21044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.319278+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.319278+0000","last_peered":"2024-09-10T20:30:05.319278+0000","last_clean":"2024-09-10T20:30:05.319278+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.319278+0000","last_undegraded":"2024-09-10T20:30:05.319278+0000","last_fullsized":"2024-09-10T20:30:05.319278+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8077","reported_seq":8205,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:13.377913+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:32:13.377913+0000","last_peered":"2024-09-10T20:32:13.377913+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:32:13.377913+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":8077,"log_dups_size":0,"ondisk_log_size":8077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33054720,"num_objects":8070,"num_object_clones":0,"num_object_copies":16140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8070,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6070,"num_write_kb":24280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":4544}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355479+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355479+0000","last_peered":"2024-09-10T20:30:05.355479+0000","last_clean":"2024-09-10T20:30:05.355479+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355479+0000","last_undegraded":"2024-09-10T20:30:05.355479+0000","last_fullsized":"2024-09-10T20:30:05.355479+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8205","reported_seq":8309,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:11.860288+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:32:11.860288+0000","last_peered":"2024-09-10T20:32:11.860288+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:32:11.860288+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":8205,"log_dups_size":0,"ondisk_log_size":8205,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33575878,"num_objects":8199,"num_object_clones":0,"num_object_copies":16398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8203,"num_write_kb":32792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4584},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8136","reported_seq":8260,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:13.272749+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:32:13.272749+0000","last_peered":"2024-09-10T20:32:13.272749+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:32:13.272749+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":8136,"log_dups_size":0,"ondisk_log_size":8136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33304576,"num_objects":8131,"num_object_clones":0,"num_object_copies":16262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5964,"num_write_kb":23856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":4498}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.703564+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.703564+0000","last_peered":"2024-09-10T20:30:05.703564+0000","last_clean":"2024-09-10T20:30:05.703564+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.703564+0000","last_undegraded":"2024-09-10T20:30:05.703564+0000","last_fullsized":"2024-09-10T20:30:05.703564+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8252","reported_seq":8376,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:13.330155+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:32:13.330155+0000","last_peered":"2024-09-10T20:32:13.330155+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:32:13.330155+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":8252,"log_dups_size":0,"ondisk_log_size":8252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33775616,"num_objects":8246,"num_object_clones":0,"num_object_copies":16492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8246,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6038,"num_write_kb":24152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":4636}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8167","reported_seq":8298,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:11.869930+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:32:11.869930+0000","last_peered":"2024-09-10T20:32:11.869930+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:32:11.869930+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":8167,"log_dups_size":0,"ondisk_log_size":8167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33443840,"num_objects":8165,"num_object_clones":0,"num_object_copies":16330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4890,"num_write_kb":19560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4517},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.473122+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.473122+0000","last_peered":"2024-09-10T20:30:05.473122+0000","last_clean":"2024-09-10T20:30:05.473122+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.473122+0000","last_undegraded":"2024-09-10T20:30:05.473122+0000","last_fullsized":"2024-09-10T20:30:05.473122+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8167","reported_seq":8301,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:11.867146+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:32:11.867146+0000","last_peered":"2024-09-10T20:32:11.867146+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:32:11.867146+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":8167,"log_dups_size":0,"ondisk_log_size":8167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33419264,"num_objects":8159,"num_object_clones":0,"num_object_copies":16318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4861,"num_write_kb":19444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4627},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345743+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345743+0000","last_peered":"2024-09-10T20:30:05.345743+0000","last_clean":"2024-09-10T20:30:05.345743+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345743+0000","last_undegraded":"2024-09-10T20:30:05.345743+0000","last_fullsized":"2024-09-10T20:30:05.345743+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8314","reported_seq":8439,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:11.908590+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:32:11.908590+0000","last_peered":"2024-09-10T20:32:11.908590+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:32:11.908590+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":8314,"log_dups_size":0,"ondisk_log_size":8314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34037760,"num_objects":8310,"num_object_clones":0,"num_object_copies":16620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5141,"num_write_kb":20564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4597},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.696279+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.696279+0000","last_peered":"2024-09-10T20:30:05.696279+0000","last_clean":"2024-09-10T20:30:05.696279+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.696279+0000","last_undegraded":"2024-09-10T20:30:05.696279+0000","last_fullsized":"2024-09-10T20:30:05.696279+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316528+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.316528+0000","last_peered":"2024-09-10T20:30:05.316528+0000","last_clean":"2024-09-10T20:30:05.316528+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.316528+0000","last_undegraded":"2024-09-10T20:30:05.316528+0000","last_fullsized":"2024-09-10T20:30:05.316528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8091","reported_seq":8251,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:11.895378+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:32:11.895378+0000","last_peered":"2024-09-10T20:32:11.895378+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:32:11.895378+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":8091,"log_dups_size":0,"ondisk_log_size":8091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33092067,"num_objects":8080,"num_object_clones":0,"num_object_copies":16160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4698,"num_write_kb":18792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4612},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":535414185,"num_objects":130719,"num_object_clones":0,"num_object_copies":261438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":160,"num_objects_unfound":0,"num_objects_dirty":130719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95094,"num_write_kb":380356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3447,"num_bytes_recovered":14106624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1086545920,"data_stored":1086459208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130798,"ondisk_log_size":130798,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182444,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":737968,"kb_used_data":449504,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93633872,"statfs":{"total":96636764160,"available":95881084928,"internally_reserved":0,"allocated":460292096,"data_stored":460058963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":295366314},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,9,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986216,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":575832,"kb_used_data":377544,"kb_used_omap":4,"kb_used_meta":198267,"kb_avail":93796008,"statfs":{"total":96636764160,"available":96047112192,"internally_reserved":0,"allocated":386605056,"data_stored":386376000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":203026090},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,0,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659163,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":424172,"kb_used_data":234204,"kb_used_omap":4,"kb_used_meta":189947,"kb_avail":93947668,"statfs":{"total":96636764160,"available":96202412032,"internally_reserved":0,"allocated":239824896,"data_stored":239515890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":194506414},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,2,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":460066816,"data_stored":460063203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":239611904,"data_stored":239532418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":386867200,"data_stored":386863587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:32:15.243 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:32:15.243 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'8131", 'reported_seq': 8207, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:11.838082+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:32:11.838082+0000', 'last_peered': '2024-09-10T20:32:11.838082+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:32:11.838082+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 8131, 'log_dups_size': 0, 'ondisk_log_size': 8131, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33304576, 'num_objects': 8131, 'num_object_clones': 0, 'num_object_copies': 16262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3607, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8131, 'num_write_kb': 32524, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4524}, {'shards': '0,2', 'objects': 3607}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:15.243 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T20:32:15.243 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'8227", 'reported_seq': 15241, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:32:13.425030+0000', 'last_change': '2024-09-10T20:30:05.653560+0000', 'last_active': '2024-09-10T20:32:13.425030+0000', 'last_peered': '2024-09-10T20:32:13.425030+0000', 'last_clean': '2024-09-10T20:30:03.279259+0000', 'last_became_active': '2024-09-10T20:30:05.629664+0000', 'last_became_peered': '2024-09-10T20:30:05.629664+0000', 'last_unstale': '2024-09-10T20:32:13.425030+0000', 'last_undegraded': '2024-09-10T20:32:13.425030+0000', 'last_fullsized': '2024-09-10T20:30:05.312061+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1910", 'last_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:41.318256+0000', 'objects_scrubbed': 1868, 'log_size': 8227, 'log_dups_size': 0, 'ondisk_log_size': 8227, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 673, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33681408, 'num_objects': 8223, 'num_object_clones': 0, 'num_object_copies': 16446, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 160, 'num_objects_unfound': 0, 'num_objects_dirty': 8223, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6355, 'num_write_kb': 25420, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3444, 'num_bytes_recovered': 14102528, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 160}, {'shards': '2', 'objects': 8063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:32:15.244 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:32:15.244 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'8048", 'reported_seq': 8180, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:11.858939+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:32:11.858939+0000', 'last_peered': '2024-09-10T20:32:11.858939+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:32:11.858939+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 8048, 'log_dups_size': 0, 'ondisk_log_size': 8048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32944128, 'num_objects': 8043, 'num_object_clones': 0, 'num_object_copies': 16086, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3616, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8043, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4912, 'num_write_kb': 19648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4429}, {'shards': '0,2', 'objects': 3614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:15.244 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:32:15.244 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'8047", 'reported_seq': 8180, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:11.888726+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:32:11.888726+0000', 'last_peered': '2024-09-10T20:32:11.888726+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:32:11.888726+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 8047, 'log_dups_size': 0, 'ondisk_log_size': 8047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32935936, 'num_objects': 8041, 'num_object_clones': 0, 'num_object_copies': 16082, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3557, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8041, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4642, 'num_write_kb': 18568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4486}, {'shards': '0,2', 'objects': 3555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:15.244 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:32:15.244 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'8077", 'reported_seq': 8205, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:13.377913+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:32:13.377913+0000', 'last_peered': '2024-09-10T20:32:13.377913+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:32:13.377913+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 8077, 'log_dups_size': 0, 'ondisk_log_size': 8077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33054720, 'num_objects': 8070, 'num_object_clones': 0, 'num_object_copies': 16140, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3528, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8070, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6070, 'num_write_kb': 24280, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 4544}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:32:15.244 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:32:15.244 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'8205", 'reported_seq': 8309, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:11.860288+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:32:11.860288+0000', 'last_peered': '2024-09-10T20:32:11.860288+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:32:11.860288+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 8205, 'log_dups_size': 0, 'ondisk_log_size': 8205, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33575878, 'num_objects': 8199, 'num_object_clones': 0, 'num_object_copies': 16398, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3617, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8199, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8203, 'num_write_kb': 32792, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4584}, {'shards': '0,2', 'objects': 3615}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:15.244 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:32:15.244 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'8136", 'reported_seq': 8260, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:13.272749+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:32:13.272749+0000', 'last_peered': '2024-09-10T20:32:13.272749+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:32:13.272749+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 8136, 'log_dups_size': 0, 'ondisk_log_size': 8136, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33304576, 'num_objects': 8131, 'num_object_clones': 0, 'num_object_copies': 16262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3635, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5964, 'num_write_kb': 23856, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3633}, {'shards': '2', 'objects': 4498}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:32:15.245 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:32:15.245 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'8252", 'reported_seq': 8376, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:13.330155+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:32:13.330155+0000', 'last_peered': '2024-09-10T20:32:13.330155+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:32:13.330155+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 8252, 'log_dups_size': 0, 'ondisk_log_size': 8252, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33775616, 'num_objects': 8246, 'num_object_clones': 0, 'num_object_copies': 16492, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8246, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6038, 'num_write_kb': 24152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3610}, {'shards': '2', 'objects': 4636}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:32:15.245 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:32:15.245 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'8167", 'reported_seq': 8298, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:11.869930+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:32:11.869930+0000', 'last_peered': '2024-09-10T20:32:11.869930+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:32:11.869930+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 8167, 'log_dups_size': 0, 'ondisk_log_size': 8167, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33443840, 'num_objects': 8165, 'num_object_clones': 0, 'num_object_copies': 16330, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3649, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8165, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4890, 'num_write_kb': 19560, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4517}, {'shards': '0,2', 'objects': 3648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:15.245 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:32:15.245 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'8167", 'reported_seq': 8301, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:11.867146+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:32:11.867146+0000', 'last_peered': '2024-09-10T20:32:11.867146+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:32:11.867146+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 8167, 'log_dups_size': 0, 'ondisk_log_size': 8167, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33419264, 'num_objects': 8159, 'num_object_clones': 0, 'num_object_copies': 16318, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3535, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8159, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4861, 'num_write_kb': 19444, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4627}, {'shards': '0,2', 'objects': 3532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:15.245 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:32:15.245 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'8314", 'reported_seq': 8439, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:11.908590+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:32:11.908590+0000', 'last_peered': '2024-09-10T20:32:11.908590+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:32:11.908590+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 8314, 'log_dups_size': 0, 'ondisk_log_size': 8314, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34037760, 'num_objects': 8310, 'num_object_clones': 0, 'num_object_copies': 16620, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3715, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8310, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5141, 'num_write_kb': 20564, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4597}, {'shards': '0,2', 'objects': 3713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:15.246 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:32:15.246 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'8091", 'reported_seq': 8251, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:11.895378+0000', 'last_change': '2024-09-10T20:30:05.690055+0000', 'last_active': '2024-09-10T20:32:11.895378+0000', 'last_peered': '2024-09-10T20:32:11.895378+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:32:11.895378+0000', 'last_undegraded': '2024-09-10T20:30:05.378941+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 8091, 'log_dups_size': 0, 'ondisk_log_size': 8091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33092067, 'num_objects': 8080, 'num_object_clones': 0, 'num_object_copies': 16160, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3471, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8080, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4698, 'num_write_kb': 18792, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4612}, {'shards': '0,2', 'objects': 3468}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:15.246 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:32:15.782 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:15.782 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39542/261442 objects degraded (15.125%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":247,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":130721,"data_bytes":535414204,"bytes_used":1779683328,"bytes_avail":288130609152,"bytes_total":289910292480,"degraded_objects":39542,"degraded_total":261442,"degraded_ratio":0.151245783003496,"misplaced_objects":160,"misplaced_total":261442,"misplaced_ratio":0.00061199042234989028,"recovering_objects_per_sec":41,"recovering_bytes_per_sec":169548,"recovering_keys_per_sec":0,"num_objects_recovered":497,"num_bytes_recovered":2035712,"num_keys_recovered":0,"write_bytes_sec":2080291,"read_op_per_sec":0,"write_op_per_sec":507},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:32:15.806 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:32:15.807 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:32:16.069 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 168 20 132422 132402 3.07782 1.47656 0.0189629 0.0253746 2024-09-10T20:32:16.342 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:16.345 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:32:16.365 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-09-10T20:32:14.876048+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":535414204,"num_objects":130721,"num_object_clones":0,"num_object_copies":261442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":160,"num_objects_unfound":0,"num_objects_dirty":130721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95096,"num_write_kb":380358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3449,"num_bytes_recovered":14106643,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130801,"ondisk_log_size":130801,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1737972,"kb_used_data":1061252,"kb_used_omap":12,"kb_used_meta":676659,"kb_avail":281377548,"statfs":{"total":289910292480,"available":288130609152,"internally_reserved":0,"allocated":1086722048,"data_stored":1085950853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":692898818},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,4,9,13,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24977408,"num_objects":6098,"num_object_clones":0,"num_object_copies":12196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-497,"num_objects_unfound":0,"num_objects_dirty":6098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6098,"num_write_kb":24392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":2035712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006687"},"pg_stats":[{"pgid":"2.f","version":"43'8131","reported_seq":8207,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:11.838082+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:32:11.838082+0000","last_peered":"2024-09-10T20:32:11.838082+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:32:11.838082+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":8131,"log_dups_size":0,"ondisk_log_size":8131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33304576,"num_objects":8131,"num_object_clones":0,"num_object_copies":16262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8131,"num_write_kb":32524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4524},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8346","reported_seq":8452,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:32:13.345776+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:32:13.345776+0000","last_peered":"2024-09-10T20:32:13.345776+0000","last_clean":"2024-09-10T20:32:13.345776+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:32:13.345776+0000","last_undegraded":"2024-09-10T20:32:13.345776+0000","last_fullsized":"2024-09-10T20:32:13.345776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":8346,"log_dups_size":0,"ondisk_log_size":8346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34160640,"num_objects":8340,"num_object_clones":0,"num_object_copies":16680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6532,"num_write_kb":26128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8227","reported_seq":15241,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:32:13.425030+0000","last_change":"2024-09-10T20:30:05.653560+0000","last_active":"2024-09-10T20:32:13.425030+0000","last_peered":"2024-09-10T20:32:13.425030+0000","last_clean":"2024-09-10T20:30:03.279259+0000","last_became_active":"2024-09-10T20:30:05.629664+0000","last_became_peered":"2024-09-10T20:30:05.629664+0000","last_unstale":"2024-09-10T20:32:13.425030+0000","last_undegraded":"2024-09-10T20:32:13.425030+0000","last_fullsized":"2024-09-10T20:30:05.312061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":8227,"log_dups_size":0,"ondisk_log_size":8227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33681408,"num_objects":8223,"num_object_clones":0,"num_object_copies":16446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":160,"num_objects_unfound":0,"num_objects_dirty":8223,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6355,"num_write_kb":25420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":14102528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":160},{"shards":"2","objects":8063}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8091","reported_seq":8167,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:32:13.401843+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:32:13.401843+0000","last_peered":"2024-09-10T20:32:13.401843+0000","last_clean":"2024-09-10T20:32:13.401843+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:32:13.401843+0000","last_undegraded":"2024-09-10T20:32:13.401843+0000","last_fullsized":"2024-09-10T20:32:13.401843+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":8091,"log_dups_size":0,"ondisk_log_size":8091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33140736,"num_objects":8091,"num_object_clones":0,"num_object_copies":16182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8091,"num_write_kb":32364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8309","reported_seq":8430,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:32:11.864940+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:32:11.864940+0000","last_peered":"2024-09-10T20:32:11.864940+0000","last_clean":"2024-09-10T20:32:11.864940+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:32:11.864940+0000","last_undegraded":"2024-09-10T20:32:11.864940+0000","last_fullsized":"2024-09-10T20:32:11.864940+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":8309,"log_dups_size":0,"ondisk_log_size":8309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34004992,"num_objects":8302,"num_object_clones":0,"num_object_copies":16604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5305,"num_write_kb":21220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8048","reported_seq":8180,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:11.858939+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:32:11.858939+0000","last_peered":"2024-09-10T20:32:11.858939+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:32:11.858939+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":8048,"log_dups_size":0,"ondisk_log_size":8048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32944128,"num_objects":8043,"num_object_clones":0,"num_object_copies":16086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4912,"num_write_kb":19648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4429},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8047","reported_seq":8180,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:11.888726+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:32:11.888726+0000","last_peered":"2024-09-10T20:32:11.888726+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:32:11.888726+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":8047,"log_dups_size":0,"ondisk_log_size":8047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32935936,"num_objects":8041,"num_object_clones":0,"num_object_copies":16082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4642,"num_write_kb":18568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4486},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8190","reported_seq":8310,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:32:11.806233+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:32:11.806233+0000","last_peered":"2024-09-10T20:32:11.806233+0000","last_clean":"2024-09-10T20:32:11.806233+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:32:11.806233+0000","last_undegraded":"2024-09-10T20:32:11.806233+0000","last_fullsized":"2024-09-10T20:32:11.806233+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":8190,"log_dups_size":0,"ondisk_log_size":8190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33538048,"num_objects":8188,"num_object_clones":0,"num_object_copies":16376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5261,"num_write_kb":21044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.319278+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:30:05.319278+0000","last_peered":"2024-09-10T20:30:05.319278+0000","last_clean":"2024-09-10T20:30:05.319278+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:30:05.319278+0000","last_undegraded":"2024-09-10T20:30:05.319278+0000","last_fullsized":"2024-09-10T20:30:05.319278+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8077","reported_seq":8205,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:13.377913+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:32:13.377913+0000","last_peered":"2024-09-10T20:32:13.377913+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:32:13.377913+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":8077,"log_dups_size":0,"ondisk_log_size":8077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33054720,"num_objects":8070,"num_object_clones":0,"num_object_copies":16140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8070,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6070,"num_write_kb":24280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":4544}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.355479+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:30:05.355479+0000","last_peered":"2024-09-10T20:30:05.355479+0000","last_clean":"2024-09-10T20:30:05.355479+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:30:05.355479+0000","last_undegraded":"2024-09-10T20:30:05.355479+0000","last_fullsized":"2024-09-10T20:30:05.355479+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8205","reported_seq":8309,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:11.860288+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:32:11.860288+0000","last_peered":"2024-09-10T20:32:11.860288+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:32:11.860288+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":8205,"log_dups_size":0,"ondisk_log_size":8205,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33575878,"num_objects":8199,"num_object_clones":0,"num_object_copies":16398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8203,"num_write_kb":32792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4584},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8136","reported_seq":8260,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:13.272749+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:32:13.272749+0000","last_peered":"2024-09-10T20:32:13.272749+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:32:13.272749+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":8136,"log_dups_size":0,"ondisk_log_size":8136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33304576,"num_objects":8131,"num_object_clones":0,"num_object_copies":16262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5964,"num_write_kb":23856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":4498}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.703564+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:30:05.703564+0000","last_peered":"2024-09-10T20:30:05.703564+0000","last_clean":"2024-09-10T20:30:05.703564+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:30:05.703564+0000","last_undegraded":"2024-09-10T20:30:05.703564+0000","last_fullsized":"2024-09-10T20:30:05.703564+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8252","reported_seq":8376,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:13.330155+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:32:13.330155+0000","last_peered":"2024-09-10T20:32:13.330155+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:32:13.330155+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":8252,"log_dups_size":0,"ondisk_log_size":8252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33775616,"num_objects":8246,"num_object_clones":0,"num_object_copies":16492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8246,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6038,"num_write_kb":24152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":4636}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8167","reported_seq":8298,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:11.869930+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:32:11.869930+0000","last_peered":"2024-09-10T20:32:11.869930+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:32:11.869930+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":8167,"log_dups_size":0,"ondisk_log_size":8167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33443840,"num_objects":8165,"num_object_clones":0,"num_object_copies":16330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4890,"num_write_kb":19560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4517},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.473122+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:30:05.473122+0000","last_peered":"2024-09-10T20:30:05.473122+0000","last_clean":"2024-09-10T20:30:05.473122+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:30:05.473122+0000","last_undegraded":"2024-09-10T20:30:05.473122+0000","last_fullsized":"2024-09-10T20:30:05.473122+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8167","reported_seq":8301,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:11.867146+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:32:11.867146+0000","last_peered":"2024-09-10T20:32:11.867146+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:32:11.867146+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":8167,"log_dups_size":0,"ondisk_log_size":8167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33419264,"num_objects":8159,"num_object_clones":0,"num_object_copies":16318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4861,"num_write_kb":19444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4627},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.345743+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:30:05.345743+0000","last_peered":"2024-09-10T20:30:05.345743+0000","last_clean":"2024-09-10T20:30:05.345743+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:30:05.345743+0000","last_undegraded":"2024-09-10T20:30:05.345743+0000","last_fullsized":"2024-09-10T20:30:05.345743+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8314","reported_seq":8439,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:11.908590+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:32:11.908590+0000","last_peered":"2024-09-10T20:32:11.908590+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:32:11.908590+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":8314,"log_dups_size":0,"ondisk_log_size":8314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34037760,"num_objects":8310,"num_object_clones":0,"num_object_copies":16620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5141,"num_write_kb":20564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4597},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.696279+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:30:05.696279+0000","last_peered":"2024-09-10T20:30:05.696279+0000","last_clean":"2024-09-10T20:30:05.696279+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:30:05.696279+0000","last_undegraded":"2024-09-10T20:30:05.696279+0000","last_fullsized":"2024-09-10T20:30:05.696279+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.316528+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:30:05.316528+0000","last_peered":"2024-09-10T20:30:05.316528+0000","last_clean":"2024-09-10T20:30:05.316528+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:30:05.316528+0000","last_undegraded":"2024-09-10T20:30:05.316528+0000","last_fullsized":"2024-09-10T20:30:05.316528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8091","reported_seq":8251,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:11.895378+0000","last_change":"2024-09-10T20:30:05.690055+0000","last_active":"2024-09-10T20:32:11.895378+0000","last_peered":"2024-09-10T20:32:11.895378+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:32:11.895378+0000","last_undegraded":"2024-09-10T20:30:05.378941+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":8091,"log_dups_size":0,"ondisk_log_size":8091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33092067,"num_objects":8080,"num_object_clones":0,"num_object_copies":16160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4698,"num_write_kb":18792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4612},{"shards":"0,2","objects":3468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":535414185,"num_objects":130719,"num_object_clones":0,"num_object_copies":261438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39542,"num_objects_misplaced":160,"num_objects_unfound":0,"num_objects_dirty":130719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95094,"num_write_kb":380356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3447,"num_bytes_recovered":14106624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1086545920,"data_stored":1086459208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130798,"ondisk_log_size":130798,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182444,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":737968,"kb_used_data":449504,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93633872,"statfs":{"total":96636764160,"available":95881084928,"internally_reserved":0,"allocated":460292096,"data_stored":460058963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":295366314},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,9,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986216,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":575832,"kb_used_data":377544,"kb_used_omap":4,"kb_used_meta":198267,"kb_avail":93796008,"statfs":{"total":96636764160,"available":96047112192,"internally_reserved":0,"allocated":386605056,"data_stored":386376000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":203026090},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,0,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659163,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":424172,"kb_used_data":234204,"kb_used_omap":4,"kb_used_meta":189947,"kb_avail":93947668,"statfs":{"total":96636764160,"available":96202412032,"internally_reserved":0,"allocated":239824896,"data_stored":239515890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":194506414},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,2,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":460066816,"data_stored":460063203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":239611904,"data_stored":239532418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":386867200,"data_stored":386863587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:32:17.069 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 169 20 132923 132903 3.07119 1.95703 0.0422795 0.0254311 2024-09-10T20:32:18.069 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 170 20 133318 133298 3.0622 1.54297 0.00619153 0.0254252 2024-09-10T20:32:19.069 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 171 20 133761 133741 3.05441 1.73047 0.0661306 0.0255708 2024-09-10T20:32:19.367 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:32:19.887 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:19.889 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:32:19.923 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2024-09-10T20:32:18.877790+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":545072572,"num_objects":133079,"num_object_clones":0,"num_object_copies":266158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":3461,"num_objects_unfound":0,"num_objects_dirty":133079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97454,"num_write_kb":389790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3616,"num_bytes_recovered":14790675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133159,"ondisk_log_size":133159,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1765896,"kb_used_data":1080844,"kb_used_omap":12,"kb_used_meta":684979,"kb_avail":281349624,"statfs":{"total":289910292480,"available":288102014976,"internally_reserved":0,"allocated":1106784256,"data_stored":1105998739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":701418498},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,9,6,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19927040,"num_objects":4865,"num_object_clones":0,"num_object_copies":9730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3471,"num_objects_misplaced":3104,"num_objects_unfound":0,"num_objects_dirty":4865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4865,"num_write_kb":19460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":364,"num_bytes_recovered":1490944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003385"},"pg_stats":[{"pgid":"2.f","version":"43'8270","reported_seq":8346,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:16.911709+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:32:16.911709+0000","last_peered":"2024-09-10T20:32:16.911709+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:32:16.911709+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":8270,"log_dups_size":0,"ondisk_log_size":8270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33873920,"num_objects":8270,"num_object_clones":0,"num_object_copies":16540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8270,"num_write_kb":33080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4663},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'8491","reported_seq":8601,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.424494+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:32:18.424494+0000","last_peered":"2024-09-10T20:32:18.424494+0000","last_clean":"2024-09-10T20:32:18.424494+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:32:18.424494+0000","last_undegraded":"2024-09-10T20:32:18.424494+0000","last_fullsized":"2024-09-10T20:32:18.424494+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":8491,"log_dups_size":0,"ondisk_log_size":8491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34754560,"num_objects":8485,"num_object_clones":0,"num_object_copies":16970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6677,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'8356","reported_seq":15703,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.312624+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:32:18.312624+0000","last_peered":"2024-09-10T20:32:18.312624+0000","last_clean":"2024-09-10T20:32:18.312624+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:32:18.312624+0000","last_undegraded":"2024-09-10T20:32:18.312624+0000","last_fullsized":"2024-09-10T20:32:18.312624+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":8356,"log_dups_size":0,"ondisk_log_size":8356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34209792,"num_objects":8352,"num_object_clones":0,"num_object_copies":16704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6484,"num_write_kb":25936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'8230","reported_seq":8310,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.378738+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:32:18.378738+0000","last_peered":"2024-09-10T20:32:18.378738+0000","last_clean":"2024-09-10T20:32:18.378738+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:32:18.378738+0000","last_undegraded":"2024-09-10T20:32:18.378738+0000","last_fullsized":"2024-09-10T20:32:18.378738+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":8230,"log_dups_size":0,"ondisk_log_size":8230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33710080,"num_objects":8230,"num_object_clones":0,"num_object_copies":16460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8230,"num_write_kb":32920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8463","reported_seq":8585,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:32:16.891777+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:32:16.891777+0000","last_peered":"2024-09-10T20:32:16.891777+0000","last_clean":"2024-09-10T20:32:16.891777+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:32:16.891777+0000","last_undegraded":"2024-09-10T20:32:16.891777+0000","last_fullsized":"2024-09-10T20:32:16.891777+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":8463,"log_dups_size":0,"ondisk_log_size":8463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34635776,"num_objects":8456,"num_object_clones":0,"num_object_copies":16912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5459,"num_write_kb":21836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8200","reported_seq":8332,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:16.879273+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:32:16.879273+0000","last_peered":"2024-09-10T20:32:16.879273+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:32:16.879273+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":8200,"log_dups_size":0,"ondisk_log_size":8200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33566720,"num_objects":8195,"num_object_clones":0,"num_object_copies":16390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5064,"num_write_kb":20256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4581},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8235","reported_seq":8369,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:16.909622+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:32:16.909622+0000","last_peered":"2024-09-10T20:32:16.909622+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:32:16.909622+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":8235,"log_dups_size":0,"ondisk_log_size":8235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33705984,"num_objects":8229,"num_object_clones":0,"num_object_copies":16458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4830,"num_write_kb":19320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4674},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8339","reported_seq":8454,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:32:16.847025+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:32:16.847025+0000","last_peered":"2024-09-10T20:32:16.847025+0000","last_clean":"2024-09-10T20:32:16.847025+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:32:16.847025+0000","last_undegraded":"2024-09-10T20:32:16.847025+0000","last_fullsized":"2024-09-10T20:32:16.847025+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":8339,"log_dups_size":0,"ondisk_log_size":8339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34148352,"num_objects":8337,"num_object_clones":0,"num_object_copies":16674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5410,"num_write_kb":21640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8209","reported_seq":8341,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:18.403599+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:32:18.403599+0000","last_peered":"2024-09-10T20:32:18.403599+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:32:18.403599+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":8209,"log_dups_size":0,"ondisk_log_size":8209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33595392,"num_objects":8202,"num_object_clones":0,"num_object_copies":16404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6202,"num_write_kb":24808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":4676}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190232+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190232+0000","last_peered":"2024-09-10T20:32:18.190232+0000","last_clean":"2024-09-10T20:32:18.190232+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190232+0000","last_undegraded":"2024-09-10T20:32:18.190232+0000","last_fullsized":"2024-09-10T20:32:18.190232+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8369","reported_seq":8473,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:16.829642+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:32:16.829642+0000","last_peered":"2024-09-10T20:32:16.829642+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:32:16.829642+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":8369,"log_dups_size":0,"ondisk_log_size":8369,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34247622,"num_objects":8363,"num_object_clones":0,"num_object_copies":16726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8367,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4748},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8284","reported_seq":8410,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:18.420541+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:32:18.420541+0000","last_peered":"2024-09-10T20:32:18.420541+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:32:18.420541+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":8284,"log_dups_size":0,"ondisk_log_size":8284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33910784,"num_objects":8279,"num_object_clones":0,"num_object_copies":16558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6112,"num_write_kb":24448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":4646}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8378","reported_seq":8506,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:18.331446+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:32:18.331446+0000","last_peered":"2024-09-10T20:32:18.331446+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:32:18.331446+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":8378,"log_dups_size":0,"ondisk_log_size":8378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34291712,"num_objects":8372,"num_object_clones":0,"num_object_copies":16744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6164,"num_write_kb":24656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":4762}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8336","reported_seq":8467,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:16.935946+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:32:16.935946+0000","last_peered":"2024-09-10T20:32:16.935946+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:32:16.935946+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":8336,"log_dups_size":0,"ondisk_log_size":8336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34136064,"num_objects":8334,"num_object_clones":0,"num_object_copies":16668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5059,"num_write_kb":20236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4686},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8314","reported_seq":8448,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:16.887241+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:32:16.887241+0000","last_peered":"2024-09-10T20:32:16.887241+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:32:16.887241+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":8314,"log_dups_size":0,"ondisk_log_size":8314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34021376,"num_objects":8306,"num_object_clones":0,"num_object_copies":16612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5008,"num_write_kb":20032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4774},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193200+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193200+0000","last_peered":"2024-09-10T20:32:18.193200+0000","last_clean":"2024-09-10T20:32:18.193200+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193200+0000","last_undegraded":"2024-09-10T20:32:18.193200+0000","last_fullsized":"2024-09-10T20:32:18.193200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8455","reported_seq":8582,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:16.842900+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:32:16.842900+0000","last_peered":"2024-09-10T20:32:16.842900+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:32:16.842900+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":8455,"log_dups_size":0,"ondisk_log_size":8455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34615296,"num_objects":8451,"num_object_clones":0,"num_object_copies":16902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5282,"num_write_kb":21128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4738},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8227","reported_seq":8402,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:32:16.954906+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:32:16.954906+0000","last_peered":"2024-09-10T20:32:16.954906+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:32:16.954906+0000","last_undegraded":"2024-09-10T20:32:16.954906+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":8227,"log_dups_size":0,"ondisk_log_size":8227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33649123,"num_objects":8216,"num_object_clones":0,"num_object_copies":16432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3461,"num_objects_unfound":0,"num_objects_dirty":8216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4834,"num_write_kb":19336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":28672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4755},{"shards":"0,2","objects":3461}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":545072553,"num_objects":133077,"num_object_clones":0,"num_object_copies":266154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":3461,"num_objects_unfound":0,"num_objects_dirty":133077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97452,"num_write_kb":389788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14790656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1106259968,"data_stored":1106173256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133156,"ondisk_log_size":133156,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182445,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":749464,"kb_used_data":456840,"kb_used_omap":4,"kb_used_meta":292603,"kb_avail":93622376,"statfs":{"total":96636764160,"available":95869313024,"internally_reserved":0,"allocated":467804160,"data_stored":467571027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":299626154},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,3,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986217,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":580480,"kb_used_data":382188,"kb_used_omap":4,"kb_used_meta":198267,"kb_avail":93791360,"statfs":{"total":96636764160,"available":96042352640,"internally_reserved":0,"allocated":391360512,"data_stored":391126689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":203026090},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659164,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":435952,"kb_used_data":241816,"kb_used_omap":4,"kb_used_meta":194107,"kb_avail":93935888,"statfs":{"total":96636764160,"available":96190349312,"internally_reserved":0,"allocated":247619584,"data_stored":247301023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":198766254},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,6,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":467611648,"data_stored":467608035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":247279616,"data_stored":247200130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":391368704,"data_stored":391365091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:32:19.925 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:32:20.070 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 172 20 134206 134186 3.04676 1.73828 0.0290293 0.0256338 2024-09-10T20:32:20.494 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:20.498 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:32:20.518 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2024-09-10T20:32:18.877790+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":545072572,"num_objects":133079,"num_object_clones":0,"num_object_copies":266158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":3461,"num_objects_unfound":0,"num_objects_dirty":133079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97454,"num_write_kb":389790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3616,"num_bytes_recovered":14790675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133159,"ondisk_log_size":133159,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1765896,"kb_used_data":1080844,"kb_used_omap":12,"kb_used_meta":684979,"kb_avail":281349624,"statfs":{"total":289910292480,"available":288102014976,"internally_reserved":0,"allocated":1106784256,"data_stored":1105998739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":701418498},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,9,6,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19927040,"num_objects":4865,"num_object_clones":0,"num_object_copies":9730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3471,"num_objects_misplaced":3104,"num_objects_unfound":0,"num_objects_dirty":4865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4865,"num_write_kb":19460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":364,"num_bytes_recovered":1490944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003385"},"pg_stats":[{"pgid":"2.f","version":"43'8270","reported_seq":8346,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:16.911709+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:32:16.911709+0000","last_peered":"2024-09-10T20:32:16.911709+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:32:16.911709+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":8270,"log_dups_size":0,"ondisk_log_size":8270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33873920,"num_objects":8270,"num_object_clones":0,"num_object_copies":16540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8270,"num_write_kb":33080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4663},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'8491","reported_seq":8601,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.424494+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:32:18.424494+0000","last_peered":"2024-09-10T20:32:18.424494+0000","last_clean":"2024-09-10T20:32:18.424494+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:32:18.424494+0000","last_undegraded":"2024-09-10T20:32:18.424494+0000","last_fullsized":"2024-09-10T20:32:18.424494+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":8491,"log_dups_size":0,"ondisk_log_size":8491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34754560,"num_objects":8485,"num_object_clones":0,"num_object_copies":16970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6677,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'8356","reported_seq":15703,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.312624+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:32:18.312624+0000","last_peered":"2024-09-10T20:32:18.312624+0000","last_clean":"2024-09-10T20:32:18.312624+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:32:18.312624+0000","last_undegraded":"2024-09-10T20:32:18.312624+0000","last_fullsized":"2024-09-10T20:32:18.312624+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":8356,"log_dups_size":0,"ondisk_log_size":8356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34209792,"num_objects":8352,"num_object_clones":0,"num_object_copies":16704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6484,"num_write_kb":25936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'8230","reported_seq":8310,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.378738+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:32:18.378738+0000","last_peered":"2024-09-10T20:32:18.378738+0000","last_clean":"2024-09-10T20:32:18.378738+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:32:18.378738+0000","last_undegraded":"2024-09-10T20:32:18.378738+0000","last_fullsized":"2024-09-10T20:32:18.378738+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":8230,"log_dups_size":0,"ondisk_log_size":8230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33710080,"num_objects":8230,"num_object_clones":0,"num_object_copies":16460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8230,"num_write_kb":32920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8463","reported_seq":8585,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:32:16.891777+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:32:16.891777+0000","last_peered":"2024-09-10T20:32:16.891777+0000","last_clean":"2024-09-10T20:32:16.891777+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:32:16.891777+0000","last_undegraded":"2024-09-10T20:32:16.891777+0000","last_fullsized":"2024-09-10T20:32:16.891777+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":8463,"log_dups_size":0,"ondisk_log_size":8463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34635776,"num_objects":8456,"num_object_clones":0,"num_object_copies":16912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5459,"num_write_kb":21836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8200","reported_seq":8332,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:16.879273+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:32:16.879273+0000","last_peered":"2024-09-10T20:32:16.879273+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:32:16.879273+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":8200,"log_dups_size":0,"ondisk_log_size":8200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33566720,"num_objects":8195,"num_object_clones":0,"num_object_copies":16390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5064,"num_write_kb":20256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4581},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8235","reported_seq":8369,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:16.909622+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:32:16.909622+0000","last_peered":"2024-09-10T20:32:16.909622+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:32:16.909622+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":8235,"log_dups_size":0,"ondisk_log_size":8235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33705984,"num_objects":8229,"num_object_clones":0,"num_object_copies":16458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4830,"num_write_kb":19320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4674},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8339","reported_seq":8454,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:32:16.847025+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:32:16.847025+0000","last_peered":"2024-09-10T20:32:16.847025+0000","last_clean":"2024-09-10T20:32:16.847025+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:32:16.847025+0000","last_undegraded":"2024-09-10T20:32:16.847025+0000","last_fullsized":"2024-09-10T20:32:16.847025+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":8339,"log_dups_size":0,"ondisk_log_size":8339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34148352,"num_objects":8337,"num_object_clones":0,"num_object_copies":16674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5410,"num_write_kb":21640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8209","reported_seq":8341,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:18.403599+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:32:18.403599+0000","last_peered":"2024-09-10T20:32:18.403599+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:32:18.403599+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":8209,"log_dups_size":0,"ondisk_log_size":8209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33595392,"num_objects":8202,"num_object_clones":0,"num_object_copies":16404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6202,"num_write_kb":24808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":4676}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190232+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190232+0000","last_peered":"2024-09-10T20:32:18.190232+0000","last_clean":"2024-09-10T20:32:18.190232+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190232+0000","last_undegraded":"2024-09-10T20:32:18.190232+0000","last_fullsized":"2024-09-10T20:32:18.190232+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8369","reported_seq":8473,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:16.829642+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:32:16.829642+0000","last_peered":"2024-09-10T20:32:16.829642+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:32:16.829642+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":8369,"log_dups_size":0,"ondisk_log_size":8369,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34247622,"num_objects":8363,"num_object_clones":0,"num_object_copies":16726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8367,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4748},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8284","reported_seq":8410,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:18.420541+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:32:18.420541+0000","last_peered":"2024-09-10T20:32:18.420541+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:32:18.420541+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":8284,"log_dups_size":0,"ondisk_log_size":8284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33910784,"num_objects":8279,"num_object_clones":0,"num_object_copies":16558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6112,"num_write_kb":24448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":4646}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8378","reported_seq":8506,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:18.331446+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:32:18.331446+0000","last_peered":"2024-09-10T20:32:18.331446+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:32:18.331446+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":8378,"log_dups_size":0,"ondisk_log_size":8378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34291712,"num_objects":8372,"num_object_clones":0,"num_object_copies":16744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6164,"num_write_kb":24656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":4762}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8336","reported_seq":8467,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:16.935946+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:32:16.935946+0000","last_peered":"2024-09-10T20:32:16.935946+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:32:16.935946+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":8336,"log_dups_size":0,"ondisk_log_size":8336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34136064,"num_objects":8334,"num_object_clones":0,"num_object_copies":16668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5059,"num_write_kb":20236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4686},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8314","reported_seq":8448,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:16.887241+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:32:16.887241+0000","last_peered":"2024-09-10T20:32:16.887241+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:32:16.887241+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":8314,"log_dups_size":0,"ondisk_log_size":8314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34021376,"num_objects":8306,"num_object_clones":0,"num_object_copies":16612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5008,"num_write_kb":20032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4774},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193200+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193200+0000","last_peered":"2024-09-10T20:32:18.193200+0000","last_clean":"2024-09-10T20:32:18.193200+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193200+0000","last_undegraded":"2024-09-10T20:32:18.193200+0000","last_fullsized":"2024-09-10T20:32:18.193200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8455","reported_seq":8582,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:16.842900+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:32:16.842900+0000","last_peered":"2024-09-10T20:32:16.842900+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:32:16.842900+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":8455,"log_dups_size":0,"ondisk_log_size":8455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34615296,"num_objects":8451,"num_object_clones":0,"num_object_copies":16902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5282,"num_write_kb":21128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4738},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8227","reported_seq":8402,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:32:16.954906+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:32:16.954906+0000","last_peered":"2024-09-10T20:32:16.954906+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:32:16.954906+0000","last_undegraded":"2024-09-10T20:32:16.954906+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":8227,"log_dups_size":0,"ondisk_log_size":8227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33649123,"num_objects":8216,"num_object_clones":0,"num_object_copies":16432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3461,"num_objects_unfound":0,"num_objects_dirty":8216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4834,"num_write_kb":19336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":28672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4755},{"shards":"0,2","objects":3461}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":545072553,"num_objects":133077,"num_object_clones":0,"num_object_copies":266154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":3461,"num_objects_unfound":0,"num_objects_dirty":133077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97452,"num_write_kb":389788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14790656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1106259968,"data_stored":1106173256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133156,"ondisk_log_size":133156,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182445,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":749464,"kb_used_data":456840,"kb_used_omap":4,"kb_used_meta":292603,"kb_avail":93622376,"statfs":{"total":96636764160,"available":95869313024,"internally_reserved":0,"allocated":467804160,"data_stored":467571027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":299626154},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,3,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986217,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":580480,"kb_used_data":382188,"kb_used_omap":4,"kb_used_meta":198267,"kb_avail":93791360,"statfs":{"total":96636764160,"available":96042352640,"internally_reserved":0,"allocated":391360512,"data_stored":391126689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":203026090},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659164,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":435952,"kb_used_data":241816,"kb_used_omap":4,"kb_used_meta":194107,"kb_avail":93935888,"statfs":{"total":96636764160,"available":96190349312,"internally_reserved":0,"allocated":247619584,"data_stored":247301023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":198766254},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,6,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":467611648,"data_stored":467608035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":247279616,"data_stored":247200130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":391368704,"data_stored":391365091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:32:20.519 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:32:20.520 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'8270", 'reported_seq': 8346, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:16.911709+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:32:16.911709+0000', 'last_peered': '2024-09-10T20:32:16.911709+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:32:16.911709+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 8270, 'log_dups_size': 0, 'ondisk_log_size': 8270, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33873920, 'num_objects': 8270, 'num_object_clones': 0, 'num_object_copies': 16540, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3607, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8270, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8270, 'num_write_kb': 33080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4663}, {'shards': '0,2', 'objects': 3607}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:20.520 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:32:20.520 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'8200", 'reported_seq': 8332, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:16.879273+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:32:16.879273+0000', 'last_peered': '2024-09-10T20:32:16.879273+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:32:16.879273+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 8200, 'log_dups_size': 0, 'ondisk_log_size': 8200, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33566720, 'num_objects': 8195, 'num_object_clones': 0, 'num_object_copies': 16390, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3616, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8195, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5064, 'num_write_kb': 20256, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4581}, {'shards': '0,2', 'objects': 3614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:20.520 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:32:20.520 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'8235", 'reported_seq': 8369, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:16.909622+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:32:16.909622+0000', 'last_peered': '2024-09-10T20:32:16.909622+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:32:16.909622+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 8235, 'log_dups_size': 0, 'ondisk_log_size': 8235, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33705984, 'num_objects': 8229, 'num_object_clones': 0, 'num_object_copies': 16458, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3557, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8229, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4830, 'num_write_kb': 19320, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4674}, {'shards': '0,2', 'objects': 3555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:20.520 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:32:20.520 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'8209", 'reported_seq': 8341, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:18.403599+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:32:18.403599+0000', 'last_peered': '2024-09-10T20:32:18.403599+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:32:18.403599+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 8209, 'log_dups_size': 0, 'ondisk_log_size': 8209, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33595392, 'num_objects': 8202, 'num_object_clones': 0, 'num_object_copies': 16404, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3528, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8202, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6202, 'num_write_kb': 24808, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 4676}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:32:20.520 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:32:20.520 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'8369", 'reported_seq': 8473, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:16.829642+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:32:16.829642+0000', 'last_peered': '2024-09-10T20:32:16.829642+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:32:16.829642+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 8369, 'log_dups_size': 0, 'ondisk_log_size': 8369, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34247622, 'num_objects': 8363, 'num_object_clones': 0, 'num_object_copies': 16726, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3617, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8363, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8367, 'num_write_kb': 33448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4748}, {'shards': '0,2', 'objects': 3615}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:20.521 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:32:20.521 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'8284", 'reported_seq': 8410, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:18.420541+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:32:18.420541+0000', 'last_peered': '2024-09-10T20:32:18.420541+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:32:18.420541+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 8284, 'log_dups_size': 0, 'ondisk_log_size': 8284, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33910784, 'num_objects': 8279, 'num_object_clones': 0, 'num_object_copies': 16558, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3635, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8279, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6112, 'num_write_kb': 24448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3633}, {'shards': '2', 'objects': 4646}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:32:20.521 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:32:20.521 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'8378", 'reported_seq': 8506, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:18.331446+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:32:18.331446+0000', 'last_peered': '2024-09-10T20:32:18.331446+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:32:18.331446+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 8378, 'log_dups_size': 0, 'ondisk_log_size': 8378, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34291712, 'num_objects': 8372, 'num_object_clones': 0, 'num_object_copies': 16744, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8372, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6164, 'num_write_kb': 24656, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3610}, {'shards': '2', 'objects': 4762}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:32:20.521 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:32:20.521 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'8336", 'reported_seq': 8467, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:16.935946+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:32:16.935946+0000', 'last_peered': '2024-09-10T20:32:16.935946+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:32:16.935946+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 8336, 'log_dups_size': 0, 'ondisk_log_size': 8336, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34136064, 'num_objects': 8334, 'num_object_clones': 0, 'num_object_copies': 16668, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3649, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8334, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5059, 'num_write_kb': 20236, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4686}, {'shards': '0,2', 'objects': 3648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:20.521 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:32:20.522 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'8314", 'reported_seq': 8448, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:16.887241+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:32:16.887241+0000', 'last_peered': '2024-09-10T20:32:16.887241+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:32:16.887241+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 8314, 'log_dups_size': 0, 'ondisk_log_size': 8314, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34021376, 'num_objects': 8306, 'num_object_clones': 0, 'num_object_copies': 16612, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3535, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8306, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5008, 'num_write_kb': 20032, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4774}, {'shards': '0,2', 'objects': 3532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:20.522 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:32:20.522 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'8455", 'reported_seq': 8582, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:16.842900+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:32:16.842900+0000', 'last_peered': '2024-09-10T20:32:16.842900+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:32:16.842900+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 8455, 'log_dups_size': 0, 'ondisk_log_size': 8455, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34615296, 'num_objects': 8451, 'num_object_clones': 0, 'num_object_copies': 16902, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3715, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8451, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5282, 'num_write_kb': 21128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4738}, {'shards': '0,2', 'objects': 3713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:20.522 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:32:20.522 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'8227", 'reported_seq': 8402, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:32:16.954906+0000', 'last_change': '2024-09-10T20:32:16.657652+0000', 'last_active': '2024-09-10T20:32:16.954906+0000', 'last_peered': '2024-09-10T20:32:16.954906+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:32:16.954906+0000', 'last_undegraded': '2024-09-10T20:32:16.954906+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 8227, 'log_dups_size': 0, 'ondisk_log_size': 8227, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33649123, 'num_objects': 8216, 'num_object_clones': 0, 'num_object_copies': 16432, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3461, 'num_objects_unfound': 0, 'num_objects_dirty': 8216, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4834, 'num_write_kb': 19336, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 7, 'num_bytes_recovered': 28672, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4755}, {'shards': '0,2', 'objects': 3461}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:20.522 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:32:21.070 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 173 20 134655 134635 3.03929 1.75391 0.0316214 0.0256974 2024-09-10T20:32:21.221 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:21.222 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36071/266158 objects degraded (13.552%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":252,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":133079,"data_bytes":545072572,"bytes_used":1808277504,"bytes_avail":288102014976,"bytes_total":289910292480,"degraded_objects":36071,"degraded_total":266158,"degraded_ratio":0.1355247634863502,"misplaced_objects":3461,"misplaced_total":266158,"misplaced_ratio":0.013003554279788696,"recovering_objects_per_sec":45,"recovering_bytes_per_sec":186289,"recovering_keys_per_sec":0,"num_objects_recovered":364,"num_bytes_recovered":1490944,"num_keys_recovered":0,"write_bytes_sec":2489826,"read_op_per_sec":0,"write_op_per_sec":607},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:32:21.251 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:32:21.251 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:32:21.835 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:21.835 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:32:21.857 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":165,"stamp":"2024-09-10T20:32:20.878640+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":545072572,"num_objects":133079,"num_object_clones":0,"num_object_copies":266158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":3461,"num_objects_unfound":0,"num_objects_dirty":133079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97454,"num_write_kb":389790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3616,"num_bytes_recovered":14790675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133159,"ondisk_log_size":133159,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1765896,"kb_used_data":1080844,"kb_used_omap":12,"kb_used_meta":684979,"kb_avail":281349624,"statfs":{"total":289910292480,"available":288102014976,"internally_reserved":0,"allocated":1106784256,"data_stored":1105998739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":701418498},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,9,6,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":13516800,"num_objects":3300,"num_object_clones":0,"num_object_copies":6600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3471,"num_objects_misplaced":3104,"num_objects_unfound":0,"num_objects_dirty":3300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3300,"num_write_kb":13200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":364,"num_bytes_recovered":1490944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003276"},"pg_stats":[{"pgid":"2.f","version":"43'8270","reported_seq":8346,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:16.911709+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:32:16.911709+0000","last_peered":"2024-09-10T20:32:16.911709+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:32:16.911709+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":8270,"log_dups_size":0,"ondisk_log_size":8270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33873920,"num_objects":8270,"num_object_clones":0,"num_object_copies":16540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8270,"num_write_kb":33080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4663},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'8491","reported_seq":8601,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.424494+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:32:18.424494+0000","last_peered":"2024-09-10T20:32:18.424494+0000","last_clean":"2024-09-10T20:32:18.424494+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:32:18.424494+0000","last_undegraded":"2024-09-10T20:32:18.424494+0000","last_fullsized":"2024-09-10T20:32:18.424494+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":8491,"log_dups_size":0,"ondisk_log_size":8491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34754560,"num_objects":8485,"num_object_clones":0,"num_object_copies":16970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6677,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'8356","reported_seq":15703,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.312624+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:32:18.312624+0000","last_peered":"2024-09-10T20:32:18.312624+0000","last_clean":"2024-09-10T20:32:18.312624+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:32:18.312624+0000","last_undegraded":"2024-09-10T20:32:18.312624+0000","last_fullsized":"2024-09-10T20:32:18.312624+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":8356,"log_dups_size":0,"ondisk_log_size":8356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34209792,"num_objects":8352,"num_object_clones":0,"num_object_copies":16704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6484,"num_write_kb":25936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'8230","reported_seq":8310,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.378738+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:32:18.378738+0000","last_peered":"2024-09-10T20:32:18.378738+0000","last_clean":"2024-09-10T20:32:18.378738+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:32:18.378738+0000","last_undegraded":"2024-09-10T20:32:18.378738+0000","last_fullsized":"2024-09-10T20:32:18.378738+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":8230,"log_dups_size":0,"ondisk_log_size":8230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33710080,"num_objects":8230,"num_object_clones":0,"num_object_copies":16460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8230,"num_write_kb":32920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8463","reported_seq":8585,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:32:16.891777+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:32:16.891777+0000","last_peered":"2024-09-10T20:32:16.891777+0000","last_clean":"2024-09-10T20:32:16.891777+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:32:16.891777+0000","last_undegraded":"2024-09-10T20:32:16.891777+0000","last_fullsized":"2024-09-10T20:32:16.891777+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":8463,"log_dups_size":0,"ondisk_log_size":8463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34635776,"num_objects":8456,"num_object_clones":0,"num_object_copies":16912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5459,"num_write_kb":21836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8200","reported_seq":8332,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:16.879273+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:32:16.879273+0000","last_peered":"2024-09-10T20:32:16.879273+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:32:16.879273+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":8200,"log_dups_size":0,"ondisk_log_size":8200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33566720,"num_objects":8195,"num_object_clones":0,"num_object_copies":16390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5064,"num_write_kb":20256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4581},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8235","reported_seq":8369,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:16.909622+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:32:16.909622+0000","last_peered":"2024-09-10T20:32:16.909622+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:32:16.909622+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":8235,"log_dups_size":0,"ondisk_log_size":8235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33705984,"num_objects":8229,"num_object_clones":0,"num_object_copies":16458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4830,"num_write_kb":19320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4674},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8339","reported_seq":8454,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:32:16.847025+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:32:16.847025+0000","last_peered":"2024-09-10T20:32:16.847025+0000","last_clean":"2024-09-10T20:32:16.847025+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:32:16.847025+0000","last_undegraded":"2024-09-10T20:32:16.847025+0000","last_fullsized":"2024-09-10T20:32:16.847025+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":8339,"log_dups_size":0,"ondisk_log_size":8339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34148352,"num_objects":8337,"num_object_clones":0,"num_object_copies":16674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5410,"num_write_kb":21640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8209","reported_seq":8341,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:18.403599+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:32:18.403599+0000","last_peered":"2024-09-10T20:32:18.403599+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:32:18.403599+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":8209,"log_dups_size":0,"ondisk_log_size":8209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33595392,"num_objects":8202,"num_object_clones":0,"num_object_copies":16404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6202,"num_write_kb":24808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":4676}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190232+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190232+0000","last_peered":"2024-09-10T20:32:18.190232+0000","last_clean":"2024-09-10T20:32:18.190232+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190232+0000","last_undegraded":"2024-09-10T20:32:18.190232+0000","last_fullsized":"2024-09-10T20:32:18.190232+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8369","reported_seq":8473,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:16.829642+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:32:16.829642+0000","last_peered":"2024-09-10T20:32:16.829642+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:32:16.829642+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":8369,"log_dups_size":0,"ondisk_log_size":8369,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34247622,"num_objects":8363,"num_object_clones":0,"num_object_copies":16726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8367,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4748},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T20:30:05.419964+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:30:05.419964+0000","last_peered":"2024-09-10T20:30:05.419964+0000","last_clean":"2024-09-10T20:30:05.419964+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:30:05.419964+0000","last_undegraded":"2024-09-10T20:30:05.419964+0000","last_fullsized":"2024-09-10T20:30:05.419964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8284","reported_seq":8410,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:18.420541+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:32:18.420541+0000","last_peered":"2024-09-10T20:32:18.420541+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:32:18.420541+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":8284,"log_dups_size":0,"ondisk_log_size":8284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33910784,"num_objects":8279,"num_object_clones":0,"num_object_copies":16558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6112,"num_write_kb":24448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":4646}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8378","reported_seq":8506,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:18.331446+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:32:18.331446+0000","last_peered":"2024-09-10T20:32:18.331446+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:32:18.331446+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":8378,"log_dups_size":0,"ondisk_log_size":8378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34291712,"num_objects":8372,"num_object_clones":0,"num_object_copies":16744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6164,"num_write_kb":24656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":4762}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8336","reported_seq":8467,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:16.935946+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:32:16.935946+0000","last_peered":"2024-09-10T20:32:16.935946+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:32:16.935946+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":8336,"log_dups_size":0,"ondisk_log_size":8336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34136064,"num_objects":8334,"num_object_clones":0,"num_object_copies":16668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5059,"num_write_kb":20236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4686},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8314","reported_seq":8448,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:16.887241+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:32:16.887241+0000","last_peered":"2024-09-10T20:32:16.887241+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:32:16.887241+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":8314,"log_dups_size":0,"ondisk_log_size":8314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34021376,"num_objects":8306,"num_object_clones":0,"num_object_copies":16612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5008,"num_write_kb":20032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4774},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193200+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193200+0000","last_peered":"2024-09-10T20:32:18.193200+0000","last_clean":"2024-09-10T20:32:18.193200+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193200+0000","last_undegraded":"2024-09-10T20:32:18.193200+0000","last_fullsized":"2024-09-10T20:32:18.193200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8455","reported_seq":8582,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:16.842900+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:32:16.842900+0000","last_peered":"2024-09-10T20:32:16.842900+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:32:16.842900+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":8455,"log_dups_size":0,"ondisk_log_size":8455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34615296,"num_objects":8451,"num_object_clones":0,"num_object_copies":16902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5282,"num_write_kb":21128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4738},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8227","reported_seq":8402,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:32:16.954906+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:32:16.954906+0000","last_peered":"2024-09-10T20:32:16.954906+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:32:16.954906+0000","last_undegraded":"2024-09-10T20:32:16.954906+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":8227,"log_dups_size":0,"ondisk_log_size":8227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33649123,"num_objects":8216,"num_object_clones":0,"num_object_copies":16432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3461,"num_objects_unfound":0,"num_objects_dirty":8216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4834,"num_write_kb":19336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":28672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4755},{"shards":"0,2","objects":3461}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":545072553,"num_objects":133077,"num_object_clones":0,"num_object_copies":266154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":3461,"num_objects_unfound":0,"num_objects_dirty":133077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97452,"num_write_kb":389788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14790656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1106259968,"data_stored":1106173256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133156,"ondisk_log_size":133156,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182445,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":749464,"kb_used_data":456840,"kb_used_omap":4,"kb_used_meta":292603,"kb_avail":93622376,"statfs":{"total":96636764160,"available":95869313024,"internally_reserved":0,"allocated":467804160,"data_stored":467571027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":299626154},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,3,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986217,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":580480,"kb_used_data":382188,"kb_used_omap":4,"kb_used_meta":198267,"kb_avail":93791360,"statfs":{"total":96636764160,"available":96042352640,"internally_reserved":0,"allocated":391360512,"data_stored":391126689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":203026090},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659164,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":435952,"kb_used_data":241816,"kb_used_omap":4,"kb_used_meta":194107,"kb_avail":93935888,"statfs":{"total":96636764160,"available":96190349312,"internally_reserved":0,"allocated":247619584,"data_stored":247301023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":198766254},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,6,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":467611648,"data_stored":467608035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":247279616,"data_stored":247200130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":391368704,"data_stored":391365091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:32:22.070 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 174 20 135111 135091 3.03206 1.78125 0.0369081 0.0257589 2024-09-10T20:32:23.070 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 175 20 135609 135589 3.02585 1.94531 0.0221369 0.0258109 2024-09-10T20:32:24.072 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 176 20 136134 136114 3.02031 2.05078 0.0407319 0.0258605 2024-09-10T20:32:24.860 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:32:25.070 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 177 20 136627 136607 3.01412 1.92578 0.0711143 0.0259126 2024-09-10T20:32:25.405 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:25.408 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:32:25.431 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":167,"stamp":"2024-09-10T20:32:24.879878+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":554341820,"num_objects":135342,"num_object_clones":0,"num_object_copies":270684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":3319,"num_objects_unfound":0,"num_objects_dirty":135342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99717,"num_write_kb":398842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3758,"num_bytes_recovered":15372307,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135422,"ondisk_log_size":135422,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1819304,"kb_used_data":1098168,"kb_used_omap":12,"kb_used_meta":721075,"kb_avail":281296216,"statfs":{"total":289910292480,"available":288047325184,"internally_reserved":0,"allocated":1124524032,"data_stored":1123724172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":738380802},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,6,7,8,14,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18927616,"num_objects":4621,"num_object_clones":0,"num_object_copies":9242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3471,"num_objects_misplaced":3159,"num_objects_unfound":0,"num_objects_dirty":4621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4621,"num_write_kb":18484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":1265664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002957"},"pg_stats":[{"pgid":"2.f","version":"45'8415","reported_seq":8494,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:22.000999+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:32:22.000999+0000","last_peered":"2024-09-10T20:32:22.000999+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:32:22.000999+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":8415,"log_dups_size":0,"ondisk_log_size":8415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34467840,"num_objects":8415,"num_object_clones":0,"num_object_copies":16830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8415,"num_write_kb":33660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4808},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'8636","reported_seq":8746,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:23.391800+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:32:23.391800+0000","last_peered":"2024-09-10T20:32:23.391800+0000","last_clean":"2024-09-10T20:32:23.391800+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:32:23.391800+0000","last_undegraded":"2024-09-10T20:32:23.391800+0000","last_fullsized":"2024-09-10T20:32:23.391800+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":8636,"log_dups_size":0,"ondisk_log_size":8636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35348480,"num_objects":8630,"num_object_clones":0,"num_object_copies":17260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6822,"num_write_kb":27288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'8507","reported_seq":15854,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:23.175400+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:32:23.175400+0000","last_peered":"2024-09-10T20:32:23.175400+0000","last_clean":"2024-09-10T20:32:23.175400+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:32:23.175400+0000","last_undegraded":"2024-09-10T20:32:23.175400+0000","last_fullsized":"2024-09-10T20:32:23.175400+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":8507,"log_dups_size":0,"ondisk_log_size":8507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34828288,"num_objects":8503,"num_object_clones":0,"num_object_copies":17006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6635,"num_write_kb":26540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'8390","reported_seq":8469,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:23.436953+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:32:23.436953+0000","last_peered":"2024-09-10T20:32:23.436953+0000","last_clean":"2024-09-10T20:32:23.436953+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:32:23.436953+0000","last_undegraded":"2024-09-10T20:32:23.436953+0000","last_fullsized":"2024-09-10T20:32:23.436953+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":8390,"log_dups_size":0,"ondisk_log_size":8390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34365440,"num_objects":8390,"num_object_clones":0,"num_object_copies":16780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8390,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'8614","reported_seq":8740,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:21.982671+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:32:21.982671+0000","last_peered":"2024-09-10T20:32:21.982671+0000","last_clean":"2024-09-10T20:32:21.982671+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:32:21.982671+0000","last_undegraded":"2024-09-10T20:32:21.982671+0000","last_fullsized":"2024-09-10T20:32:21.982671+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":8614,"log_dups_size":0,"ondisk_log_size":8614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35254272,"num_objects":8607,"num_object_clones":0,"num_object_copies":17214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5610,"num_write_kb":22440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'8330","reported_seq":8466,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:21.957946+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:32:21.957946+0000","last_peered":"2024-09-10T20:32:21.957946+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:32:21.957946+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":8330,"log_dups_size":0,"ondisk_log_size":8330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34099200,"num_objects":8325,"num_object_clones":0,"num_object_copies":16650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5194,"num_write_kb":20776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4711},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'8380","reported_seq":8518,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:21.949033+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:32:21.949033+0000","last_peered":"2024-09-10T20:32:21.949033+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:32:21.949033+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":8380,"log_dups_size":0,"ondisk_log_size":8380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34299904,"num_objects":8374,"num_object_clones":0,"num_object_copies":16748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8374,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4975,"num_write_kb":19900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4819},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'8477","reported_seq":8598,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:21.979070+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:32:21.979070+0000","last_peered":"2024-09-10T20:32:21.979070+0000","last_clean":"2024-09-10T20:32:21.979070+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:32:21.979070+0000","last_undegraded":"2024-09-10T20:32:21.979070+0000","last_fullsized":"2024-09-10T20:32:21.979070+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":8477,"log_dups_size":0,"ondisk_log_size":8477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34713600,"num_objects":8475,"num_object_clones":0,"num_object_copies":16950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5548,"num_write_kb":22192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8356","reported_seq":8488,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:23.435777+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:32:23.435777+0000","last_peered":"2024-09-10T20:32:23.435777+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:32:23.435777+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":8356,"log_dups_size":0,"ondisk_log_size":8356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34197504,"num_objects":8349,"num_object_clones":0,"num_object_copies":16698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6349,"num_write_kb":25396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":4823}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190232+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190232+0000","last_peered":"2024-09-10T20:32:18.190232+0000","last_clean":"2024-09-10T20:32:18.190232+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190232+0000","last_undegraded":"2024-09-10T20:32:18.190232+0000","last_fullsized":"2024-09-10T20:32:18.190232+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8499","reported_seq":8607,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:21.930465+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:32:21.930465+0000","last_peered":"2024-09-10T20:32:21.930465+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:32:21.930465+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":8499,"log_dups_size":0,"ondisk_log_size":8499,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34780102,"num_objects":8493,"num_object_clones":0,"num_object_copies":16986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8497,"num_write_kb":33968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4878},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182737+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182737+0000","last_peered":"2024-09-10T20:32:18.182737+0000","last_clean":"2024-09-10T20:32:18.182737+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182737+0000","last_undegraded":"2024-09-10T20:32:18.182737+0000","last_fullsized":"2024-09-10T20:32:18.182737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8449","reported_seq":8577,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:23.406210+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:32:23.406210+0000","last_peered":"2024-09-10T20:32:23.406210+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:32:23.406210+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":8449,"log_dups_size":0,"ondisk_log_size":8449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34586624,"num_objects":8444,"num_object_clones":0,"num_object_copies":16888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6277,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":4811}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8509","reported_seq":8637,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:23.296726+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:32:23.296726+0000","last_peered":"2024-09-10T20:32:23.296726+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:32:23.296726+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":8509,"log_dups_size":0,"ondisk_log_size":8509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34828288,"num_objects":8503,"num_object_clones":0,"num_object_copies":17006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6295,"num_write_kb":25180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":4893}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'8466","reported_seq":8601,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:21.895851+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:32:21.895851+0000","last_peered":"2024-09-10T20:32:21.895851+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:32:21.895851+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":8466,"log_dups_size":0,"ondisk_log_size":8466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34668544,"num_objects":8464,"num_object_clones":0,"num_object_copies":16928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5189,"num_write_kb":20756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4816},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'8438","reported_seq":8576,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:22.011917+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:32:22.011917+0000","last_peered":"2024-09-10T20:32:22.011917+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:32:22.011917+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":8438,"log_dups_size":0,"ondisk_log_size":8438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34529280,"num_objects":8430,"num_object_clones":0,"num_object_copies":16860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5132,"num_write_kb":20528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4898},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193200+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193200+0000","last_peered":"2024-09-10T20:32:18.193200+0000","last_clean":"2024-09-10T20:32:18.193200+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193200+0000","last_undegraded":"2024-09-10T20:32:18.193200+0000","last_fullsized":"2024-09-10T20:32:18.193200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'8607","reported_seq":8738,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:21.993436+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:32:21.993436+0000","last_peered":"2024-09-10T20:32:21.993436+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:32:21.993436+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":8607,"log_dups_size":0,"ondisk_log_size":8607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35237888,"num_objects":8603,"num_object_clones":0,"num_object_copies":17206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5434,"num_write_kb":21736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4890},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'8346","reported_seq":8810,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:32:22.015463+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:32:22.015463+0000","last_peered":"2024-09-10T20:32:22.015463+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:32:22.015463+0000","last_undegraded":"2024-09-10T20:32:22.015463+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":8346,"log_dups_size":0,"ondisk_log_size":8346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34136547,"num_objects":8335,"num_object_clones":0,"num_object_copies":16670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3319,"num_objects_unfound":0,"num_objects_dirty":8335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4953,"num_write_kb":19812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":610304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5016},{"shards":"0,2","objects":3319}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":554341801,"num_objects":135340,"num_object_clones":0,"num_object_copies":270680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":3319,"num_objects_unfound":0,"num_objects_dirty":135340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99715,"num_write_kb":398840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3756,"num_bytes_recovered":15372288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1125711872,"data_stored":1125625160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135419,"ondisk_log_size":135419,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182446,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":747820,"kb_used_data":463516,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93624020,"statfs":{"total":96636764160,"available":95870996480,"internally_reserved":0,"allocated":474640384,"data_stored":474397696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":291106474},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,2,2,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986218,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":625420,"kb_used_data":386876,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93746420,"statfs":{"total":96636764160,"available":95996334080,"internally_reserved":0,"allocated":396161024,"data_stored":395922413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":244248234},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659165,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":446064,"kb_used_data":247776,"kb_used_omap":4,"kb_used_meta":198267,"kb_avail":93925776,"statfs":{"total":96636764160,"available":96179994624,"internally_reserved":0,"allocated":253722624,"data_stored":253404063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":203026094},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,4,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":474480640,"data_stored":474477027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":254976000,"data_stored":254896514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":396255232,"data_stored":396251619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:32:25.433 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:32:26.006 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:26.007 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:32:26.033 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":167,"stamp":"2024-09-10T20:32:24.879878+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":554341820,"num_objects":135342,"num_object_clones":0,"num_object_copies":270684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":3319,"num_objects_unfound":0,"num_objects_dirty":135342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99717,"num_write_kb":398842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3758,"num_bytes_recovered":15372307,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135422,"ondisk_log_size":135422,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1819304,"kb_used_data":1098168,"kb_used_omap":12,"kb_used_meta":721075,"kb_avail":281296216,"statfs":{"total":289910292480,"available":288047325184,"internally_reserved":0,"allocated":1124524032,"data_stored":1123724172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":738380802},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,6,7,8,14,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18927616,"num_objects":4621,"num_object_clones":0,"num_object_copies":9242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3471,"num_objects_misplaced":3159,"num_objects_unfound":0,"num_objects_dirty":4621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4621,"num_write_kb":18484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":1265664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002957"},"pg_stats":[{"pgid":"2.f","version":"45'8415","reported_seq":8494,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:22.000999+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:32:22.000999+0000","last_peered":"2024-09-10T20:32:22.000999+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:32:22.000999+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":8415,"log_dups_size":0,"ondisk_log_size":8415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34467840,"num_objects":8415,"num_object_clones":0,"num_object_copies":16830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8415,"num_write_kb":33660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4808},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'8636","reported_seq":8746,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:23.391800+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:32:23.391800+0000","last_peered":"2024-09-10T20:32:23.391800+0000","last_clean":"2024-09-10T20:32:23.391800+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:32:23.391800+0000","last_undegraded":"2024-09-10T20:32:23.391800+0000","last_fullsized":"2024-09-10T20:32:23.391800+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":8636,"log_dups_size":0,"ondisk_log_size":8636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35348480,"num_objects":8630,"num_object_clones":0,"num_object_copies":17260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6822,"num_write_kb":27288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'8507","reported_seq":15854,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:23.175400+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:32:23.175400+0000","last_peered":"2024-09-10T20:32:23.175400+0000","last_clean":"2024-09-10T20:32:23.175400+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:32:23.175400+0000","last_undegraded":"2024-09-10T20:32:23.175400+0000","last_fullsized":"2024-09-10T20:32:23.175400+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":8507,"log_dups_size":0,"ondisk_log_size":8507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34828288,"num_objects":8503,"num_object_clones":0,"num_object_copies":17006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6635,"num_write_kb":26540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'8390","reported_seq":8469,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:23.436953+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:32:23.436953+0000","last_peered":"2024-09-10T20:32:23.436953+0000","last_clean":"2024-09-10T20:32:23.436953+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:32:23.436953+0000","last_undegraded":"2024-09-10T20:32:23.436953+0000","last_fullsized":"2024-09-10T20:32:23.436953+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":8390,"log_dups_size":0,"ondisk_log_size":8390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34365440,"num_objects":8390,"num_object_clones":0,"num_object_copies":16780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8390,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'8614","reported_seq":8740,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:21.982671+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:32:21.982671+0000","last_peered":"2024-09-10T20:32:21.982671+0000","last_clean":"2024-09-10T20:32:21.982671+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:32:21.982671+0000","last_undegraded":"2024-09-10T20:32:21.982671+0000","last_fullsized":"2024-09-10T20:32:21.982671+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":8614,"log_dups_size":0,"ondisk_log_size":8614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35254272,"num_objects":8607,"num_object_clones":0,"num_object_copies":17214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5610,"num_write_kb":22440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'8330","reported_seq":8466,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:21.957946+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:32:21.957946+0000","last_peered":"2024-09-10T20:32:21.957946+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:32:21.957946+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":8330,"log_dups_size":0,"ondisk_log_size":8330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34099200,"num_objects":8325,"num_object_clones":0,"num_object_copies":16650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5194,"num_write_kb":20776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4711},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'8380","reported_seq":8518,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:21.949033+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:32:21.949033+0000","last_peered":"2024-09-10T20:32:21.949033+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:32:21.949033+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":8380,"log_dups_size":0,"ondisk_log_size":8380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34299904,"num_objects":8374,"num_object_clones":0,"num_object_copies":16748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8374,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4975,"num_write_kb":19900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4819},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'8477","reported_seq":8598,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:21.979070+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:32:21.979070+0000","last_peered":"2024-09-10T20:32:21.979070+0000","last_clean":"2024-09-10T20:32:21.979070+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:32:21.979070+0000","last_undegraded":"2024-09-10T20:32:21.979070+0000","last_fullsized":"2024-09-10T20:32:21.979070+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":8477,"log_dups_size":0,"ondisk_log_size":8477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34713600,"num_objects":8475,"num_object_clones":0,"num_object_copies":16950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5548,"num_write_kb":22192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8356","reported_seq":8488,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:23.435777+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:32:23.435777+0000","last_peered":"2024-09-10T20:32:23.435777+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:32:23.435777+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":8356,"log_dups_size":0,"ondisk_log_size":8356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34197504,"num_objects":8349,"num_object_clones":0,"num_object_copies":16698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6349,"num_write_kb":25396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":4823}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190232+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190232+0000","last_peered":"2024-09-10T20:32:18.190232+0000","last_clean":"2024-09-10T20:32:18.190232+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190232+0000","last_undegraded":"2024-09-10T20:32:18.190232+0000","last_fullsized":"2024-09-10T20:32:18.190232+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8499","reported_seq":8607,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:21.930465+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:32:21.930465+0000","last_peered":"2024-09-10T20:32:21.930465+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:32:21.930465+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":8499,"log_dups_size":0,"ondisk_log_size":8499,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34780102,"num_objects":8493,"num_object_clones":0,"num_object_copies":16986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8497,"num_write_kb":33968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4878},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182737+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182737+0000","last_peered":"2024-09-10T20:32:18.182737+0000","last_clean":"2024-09-10T20:32:18.182737+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182737+0000","last_undegraded":"2024-09-10T20:32:18.182737+0000","last_fullsized":"2024-09-10T20:32:18.182737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8449","reported_seq":8577,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:23.406210+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:32:23.406210+0000","last_peered":"2024-09-10T20:32:23.406210+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:32:23.406210+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":8449,"log_dups_size":0,"ondisk_log_size":8449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34586624,"num_objects":8444,"num_object_clones":0,"num_object_copies":16888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6277,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":4811}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8509","reported_seq":8637,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:23.296726+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:32:23.296726+0000","last_peered":"2024-09-10T20:32:23.296726+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:32:23.296726+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":8509,"log_dups_size":0,"ondisk_log_size":8509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34828288,"num_objects":8503,"num_object_clones":0,"num_object_copies":17006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6295,"num_write_kb":25180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":4893}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'8466","reported_seq":8601,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:21.895851+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:32:21.895851+0000","last_peered":"2024-09-10T20:32:21.895851+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:32:21.895851+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":8466,"log_dups_size":0,"ondisk_log_size":8466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34668544,"num_objects":8464,"num_object_clones":0,"num_object_copies":16928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5189,"num_write_kb":20756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4816},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'8438","reported_seq":8576,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:22.011917+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:32:22.011917+0000","last_peered":"2024-09-10T20:32:22.011917+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:32:22.011917+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":8438,"log_dups_size":0,"ondisk_log_size":8438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34529280,"num_objects":8430,"num_object_clones":0,"num_object_copies":16860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5132,"num_write_kb":20528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4898},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193200+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193200+0000","last_peered":"2024-09-10T20:32:18.193200+0000","last_clean":"2024-09-10T20:32:18.193200+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193200+0000","last_undegraded":"2024-09-10T20:32:18.193200+0000","last_fullsized":"2024-09-10T20:32:18.193200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'8607","reported_seq":8738,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:21.993436+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:32:21.993436+0000","last_peered":"2024-09-10T20:32:21.993436+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:32:21.993436+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":8607,"log_dups_size":0,"ondisk_log_size":8607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35237888,"num_objects":8603,"num_object_clones":0,"num_object_copies":17206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5434,"num_write_kb":21736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4890},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'8346","reported_seq":8810,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:32:22.015463+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:32:22.015463+0000","last_peered":"2024-09-10T20:32:22.015463+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:32:22.015463+0000","last_undegraded":"2024-09-10T20:32:22.015463+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":8346,"log_dups_size":0,"ondisk_log_size":8346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34136547,"num_objects":8335,"num_object_clones":0,"num_object_copies":16670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3319,"num_objects_unfound":0,"num_objects_dirty":8335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4953,"num_write_kb":19812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":610304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5016},{"shards":"0,2","objects":3319}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":554341801,"num_objects":135340,"num_object_clones":0,"num_object_copies":270680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":3319,"num_objects_unfound":0,"num_objects_dirty":135340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99715,"num_write_kb":398840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3756,"num_bytes_recovered":15372288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1125711872,"data_stored":1125625160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135419,"ondisk_log_size":135419,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182446,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":747820,"kb_used_data":463516,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93624020,"statfs":{"total":96636764160,"available":95870996480,"internally_reserved":0,"allocated":474640384,"data_stored":474397696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":291106474},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,2,2,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986218,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":625420,"kb_used_data":386876,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93746420,"statfs":{"total":96636764160,"available":95996334080,"internally_reserved":0,"allocated":396161024,"data_stored":395922413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":244248234},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659165,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":446064,"kb_used_data":247776,"kb_used_omap":4,"kb_used_meta":198267,"kb_avail":93925776,"statfs":{"total":96636764160,"available":96179994624,"internally_reserved":0,"allocated":253722624,"data_stored":253404063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":203026094},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,4,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":474480640,"data_stored":474477027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":254976000,"data_stored":254896514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":396255232,"data_stored":396251619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:32:26.036 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:32:26.036 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'8415", 'reported_seq': 8494, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:22.000999+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:32:22.000999+0000', 'last_peered': '2024-09-10T20:32:22.000999+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:32:22.000999+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 8415, 'log_dups_size': 0, 'ondisk_log_size': 8415, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34467840, 'num_objects': 8415, 'num_object_clones': 0, 'num_object_copies': 16830, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3607, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8415, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8415, 'num_write_kb': 33660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4808}, {'shards': '0,2', 'objects': 3607}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:26.036 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:32:26.036 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'8330", 'reported_seq': 8466, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:21.957946+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:32:21.957946+0000', 'last_peered': '2024-09-10T20:32:21.957946+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:32:21.957946+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 8330, 'log_dups_size': 0, 'ondisk_log_size': 8330, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34099200, 'num_objects': 8325, 'num_object_clones': 0, 'num_object_copies': 16650, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3616, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8325, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5194, 'num_write_kb': 20776, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4711}, {'shards': '0,2', 'objects': 3614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:26.036 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:32:26.036 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'8380", 'reported_seq': 8518, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:21.949033+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:32:21.949033+0000', 'last_peered': '2024-09-10T20:32:21.949033+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:32:21.949033+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 8380, 'log_dups_size': 0, 'ondisk_log_size': 8380, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34299904, 'num_objects': 8374, 'num_object_clones': 0, 'num_object_copies': 16748, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3557, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8374, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4975, 'num_write_kb': 19900, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4819}, {'shards': '0,2', 'objects': 3555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:26.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:32:26.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'8356", 'reported_seq': 8488, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:23.435777+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:32:23.435777+0000', 'last_peered': '2024-09-10T20:32:23.435777+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:32:23.435777+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 8356, 'log_dups_size': 0, 'ondisk_log_size': 8356, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34197504, 'num_objects': 8349, 'num_object_clones': 0, 'num_object_copies': 16698, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3528, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8349, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6349, 'num_write_kb': 25396, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 4823}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:32:26.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:32:26.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'8499", 'reported_seq': 8607, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:21.930465+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:32:21.930465+0000', 'last_peered': '2024-09-10T20:32:21.930465+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:32:21.930465+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 8499, 'log_dups_size': 0, 'ondisk_log_size': 8499, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34780102, 'num_objects': 8493, 'num_object_clones': 0, 'num_object_copies': 16986, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3617, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8493, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8497, 'num_write_kb': 33968, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4878}, {'shards': '0,2', 'objects': 3615}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:26.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:32:26.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'8449", 'reported_seq': 8577, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:23.406210+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:32:23.406210+0000', 'last_peered': '2024-09-10T20:32:23.406210+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:32:23.406210+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 8449, 'log_dups_size': 0, 'ondisk_log_size': 8449, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34586624, 'num_objects': 8444, 'num_object_clones': 0, 'num_object_copies': 16888, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3635, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8444, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6277, 'num_write_kb': 25108, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3633}, {'shards': '2', 'objects': 4811}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:32:26.038 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:32:26.038 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'8509", 'reported_seq': 8637, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:23.296726+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:32:23.296726+0000', 'last_peered': '2024-09-10T20:32:23.296726+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:32:23.296726+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 8509, 'log_dups_size': 0, 'ondisk_log_size': 8509, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34828288, 'num_objects': 8503, 'num_object_clones': 0, 'num_object_copies': 17006, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8503, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6295, 'num_write_kb': 25180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3610}, {'shards': '2', 'objects': 4893}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:32:26.038 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:32:26.038 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'8466", 'reported_seq': 8601, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:21.895851+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:32:21.895851+0000', 'last_peered': '2024-09-10T20:32:21.895851+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:32:21.895851+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 8466, 'log_dups_size': 0, 'ondisk_log_size': 8466, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34668544, 'num_objects': 8464, 'num_object_clones': 0, 'num_object_copies': 16928, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3649, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8464, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5189, 'num_write_kb': 20756, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4816}, {'shards': '0,2', 'objects': 3648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:26.038 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:32:26.038 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'8438", 'reported_seq': 8576, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:22.011917+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:32:22.011917+0000', 'last_peered': '2024-09-10T20:32:22.011917+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:32:22.011917+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 8438, 'log_dups_size': 0, 'ondisk_log_size': 8438, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34529280, 'num_objects': 8430, 'num_object_clones': 0, 'num_object_copies': 16860, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3535, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8430, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5132, 'num_write_kb': 20528, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4898}, {'shards': '0,2', 'objects': 3532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:26.038 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:32:26.038 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'8607", 'reported_seq': 8738, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:21.993436+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:32:21.993436+0000', 'last_peered': '2024-09-10T20:32:21.993436+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:32:21.993436+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 8607, 'log_dups_size': 0, 'ondisk_log_size': 8607, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35237888, 'num_objects': 8603, 'num_object_clones': 0, 'num_object_copies': 17206, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3715, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8603, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5434, 'num_write_kb': 21736, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4890}, {'shards': '0,2', 'objects': 3713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:26.039 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:32:26.039 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'8346", 'reported_seq': 8810, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:32:22.015463+0000', 'last_change': '2024-09-10T20:32:16.657652+0000', 'last_active': '2024-09-10T20:32:22.015463+0000', 'last_peered': '2024-09-10T20:32:22.015463+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:32:22.015463+0000', 'last_undegraded': '2024-09-10T20:32:22.015463+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 8346, 'log_dups_size': 0, 'ondisk_log_size': 8346, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34136547, 'num_objects': 8335, 'num_object_clones': 0, 'num_object_copies': 16670, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3319, 'num_objects_unfound': 0, 'num_objects_dirty': 8335, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4953, 'num_write_kb': 19812, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 149, 'num_bytes_recovered': 610304, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5016}, {'shards': '0,2', 'objects': 3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:26.039 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:32:26.070 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 178 20 137070 137050 3.00691 1.73047 0.0518254 0.0259743 2024-09-10T20:32:26.733 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:26.733 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36071/270684 objects degraded (13.326%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":258,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":135342,"data_bytes":554341820,"bytes_used":1862967296,"bytes_avail":288047325184,"bytes_total":289910292480,"degraded_objects":36071,"degraded_total":270684,"degraded_ratio":0.13325870757045116,"misplaced_objects":3319,"misplaced_total":270684,"misplaced_ratio":0.012261530049799766,"recovering_objects_per_sec":38,"recovering_bytes_per_sec":158149,"recovering_keys_per_sec":0,"num_objects_recovered":309,"num_bytes_recovered":1265664,"num_keys_recovered":0,"write_bytes_sec":2365077,"read_op_per_sec":0,"write_op_per_sec":577},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:32:26.761 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:32:26.761 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:32:27.070 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 179 20 137524 137504 3.00002 1.77344 0.0703323 0.0260337 2024-09-10T20:32:27.323 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:27.323 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:32:27.354 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":168,"stamp":"2024-09-10T20:32:26.880653+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":554341820,"num_objects":135342,"num_object_clones":0,"num_object_copies":270684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":3319,"num_objects_unfound":0,"num_objects_dirty":135342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99717,"num_write_kb":398842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3758,"num_bytes_recovered":15372307,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135422,"ondisk_log_size":135422,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1819304,"kb_used_data":1098168,"kb_used_omap":12,"kb_used_meta":721075,"kb_avail":281296216,"statfs":{"total":289910292480,"available":288047325184,"internally_reserved":0,"allocated":1124524032,"data_stored":1123724172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":738380802},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,6,7,8,14,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":12623872,"num_objects":3082,"num_object_clones":0,"num_object_copies":6164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-302,"num_objects_unfound":0,"num_objects_dirty":3082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3082,"num_write_kb":12328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":1236992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.696040"},"pg_stats":[{"pgid":"2.f","version":"45'8415","reported_seq":8494,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:22.000999+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:32:22.000999+0000","last_peered":"2024-09-10T20:32:22.000999+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:32:22.000999+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":8415,"log_dups_size":0,"ondisk_log_size":8415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34467840,"num_objects":8415,"num_object_clones":0,"num_object_copies":16830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8415,"num_write_kb":33660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4808},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'8636","reported_seq":8746,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:23.391800+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:32:23.391800+0000","last_peered":"2024-09-10T20:32:23.391800+0000","last_clean":"2024-09-10T20:32:23.391800+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:32:23.391800+0000","last_undegraded":"2024-09-10T20:32:23.391800+0000","last_fullsized":"2024-09-10T20:32:23.391800+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":8636,"log_dups_size":0,"ondisk_log_size":8636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35348480,"num_objects":8630,"num_object_clones":0,"num_object_copies":17260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6822,"num_write_kb":27288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'8507","reported_seq":15854,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:23.175400+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:32:23.175400+0000","last_peered":"2024-09-10T20:32:23.175400+0000","last_clean":"2024-09-10T20:32:23.175400+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:32:23.175400+0000","last_undegraded":"2024-09-10T20:32:23.175400+0000","last_fullsized":"2024-09-10T20:32:23.175400+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":8507,"log_dups_size":0,"ondisk_log_size":8507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34828288,"num_objects":8503,"num_object_clones":0,"num_object_copies":17006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6635,"num_write_kb":26540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'8390","reported_seq":8469,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:23.436953+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:32:23.436953+0000","last_peered":"2024-09-10T20:32:23.436953+0000","last_clean":"2024-09-10T20:32:23.436953+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:32:23.436953+0000","last_undegraded":"2024-09-10T20:32:23.436953+0000","last_fullsized":"2024-09-10T20:32:23.436953+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":8390,"log_dups_size":0,"ondisk_log_size":8390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34365440,"num_objects":8390,"num_object_clones":0,"num_object_copies":16780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8390,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'8614","reported_seq":8740,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:21.982671+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:32:21.982671+0000","last_peered":"2024-09-10T20:32:21.982671+0000","last_clean":"2024-09-10T20:32:21.982671+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:32:21.982671+0000","last_undegraded":"2024-09-10T20:32:21.982671+0000","last_fullsized":"2024-09-10T20:32:21.982671+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":8614,"log_dups_size":0,"ondisk_log_size":8614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35254272,"num_objects":8607,"num_object_clones":0,"num_object_copies":17214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5610,"num_write_kb":22440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'8330","reported_seq":8466,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:21.957946+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:32:21.957946+0000","last_peered":"2024-09-10T20:32:21.957946+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:32:21.957946+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":8330,"log_dups_size":0,"ondisk_log_size":8330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34099200,"num_objects":8325,"num_object_clones":0,"num_object_copies":16650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5194,"num_write_kb":20776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4711},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'8380","reported_seq":8518,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:21.949033+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:32:21.949033+0000","last_peered":"2024-09-10T20:32:21.949033+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:32:21.949033+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":8380,"log_dups_size":0,"ondisk_log_size":8380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34299904,"num_objects":8374,"num_object_clones":0,"num_object_copies":16748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8374,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4975,"num_write_kb":19900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4819},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'8477","reported_seq":8598,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:21.979070+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:32:21.979070+0000","last_peered":"2024-09-10T20:32:21.979070+0000","last_clean":"2024-09-10T20:32:21.979070+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:32:21.979070+0000","last_undegraded":"2024-09-10T20:32:21.979070+0000","last_fullsized":"2024-09-10T20:32:21.979070+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":8477,"log_dups_size":0,"ondisk_log_size":8477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34713600,"num_objects":8475,"num_object_clones":0,"num_object_copies":16950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5548,"num_write_kb":22192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8356","reported_seq":8488,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:23.435777+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:32:23.435777+0000","last_peered":"2024-09-10T20:32:23.435777+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:32:23.435777+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":8356,"log_dups_size":0,"ondisk_log_size":8356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34197504,"num_objects":8349,"num_object_clones":0,"num_object_copies":16698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6349,"num_write_kb":25396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":4823}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190232+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190232+0000","last_peered":"2024-09-10T20:32:18.190232+0000","last_clean":"2024-09-10T20:32:18.190232+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190232+0000","last_undegraded":"2024-09-10T20:32:18.190232+0000","last_fullsized":"2024-09-10T20:32:18.190232+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8499","reported_seq":8607,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:21.930465+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:32:21.930465+0000","last_peered":"2024-09-10T20:32:21.930465+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:32:21.930465+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":8499,"log_dups_size":0,"ondisk_log_size":8499,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34780102,"num_objects":8493,"num_object_clones":0,"num_object_copies":16986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8497,"num_write_kb":33968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4878},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182737+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182737+0000","last_peered":"2024-09-10T20:32:18.182737+0000","last_clean":"2024-09-10T20:32:18.182737+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182737+0000","last_undegraded":"2024-09-10T20:32:18.182737+0000","last_fullsized":"2024-09-10T20:32:18.182737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8449","reported_seq":8577,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:23.406210+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:32:23.406210+0000","last_peered":"2024-09-10T20:32:23.406210+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:32:23.406210+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":8449,"log_dups_size":0,"ondisk_log_size":8449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34586624,"num_objects":8444,"num_object_clones":0,"num_object_copies":16888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6277,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":4811}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8509","reported_seq":8637,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:23.296726+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:32:23.296726+0000","last_peered":"2024-09-10T20:32:23.296726+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:32:23.296726+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":8509,"log_dups_size":0,"ondisk_log_size":8509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34828288,"num_objects":8503,"num_object_clones":0,"num_object_copies":17006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6295,"num_write_kb":25180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":4893}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'8466","reported_seq":8601,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:21.895851+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:32:21.895851+0000","last_peered":"2024-09-10T20:32:21.895851+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:32:21.895851+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":8466,"log_dups_size":0,"ondisk_log_size":8466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34668544,"num_objects":8464,"num_object_clones":0,"num_object_copies":16928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5189,"num_write_kb":20756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4816},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'8438","reported_seq":8576,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:22.011917+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:32:22.011917+0000","last_peered":"2024-09-10T20:32:22.011917+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:32:22.011917+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":8438,"log_dups_size":0,"ondisk_log_size":8438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34529280,"num_objects":8430,"num_object_clones":0,"num_object_copies":16860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5132,"num_write_kb":20528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4898},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193200+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193200+0000","last_peered":"2024-09-10T20:32:18.193200+0000","last_clean":"2024-09-10T20:32:18.193200+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193200+0000","last_undegraded":"2024-09-10T20:32:18.193200+0000","last_fullsized":"2024-09-10T20:32:18.193200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'8607","reported_seq":8738,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:21.993436+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:32:21.993436+0000","last_peered":"2024-09-10T20:32:21.993436+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:32:21.993436+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":8607,"log_dups_size":0,"ondisk_log_size":8607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35237888,"num_objects":8603,"num_object_clones":0,"num_object_copies":17206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5434,"num_write_kb":21736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4890},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'8346","reported_seq":8810,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:32:22.015463+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:32:22.015463+0000","last_peered":"2024-09-10T20:32:22.015463+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:32:22.015463+0000","last_undegraded":"2024-09-10T20:32:22.015463+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":8346,"log_dups_size":0,"ondisk_log_size":8346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34136547,"num_objects":8335,"num_object_clones":0,"num_object_copies":16670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3319,"num_objects_unfound":0,"num_objects_dirty":8335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4953,"num_write_kb":19812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":610304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5016},{"shards":"0,2","objects":3319}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":554341801,"num_objects":135340,"num_object_clones":0,"num_object_copies":270680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":3319,"num_objects_unfound":0,"num_objects_dirty":135340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99715,"num_write_kb":398840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3756,"num_bytes_recovered":15372288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1125711872,"data_stored":1125625160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135419,"ondisk_log_size":135419,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182446,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":747820,"kb_used_data":463516,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93624020,"statfs":{"total":96636764160,"available":95870996480,"internally_reserved":0,"allocated":474640384,"data_stored":474397696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":291106474},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,2,2,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986218,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":625420,"kb_used_data":386876,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93746420,"statfs":{"total":96636764160,"available":95996334080,"internally_reserved":0,"allocated":396161024,"data_stored":395922413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":244248234},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659165,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":446064,"kb_used_data":247776,"kb_used_omap":4,"kb_used_meta":198267,"kb_avail":93925776,"statfs":{"total":96636764160,"available":96179994624,"internally_reserved":0,"allocated":253722624,"data_stored":253404063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":203026094},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,4,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":474480640,"data_stored":474477027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":254976000,"data_stored":254896514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":396255232,"data_stored":396251619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:32:28.070 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout:2024-09-10T20:32:28.067622+0000 min lat: 0.00191414 max lat: 2.51074 avg lat: 0.0260856 2024-09-10T20:32:28.070 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T20:32:28.070 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 180 20 138022 138002 2.99416 1.94531 0.0346021 0.0260856 2024-09-10T20:32:29.070 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 181 20 138525 138505 2.98847 1.96484 0.0167781 0.026134 2024-09-10T20:32:30.070 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 182 20 139023 139003 2.98274 1.94531 0.0773376 0.0261849 2024-09-10T20:32:30.356 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:32:30.911 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:30.912 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:32:30.937 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2024-09-10T20:32:30.885766+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":564323772,"num_objects":137779,"num_object_clones":0,"num_object_copies":275558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":3233,"num_objects_unfound":0,"num_objects_dirty":137779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102154,"num_write_kb":408590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3844,"num_bytes_recovered":15724563,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137859,"ondisk_log_size":137859,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1871084,"kb_used_data":1118008,"kb_used_omap":12,"kb_used_meta":753011,"kb_avail":281244436,"statfs":{"total":289910292480,"available":287994302464,"internally_reserved":0,"allocated":1144840192,"data_stored":1144040332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":771083266},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,4,10,8,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19251200,"num_objects":4700,"num_object_clones":0,"num_object_copies":9400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-228,"num_objects_unfound":0,"num_objects_dirty":4700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4700,"num_write_kb":18800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":933888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007976"},"pg_stats":[{"pgid":"2.f","version":"45'8573","reported_seq":8653,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:27.021627+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:32:27.021627+0000","last_peered":"2024-09-10T20:32:27.021627+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:32:27.021627+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":8573,"log_dups_size":0,"ondisk_log_size":8573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35115008,"num_objects":8573,"num_object_clones":0,"num_object_copies":17146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8573,"num_write_kb":34292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4966},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'8794","reported_seq":8904,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:28.462753+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:32:28.462753+0000","last_peered":"2024-09-10T20:32:28.462753+0000","last_clean":"2024-09-10T20:32:28.462753+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:32:28.462753+0000","last_undegraded":"2024-09-10T20:32:28.462753+0000","last_fullsized":"2024-09-10T20:32:28.462753+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":8794,"log_dups_size":0,"ondisk_log_size":8794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35995648,"num_objects":8788,"num_object_clones":0,"num_object_copies":17576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6980,"num_write_kb":27920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'8658","reported_seq":16005,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:28.375715+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:32:28.375715+0000","last_peered":"2024-09-10T20:32:28.375715+0000","last_clean":"2024-09-10T20:32:28.375715+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:32:28.375715+0000","last_undegraded":"2024-09-10T20:32:28.375715+0000","last_fullsized":"2024-09-10T20:32:28.375715+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":8658,"log_dups_size":0,"ondisk_log_size":8658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35446784,"num_objects":8654,"num_object_clones":0,"num_object_copies":17308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6786,"num_write_kb":27144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'8542","reported_seq":8622,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:28.344999+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:32:28.344999+0000","last_peered":"2024-09-10T20:32:28.344999+0000","last_clean":"2024-09-10T20:32:28.344999+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:32:28.344999+0000","last_undegraded":"2024-09-10T20:32:28.344999+0000","last_fullsized":"2024-09-10T20:32:28.344999+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":8542,"log_dups_size":0,"ondisk_log_size":8542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34988032,"num_objects":8542,"num_object_clones":0,"num_object_copies":17084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8542,"num_write_kb":34168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'8760","reported_seq":8886,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:26.980091+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:32:26.980091+0000","last_peered":"2024-09-10T20:32:26.980091+0000","last_clean":"2024-09-10T20:32:26.980091+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:32:26.980091+0000","last_undegraded":"2024-09-10T20:32:26.980091+0000","last_fullsized":"2024-09-10T20:32:26.980091+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":8760,"log_dups_size":0,"ondisk_log_size":8760,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35852288,"num_objects":8753,"num_object_clones":0,"num_object_copies":17506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5756,"num_write_kb":23024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'8494","reported_seq":8628,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:27.050655+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:32:27.050655+0000","last_peered":"2024-09-10T20:32:27.050655+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:32:27.050655+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":8494,"log_dups_size":0,"ondisk_log_size":8494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34770944,"num_objects":8489,"num_object_clones":0,"num_object_copies":16978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8489,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5358,"num_write_kb":21432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4875},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'8549","reported_seq":8687,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:26.969322+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:32:26.969322+0000","last_peered":"2024-09-10T20:32:26.969322+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:32:26.969322+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":8549,"log_dups_size":0,"ondisk_log_size":8549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34992128,"num_objects":8543,"num_object_clones":0,"num_object_copies":17086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5144,"num_write_kb":20576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4988},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'8615","reported_seq":8739,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:27.040046+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:32:27.040046+0000","last_peered":"2024-09-10T20:32:27.040046+0000","last_clean":"2024-09-10T20:32:27.040046+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:32:27.040046+0000","last_undegraded":"2024-09-10T20:32:27.040046+0000","last_fullsized":"2024-09-10T20:32:27.040046+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":8615,"log_dups_size":0,"ondisk_log_size":8615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35278848,"num_objects":8613,"num_object_clones":0,"num_object_copies":17226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5686,"num_write_kb":22744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8498","reported_seq":8630,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:28.421267+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:32:28.421267+0000","last_peered":"2024-09-10T20:32:28.421267+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:32:28.421267+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":8498,"log_dups_size":0,"ondisk_log_size":8498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34779136,"num_objects":8491,"num_object_clones":0,"num_object_copies":16982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6491,"num_write_kb":25964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":4965}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190232+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190232+0000","last_peered":"2024-09-10T20:32:18.190232+0000","last_clean":"2024-09-10T20:32:18.190232+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190232+0000","last_undegraded":"2024-09-10T20:32:18.190232+0000","last_fullsized":"2024-09-10T20:32:18.190232+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8663","reported_seq":8771,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:27.010779+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:32:27.010779+0000","last_peered":"2024-09-10T20:32:27.010779+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:32:27.010779+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":8663,"log_dups_size":0,"ondisk_log_size":8663,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35451846,"num_objects":8657,"num_object_clones":0,"num_object_copies":17314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8661,"num_write_kb":34624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5042},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182737+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182737+0000","last_peered":"2024-09-10T20:32:18.182737+0000","last_clean":"2024-09-10T20:32:18.182737+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182737+0000","last_undegraded":"2024-09-10T20:32:18.182737+0000","last_fullsized":"2024-09-10T20:32:18.182737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8596","reported_seq":8722,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:28.411937+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:32:28.411937+0000","last_peered":"2024-09-10T20:32:28.411937+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:32:28.411937+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":8596,"log_dups_size":0,"ondisk_log_size":8596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35188736,"num_objects":8591,"num_object_clones":0,"num_object_copies":17182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6424,"num_write_kb":25696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":4958}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8656","reported_seq":8784,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:28.369542+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:32:28.369542+0000","last_peered":"2024-09-10T20:32:28.369542+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:32:28.369542+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":8656,"log_dups_size":0,"ondisk_log_size":8656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35430400,"num_objects":8650,"num_object_clones":0,"num_object_copies":17300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6442,"num_write_kb":25768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":5040}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'8626","reported_seq":8761,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:26.972040+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:32:26.972040+0000","last_peered":"2024-09-10T20:32:26.972040+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:32:26.972040+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":8626,"log_dups_size":0,"ondisk_log_size":8626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35323904,"num_objects":8624,"num_object_clones":0,"num_object_copies":17248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5349,"num_write_kb":21396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4976},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'8579","reported_seq":8717,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:27.043884+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:32:27.043884+0000","last_peered":"2024-09-10T20:32:27.043884+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:32:27.043884+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":8579,"log_dups_size":0,"ondisk_log_size":8579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35106816,"num_objects":8571,"num_object_clones":0,"num_object_copies":17142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5273,"num_write_kb":21092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5039},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193200+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193200+0000","last_peered":"2024-09-10T20:32:18.193200+0000","last_clean":"2024-09-10T20:32:18.193200+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193200+0000","last_undegraded":"2024-09-10T20:32:18.193200+0000","last_fullsized":"2024-09-10T20:32:18.193200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'8740","reported_seq":8871,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:26.944498+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:32:26.944498+0000","last_peered":"2024-09-10T20:32:26.944498+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:32:26.944498+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":8740,"log_dups_size":0,"ondisk_log_size":8740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35782656,"num_objects":8736,"num_object_clones":0,"num_object_copies":17472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5567,"num_write_kb":22268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5023},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'8513","reported_seq":9149,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:32:27.068777+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:32:27.068777+0000","last_peered":"2024-09-10T20:32:27.068777+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:32:27.068777+0000","last_undegraded":"2024-09-10T20:32:27.068777+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":8513,"log_dups_size":0,"ondisk_log_size":8513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34820579,"num_objects":8502,"num_object_clones":0,"num_object_copies":17004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3233,"num_objects_unfound":0,"num_objects_dirty":8502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5120,"num_write_kb":20480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":235,"num_bytes_recovered":962560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5269},{"shards":"0,2","objects":3233}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":564323753,"num_objects":137777,"num_object_clones":0,"num_object_copies":275554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":3233,"num_objects_unfound":0,"num_objects_dirty":137777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102152,"num_write_kb":408588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3842,"num_bytes_recovered":15724544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1145974784,"data_stored":1145888072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137856,"ondisk_log_size":137856,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182447,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":777340,"kb_used_data":470828,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93594500,"statfs":{"total":96636764160,"available":95840768000,"internally_reserved":0,"allocated":482127872,"data_stored":481885184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,4,5,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986219,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":635736,"kb_used_data":391624,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93736104,"statfs":{"total":96636764160,"available":95985770496,"internally_reserved":0,"allocated":401022976,"data_stored":400784365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":249949866},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659166,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":458008,"kb_used_data":255556,"kb_used_omap":4,"kb_used_meta":202427,"kb_avail":93913832,"statfs":{"total":96636764160,"available":96167763968,"internally_reserved":0,"allocated":261689344,"data_stored":261370783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":207285934},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":481984512,"data_stored":481980899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":262864896,"data_stored":262785410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401125376,"data_stored":401121763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:32:30.939 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:32:31.070 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 183 20 139480 139460 2.9762 1.78516 0.0121128 0.0262418 2024-09-10T20:32:31.473 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:31.474 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:32:31.498 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2024-09-10T20:32:30.885766+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":564323772,"num_objects":137779,"num_object_clones":0,"num_object_copies":275558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":3233,"num_objects_unfound":0,"num_objects_dirty":137779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102154,"num_write_kb":408590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3844,"num_bytes_recovered":15724563,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137859,"ondisk_log_size":137859,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1871084,"kb_used_data":1118008,"kb_used_omap":12,"kb_used_meta":753011,"kb_avail":281244436,"statfs":{"total":289910292480,"available":287994302464,"internally_reserved":0,"allocated":1144840192,"data_stored":1144040332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":771083266},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,4,10,8,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19251200,"num_objects":4700,"num_object_clones":0,"num_object_copies":9400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-228,"num_objects_unfound":0,"num_objects_dirty":4700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4700,"num_write_kb":18800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":933888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007976"},"pg_stats":[{"pgid":"2.f","version":"45'8573","reported_seq":8653,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:27.021627+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:32:27.021627+0000","last_peered":"2024-09-10T20:32:27.021627+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:32:27.021627+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":8573,"log_dups_size":0,"ondisk_log_size":8573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35115008,"num_objects":8573,"num_object_clones":0,"num_object_copies":17146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8573,"num_write_kb":34292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4966},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'8794","reported_seq":8904,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:28.462753+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:32:28.462753+0000","last_peered":"2024-09-10T20:32:28.462753+0000","last_clean":"2024-09-10T20:32:28.462753+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:32:28.462753+0000","last_undegraded":"2024-09-10T20:32:28.462753+0000","last_fullsized":"2024-09-10T20:32:28.462753+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":8794,"log_dups_size":0,"ondisk_log_size":8794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35995648,"num_objects":8788,"num_object_clones":0,"num_object_copies":17576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6980,"num_write_kb":27920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'8658","reported_seq":16005,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:28.375715+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:32:28.375715+0000","last_peered":"2024-09-10T20:32:28.375715+0000","last_clean":"2024-09-10T20:32:28.375715+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:32:28.375715+0000","last_undegraded":"2024-09-10T20:32:28.375715+0000","last_fullsized":"2024-09-10T20:32:28.375715+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":8658,"log_dups_size":0,"ondisk_log_size":8658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35446784,"num_objects":8654,"num_object_clones":0,"num_object_copies":17308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6786,"num_write_kb":27144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'8542","reported_seq":8622,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:28.344999+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:32:28.344999+0000","last_peered":"2024-09-10T20:32:28.344999+0000","last_clean":"2024-09-10T20:32:28.344999+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:32:28.344999+0000","last_undegraded":"2024-09-10T20:32:28.344999+0000","last_fullsized":"2024-09-10T20:32:28.344999+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":8542,"log_dups_size":0,"ondisk_log_size":8542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34988032,"num_objects":8542,"num_object_clones":0,"num_object_copies":17084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8542,"num_write_kb":34168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'8760","reported_seq":8886,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:26.980091+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:32:26.980091+0000","last_peered":"2024-09-10T20:32:26.980091+0000","last_clean":"2024-09-10T20:32:26.980091+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:32:26.980091+0000","last_undegraded":"2024-09-10T20:32:26.980091+0000","last_fullsized":"2024-09-10T20:32:26.980091+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":8760,"log_dups_size":0,"ondisk_log_size":8760,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35852288,"num_objects":8753,"num_object_clones":0,"num_object_copies":17506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5756,"num_write_kb":23024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'8494","reported_seq":8628,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:27.050655+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:32:27.050655+0000","last_peered":"2024-09-10T20:32:27.050655+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:32:27.050655+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":8494,"log_dups_size":0,"ondisk_log_size":8494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34770944,"num_objects":8489,"num_object_clones":0,"num_object_copies":16978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8489,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5358,"num_write_kb":21432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4875},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'8549","reported_seq":8687,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:26.969322+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:32:26.969322+0000","last_peered":"2024-09-10T20:32:26.969322+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:32:26.969322+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":8549,"log_dups_size":0,"ondisk_log_size":8549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34992128,"num_objects":8543,"num_object_clones":0,"num_object_copies":17086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5144,"num_write_kb":20576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4988},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'8615","reported_seq":8739,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:27.040046+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:32:27.040046+0000","last_peered":"2024-09-10T20:32:27.040046+0000","last_clean":"2024-09-10T20:32:27.040046+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:32:27.040046+0000","last_undegraded":"2024-09-10T20:32:27.040046+0000","last_fullsized":"2024-09-10T20:32:27.040046+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":8615,"log_dups_size":0,"ondisk_log_size":8615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35278848,"num_objects":8613,"num_object_clones":0,"num_object_copies":17226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5686,"num_write_kb":22744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8498","reported_seq":8630,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:28.421267+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:32:28.421267+0000","last_peered":"2024-09-10T20:32:28.421267+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:32:28.421267+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":8498,"log_dups_size":0,"ondisk_log_size":8498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34779136,"num_objects":8491,"num_object_clones":0,"num_object_copies":16982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6491,"num_write_kb":25964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":4965}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190232+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190232+0000","last_peered":"2024-09-10T20:32:18.190232+0000","last_clean":"2024-09-10T20:32:18.190232+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190232+0000","last_undegraded":"2024-09-10T20:32:18.190232+0000","last_fullsized":"2024-09-10T20:32:18.190232+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8663","reported_seq":8771,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:27.010779+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:32:27.010779+0000","last_peered":"2024-09-10T20:32:27.010779+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:32:27.010779+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":8663,"log_dups_size":0,"ondisk_log_size":8663,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35451846,"num_objects":8657,"num_object_clones":0,"num_object_copies":17314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8661,"num_write_kb":34624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5042},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182737+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182737+0000","last_peered":"2024-09-10T20:32:18.182737+0000","last_clean":"2024-09-10T20:32:18.182737+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182737+0000","last_undegraded":"2024-09-10T20:32:18.182737+0000","last_fullsized":"2024-09-10T20:32:18.182737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8596","reported_seq":8722,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:28.411937+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:32:28.411937+0000","last_peered":"2024-09-10T20:32:28.411937+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:32:28.411937+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":8596,"log_dups_size":0,"ondisk_log_size":8596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35188736,"num_objects":8591,"num_object_clones":0,"num_object_copies":17182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6424,"num_write_kb":25696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":4958}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8656","reported_seq":8784,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:28.369542+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:32:28.369542+0000","last_peered":"2024-09-10T20:32:28.369542+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:32:28.369542+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":8656,"log_dups_size":0,"ondisk_log_size":8656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35430400,"num_objects":8650,"num_object_clones":0,"num_object_copies":17300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6442,"num_write_kb":25768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":5040}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'8626","reported_seq":8761,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:26.972040+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:32:26.972040+0000","last_peered":"2024-09-10T20:32:26.972040+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:32:26.972040+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":8626,"log_dups_size":0,"ondisk_log_size":8626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35323904,"num_objects":8624,"num_object_clones":0,"num_object_copies":17248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5349,"num_write_kb":21396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4976},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'8579","reported_seq":8717,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:27.043884+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:32:27.043884+0000","last_peered":"2024-09-10T20:32:27.043884+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:32:27.043884+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":8579,"log_dups_size":0,"ondisk_log_size":8579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35106816,"num_objects":8571,"num_object_clones":0,"num_object_copies":17142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5273,"num_write_kb":21092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5039},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193200+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193200+0000","last_peered":"2024-09-10T20:32:18.193200+0000","last_clean":"2024-09-10T20:32:18.193200+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193200+0000","last_undegraded":"2024-09-10T20:32:18.193200+0000","last_fullsized":"2024-09-10T20:32:18.193200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'8740","reported_seq":8871,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:26.944498+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:32:26.944498+0000","last_peered":"2024-09-10T20:32:26.944498+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:32:26.944498+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":8740,"log_dups_size":0,"ondisk_log_size":8740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35782656,"num_objects":8736,"num_object_clones":0,"num_object_copies":17472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5567,"num_write_kb":22268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5023},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'8513","reported_seq":9149,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:32:27.068777+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:32:27.068777+0000","last_peered":"2024-09-10T20:32:27.068777+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:32:27.068777+0000","last_undegraded":"2024-09-10T20:32:27.068777+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":8513,"log_dups_size":0,"ondisk_log_size":8513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34820579,"num_objects":8502,"num_object_clones":0,"num_object_copies":17004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3233,"num_objects_unfound":0,"num_objects_dirty":8502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5120,"num_write_kb":20480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":235,"num_bytes_recovered":962560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5269},{"shards":"0,2","objects":3233}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":564323753,"num_objects":137777,"num_object_clones":0,"num_object_copies":275554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":3233,"num_objects_unfound":0,"num_objects_dirty":137777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102152,"num_write_kb":408588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3842,"num_bytes_recovered":15724544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1145974784,"data_stored":1145888072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137856,"ondisk_log_size":137856,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182447,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":777340,"kb_used_data":470828,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93594500,"statfs":{"total":96636764160,"available":95840768000,"internally_reserved":0,"allocated":482127872,"data_stored":481885184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,4,5,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986219,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":635736,"kb_used_data":391624,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93736104,"statfs":{"total":96636764160,"available":95985770496,"internally_reserved":0,"allocated":401022976,"data_stored":400784365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":249949866},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659166,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":458008,"kb_used_data":255556,"kb_used_omap":4,"kb_used_meta":202427,"kb_avail":93913832,"statfs":{"total":96636764160,"available":96167763968,"internally_reserved":0,"allocated":261689344,"data_stored":261370783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":207285934},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":481984512,"data_stored":481980899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":262864896,"data_stored":262785410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401125376,"data_stored":401121763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:32:31.499 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:32:31.499 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'8573", 'reported_seq': 8653, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:27.021627+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:32:27.021627+0000', 'last_peered': '2024-09-10T20:32:27.021627+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:32:27.021627+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 8573, 'log_dups_size': 0, 'ondisk_log_size': 8573, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35115008, 'num_objects': 8573, 'num_object_clones': 0, 'num_object_copies': 17146, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3607, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8573, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8573, 'num_write_kb': 34292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4966}, {'shards': '0,2', 'objects': 3607}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:31.500 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:32:31.500 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'8494", 'reported_seq': 8628, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:27.050655+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:32:27.050655+0000', 'last_peered': '2024-09-10T20:32:27.050655+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:32:27.050655+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 8494, 'log_dups_size': 0, 'ondisk_log_size': 8494, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34770944, 'num_objects': 8489, 'num_object_clones': 0, 'num_object_copies': 16978, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3616, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8489, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5358, 'num_write_kb': 21432, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4875}, {'shards': '0,2', 'objects': 3614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:31.500 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:32:31.500 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'8549", 'reported_seq': 8687, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:26.969322+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:32:26.969322+0000', 'last_peered': '2024-09-10T20:32:26.969322+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:32:26.969322+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 8549, 'log_dups_size': 0, 'ondisk_log_size': 8549, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34992128, 'num_objects': 8543, 'num_object_clones': 0, 'num_object_copies': 17086, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3557, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8543, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5144, 'num_write_kb': 20576, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4988}, {'shards': '0,2', 'objects': 3555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:31.500 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:32:31.500 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'8498", 'reported_seq': 8630, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:28.421267+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:32:28.421267+0000', 'last_peered': '2024-09-10T20:32:28.421267+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:32:28.421267+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 8498, 'log_dups_size': 0, 'ondisk_log_size': 8498, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34779136, 'num_objects': 8491, 'num_object_clones': 0, 'num_object_copies': 16982, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3528, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8491, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6491, 'num_write_kb': 25964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 4965}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:32:31.500 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:32:31.500 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'8663", 'reported_seq': 8771, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:27.010779+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:32:27.010779+0000', 'last_peered': '2024-09-10T20:32:27.010779+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:32:27.010779+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 8663, 'log_dups_size': 0, 'ondisk_log_size': 8663, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35451846, 'num_objects': 8657, 'num_object_clones': 0, 'num_object_copies': 17314, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3617, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8657, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8661, 'num_write_kb': 34624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5042}, {'shards': '0,2', 'objects': 3615}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:31.501 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:32:31.501 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'8596", 'reported_seq': 8722, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:28.411937+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:32:28.411937+0000', 'last_peered': '2024-09-10T20:32:28.411937+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:32:28.411937+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 8596, 'log_dups_size': 0, 'ondisk_log_size': 8596, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35188736, 'num_objects': 8591, 'num_object_clones': 0, 'num_object_copies': 17182, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3635, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8591, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6424, 'num_write_kb': 25696, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3633}, {'shards': '2', 'objects': 4958}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:32:31.501 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:32:31.501 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'8656", 'reported_seq': 8784, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:28.369542+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:32:28.369542+0000', 'last_peered': '2024-09-10T20:32:28.369542+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:32:28.369542+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 8656, 'log_dups_size': 0, 'ondisk_log_size': 8656, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35430400, 'num_objects': 8650, 'num_object_clones': 0, 'num_object_copies': 17300, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8650, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6442, 'num_write_kb': 25768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3610}, {'shards': '2', 'objects': 5040}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:32:31.501 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:32:31.501 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'8626", 'reported_seq': 8761, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:26.972040+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:32:26.972040+0000', 'last_peered': '2024-09-10T20:32:26.972040+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:32:26.972040+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 8626, 'log_dups_size': 0, 'ondisk_log_size': 8626, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35323904, 'num_objects': 8624, 'num_object_clones': 0, 'num_object_copies': 17248, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3649, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8624, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5349, 'num_write_kb': 21396, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4976}, {'shards': '0,2', 'objects': 3648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:31.501 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:32:31.501 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'8579", 'reported_seq': 8717, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:27.043884+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:32:27.043884+0000', 'last_peered': '2024-09-10T20:32:27.043884+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:32:27.043884+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 8579, 'log_dups_size': 0, 'ondisk_log_size': 8579, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35106816, 'num_objects': 8571, 'num_object_clones': 0, 'num_object_copies': 17142, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3535, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8571, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5273, 'num_write_kb': 21092, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5039}, {'shards': '0,2', 'objects': 3532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:31.501 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:32:31.501 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'8740", 'reported_seq': 8871, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:26.944498+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:32:26.944498+0000', 'last_peered': '2024-09-10T20:32:26.944498+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:32:26.944498+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 8740, 'log_dups_size': 0, 'ondisk_log_size': 8740, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35782656, 'num_objects': 8736, 'num_object_clones': 0, 'num_object_copies': 17472, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3715, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8736, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5567, 'num_write_kb': 22268, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5023}, {'shards': '0,2', 'objects': 3713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:31.501 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:32:31.502 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'8513", 'reported_seq': 9149, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:32:27.068777+0000', 'last_change': '2024-09-10T20:32:16.657652+0000', 'last_active': '2024-09-10T20:32:27.068777+0000', 'last_peered': '2024-09-10T20:32:27.068777+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:32:27.068777+0000', 'last_undegraded': '2024-09-10T20:32:27.068777+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 8513, 'log_dups_size': 0, 'ondisk_log_size': 8513, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34820579, 'num_objects': 8502, 'num_object_clones': 0, 'num_object_copies': 17004, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3233, 'num_objects_unfound': 0, 'num_objects_dirty': 8502, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5120, 'num_write_kb': 20480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 235, 'num_bytes_recovered': 962560, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5269}, {'shards': '0,2', 'objects': 3233}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:31.502 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:32:32.071 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 184 20 139922 139902 2.96941 1.72656 0.0603826 0.0263019 2024-09-10T20:32:32.198 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:32.198 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36071/275558 objects degraded (13.090%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":263,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":137779,"data_bytes":564323772,"bytes_used":1915990016,"bytes_avail":287994302464,"bytes_total":289910292480,"degraded_objects":36071,"degraded_total":275558,"degraded_ratio":0.13090166135622991,"misplaced_objects":3233,"misplaced_total":275558,"misplaced_ratio":0.01173255721118603,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":77772,"recovering_keys_per_sec":0,"num_objects_recovered":228,"num_bytes_recovered":933888,"num_keys_recovered":0,"write_bytes_sec":1603201,"read_op_per_sec":0,"write_op_per_sec":391},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:32:32.224 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:32:32.224 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:32:32.759 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:32.759 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:32:32.790 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2024-09-10T20:32:30.885766+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":564323772,"num_objects":137779,"num_object_clones":0,"num_object_copies":275558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":3233,"num_objects_unfound":0,"num_objects_dirty":137779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102154,"num_write_kb":408590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3844,"num_bytes_recovered":15724563,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137859,"ondisk_log_size":137859,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1871084,"kb_used_data":1118008,"kb_used_omap":12,"kb_used_meta":753011,"kb_avail":281244436,"statfs":{"total":289910292480,"available":287994302464,"internally_reserved":0,"allocated":1144840192,"data_stored":1144040332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":771083266},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,4,10,8,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19251200,"num_objects":4700,"num_object_clones":0,"num_object_copies":9400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-228,"num_objects_unfound":0,"num_objects_dirty":4700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4700,"num_write_kb":18800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":933888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007976"},"pg_stats":[{"pgid":"2.f","version":"45'8573","reported_seq":8653,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:27.021627+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:32:27.021627+0000","last_peered":"2024-09-10T20:32:27.021627+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:32:27.021627+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":8573,"log_dups_size":0,"ondisk_log_size":8573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35115008,"num_objects":8573,"num_object_clones":0,"num_object_copies":17146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8573,"num_write_kb":34292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4966},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'8794","reported_seq":8904,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:28.462753+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:32:28.462753+0000","last_peered":"2024-09-10T20:32:28.462753+0000","last_clean":"2024-09-10T20:32:28.462753+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:32:28.462753+0000","last_undegraded":"2024-09-10T20:32:28.462753+0000","last_fullsized":"2024-09-10T20:32:28.462753+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":8794,"log_dups_size":0,"ondisk_log_size":8794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35995648,"num_objects":8788,"num_object_clones":0,"num_object_copies":17576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6980,"num_write_kb":27920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'8658","reported_seq":16005,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:28.375715+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:32:28.375715+0000","last_peered":"2024-09-10T20:32:28.375715+0000","last_clean":"2024-09-10T20:32:28.375715+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:32:28.375715+0000","last_undegraded":"2024-09-10T20:32:28.375715+0000","last_fullsized":"2024-09-10T20:32:28.375715+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":8658,"log_dups_size":0,"ondisk_log_size":8658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35446784,"num_objects":8654,"num_object_clones":0,"num_object_copies":17308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6786,"num_write_kb":27144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'8542","reported_seq":8622,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:28.344999+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:32:28.344999+0000","last_peered":"2024-09-10T20:32:28.344999+0000","last_clean":"2024-09-10T20:32:28.344999+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:32:28.344999+0000","last_undegraded":"2024-09-10T20:32:28.344999+0000","last_fullsized":"2024-09-10T20:32:28.344999+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":8542,"log_dups_size":0,"ondisk_log_size":8542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34988032,"num_objects":8542,"num_object_clones":0,"num_object_copies":17084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8542,"num_write_kb":34168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'8760","reported_seq":8886,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:26.980091+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:32:26.980091+0000","last_peered":"2024-09-10T20:32:26.980091+0000","last_clean":"2024-09-10T20:32:26.980091+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:32:26.980091+0000","last_undegraded":"2024-09-10T20:32:26.980091+0000","last_fullsized":"2024-09-10T20:32:26.980091+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":8760,"log_dups_size":0,"ondisk_log_size":8760,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35852288,"num_objects":8753,"num_object_clones":0,"num_object_copies":17506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5756,"num_write_kb":23024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'8494","reported_seq":8628,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:27.050655+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:32:27.050655+0000","last_peered":"2024-09-10T20:32:27.050655+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:32:27.050655+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":8494,"log_dups_size":0,"ondisk_log_size":8494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34770944,"num_objects":8489,"num_object_clones":0,"num_object_copies":16978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8489,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5358,"num_write_kb":21432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4875},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'8549","reported_seq":8687,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:26.969322+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:32:26.969322+0000","last_peered":"2024-09-10T20:32:26.969322+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:32:26.969322+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":8549,"log_dups_size":0,"ondisk_log_size":8549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34992128,"num_objects":8543,"num_object_clones":0,"num_object_copies":17086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5144,"num_write_kb":20576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4988},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'8615","reported_seq":8739,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:27.040046+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:32:27.040046+0000","last_peered":"2024-09-10T20:32:27.040046+0000","last_clean":"2024-09-10T20:32:27.040046+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:32:27.040046+0000","last_undegraded":"2024-09-10T20:32:27.040046+0000","last_fullsized":"2024-09-10T20:32:27.040046+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":8615,"log_dups_size":0,"ondisk_log_size":8615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35278848,"num_objects":8613,"num_object_clones":0,"num_object_copies":17226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5686,"num_write_kb":22744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8498","reported_seq":8630,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:28.421267+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:32:28.421267+0000","last_peered":"2024-09-10T20:32:28.421267+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:32:28.421267+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":8498,"log_dups_size":0,"ondisk_log_size":8498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34779136,"num_objects":8491,"num_object_clones":0,"num_object_copies":16982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6491,"num_write_kb":25964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":4965}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190232+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190232+0000","last_peered":"2024-09-10T20:32:18.190232+0000","last_clean":"2024-09-10T20:32:18.190232+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190232+0000","last_undegraded":"2024-09-10T20:32:18.190232+0000","last_fullsized":"2024-09-10T20:32:18.190232+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8663","reported_seq":8771,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:27.010779+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:32:27.010779+0000","last_peered":"2024-09-10T20:32:27.010779+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:32:27.010779+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":8663,"log_dups_size":0,"ondisk_log_size":8663,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35451846,"num_objects":8657,"num_object_clones":0,"num_object_copies":17314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8661,"num_write_kb":34624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5042},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182737+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182737+0000","last_peered":"2024-09-10T20:32:18.182737+0000","last_clean":"2024-09-10T20:32:18.182737+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182737+0000","last_undegraded":"2024-09-10T20:32:18.182737+0000","last_fullsized":"2024-09-10T20:32:18.182737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8596","reported_seq":8722,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:28.411937+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:32:28.411937+0000","last_peered":"2024-09-10T20:32:28.411937+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:32:28.411937+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":8596,"log_dups_size":0,"ondisk_log_size":8596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35188736,"num_objects":8591,"num_object_clones":0,"num_object_copies":17182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6424,"num_write_kb":25696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":4958}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8656","reported_seq":8784,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:28.369542+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:32:28.369542+0000","last_peered":"2024-09-10T20:32:28.369542+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:32:28.369542+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":8656,"log_dups_size":0,"ondisk_log_size":8656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35430400,"num_objects":8650,"num_object_clones":0,"num_object_copies":17300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6442,"num_write_kb":25768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":5040}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'8626","reported_seq":8761,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:26.972040+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:32:26.972040+0000","last_peered":"2024-09-10T20:32:26.972040+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:32:26.972040+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":8626,"log_dups_size":0,"ondisk_log_size":8626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35323904,"num_objects":8624,"num_object_clones":0,"num_object_copies":17248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5349,"num_write_kb":21396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4976},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'8579","reported_seq":8717,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:27.043884+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:32:27.043884+0000","last_peered":"2024-09-10T20:32:27.043884+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:32:27.043884+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":8579,"log_dups_size":0,"ondisk_log_size":8579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35106816,"num_objects":8571,"num_object_clones":0,"num_object_copies":17142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5273,"num_write_kb":21092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5039},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193200+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193200+0000","last_peered":"2024-09-10T20:32:18.193200+0000","last_clean":"2024-09-10T20:32:18.193200+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193200+0000","last_undegraded":"2024-09-10T20:32:18.193200+0000","last_fullsized":"2024-09-10T20:32:18.193200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'8740","reported_seq":8871,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:26.944498+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:32:26.944498+0000","last_peered":"2024-09-10T20:32:26.944498+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:32:26.944498+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":8740,"log_dups_size":0,"ondisk_log_size":8740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35782656,"num_objects":8736,"num_object_clones":0,"num_object_copies":17472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5567,"num_write_kb":22268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5023},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'8513","reported_seq":9149,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:32:27.068777+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:32:27.068777+0000","last_peered":"2024-09-10T20:32:27.068777+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:32:27.068777+0000","last_undegraded":"2024-09-10T20:32:27.068777+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":8513,"log_dups_size":0,"ondisk_log_size":8513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34820579,"num_objects":8502,"num_object_clones":0,"num_object_copies":17004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3233,"num_objects_unfound":0,"num_objects_dirty":8502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5120,"num_write_kb":20480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":235,"num_bytes_recovered":962560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5269},{"shards":"0,2","objects":3233}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":564323753,"num_objects":137777,"num_object_clones":0,"num_object_copies":275554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":3233,"num_objects_unfound":0,"num_objects_dirty":137777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102152,"num_write_kb":408588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3842,"num_bytes_recovered":15724544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1145974784,"data_stored":1145888072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137856,"ondisk_log_size":137856,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182447,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":777340,"kb_used_data":470828,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93594500,"statfs":{"total":96636764160,"available":95840768000,"internally_reserved":0,"allocated":482127872,"data_stored":481885184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,4,5,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986219,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":635736,"kb_used_data":391624,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93736104,"statfs":{"total":96636764160,"available":95985770496,"internally_reserved":0,"allocated":401022976,"data_stored":400784365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":249949866},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659166,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":458008,"kb_used_data":255556,"kb_used_omap":4,"kb_used_meta":202427,"kb_avail":93913832,"statfs":{"total":96636764160,"available":96167763968,"internally_reserved":0,"allocated":261689344,"data_stored":261370783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":207285934},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":481984512,"data_stored":481980899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":262864896,"data_stored":262785410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401125376,"data_stored":401121763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:32:33.071 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 185 20 140380 140360 2.96303 1.78906 0.0219646 0.0263579 2024-09-10T20:32:34.071 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 186 20 140897 140877 2.95795 2.01953 0.035226 0.0264051 2024-09-10T20:32:35.071 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 187 20 141399 141379 2.95262 1.96094 0.027753 0.0264516 2024-09-10T20:32:35.792 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:32:36.071 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 188 20 141885 141865 2.94701 1.89844 0.0286638 0.0265021 2024-09-10T20:32:36.353 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:36.355 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:32:36.376 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":172,"stamp":"2024-09-10T20:32:34.887326+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":574150076,"num_objects":140178,"num_object_clones":0,"num_object_copies":280356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":3124,"num_objects_unfound":0,"num_objects_dirty":140178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104553,"num_write_kb":418186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":16171027,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140258,"ondisk_log_size":140258,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1890648,"kb_used_data":1137576,"kb_used_omap":12,"kb_used_meta":753011,"kb_avail":281224872,"statfs":{"total":289910292480,"available":287974268928,"internally_reserved":0,"allocated":1164877824,"data_stored":1164077964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":771083266},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,7,10,10,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23490560,"num_objects":5735,"num_object_clones":0,"num_object_copies":11470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-195,"num_objects_unfound":0,"num_objects_dirty":5735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5735,"num_write_kb":22940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":195,"num_bytes_recovered":798720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008016"},"pg_stats":[{"pgid":"2.f","version":"45'8743","reported_seq":8820,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:32.027635+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:32:32.027635+0000","last_peered":"2024-09-10T20:32:32.027635+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:32:32.027635+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":8743,"log_dups_size":0,"ondisk_log_size":8743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35811328,"num_objects":8743,"num_object_clones":0,"num_object_copies":17486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8743,"num_write_kb":34972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5136},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'8931","reported_seq":9041,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:33.415668+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:32:33.415668+0000","last_peered":"2024-09-10T20:32:33.415668+0000","last_clean":"2024-09-10T20:32:33.415668+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:32:33.415668+0000","last_undegraded":"2024-09-10T20:32:33.415668+0000","last_fullsized":"2024-09-10T20:32:33.415668+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":8931,"log_dups_size":0,"ondisk_log_size":8931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36556800,"num_objects":8925,"num_object_clones":0,"num_object_copies":17850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7117,"num_write_kb":28468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'8811","reported_seq":16158,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:33.450361+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:32:33.450361+0000","last_peered":"2024-09-10T20:32:33.450361+0000","last_clean":"2024-09-10T20:32:33.450361+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:32:33.450361+0000","last_undegraded":"2024-09-10T20:32:33.450361+0000","last_fullsized":"2024-09-10T20:32:33.450361+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":8811,"log_dups_size":0,"ondisk_log_size":8811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36073472,"num_objects":8807,"num_object_clones":0,"num_object_copies":17614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6939,"num_write_kb":27756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'8668","reported_seq":8748,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:33.456874+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:32:33.456874+0000","last_peered":"2024-09-10T20:32:33.456874+0000","last_clean":"2024-09-10T20:32:33.456874+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:32:33.456874+0000","last_undegraded":"2024-09-10T20:32:33.456874+0000","last_fullsized":"2024-09-10T20:32:33.456874+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":8668,"log_dups_size":0,"ondisk_log_size":8668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35504128,"num_objects":8668,"num_object_clones":0,"num_object_copies":17336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8668,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8668,"num_write_kb":34672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'8913","reported_seq":9039,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:32.049684+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:32:32.049684+0000","last_peered":"2024-09-10T20:32:32.049684+0000","last_clean":"2024-09-10T20:32:32.049684+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:32:32.049684+0000","last_undegraded":"2024-09-10T20:32:32.049684+0000","last_fullsized":"2024-09-10T20:32:32.049684+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":8913,"log_dups_size":0,"ondisk_log_size":8913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36478976,"num_objects":8906,"num_object_clones":0,"num_object_copies":17812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5909,"num_write_kb":23636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'8622","reported_seq":8758,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:31.940835+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:32:31.940835+0000","last_peered":"2024-09-10T20:32:31.940835+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:32:31.940835+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":8622,"log_dups_size":0,"ondisk_log_size":8622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35295232,"num_objects":8617,"num_object_clones":0,"num_object_copies":17234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5486,"num_write_kb":21944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5003},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'8700","reported_seq":8838,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:32.016535+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:32:32.016535+0000","last_peered":"2024-09-10T20:32:32.016535+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:32:32.016535+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":8700,"log_dups_size":0,"ondisk_log_size":8700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35610624,"num_objects":8694,"num_object_clones":0,"num_object_copies":17388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5295,"num_write_kb":21180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5139},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'8774","reported_seq":8898,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:31.924954+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:32:31.924954+0000","last_peered":"2024-09-10T20:32:31.924954+0000","last_clean":"2024-09-10T20:32:31.924954+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:32:31.924954+0000","last_undegraded":"2024-09-10T20:32:31.924954+0000","last_fullsized":"2024-09-10T20:32:31.924954+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":8774,"log_dups_size":0,"ondisk_log_size":8774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35930112,"num_objects":8772,"num_object_clones":0,"num_object_copies":17544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5845,"num_write_kb":23380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8668","reported_seq":8798,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:33.477352+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:32:33.477352+0000","last_peered":"2024-09-10T20:32:33.477352+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:32:33.477352+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":8668,"log_dups_size":0,"ondisk_log_size":8668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35475456,"num_objects":8661,"num_object_clones":0,"num_object_copies":17322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6661,"num_write_kb":26644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":5135}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190232+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190232+0000","last_peered":"2024-09-10T20:32:18.190232+0000","last_clean":"2024-09-10T20:32:18.190232+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190232+0000","last_undegraded":"2024-09-10T20:32:18.190232+0000","last_fullsized":"2024-09-10T20:32:18.190232+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8800","reported_seq":8908,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:31.947676+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:32:31.947676+0000","last_peered":"2024-09-10T20:32:31.947676+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:32:31.947676+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":8800,"log_dups_size":0,"ondisk_log_size":8800,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36012998,"num_objects":8794,"num_object_clones":0,"num_object_copies":17588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8794,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8798,"num_write_kb":35172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5179},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182737+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182737+0000","last_peered":"2024-09-10T20:32:18.182737+0000","last_clean":"2024-09-10T20:32:18.182737+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182737+0000","last_undegraded":"2024-09-10T20:32:18.182737+0000","last_fullsized":"2024-09-10T20:32:18.182737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8747","reported_seq":8875,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:33.413764+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:32:33.413764+0000","last_peered":"2024-09-10T20:32:33.413764+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:32:33.413764+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":8747,"log_dups_size":0,"ondisk_log_size":8747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35807232,"num_objects":8742,"num_object_clones":0,"num_object_copies":17484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6575,"num_write_kb":26300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":5109}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8827","reported_seq":8955,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:33.442226+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:32:33.442226+0000","last_peered":"2024-09-10T20:32:33.442226+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:32:33.442226+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":8827,"log_dups_size":0,"ondisk_log_size":8827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36130816,"num_objects":8821,"num_object_clones":0,"num_object_copies":17642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8821,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6613,"num_write_kb":26452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":5211}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'8799","reported_seq":8931,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:32.047763+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:32:32.047763+0000","last_peered":"2024-09-10T20:32:32.047763+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:32:32.047763+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":8799,"log_dups_size":0,"ondisk_log_size":8799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36032512,"num_objects":8797,"num_object_clones":0,"num_object_copies":17594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5522,"num_write_kb":22088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5149},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'8709","reported_seq":8846,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:32.054171+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:32:32.054171+0000","last_peered":"2024-09-10T20:32:32.054171+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:32:32.054171+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":8709,"log_dups_size":0,"ondisk_log_size":8709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35639296,"num_objects":8701,"num_object_clones":0,"num_object_copies":17402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8701,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5403,"num_write_kb":21612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5169},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193200+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193200+0000","last_peered":"2024-09-10T20:32:18.193200+0000","last_clean":"2024-09-10T20:32:18.193200+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193200+0000","last_undegraded":"2024-09-10T20:32:18.193200+0000","last_fullsized":"2024-09-10T20:32:18.193200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'8908","reported_seq":9038,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:32.048685+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:32:32.048685+0000","last_peered":"2024-09-10T20:32:32.048685+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:32:32.048685+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":8908,"log_dups_size":0,"ondisk_log_size":8908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36470784,"num_objects":8904,"num_object_clones":0,"num_object_copies":17808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5735,"num_write_kb":22940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5191},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'8635","reported_seq":9489,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:32:32.070456+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:32:32.070456+0000","last_peered":"2024-09-10T20:32:32.070456+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:32:32.070456+0000","last_undegraded":"2024-09-10T20:32:32.070456+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":8635,"log_dups_size":0,"ondisk_log_size":8635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35320291,"num_objects":8624,"num_object_clones":0,"num_object_copies":17248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3124,"num_objects_unfound":0,"num_objects_dirty":8624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5242,"num_write_kb":20968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":1409024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5500},{"shards":"0,2","objects":3124}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":574150057,"num_objects":140176,"num_object_clones":0,"num_object_copies":280352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":3124,"num_objects_unfound":0,"num_objects_dirty":140176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104551,"num_write_kb":418184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":16171008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1166049280,"data_stored":1165962568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140255,"ondisk_log_size":140255,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182448,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":788492,"kb_used_data":477820,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93583348,"statfs":{"total":96636764160,"available":95829348352,"internally_reserved":0,"allocated":489287680,"data_stored":489044992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,4,4,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986220,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":644704,"kb_used_data":396432,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93727136,"statfs":{"total":96636764160,"available":95976587264,"internally_reserved":0,"allocated":405946368,"data_stored":405707757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":254209706},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659167,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":457452,"kb_used_data":263324,"kb_used_omap":4,"kb_used_meta":194107,"kb_avail":93914388,"statfs":{"total":96636764160,"available":96168333312,"internally_reserved":0,"allocated":269643776,"data_stored":269325215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":198766254},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":489197568,"data_stored":489193955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":270766080,"data_stored":270686594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":406085632,"data_stored":406082019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:32:36.377 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:32:36.926 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:36.926 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:32:36.948 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":173,"stamp":"2024-09-10T20:32:36.888230+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":574150076,"num_objects":140178,"num_object_clones":0,"num_object_copies":280356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":3124,"num_objects_unfound":0,"num_objects_dirty":140178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104553,"num_write_kb":418186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":16171027,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140258,"ondisk_log_size":140258,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1890648,"kb_used_data":1137576,"kb_used_omap":12,"kb_used_meta":753011,"kb_avail":281224872,"statfs":{"total":289910292480,"available":287974268928,"internally_reserved":0,"allocated":1164877824,"data_stored":1164077964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":771083266},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,7,10,10,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19808256,"num_objects":4836,"num_object_clones":0,"num_object_copies":9672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-195,"num_objects_unfound":0,"num_objects_dirty":4836,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4836,"num_write_kb":19344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":195,"num_bytes_recovered":798720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008351"},"pg_stats":[{"pgid":"2.f","version":"45'8743","reported_seq":8820,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:32.027635+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:32:32.027635+0000","last_peered":"2024-09-10T20:32:32.027635+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:32:32.027635+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":8743,"log_dups_size":0,"ondisk_log_size":8743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35811328,"num_objects":8743,"num_object_clones":0,"num_object_copies":17486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8743,"num_write_kb":34972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5136},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'8931","reported_seq":9041,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:33.415668+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:32:33.415668+0000","last_peered":"2024-09-10T20:32:33.415668+0000","last_clean":"2024-09-10T20:32:33.415668+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:32:33.415668+0000","last_undegraded":"2024-09-10T20:32:33.415668+0000","last_fullsized":"2024-09-10T20:32:33.415668+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":8931,"log_dups_size":0,"ondisk_log_size":8931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36556800,"num_objects":8925,"num_object_clones":0,"num_object_copies":17850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7117,"num_write_kb":28468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'8811","reported_seq":16158,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:33.450361+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:32:33.450361+0000","last_peered":"2024-09-10T20:32:33.450361+0000","last_clean":"2024-09-10T20:32:33.450361+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:32:33.450361+0000","last_undegraded":"2024-09-10T20:32:33.450361+0000","last_fullsized":"2024-09-10T20:32:33.450361+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":8811,"log_dups_size":0,"ondisk_log_size":8811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36073472,"num_objects":8807,"num_object_clones":0,"num_object_copies":17614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6939,"num_write_kb":27756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'8668","reported_seq":8748,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:33.456874+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:32:33.456874+0000","last_peered":"2024-09-10T20:32:33.456874+0000","last_clean":"2024-09-10T20:32:33.456874+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:32:33.456874+0000","last_undegraded":"2024-09-10T20:32:33.456874+0000","last_fullsized":"2024-09-10T20:32:33.456874+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":8668,"log_dups_size":0,"ondisk_log_size":8668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35504128,"num_objects":8668,"num_object_clones":0,"num_object_copies":17336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8668,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8668,"num_write_kb":34672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'8913","reported_seq":9039,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:32.049684+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:32:32.049684+0000","last_peered":"2024-09-10T20:32:32.049684+0000","last_clean":"2024-09-10T20:32:32.049684+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:32:32.049684+0000","last_undegraded":"2024-09-10T20:32:32.049684+0000","last_fullsized":"2024-09-10T20:32:32.049684+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":8913,"log_dups_size":0,"ondisk_log_size":8913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36478976,"num_objects":8906,"num_object_clones":0,"num_object_copies":17812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5909,"num_write_kb":23636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'8622","reported_seq":8758,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:31.940835+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:32:31.940835+0000","last_peered":"2024-09-10T20:32:31.940835+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:32:31.940835+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":8622,"log_dups_size":0,"ondisk_log_size":8622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35295232,"num_objects":8617,"num_object_clones":0,"num_object_copies":17234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5486,"num_write_kb":21944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5003},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'8700","reported_seq":8838,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:32.016535+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:32:32.016535+0000","last_peered":"2024-09-10T20:32:32.016535+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:32:32.016535+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":8700,"log_dups_size":0,"ondisk_log_size":8700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35610624,"num_objects":8694,"num_object_clones":0,"num_object_copies":17388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5295,"num_write_kb":21180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5139},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'8774","reported_seq":8898,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:31.924954+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:32:31.924954+0000","last_peered":"2024-09-10T20:32:31.924954+0000","last_clean":"2024-09-10T20:32:31.924954+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:32:31.924954+0000","last_undegraded":"2024-09-10T20:32:31.924954+0000","last_fullsized":"2024-09-10T20:32:31.924954+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":8774,"log_dups_size":0,"ondisk_log_size":8774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35930112,"num_objects":8772,"num_object_clones":0,"num_object_copies":17544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5845,"num_write_kb":23380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8668","reported_seq":8798,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:33.477352+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:32:33.477352+0000","last_peered":"2024-09-10T20:32:33.477352+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:32:33.477352+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":8668,"log_dups_size":0,"ondisk_log_size":8668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35475456,"num_objects":8661,"num_object_clones":0,"num_object_copies":17322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6661,"num_write_kb":26644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":5135}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190232+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190232+0000","last_peered":"2024-09-10T20:32:18.190232+0000","last_clean":"2024-09-10T20:32:18.190232+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190232+0000","last_undegraded":"2024-09-10T20:32:18.190232+0000","last_fullsized":"2024-09-10T20:32:18.190232+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8800","reported_seq":8908,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:31.947676+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:32:31.947676+0000","last_peered":"2024-09-10T20:32:31.947676+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:32:31.947676+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":8800,"log_dups_size":0,"ondisk_log_size":8800,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36012998,"num_objects":8794,"num_object_clones":0,"num_object_copies":17588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8794,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8798,"num_write_kb":35172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5179},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182737+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182737+0000","last_peered":"2024-09-10T20:32:18.182737+0000","last_clean":"2024-09-10T20:32:18.182737+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182737+0000","last_undegraded":"2024-09-10T20:32:18.182737+0000","last_fullsized":"2024-09-10T20:32:18.182737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8747","reported_seq":8875,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:33.413764+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:32:33.413764+0000","last_peered":"2024-09-10T20:32:33.413764+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:32:33.413764+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":8747,"log_dups_size":0,"ondisk_log_size":8747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35807232,"num_objects":8742,"num_object_clones":0,"num_object_copies":17484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6575,"num_write_kb":26300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":5109}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8827","reported_seq":8955,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:33.442226+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:32:33.442226+0000","last_peered":"2024-09-10T20:32:33.442226+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:32:33.442226+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":8827,"log_dups_size":0,"ondisk_log_size":8827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36130816,"num_objects":8821,"num_object_clones":0,"num_object_copies":17642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8821,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6613,"num_write_kb":26452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":5211}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'8799","reported_seq":8931,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:32.047763+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:32:32.047763+0000","last_peered":"2024-09-10T20:32:32.047763+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:32:32.047763+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":8799,"log_dups_size":0,"ondisk_log_size":8799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36032512,"num_objects":8797,"num_object_clones":0,"num_object_copies":17594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5522,"num_write_kb":22088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5149},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'8709","reported_seq":8846,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:32.054171+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:32:32.054171+0000","last_peered":"2024-09-10T20:32:32.054171+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:32:32.054171+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":8709,"log_dups_size":0,"ondisk_log_size":8709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35639296,"num_objects":8701,"num_object_clones":0,"num_object_copies":17402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8701,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5403,"num_write_kb":21612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5169},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193200+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193200+0000","last_peered":"2024-09-10T20:32:18.193200+0000","last_clean":"2024-09-10T20:32:18.193200+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193200+0000","last_undegraded":"2024-09-10T20:32:18.193200+0000","last_fullsized":"2024-09-10T20:32:18.193200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'8908","reported_seq":9038,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:32.048685+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:32:32.048685+0000","last_peered":"2024-09-10T20:32:32.048685+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:32:32.048685+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":8908,"log_dups_size":0,"ondisk_log_size":8908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36470784,"num_objects":8904,"num_object_clones":0,"num_object_copies":17808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5735,"num_write_kb":22940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5191},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'8635","reported_seq":9489,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:32:32.070456+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:32:32.070456+0000","last_peered":"2024-09-10T20:32:32.070456+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:32:32.070456+0000","last_undegraded":"2024-09-10T20:32:32.070456+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":8635,"log_dups_size":0,"ondisk_log_size":8635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35320291,"num_objects":8624,"num_object_clones":0,"num_object_copies":17248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3124,"num_objects_unfound":0,"num_objects_dirty":8624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5242,"num_write_kb":20968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":1409024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5500},{"shards":"0,2","objects":3124}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":574150057,"num_objects":140176,"num_object_clones":0,"num_object_copies":280352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":3124,"num_objects_unfound":0,"num_objects_dirty":140176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104551,"num_write_kb":418184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":16171008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1166049280,"data_stored":1165962568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140255,"ondisk_log_size":140255,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182448,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":788492,"kb_used_data":477820,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93583348,"statfs":{"total":96636764160,"available":95829348352,"internally_reserved":0,"allocated":489287680,"data_stored":489044992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,4,4,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986220,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":644704,"kb_used_data":396432,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93727136,"statfs":{"total":96636764160,"available":95976587264,"internally_reserved":0,"allocated":405946368,"data_stored":405707757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":254209706},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659167,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":457452,"kb_used_data":263324,"kb_used_omap":4,"kb_used_meta":194107,"kb_avail":93914388,"statfs":{"total":96636764160,"available":96168333312,"internally_reserved":0,"allocated":269643776,"data_stored":269325215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":198766254},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":489197568,"data_stored":489193955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":270766080,"data_stored":270686594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":406085632,"data_stored":406082019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:32:36.949 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:32:36.949 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'8743", 'reported_seq': 8820, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:32.027635+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:32:32.027635+0000', 'last_peered': '2024-09-10T20:32:32.027635+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:32:32.027635+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 8743, 'log_dups_size': 0, 'ondisk_log_size': 8743, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35811328, 'num_objects': 8743, 'num_object_clones': 0, 'num_object_copies': 17486, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3607, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8743, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8743, 'num_write_kb': 34972, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5136}, {'shards': '0,2', 'objects': 3607}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:36.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:32:36.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'8622", 'reported_seq': 8758, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:31.940835+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:32:31.940835+0000', 'last_peered': '2024-09-10T20:32:31.940835+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:32:31.940835+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 8622, 'log_dups_size': 0, 'ondisk_log_size': 8622, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35295232, 'num_objects': 8617, 'num_object_clones': 0, 'num_object_copies': 17234, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3616, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8617, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5486, 'num_write_kb': 21944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5003}, {'shards': '0,2', 'objects': 3614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:36.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:32:36.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'8700", 'reported_seq': 8838, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:32.016535+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:32:32.016535+0000', 'last_peered': '2024-09-10T20:32:32.016535+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:32:32.016535+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 8700, 'log_dups_size': 0, 'ondisk_log_size': 8700, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35610624, 'num_objects': 8694, 'num_object_clones': 0, 'num_object_copies': 17388, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3557, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8694, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5295, 'num_write_kb': 21180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5139}, {'shards': '0,2', 'objects': 3555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:36.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:32:36.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'8668", 'reported_seq': 8798, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:33.477352+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:32:33.477352+0000', 'last_peered': '2024-09-10T20:32:33.477352+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:32:33.477352+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 8668, 'log_dups_size': 0, 'ondisk_log_size': 8668, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35475456, 'num_objects': 8661, 'num_object_clones': 0, 'num_object_copies': 17322, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3528, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8661, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6661, 'num_write_kb': 26644, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 5135}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:32:36.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:32:36.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'8800", 'reported_seq': 8908, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:31.947676+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:32:31.947676+0000', 'last_peered': '2024-09-10T20:32:31.947676+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:32:31.947676+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 8800, 'log_dups_size': 0, 'ondisk_log_size': 8800, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36012998, 'num_objects': 8794, 'num_object_clones': 0, 'num_object_copies': 17588, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3617, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8794, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8798, 'num_write_kb': 35172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5179}, {'shards': '0,2', 'objects': 3615}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:36.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:32:36.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'8747", 'reported_seq': 8875, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:33.413764+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:32:33.413764+0000', 'last_peered': '2024-09-10T20:32:33.413764+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:32:33.413764+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 8747, 'log_dups_size': 0, 'ondisk_log_size': 8747, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35807232, 'num_objects': 8742, 'num_object_clones': 0, 'num_object_copies': 17484, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3635, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8742, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6575, 'num_write_kb': 26300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3633}, {'shards': '2', 'objects': 5109}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:32:36.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:32:36.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'8827", 'reported_seq': 8955, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:33.442226+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:32:33.442226+0000', 'last_peered': '2024-09-10T20:32:33.442226+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:32:33.442226+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 8827, 'log_dups_size': 0, 'ondisk_log_size': 8827, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36130816, 'num_objects': 8821, 'num_object_clones': 0, 'num_object_copies': 17642, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8821, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6613, 'num_write_kb': 26452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3610}, {'shards': '2', 'objects': 5211}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:32:36.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:32:36.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'8799", 'reported_seq': 8931, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:32.047763+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:32:32.047763+0000', 'last_peered': '2024-09-10T20:32:32.047763+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:32:32.047763+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 8799, 'log_dups_size': 0, 'ondisk_log_size': 8799, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36032512, 'num_objects': 8797, 'num_object_clones': 0, 'num_object_copies': 17594, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3649, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8797, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5522, 'num_write_kb': 22088, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5149}, {'shards': '0,2', 'objects': 3648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:36.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:32:36.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'8709", 'reported_seq': 8846, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:32.054171+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:32:32.054171+0000', 'last_peered': '2024-09-10T20:32:32.054171+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:32:32.054171+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 8709, 'log_dups_size': 0, 'ondisk_log_size': 8709, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35639296, 'num_objects': 8701, 'num_object_clones': 0, 'num_object_copies': 17402, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3535, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8701, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5403, 'num_write_kb': 21612, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5169}, {'shards': '0,2', 'objects': 3532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:36.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:32:36.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'8908", 'reported_seq': 9038, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:32.048685+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:32:32.048685+0000', 'last_peered': '2024-09-10T20:32:32.048685+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:32:32.048685+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 8908, 'log_dups_size': 0, 'ondisk_log_size': 8908, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36470784, 'num_objects': 8904, 'num_object_clones': 0, 'num_object_copies': 17808, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3715, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8904, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5735, 'num_write_kb': 22940, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5191}, {'shards': '0,2', 'objects': 3713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:36.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:32:36.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'8635", 'reported_seq': 9489, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:32:32.070456+0000', 'last_change': '2024-09-10T20:32:16.657652+0000', 'last_active': '2024-09-10T20:32:32.070456+0000', 'last_peered': '2024-09-10T20:32:32.070456+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:32:32.070456+0000', 'last_undegraded': '2024-09-10T20:32:32.070456+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 8635, 'log_dups_size': 0, 'ondisk_log_size': 8635, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35320291, 'num_objects': 8624, 'num_object_clones': 0, 'num_object_copies': 17248, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3124, 'num_objects_unfound': 0, 'num_objects_dirty': 8624, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5242, 'num_write_kb': 20968, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 1409024, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5500}, {'shards': '0,2', 'objects': 3124}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:36.952 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:32:37.071 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 189 20 142325 142305 2.94052 1.71875 0.0190379 0.0265611 2024-09-10T20:32:37.635 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:37.635 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36071/280356 objects degraded (12.866%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":269,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":140178,"data_bytes":574150076,"bytes_used":1936023552,"bytes_avail":287974268928,"bytes_total":289910292480,"degraded_objects":36071,"degraded_total":280356,"degraded_ratio":0.12866141619940361,"misplaced_objects":3124,"misplaced_total":280356,"misplaced_ratio":0.011142975359899556,"recovering_objects_per_sec":16,"recovering_bytes_per_sec":66513,"recovering_keys_per_sec":0,"num_objects_recovered":195,"num_bytes_recovered":798720,"num_keys_recovered":0,"write_bytes_sec":1649540,"read_op_per_sec":0,"write_op_per_sec":402},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:32:37.661 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:32:37.661 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:32:38.071 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 190 20 142769 142749 2.93417 1.73438 0.0335102 0.0266176 2024-09-10T20:32:38.220 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:38.221 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:32:38.248 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":173,"stamp":"2024-09-10T20:32:36.888230+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":574150076,"num_objects":140178,"num_object_clones":0,"num_object_copies":280356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":3124,"num_objects_unfound":0,"num_objects_dirty":140178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104553,"num_write_kb":418186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":16171027,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140258,"ondisk_log_size":140258,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1890648,"kb_used_data":1137576,"kb_used_omap":12,"kb_used_meta":753011,"kb_avail":281224872,"statfs":{"total":289910292480,"available":287974268928,"internally_reserved":0,"allocated":1164877824,"data_stored":1164077964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":771083266},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,7,10,10,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19808256,"num_objects":4836,"num_object_clones":0,"num_object_copies":9672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-195,"num_objects_unfound":0,"num_objects_dirty":4836,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4836,"num_write_kb":19344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":195,"num_bytes_recovered":798720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008351"},"pg_stats":[{"pgid":"2.f","version":"45'8743","reported_seq":8820,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:32.027635+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:32:32.027635+0000","last_peered":"2024-09-10T20:32:32.027635+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:32:32.027635+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":8743,"log_dups_size":0,"ondisk_log_size":8743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35811328,"num_objects":8743,"num_object_clones":0,"num_object_copies":17486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8743,"num_write_kb":34972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5136},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'8931","reported_seq":9041,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:33.415668+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:32:33.415668+0000","last_peered":"2024-09-10T20:32:33.415668+0000","last_clean":"2024-09-10T20:32:33.415668+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:32:33.415668+0000","last_undegraded":"2024-09-10T20:32:33.415668+0000","last_fullsized":"2024-09-10T20:32:33.415668+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":8931,"log_dups_size":0,"ondisk_log_size":8931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36556800,"num_objects":8925,"num_object_clones":0,"num_object_copies":17850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7117,"num_write_kb":28468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'8811","reported_seq":16158,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:33.450361+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:32:33.450361+0000","last_peered":"2024-09-10T20:32:33.450361+0000","last_clean":"2024-09-10T20:32:33.450361+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:32:33.450361+0000","last_undegraded":"2024-09-10T20:32:33.450361+0000","last_fullsized":"2024-09-10T20:32:33.450361+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":8811,"log_dups_size":0,"ondisk_log_size":8811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36073472,"num_objects":8807,"num_object_clones":0,"num_object_copies":17614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6939,"num_write_kb":27756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'8668","reported_seq":8748,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:33.456874+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:32:33.456874+0000","last_peered":"2024-09-10T20:32:33.456874+0000","last_clean":"2024-09-10T20:32:33.456874+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:32:33.456874+0000","last_undegraded":"2024-09-10T20:32:33.456874+0000","last_fullsized":"2024-09-10T20:32:33.456874+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":8668,"log_dups_size":0,"ondisk_log_size":8668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35504128,"num_objects":8668,"num_object_clones":0,"num_object_copies":17336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8668,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8668,"num_write_kb":34672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'8913","reported_seq":9039,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:32.049684+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:32:32.049684+0000","last_peered":"2024-09-10T20:32:32.049684+0000","last_clean":"2024-09-10T20:32:32.049684+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:32:32.049684+0000","last_undegraded":"2024-09-10T20:32:32.049684+0000","last_fullsized":"2024-09-10T20:32:32.049684+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":8913,"log_dups_size":0,"ondisk_log_size":8913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36478976,"num_objects":8906,"num_object_clones":0,"num_object_copies":17812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5909,"num_write_kb":23636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'8622","reported_seq":8758,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:31.940835+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:32:31.940835+0000","last_peered":"2024-09-10T20:32:31.940835+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:32:31.940835+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":8622,"log_dups_size":0,"ondisk_log_size":8622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35295232,"num_objects":8617,"num_object_clones":0,"num_object_copies":17234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5486,"num_write_kb":21944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5003},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'8700","reported_seq":8838,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:32.016535+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:32:32.016535+0000","last_peered":"2024-09-10T20:32:32.016535+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:32:32.016535+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":8700,"log_dups_size":0,"ondisk_log_size":8700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35610624,"num_objects":8694,"num_object_clones":0,"num_object_copies":17388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5295,"num_write_kb":21180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5139},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'8774","reported_seq":8898,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:31.924954+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:32:31.924954+0000","last_peered":"2024-09-10T20:32:31.924954+0000","last_clean":"2024-09-10T20:32:31.924954+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:32:31.924954+0000","last_undegraded":"2024-09-10T20:32:31.924954+0000","last_fullsized":"2024-09-10T20:32:31.924954+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":8774,"log_dups_size":0,"ondisk_log_size":8774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35930112,"num_objects":8772,"num_object_clones":0,"num_object_copies":17544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5845,"num_write_kb":23380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8668","reported_seq":8798,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:33.477352+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:32:33.477352+0000","last_peered":"2024-09-10T20:32:33.477352+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:32:33.477352+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":8668,"log_dups_size":0,"ondisk_log_size":8668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35475456,"num_objects":8661,"num_object_clones":0,"num_object_copies":17322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6661,"num_write_kb":26644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":5135}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190232+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190232+0000","last_peered":"2024-09-10T20:32:18.190232+0000","last_clean":"2024-09-10T20:32:18.190232+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190232+0000","last_undegraded":"2024-09-10T20:32:18.190232+0000","last_fullsized":"2024-09-10T20:32:18.190232+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8800","reported_seq":8908,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:31.947676+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:32:31.947676+0000","last_peered":"2024-09-10T20:32:31.947676+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:32:31.947676+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":8800,"log_dups_size":0,"ondisk_log_size":8800,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36012998,"num_objects":8794,"num_object_clones":0,"num_object_copies":17588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8794,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8798,"num_write_kb":35172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5179},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182737+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182737+0000","last_peered":"2024-09-10T20:32:18.182737+0000","last_clean":"2024-09-10T20:32:18.182737+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182737+0000","last_undegraded":"2024-09-10T20:32:18.182737+0000","last_fullsized":"2024-09-10T20:32:18.182737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8747","reported_seq":8875,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:33.413764+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:32:33.413764+0000","last_peered":"2024-09-10T20:32:33.413764+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:32:33.413764+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":8747,"log_dups_size":0,"ondisk_log_size":8747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35807232,"num_objects":8742,"num_object_clones":0,"num_object_copies":17484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6575,"num_write_kb":26300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":5109}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8827","reported_seq":8955,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:33.442226+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:32:33.442226+0000","last_peered":"2024-09-10T20:32:33.442226+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:32:33.442226+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":8827,"log_dups_size":0,"ondisk_log_size":8827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36130816,"num_objects":8821,"num_object_clones":0,"num_object_copies":17642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8821,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6613,"num_write_kb":26452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":5211}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'8799","reported_seq":8931,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:32.047763+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:32:32.047763+0000","last_peered":"2024-09-10T20:32:32.047763+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:32:32.047763+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":8799,"log_dups_size":0,"ondisk_log_size":8799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36032512,"num_objects":8797,"num_object_clones":0,"num_object_copies":17594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5522,"num_write_kb":22088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5149},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'8709","reported_seq":8846,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:32.054171+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:32:32.054171+0000","last_peered":"2024-09-10T20:32:32.054171+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:32:32.054171+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":8709,"log_dups_size":0,"ondisk_log_size":8709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35639296,"num_objects":8701,"num_object_clones":0,"num_object_copies":17402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8701,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5403,"num_write_kb":21612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5169},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193200+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193200+0000","last_peered":"2024-09-10T20:32:18.193200+0000","last_clean":"2024-09-10T20:32:18.193200+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193200+0000","last_undegraded":"2024-09-10T20:32:18.193200+0000","last_fullsized":"2024-09-10T20:32:18.193200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'8908","reported_seq":9038,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:32.048685+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:32:32.048685+0000","last_peered":"2024-09-10T20:32:32.048685+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:32:32.048685+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":8908,"log_dups_size":0,"ondisk_log_size":8908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36470784,"num_objects":8904,"num_object_clones":0,"num_object_copies":17808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5735,"num_write_kb":22940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5191},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'8635","reported_seq":9489,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:32:32.070456+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:32:32.070456+0000","last_peered":"2024-09-10T20:32:32.070456+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:32:32.070456+0000","last_undegraded":"2024-09-10T20:32:32.070456+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":8635,"log_dups_size":0,"ondisk_log_size":8635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35320291,"num_objects":8624,"num_object_clones":0,"num_object_copies":17248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3124,"num_objects_unfound":0,"num_objects_dirty":8624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5242,"num_write_kb":20968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":1409024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5500},{"shards":"0,2","objects":3124}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":574150057,"num_objects":140176,"num_object_clones":0,"num_object_copies":280352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":3124,"num_objects_unfound":0,"num_objects_dirty":140176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104551,"num_write_kb":418184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":16171008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1166049280,"data_stored":1165962568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140255,"ondisk_log_size":140255,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182448,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":788492,"kb_used_data":477820,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93583348,"statfs":{"total":96636764160,"available":95829348352,"internally_reserved":0,"allocated":489287680,"data_stored":489044992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,4,4,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986220,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":644704,"kb_used_data":396432,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93727136,"statfs":{"total":96636764160,"available":95976587264,"internally_reserved":0,"allocated":405946368,"data_stored":405707757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":254209706},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659167,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":457452,"kb_used_data":263324,"kb_used_omap":4,"kb_used_meta":194107,"kb_avail":93914388,"statfs":{"total":96636764160,"available":96168333312,"internally_reserved":0,"allocated":269643776,"data_stored":269325215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":198766254},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":489197568,"data_stored":489193955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":270766080,"data_stored":270686594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":406085632,"data_stored":406082019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:32:39.071 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 191 20 143260 143240 2.92885 1.91797 0.0226131 0.0266675 2024-09-10T20:32:40.071 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 192 20 143761 143741 2.92379 1.95703 0.0523329 0.0267131 2024-09-10T20:32:41.071 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 193 20 144260 144240 2.91874 1.94922 0.0229139 0.0267583 2024-09-10T20:32:41.250 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:32:41.844 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:41.847 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:32:41.870 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":175,"stamp":"2024-09-10T20:32:40.889841+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":583935420,"num_objects":142567,"num_object_clones":0,"num_object_copies":285134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":3027,"num_objects_unfound":0,"num_objects_dirty":142567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106942,"num_write_kb":427742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4050,"num_bytes_recovered":16568339,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142647,"ondisk_log_size":142647,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1907436,"kb_used_data":1158524,"kb_used_omap":12,"kb_used_meta":748851,"kb_avail":281208084,"statfs":{"total":289910292480,"available":287957078016,"internally_reserved":0,"allocated":1186328576,"data_stored":1185528716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":766823426},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,6,3,3,8,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19611648,"num_objects":4788,"num_object_clones":0,"num_object_copies":9576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-206,"num_objects_unfound":0,"num_objects_dirty":4788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4788,"num_write_kb":19152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":843776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008460"},"pg_stats":[{"pgid":"2.f","version":"45'8895","reported_seq":8975,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:37.079955+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:32:37.079955+0000","last_peered":"2024-09-10T20:32:37.079955+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:32:37.079955+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":8895,"log_dups_size":0,"ondisk_log_size":8895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36433920,"num_objects":8895,"num_object_clones":0,"num_object_copies":17790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8895,"num_write_kb":35580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5288},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9079","reported_seq":9188,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:38.446859+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:32:38.446859+0000","last_peered":"2024-09-10T20:32:38.446859+0000","last_clean":"2024-09-10T20:32:38.446859+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:32:38.446859+0000","last_undegraded":"2024-09-10T20:32:38.446859+0000","last_fullsized":"2024-09-10T20:32:38.446859+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":9079,"log_dups_size":0,"ondisk_log_size":9079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37163008,"num_objects":9073,"num_object_clones":0,"num_object_copies":18146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7265,"num_write_kb":29060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'8950","reported_seq":16297,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:38.499797+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:32:38.499797+0000","last_peered":"2024-09-10T20:32:38.499797+0000","last_clean":"2024-09-10T20:32:38.499797+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:32:38.499797+0000","last_undegraded":"2024-09-10T20:32:38.499797+0000","last_fullsized":"2024-09-10T20:32:38.499797+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":8950,"log_dups_size":0,"ondisk_log_size":8950,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36642816,"num_objects":8946,"num_object_clones":0,"num_object_copies":17892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7078,"num_write_kb":28312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'8829","reported_seq":8909,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:38.367832+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:32:38.367832+0000","last_peered":"2024-09-10T20:32:38.367832+0000","last_clean":"2024-09-10T20:32:38.367832+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:32:38.367832+0000","last_undegraded":"2024-09-10T20:32:38.367832+0000","last_fullsized":"2024-09-10T20:32:38.367832+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":8829,"log_dups_size":0,"ondisk_log_size":8829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36163584,"num_objects":8829,"num_object_clones":0,"num_object_copies":17658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8829,"num_write_kb":35316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9058","reported_seq":9184,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:37.064950+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:32:37.064950+0000","last_peered":"2024-09-10T20:32:37.064950+0000","last_clean":"2024-09-10T20:32:37.064950+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:32:37.064950+0000","last_undegraded":"2024-09-10T20:32:37.064950+0000","last_fullsized":"2024-09-10T20:32:37.064950+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":9058,"log_dups_size":0,"ondisk_log_size":9058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37072896,"num_objects":9051,"num_object_clones":0,"num_object_copies":18102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6054,"num_write_kb":24216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'8776","reported_seq":8912,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:36.892342+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:32:36.892342+0000","last_peered":"2024-09-10T20:32:36.892342+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:32:36.892342+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":8776,"log_dups_size":0,"ondisk_log_size":8776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35926016,"num_objects":8771,"num_object_clones":0,"num_object_copies":17542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5640,"num_write_kb":22560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5157},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'8867","reported_seq":9005,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:37.047704+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:32:37.047704+0000","last_peered":"2024-09-10T20:32:37.047704+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:32:37.047704+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":8867,"log_dups_size":0,"ondisk_log_size":8867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36294656,"num_objects":8861,"num_object_clones":0,"num_object_copies":17722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5462,"num_write_kb":21848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5306},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'8937","reported_seq":9061,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:37.018265+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:32:37.018265+0000","last_peered":"2024-09-10T20:32:37.018265+0000","last_clean":"2024-09-10T20:32:37.018265+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:32:37.018265+0000","last_undegraded":"2024-09-10T20:32:37.018265+0000","last_fullsized":"2024-09-10T20:32:37.018265+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":8937,"log_dups_size":0,"ondisk_log_size":8937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36597760,"num_objects":8935,"num_object_clones":0,"num_object_copies":17870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6008,"num_write_kb":24032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8819","reported_seq":8951,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:38.426556+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:32:38.426556+0000","last_peered":"2024-09-10T20:32:38.426556+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:32:38.426556+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":8819,"log_dups_size":0,"ondisk_log_size":8819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36093952,"num_objects":8812,"num_object_clones":0,"num_object_copies":17624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8812,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6812,"num_write_kb":27248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":5286}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190232+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190232+0000","last_peered":"2024-09-10T20:32:18.190232+0000","last_clean":"2024-09-10T20:32:18.190232+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190232+0000","last_undegraded":"2024-09-10T20:32:18.190232+0000","last_fullsized":"2024-09-10T20:32:18.190232+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8941","reported_seq":9049,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:36.922156+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:32:36.922156+0000","last_peered":"2024-09-10T20:32:36.922156+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:32:36.922156+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":8941,"log_dups_size":0,"ondisk_log_size":8941,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36590534,"num_objects":8935,"num_object_clones":0,"num_object_copies":17870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8939,"num_write_kb":35736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5320},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182737+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182737+0000","last_peered":"2024-09-10T20:32:18.182737+0000","last_clean":"2024-09-10T20:32:18.182737+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182737+0000","last_undegraded":"2024-09-10T20:32:18.182737+0000","last_fullsized":"2024-09-10T20:32:18.182737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8892","reported_seq":9020,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:38.439449+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:32:38.439449+0000","last_peered":"2024-09-10T20:32:38.439449+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:32:38.439449+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":8892,"log_dups_size":0,"ondisk_log_size":8892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36401152,"num_objects":8887,"num_object_clones":0,"num_object_copies":17774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6720,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":5254}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8971","reported_seq":9099,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:38.422834+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:32:38.422834+0000","last_peered":"2024-09-10T20:32:38.422834+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:32:38.422834+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":8971,"log_dups_size":0,"ondisk_log_size":8971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36720640,"num_objects":8965,"num_object_clones":0,"num_object_copies":17930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8965,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6757,"num_write_kb":27028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":5355}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'8948","reported_seq":9083,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:36.940530+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:32:36.940530+0000","last_peered":"2024-09-10T20:32:36.940530+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:32:36.940530+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":8948,"log_dups_size":0,"ondisk_log_size":8948,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36642816,"num_objects":8946,"num_object_clones":0,"num_object_copies":17892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5671,"num_write_kb":22684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5298},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'8858","reported_seq":8994,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:37.035661+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:32:37.035661+0000","last_peered":"2024-09-10T20:32:37.035661+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:32:37.035661+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":8858,"log_dups_size":0,"ondisk_log_size":8858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36249600,"num_objects":8850,"num_object_clones":0,"num_object_copies":17700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5552,"num_write_kb":22208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5318},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193200+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193200+0000","last_peered":"2024-09-10T20:32:18.193200+0000","last_clean":"2024-09-10T20:32:18.193200+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193200+0000","last_undegraded":"2024-09-10T20:32:18.193200+0000","last_fullsized":"2024-09-10T20:32:18.193200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9043","reported_seq":9174,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:37.053712+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:32:37.053712+0000","last_peered":"2024-09-10T20:32:37.053712+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:32:37.053712+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":9043,"log_dups_size":0,"ondisk_log_size":9043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37023744,"num_objects":9039,"num_object_clones":0,"num_object_copies":18078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5870,"num_write_kb":23480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5326},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'8781","reported_seq":9829,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:32:37.055781+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:32:37.055781+0000","last_peered":"2024-09-10T20:32:37.055781+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:32:37.055781+0000","last_undegraded":"2024-09-10T20:32:37.055781+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":8781,"log_dups_size":0,"ondisk_log_size":8781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35918307,"num_objects":8770,"num_object_clones":0,"num_object_copies":17540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3027,"num_objects_unfound":0,"num_objects_dirty":8770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5388,"num_write_kb":21552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":441,"num_bytes_recovered":1806336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5743},{"shards":"0,2","objects":3027}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":583935401,"num_objects":142565,"num_object_clones":0,"num_object_copies":285130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":3027,"num_objects_unfound":0,"num_objects_dirty":142565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106940,"num_write_kb":427740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4048,"num_bytes_recovered":16568320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1186070528,"data_stored":1185983816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142644,"ondisk_log_size":142644,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182449,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":783328,"kb_used_data":485136,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93588512,"statfs":{"total":96636764160,"available":95834636288,"internally_reserved":0,"allocated":496779264,"data_stored":496536576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305327786},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986221,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":653688,"kb_used_data":401256,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93718152,"statfs":{"total":96636764160,"available":95967387648,"internally_reserved":0,"allocated":410886144,"data_stored":410647533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":258469546},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,0,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659168,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":470420,"kb_used_data":272132,"kb_used_omap":4,"kb_used_meta":198267,"kb_avail":93901420,"statfs":{"total":96636764160,"available":96155054080,"internally_reserved":0,"allocated":278663168,"data_stored":278344607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":203026094},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":496635904,"data_stored":496632291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":278360064,"data_stored":278280578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":411074560,"data_stored":411070947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:32:41.871 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:32:42.071 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 194 20 144711 144691 2.91277 1.76172 0.0936706 0.0268144 2024-09-10T20:32:42.422 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:42.422 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:32:42.448 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":175,"stamp":"2024-09-10T20:32:40.889841+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":583935420,"num_objects":142567,"num_object_clones":0,"num_object_copies":285134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":3027,"num_objects_unfound":0,"num_objects_dirty":142567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106942,"num_write_kb":427742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4050,"num_bytes_recovered":16568339,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142647,"ondisk_log_size":142647,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1907436,"kb_used_data":1158524,"kb_used_omap":12,"kb_used_meta":748851,"kb_avail":281208084,"statfs":{"total":289910292480,"available":287957078016,"internally_reserved":0,"allocated":1186328576,"data_stored":1185528716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":766823426},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,6,3,3,8,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19611648,"num_objects":4788,"num_object_clones":0,"num_object_copies":9576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-206,"num_objects_unfound":0,"num_objects_dirty":4788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4788,"num_write_kb":19152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":843776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008460"},"pg_stats":[{"pgid":"2.f","version":"45'8895","reported_seq":8975,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:37.079955+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:32:37.079955+0000","last_peered":"2024-09-10T20:32:37.079955+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:32:37.079955+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":8895,"log_dups_size":0,"ondisk_log_size":8895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36433920,"num_objects":8895,"num_object_clones":0,"num_object_copies":17790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8895,"num_write_kb":35580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5288},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9079","reported_seq":9188,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:38.446859+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:32:38.446859+0000","last_peered":"2024-09-10T20:32:38.446859+0000","last_clean":"2024-09-10T20:32:38.446859+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:32:38.446859+0000","last_undegraded":"2024-09-10T20:32:38.446859+0000","last_fullsized":"2024-09-10T20:32:38.446859+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":9079,"log_dups_size":0,"ondisk_log_size":9079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37163008,"num_objects":9073,"num_object_clones":0,"num_object_copies":18146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7265,"num_write_kb":29060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'8950","reported_seq":16297,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:38.499797+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:32:38.499797+0000","last_peered":"2024-09-10T20:32:38.499797+0000","last_clean":"2024-09-10T20:32:38.499797+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:32:38.499797+0000","last_undegraded":"2024-09-10T20:32:38.499797+0000","last_fullsized":"2024-09-10T20:32:38.499797+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":8950,"log_dups_size":0,"ondisk_log_size":8950,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36642816,"num_objects":8946,"num_object_clones":0,"num_object_copies":17892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7078,"num_write_kb":28312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'8829","reported_seq":8909,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:38.367832+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:32:38.367832+0000","last_peered":"2024-09-10T20:32:38.367832+0000","last_clean":"2024-09-10T20:32:38.367832+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:32:38.367832+0000","last_undegraded":"2024-09-10T20:32:38.367832+0000","last_fullsized":"2024-09-10T20:32:38.367832+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":8829,"log_dups_size":0,"ondisk_log_size":8829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36163584,"num_objects":8829,"num_object_clones":0,"num_object_copies":17658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8829,"num_write_kb":35316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9058","reported_seq":9184,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:37.064950+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:32:37.064950+0000","last_peered":"2024-09-10T20:32:37.064950+0000","last_clean":"2024-09-10T20:32:37.064950+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:32:37.064950+0000","last_undegraded":"2024-09-10T20:32:37.064950+0000","last_fullsized":"2024-09-10T20:32:37.064950+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":9058,"log_dups_size":0,"ondisk_log_size":9058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37072896,"num_objects":9051,"num_object_clones":0,"num_object_copies":18102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6054,"num_write_kb":24216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'8776","reported_seq":8912,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:36.892342+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:32:36.892342+0000","last_peered":"2024-09-10T20:32:36.892342+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:32:36.892342+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":8776,"log_dups_size":0,"ondisk_log_size":8776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35926016,"num_objects":8771,"num_object_clones":0,"num_object_copies":17542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5640,"num_write_kb":22560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5157},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'8867","reported_seq":9005,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:37.047704+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:32:37.047704+0000","last_peered":"2024-09-10T20:32:37.047704+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:32:37.047704+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":8867,"log_dups_size":0,"ondisk_log_size":8867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36294656,"num_objects":8861,"num_object_clones":0,"num_object_copies":17722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5462,"num_write_kb":21848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5306},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'8937","reported_seq":9061,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:37.018265+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:32:37.018265+0000","last_peered":"2024-09-10T20:32:37.018265+0000","last_clean":"2024-09-10T20:32:37.018265+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:32:37.018265+0000","last_undegraded":"2024-09-10T20:32:37.018265+0000","last_fullsized":"2024-09-10T20:32:37.018265+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":8937,"log_dups_size":0,"ondisk_log_size":8937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36597760,"num_objects":8935,"num_object_clones":0,"num_object_copies":17870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6008,"num_write_kb":24032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8819","reported_seq":8951,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:38.426556+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:32:38.426556+0000","last_peered":"2024-09-10T20:32:38.426556+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:32:38.426556+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":8819,"log_dups_size":0,"ondisk_log_size":8819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36093952,"num_objects":8812,"num_object_clones":0,"num_object_copies":17624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8812,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6812,"num_write_kb":27248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":5286}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190232+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190232+0000","last_peered":"2024-09-10T20:32:18.190232+0000","last_clean":"2024-09-10T20:32:18.190232+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190232+0000","last_undegraded":"2024-09-10T20:32:18.190232+0000","last_fullsized":"2024-09-10T20:32:18.190232+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8941","reported_seq":9049,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:36.922156+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:32:36.922156+0000","last_peered":"2024-09-10T20:32:36.922156+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:32:36.922156+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":8941,"log_dups_size":0,"ondisk_log_size":8941,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36590534,"num_objects":8935,"num_object_clones":0,"num_object_copies":17870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8939,"num_write_kb":35736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5320},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182737+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182737+0000","last_peered":"2024-09-10T20:32:18.182737+0000","last_clean":"2024-09-10T20:32:18.182737+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182737+0000","last_undegraded":"2024-09-10T20:32:18.182737+0000","last_fullsized":"2024-09-10T20:32:18.182737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8892","reported_seq":9020,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:38.439449+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:32:38.439449+0000","last_peered":"2024-09-10T20:32:38.439449+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:32:38.439449+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":8892,"log_dups_size":0,"ondisk_log_size":8892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36401152,"num_objects":8887,"num_object_clones":0,"num_object_copies":17774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6720,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":5254}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8971","reported_seq":9099,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:38.422834+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:32:38.422834+0000","last_peered":"2024-09-10T20:32:38.422834+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:32:38.422834+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":8971,"log_dups_size":0,"ondisk_log_size":8971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36720640,"num_objects":8965,"num_object_clones":0,"num_object_copies":17930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8965,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6757,"num_write_kb":27028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":5355}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'8948","reported_seq":9083,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:36.940530+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:32:36.940530+0000","last_peered":"2024-09-10T20:32:36.940530+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:32:36.940530+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":8948,"log_dups_size":0,"ondisk_log_size":8948,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36642816,"num_objects":8946,"num_object_clones":0,"num_object_copies":17892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5671,"num_write_kb":22684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5298},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'8858","reported_seq":8994,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:37.035661+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:32:37.035661+0000","last_peered":"2024-09-10T20:32:37.035661+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:32:37.035661+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":8858,"log_dups_size":0,"ondisk_log_size":8858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36249600,"num_objects":8850,"num_object_clones":0,"num_object_copies":17700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5552,"num_write_kb":22208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5318},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193200+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193200+0000","last_peered":"2024-09-10T20:32:18.193200+0000","last_clean":"2024-09-10T20:32:18.193200+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193200+0000","last_undegraded":"2024-09-10T20:32:18.193200+0000","last_fullsized":"2024-09-10T20:32:18.193200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9043","reported_seq":9174,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:37.053712+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:32:37.053712+0000","last_peered":"2024-09-10T20:32:37.053712+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:32:37.053712+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":9043,"log_dups_size":0,"ondisk_log_size":9043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37023744,"num_objects":9039,"num_object_clones":0,"num_object_copies":18078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5870,"num_write_kb":23480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5326},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'8781","reported_seq":9829,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:32:37.055781+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:32:37.055781+0000","last_peered":"2024-09-10T20:32:37.055781+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:32:37.055781+0000","last_undegraded":"2024-09-10T20:32:37.055781+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":8781,"log_dups_size":0,"ondisk_log_size":8781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35918307,"num_objects":8770,"num_object_clones":0,"num_object_copies":17540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3027,"num_objects_unfound":0,"num_objects_dirty":8770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5388,"num_write_kb":21552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":441,"num_bytes_recovered":1806336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5743},{"shards":"0,2","objects":3027}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":583935401,"num_objects":142565,"num_object_clones":0,"num_object_copies":285130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":3027,"num_objects_unfound":0,"num_objects_dirty":142565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106940,"num_write_kb":427740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4048,"num_bytes_recovered":16568320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1186070528,"data_stored":1185983816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142644,"ondisk_log_size":142644,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182449,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":783328,"kb_used_data":485136,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93588512,"statfs":{"total":96636764160,"available":95834636288,"internally_reserved":0,"allocated":496779264,"data_stored":496536576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305327786},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986221,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":653688,"kb_used_data":401256,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93718152,"statfs":{"total":96636764160,"available":95967387648,"internally_reserved":0,"allocated":410886144,"data_stored":410647533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":258469546},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,0,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659168,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":470420,"kb_used_data":272132,"kb_used_omap":4,"kb_used_meta":198267,"kb_avail":93901420,"statfs":{"total":96636764160,"available":96155054080,"internally_reserved":0,"allocated":278663168,"data_stored":278344607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":203026094},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":496635904,"data_stored":496632291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":278360064,"data_stored":278280578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":411074560,"data_stored":411070947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:32:42.449 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:32:42.449 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'8895", 'reported_seq': 8975, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:37.079955+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:32:37.079955+0000', 'last_peered': '2024-09-10T20:32:37.079955+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:32:37.079955+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 8895, 'log_dups_size': 0, 'ondisk_log_size': 8895, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36433920, 'num_objects': 8895, 'num_object_clones': 0, 'num_object_copies': 17790, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3607, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8895, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8895, 'num_write_kb': 35580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5288}, {'shards': '0,2', 'objects': 3607}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:42.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:32:42.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'8776", 'reported_seq': 8912, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:36.892342+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:32:36.892342+0000', 'last_peered': '2024-09-10T20:32:36.892342+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:32:36.892342+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 8776, 'log_dups_size': 0, 'ondisk_log_size': 8776, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35926016, 'num_objects': 8771, 'num_object_clones': 0, 'num_object_copies': 17542, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3616, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8771, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5640, 'num_write_kb': 22560, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5157}, {'shards': '0,2', 'objects': 3614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:42.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:32:42.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'8867", 'reported_seq': 9005, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:37.047704+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:32:37.047704+0000', 'last_peered': '2024-09-10T20:32:37.047704+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:32:37.047704+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 8867, 'log_dups_size': 0, 'ondisk_log_size': 8867, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36294656, 'num_objects': 8861, 'num_object_clones': 0, 'num_object_copies': 17722, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3557, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8861, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5462, 'num_write_kb': 21848, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5306}, {'shards': '0,2', 'objects': 3555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:42.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:32:42.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'8819", 'reported_seq': 8951, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:38.426556+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:32:38.426556+0000', 'last_peered': '2024-09-10T20:32:38.426556+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:32:38.426556+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 8819, 'log_dups_size': 0, 'ondisk_log_size': 8819, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36093952, 'num_objects': 8812, 'num_object_clones': 0, 'num_object_copies': 17624, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3528, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8812, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6812, 'num_write_kb': 27248, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 5286}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:32:42.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:32:42.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'8941", 'reported_seq': 9049, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:36.922156+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:32:36.922156+0000', 'last_peered': '2024-09-10T20:32:36.922156+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:32:36.922156+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 8941, 'log_dups_size': 0, 'ondisk_log_size': 8941, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36590534, 'num_objects': 8935, 'num_object_clones': 0, 'num_object_copies': 17870, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3617, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8935, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8939, 'num_write_kb': 35736, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5320}, {'shards': '0,2', 'objects': 3615}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:42.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:32:42.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'8892", 'reported_seq': 9020, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:38.439449+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:32:38.439449+0000', 'last_peered': '2024-09-10T20:32:38.439449+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:32:38.439449+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 8892, 'log_dups_size': 0, 'ondisk_log_size': 8892, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36401152, 'num_objects': 8887, 'num_object_clones': 0, 'num_object_copies': 17774, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3635, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8887, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6720, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3633}, {'shards': '2', 'objects': 5254}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:32:42.451 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:32:42.451 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'8971", 'reported_seq': 9099, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:38.422834+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:32:38.422834+0000', 'last_peered': '2024-09-10T20:32:38.422834+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:32:38.422834+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 8971, 'log_dups_size': 0, 'ondisk_log_size': 8971, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36720640, 'num_objects': 8965, 'num_object_clones': 0, 'num_object_copies': 17930, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8965, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6757, 'num_write_kb': 27028, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3610}, {'shards': '2', 'objects': 5355}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:32:42.451 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:32:42.451 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'8948", 'reported_seq': 9083, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:36.940530+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:32:36.940530+0000', 'last_peered': '2024-09-10T20:32:36.940530+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:32:36.940530+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 8948, 'log_dups_size': 0, 'ondisk_log_size': 8948, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36642816, 'num_objects': 8946, 'num_object_clones': 0, 'num_object_copies': 17892, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3649, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8946, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5671, 'num_write_kb': 22684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5298}, {'shards': '0,2', 'objects': 3648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:42.451 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:32:42.451 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'8858", 'reported_seq': 8994, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:37.035661+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:32:37.035661+0000', 'last_peered': '2024-09-10T20:32:37.035661+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:32:37.035661+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 8858, 'log_dups_size': 0, 'ondisk_log_size': 8858, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36249600, 'num_objects': 8850, 'num_object_clones': 0, 'num_object_copies': 17700, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3535, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8850, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5552, 'num_write_kb': 22208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5318}, {'shards': '0,2', 'objects': 3532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:42.451 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:32:42.451 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'9043", 'reported_seq': 9174, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:37.053712+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:32:37.053712+0000', 'last_peered': '2024-09-10T20:32:37.053712+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:32:37.053712+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 9043, 'log_dups_size': 0, 'ondisk_log_size': 9043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37023744, 'num_objects': 9039, 'num_object_clones': 0, 'num_object_copies': 18078, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3715, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9039, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5870, 'num_write_kb': 23480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5326}, {'shards': '0,2', 'objects': 3713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:42.451 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:32:42.451 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'8781", 'reported_seq': 9829, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:32:37.055781+0000', 'last_change': '2024-09-10T20:32:16.657652+0000', 'last_active': '2024-09-10T20:32:37.055781+0000', 'last_peered': '2024-09-10T20:32:37.055781+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:32:37.055781+0000', 'last_undegraded': '2024-09-10T20:32:37.055781+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 8781, 'log_dups_size': 0, 'ondisk_log_size': 8781, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35918307, 'num_objects': 8770, 'num_object_clones': 0, 'num_object_copies': 17540, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3027, 'num_objects_unfound': 0, 'num_objects_dirty': 8770, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5388, 'num_write_kb': 21552, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 441, 'num_bytes_recovered': 1806336, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5743}, {'shards': '0,2', 'objects': 3027}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:42.452 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:32:43.071 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 195 19 145148 145129 2.90661 1.71094 0.0789348 0.0268705 2024-09-10T20:32:43.156 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:43.157 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36071/288150 objects degraded (12.518%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":274,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":144075,"data_bytes":590112188,"bytes_used":1964896256,"bytes_avail":287945396224,"bytes_total":289910292480,"degraded_objects":36071,"degraded_total":288150,"degraded_ratio":0.12518132916883568,"misplaced_objects":2931,"misplaced_total":288150,"misplaced_ratio":0.01017178552837064,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":103029,"recovering_keys_per_sec":0,"num_objects_recovered":302,"num_bytes_recovered":1236992,"num_keys_recovered":0,"write_bytes_sec":2147930,"read_op_per_sec":0,"write_op_per_sec":524},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:32:43.187 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:32:43.187 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:32:43.735 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:43.735 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:32:43.757 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":176,"stamp":"2024-09-10T20:32:42.891935+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590112188,"num_objects":144075,"num_object_clones":0,"num_object_copies":288150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2931,"num_objects_unfound":0,"num_objects_dirty":144075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108450,"num_write_kb":433774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4146,"num_bytes_recovered":16961555,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144155,"ondisk_log_size":144155,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1918844,"kb_used_data":1165772,"kb_used_omap":12,"kb_used_meta":753011,"kb_avail":281196676,"statfs":{"total":289910292480,"available":287945396224,"internally_reserved":0,"allocated":1193750528,"data_stored":1192950668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":771083266},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,6,2,3,14,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25788416,"num_objects":6296,"num_object_clones":0,"num_object_copies":12592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-302,"num_objects_unfound":0,"num_objects_dirty":6296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6296,"num_write_kb":25184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":1236992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006168"},"pg_stats":[{"pgid":"2.f","version":"45'9030","reported_seq":9109,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:42.132254+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:32:42.132254+0000","last_peered":"2024-09-10T20:32:42.132254+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:32:42.132254+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":9030,"log_dups_size":0,"ondisk_log_size":9030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36986880,"num_objects":9030,"num_object_clones":0,"num_object_copies":18060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9030,"num_write_kb":36120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5423},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9079","reported_seq":9188,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:38.446859+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:32:38.446859+0000","last_peered":"2024-09-10T20:32:38.446859+0000","last_clean":"2024-09-10T20:32:38.446859+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:32:38.446859+0000","last_undegraded":"2024-09-10T20:32:38.446859+0000","last_fullsized":"2024-09-10T20:32:38.446859+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":9079,"log_dups_size":0,"ondisk_log_size":9079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37163008,"num_objects":9073,"num_object_clones":0,"num_object_copies":18146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7265,"num_write_kb":29060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'8950","reported_seq":16297,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:38.499797+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:32:38.499797+0000","last_peered":"2024-09-10T20:32:38.499797+0000","last_clean":"2024-09-10T20:32:38.499797+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:32:38.499797+0000","last_undegraded":"2024-09-10T20:32:38.499797+0000","last_fullsized":"2024-09-10T20:32:38.499797+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":8950,"log_dups_size":0,"ondisk_log_size":8950,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36642816,"num_objects":8946,"num_object_clones":0,"num_object_copies":17892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7078,"num_write_kb":28312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'8829","reported_seq":8909,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:38.367832+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:32:38.367832+0000","last_peered":"2024-09-10T20:32:38.367832+0000","last_clean":"2024-09-10T20:32:38.367832+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:32:38.367832+0000","last_undegraded":"2024-09-10T20:32:38.367832+0000","last_fullsized":"2024-09-10T20:32:38.367832+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":8829,"log_dups_size":0,"ondisk_log_size":8829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36163584,"num_objects":8829,"num_object_clones":0,"num_object_copies":17658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8829,"num_write_kb":35316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9207","reported_seq":9332,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:42.152731+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:32:42.152731+0000","last_peered":"2024-09-10T20:32:42.152731+0000","last_clean":"2024-09-10T20:32:42.152731+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:32:42.152731+0000","last_undegraded":"2024-09-10T20:32:42.152731+0000","last_fullsized":"2024-09-10T20:32:42.152731+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":9207,"log_dups_size":0,"ondisk_log_size":9207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37683200,"num_objects":9200,"num_object_clones":0,"num_object_copies":18400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6203,"num_write_kb":24812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'8937","reported_seq":9073,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:42.078950+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:32:42.078950+0000","last_peered":"2024-09-10T20:32:42.078950+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:32:42.078950+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":8937,"log_dups_size":0,"ondisk_log_size":8937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36585472,"num_objects":8932,"num_object_clones":0,"num_object_copies":17864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8932,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5801,"num_write_kb":23204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5318},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9024","reported_seq":9162,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:42.037904+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:32:42.037904+0000","last_peered":"2024-09-10T20:32:42.037904+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:32:42.037904+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":9024,"log_dups_size":0,"ondisk_log_size":9024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36937728,"num_objects":9018,"num_object_clones":0,"num_object_copies":18036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5619,"num_write_kb":22476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5463},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9095","reported_seq":9219,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:41.952901+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:32:41.952901+0000","last_peered":"2024-09-10T20:32:41.952901+0000","last_clean":"2024-09-10T20:32:41.952901+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:32:41.952901+0000","last_undegraded":"2024-09-10T20:32:41.952901+0000","last_fullsized":"2024-09-10T20:32:41.952901+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":9095,"log_dups_size":0,"ondisk_log_size":9095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37244928,"num_objects":9093,"num_object_clones":0,"num_object_copies":18186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6166,"num_write_kb":24664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8819","reported_seq":8951,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:38.426556+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:32:38.426556+0000","last_peered":"2024-09-10T20:32:38.426556+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:32:38.426556+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":8819,"log_dups_size":0,"ondisk_log_size":8819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36093952,"num_objects":8812,"num_object_clones":0,"num_object_copies":17624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8812,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6812,"num_write_kb":27248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":5286}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190232+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190232+0000","last_peered":"2024-09-10T20:32:18.190232+0000","last_clean":"2024-09-10T20:32:18.190232+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190232+0000","last_undegraded":"2024-09-10T20:32:18.190232+0000","last_fullsized":"2024-09-10T20:32:18.190232+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9069","reported_seq":9177,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:42.058813+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:32:42.058813+0000","last_peered":"2024-09-10T20:32:42.058813+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:32:42.058813+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":9069,"log_dups_size":0,"ondisk_log_size":9069,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37114822,"num_objects":9063,"num_object_clones":0,"num_object_copies":18126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9063,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9067,"num_write_kb":36248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5448},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182737+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182737+0000","last_peered":"2024-09-10T20:32:18.182737+0000","last_clean":"2024-09-10T20:32:18.182737+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182737+0000","last_undegraded":"2024-09-10T20:32:18.182737+0000","last_fullsized":"2024-09-10T20:32:18.182737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8892","reported_seq":9020,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:38.439449+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:32:38.439449+0000","last_peered":"2024-09-10T20:32:38.439449+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:32:38.439449+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":8892,"log_dups_size":0,"ondisk_log_size":8892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36401152,"num_objects":8887,"num_object_clones":0,"num_object_copies":17774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6720,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":5254}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8971","reported_seq":9099,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:38.422834+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:32:38.422834+0000","last_peered":"2024-09-10T20:32:38.422834+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:32:38.422834+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":8971,"log_dups_size":0,"ondisk_log_size":8971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36720640,"num_objects":8965,"num_object_clones":0,"num_object_copies":17930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8965,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6757,"num_write_kb":27028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":5355}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9113","reported_seq":9245,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:42.158229+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:32:42.158229+0000","last_peered":"2024-09-10T20:32:42.158229+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:32:42.158229+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":9113,"log_dups_size":0,"ondisk_log_size":9113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37318656,"num_objects":9111,"num_object_clones":0,"num_object_copies":18222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5836,"num_write_kb":23344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5463},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9013","reported_seq":9151,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:42.098437+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:32:42.098437+0000","last_peered":"2024-09-10T20:32:42.098437+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:32:42.098437+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":9013,"log_dups_size":0,"ondisk_log_size":9013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36884480,"num_objects":9005,"num_object_clones":0,"num_object_copies":18010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5707,"num_write_kb":22828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5473},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193200+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193200+0000","last_peered":"2024-09-10T20:32:18.193200+0000","last_clean":"2024-09-10T20:32:18.193200+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193200+0000","last_undegraded":"2024-09-10T20:32:18.193200+0000","last_fullsized":"2024-09-10T20:32:18.193200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9195","reported_seq":9326,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:42.009667+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:32:42.009667+0000","last_peered":"2024-09-10T20:32:42.009667+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:32:42.009667+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":9195,"log_dups_size":0,"ondisk_log_size":9195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37646336,"num_objects":9191,"num_object_clones":0,"num_object_copies":18382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6022,"num_write_kb":24088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5478},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'8929","reported_seq":10168,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:32:42.124695+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:32:42.124695+0000","last_peered":"2024-09-10T20:32:42.124695+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:32:42.124695+0000","last_undegraded":"2024-09-10T20:32:42.124695+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":8929,"log_dups_size":0,"ondisk_log_size":8929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36524515,"num_objects":8918,"num_object_clones":0,"num_object_copies":17836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2931,"num_objects_unfound":0,"num_objects_dirty":8918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5536,"num_write_kb":22144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":537,"num_bytes_recovered":2199552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5987},{"shards":"0,2","objects":2931}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":590112169,"num_objects":144073,"num_object_clones":0,"num_object_copies":288146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2931,"num_objects_unfound":0,"num_objects_dirty":144073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108448,"num_write_kb":433772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4144,"num_bytes_recovered":16961536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1193578496,"data_stored":1193491784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144152,"ondisk_log_size":144152,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182450,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":794736,"kb_used_data":492384,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93577104,"statfs":{"total":96636764160,"available":95822954496,"internally_reserved":0,"allocated":504201216,"data_stored":503958528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,1,8,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986221,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":653688,"kb_used_data":401256,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93718152,"statfs":{"total":96636764160,"available":95967387648,"internally_reserved":0,"allocated":410886144,"data_stored":410647533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":258469546},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,0,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659168,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":470420,"kb_used_data":272132,"kb_used_omap":4,"kb_used_meta":198267,"kb_avail":93901420,"statfs":{"total":96636764160,"available":96155054080,"internally_reserved":0,"allocated":278663168,"data_stored":278344607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":203026094},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":504143872,"data_stored":504140259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":278360064,"data_stored":278280578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":411074560,"data_stored":411070947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:32:44.072 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 196 20 145601 145581 2.90079 1.76562 0.0273514 0.0269251 2024-09-10T20:32:45.072 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 197 20 146111 146091 2.89618 1.99219 0.0418074 0.0269677 2024-09-10T20:32:46.072 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 198 20 146612 146592 2.89143 1.95703 0.0218723 0.0270123 2024-09-10T20:32:46.759 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:32:47.072 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 199 20 147080 147060 2.88609 1.82812 0.048478 0.0270625 2024-09-10T20:32:47.332 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:47.332 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:32:47.357 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":178,"stamp":"2024-09-10T20:32:46.893535+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":593782204,"num_objects":144971,"num_object_clones":0,"num_object_copies":289942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2931,"num_objects_unfound":0,"num_objects_dirty":144971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109346,"num_write_kb":437358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4146,"num_bytes_recovered":16961555,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145051,"ondisk_log_size":145051,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1957560,"kb_used_data":1178120,"kb_used_omap":12,"kb_used_meta":779379,"kb_avail":281157960,"statfs":{"total":289910292480,"available":287905751040,"internally_reserved":0,"allocated":1206394880,"data_stored":1205595020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":798084098},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,4,15,10,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19632128,"num_objects":4793,"num_object_clones":0,"num_object_copies":9586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-193,"num_objects_unfound":0,"num_objects_dirty":4793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4793,"num_write_kb":19172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":790528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006208"},"pg_stats":[{"pgid":"2.f","version":"45'9030","reported_seq":9109,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:42.132254+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:32:42.132254+0000","last_peered":"2024-09-10T20:32:42.132254+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:32:42.132254+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":9030,"log_dups_size":0,"ondisk_log_size":9030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36986880,"num_objects":9030,"num_object_clones":0,"num_object_copies":18060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9030,"num_write_kb":36120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5423},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9229","reported_seq":9339,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:43.514912+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:32:43.514912+0000","last_peered":"2024-09-10T20:32:43.514912+0000","last_clean":"2024-09-10T20:32:43.514912+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:32:43.514912+0000","last_undegraded":"2024-09-10T20:32:43.514912+0000","last_fullsized":"2024-09-10T20:32:43.514912+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":9229,"log_dups_size":0,"ondisk_log_size":9229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37777408,"num_objects":9223,"num_object_clones":0,"num_object_copies":18446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9223,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7415,"num_write_kb":29660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9087","reported_seq":16434,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:43.536861+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:32:43.536861+0000","last_peered":"2024-09-10T20:32:43.536861+0000","last_clean":"2024-09-10T20:32:43.536861+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:32:43.536861+0000","last_undegraded":"2024-09-10T20:32:43.536861+0000","last_fullsized":"2024-09-10T20:32:43.536861+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":9087,"log_dups_size":0,"ondisk_log_size":9087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37203968,"num_objects":9083,"num_object_clones":0,"num_object_copies":18166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9083,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7215,"num_write_kb":28860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'9003","reported_seq":9083,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:43.447710+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:32:43.447710+0000","last_peered":"2024-09-10T20:32:43.447710+0000","last_clean":"2024-09-10T20:32:43.447710+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:32:43.447710+0000","last_undegraded":"2024-09-10T20:32:43.447710+0000","last_fullsized":"2024-09-10T20:32:43.447710+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":9003,"log_dups_size":0,"ondisk_log_size":9003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36876288,"num_objects":9003,"num_object_clones":0,"num_object_copies":18006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9003,"num_write_kb":36012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9207","reported_seq":9332,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:42.152731+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:32:42.152731+0000","last_peered":"2024-09-10T20:32:42.152731+0000","last_clean":"2024-09-10T20:32:42.152731+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:32:42.152731+0000","last_undegraded":"2024-09-10T20:32:42.152731+0000","last_fullsized":"2024-09-10T20:32:42.152731+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":9207,"log_dups_size":0,"ondisk_log_size":9207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37683200,"num_objects":9200,"num_object_clones":0,"num_object_copies":18400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6203,"num_write_kb":24812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'8937","reported_seq":9073,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:42.078950+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:32:42.078950+0000","last_peered":"2024-09-10T20:32:42.078950+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:32:42.078950+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":8937,"log_dups_size":0,"ondisk_log_size":8937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36585472,"num_objects":8932,"num_object_clones":0,"num_object_copies":17864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8932,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5801,"num_write_kb":23204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5318},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9024","reported_seq":9162,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:42.037904+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:32:42.037904+0000","last_peered":"2024-09-10T20:32:42.037904+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:32:42.037904+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":9024,"log_dups_size":0,"ondisk_log_size":9024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36937728,"num_objects":9018,"num_object_clones":0,"num_object_copies":18036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5619,"num_write_kb":22476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5463},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9095","reported_seq":9219,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:41.952901+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:32:41.952901+0000","last_peered":"2024-09-10T20:32:41.952901+0000","last_clean":"2024-09-10T20:32:41.952901+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:32:41.952901+0000","last_undegraded":"2024-09-10T20:32:41.952901+0000","last_fullsized":"2024-09-10T20:32:41.952901+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":9095,"log_dups_size":0,"ondisk_log_size":9095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37244928,"num_objects":9093,"num_object_clones":0,"num_object_copies":18186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6166,"num_write_kb":24664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8961","reported_seq":9093,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:43.427939+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:32:43.427939+0000","last_peered":"2024-09-10T20:32:43.427939+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:32:43.427939+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":8961,"log_dups_size":0,"ondisk_log_size":8961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36675584,"num_objects":8954,"num_object_clones":0,"num_object_copies":17908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6954,"num_write_kb":27816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":5428}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190232+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190232+0000","last_peered":"2024-09-10T20:32:18.190232+0000","last_clean":"2024-09-10T20:32:18.190232+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190232+0000","last_undegraded":"2024-09-10T20:32:18.190232+0000","last_fullsized":"2024-09-10T20:32:18.190232+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9069","reported_seq":9177,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:42.058813+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:32:42.058813+0000","last_peered":"2024-09-10T20:32:42.058813+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:32:42.058813+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":9069,"log_dups_size":0,"ondisk_log_size":9069,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37114822,"num_objects":9063,"num_object_clones":0,"num_object_copies":18126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9063,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9067,"num_write_kb":36248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5448},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182737+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182737+0000","last_peered":"2024-09-10T20:32:18.182737+0000","last_clean":"2024-09-10T20:32:18.182737+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182737+0000","last_undegraded":"2024-09-10T20:32:18.182737+0000","last_fullsized":"2024-09-10T20:32:18.182737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9058","reported_seq":9186,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:43.469035+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:32:43.469035+0000","last_peered":"2024-09-10T20:32:43.469035+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:32:43.469035+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":9058,"log_dups_size":0,"ondisk_log_size":9058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37081088,"num_objects":9053,"num_object_clones":0,"num_object_copies":18106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9053,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6886,"num_write_kb":27544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":5420}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9098","reported_seq":9226,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:43.372037+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:32:43.372037+0000","last_peered":"2024-09-10T20:32:43.372037+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:32:43.372037+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":9098,"log_dups_size":0,"ondisk_log_size":9098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37240832,"num_objects":9092,"num_object_clones":0,"num_object_copies":18184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9092,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6884,"num_write_kb":27536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":5482}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9113","reported_seq":9245,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:42.158229+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:32:42.158229+0000","last_peered":"2024-09-10T20:32:42.158229+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:32:42.158229+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":9113,"log_dups_size":0,"ondisk_log_size":9113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37318656,"num_objects":9111,"num_object_clones":0,"num_object_copies":18222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5836,"num_write_kb":23344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5463},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9013","reported_seq":9151,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:42.098437+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:32:42.098437+0000","last_peered":"2024-09-10T20:32:42.098437+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:32:42.098437+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":9013,"log_dups_size":0,"ondisk_log_size":9013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36884480,"num_objects":9005,"num_object_clones":0,"num_object_copies":18010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5707,"num_write_kb":22828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5473},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193200+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193200+0000","last_peered":"2024-09-10T20:32:18.193200+0000","last_clean":"2024-09-10T20:32:18.193200+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193200+0000","last_undegraded":"2024-09-10T20:32:18.193200+0000","last_fullsized":"2024-09-10T20:32:18.193200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9195","reported_seq":9326,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:42.009667+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:32:42.009667+0000","last_peered":"2024-09-10T20:32:42.009667+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:32:42.009667+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":9195,"log_dups_size":0,"ondisk_log_size":9195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37646336,"num_objects":9191,"num_object_clones":0,"num_object_copies":18382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6022,"num_write_kb":24088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5478},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'8929","reported_seq":10168,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:32:42.124695+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:32:42.124695+0000","last_peered":"2024-09-10T20:32:42.124695+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:32:42.124695+0000","last_undegraded":"2024-09-10T20:32:42.124695+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":8929,"log_dups_size":0,"ondisk_log_size":8929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36524515,"num_objects":8918,"num_object_clones":0,"num_object_copies":17836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2931,"num_objects_unfound":0,"num_objects_dirty":8918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5536,"num_write_kb":22144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":537,"num_bytes_recovered":2199552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5987},{"shards":"0,2","objects":2931}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":593782185,"num_objects":144969,"num_object_clones":0,"num_object_copies":289938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2931,"num_objects_unfound":0,"num_objects_dirty":144969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109344,"num_write_kb":437356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4144,"num_bytes_recovered":16961536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1206059008,"data_stored":1205972296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145048,"ondisk_log_size":145048,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182450,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":794736,"kb_used_data":492384,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93577104,"statfs":{"total":96636764160,"available":95822954496,"internally_reserved":0,"allocated":504201216,"data_stored":503958528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,1,8,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986222,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":680704,"kb_used_data":406064,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93691136,"statfs":{"total":96636764160,"available":95939723264,"internally_reserved":0,"allocated":415809536,"data_stored":415570925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281210538},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659169,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482120,"kb_used_data":279672,"kb_used_omap":4,"kb_used_meta":202427,"kb_avail":93889720,"statfs":{"total":96636764160,"available":96143073280,"internally_reserved":0,"allocated":286384128,"data_stored":286065567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":207285934},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":504143872,"data_stored":504140259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":286093312,"data_stored":286013826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":415821824,"data_stored":415818211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:32:47.358 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:32:47.920 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:47.923 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:32:47.948 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":178,"stamp":"2024-09-10T20:32:46.893535+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":593782204,"num_objects":144971,"num_object_clones":0,"num_object_copies":289942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2931,"num_objects_unfound":0,"num_objects_dirty":144971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109346,"num_write_kb":437358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4146,"num_bytes_recovered":16961555,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145051,"ondisk_log_size":145051,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1957560,"kb_used_data":1178120,"kb_used_omap":12,"kb_used_meta":779379,"kb_avail":281157960,"statfs":{"total":289910292480,"available":287905751040,"internally_reserved":0,"allocated":1206394880,"data_stored":1205595020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":798084098},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,4,15,10,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19632128,"num_objects":4793,"num_object_clones":0,"num_object_copies":9586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-193,"num_objects_unfound":0,"num_objects_dirty":4793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4793,"num_write_kb":19172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":790528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006208"},"pg_stats":[{"pgid":"2.f","version":"45'9030","reported_seq":9109,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:42.132254+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:32:42.132254+0000","last_peered":"2024-09-10T20:32:42.132254+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:32:42.132254+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":9030,"log_dups_size":0,"ondisk_log_size":9030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36986880,"num_objects":9030,"num_object_clones":0,"num_object_copies":18060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9030,"num_write_kb":36120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5423},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9229","reported_seq":9339,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:43.514912+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:32:43.514912+0000","last_peered":"2024-09-10T20:32:43.514912+0000","last_clean":"2024-09-10T20:32:43.514912+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:32:43.514912+0000","last_undegraded":"2024-09-10T20:32:43.514912+0000","last_fullsized":"2024-09-10T20:32:43.514912+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":9229,"log_dups_size":0,"ondisk_log_size":9229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37777408,"num_objects":9223,"num_object_clones":0,"num_object_copies":18446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9223,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7415,"num_write_kb":29660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9087","reported_seq":16434,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:43.536861+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:32:43.536861+0000","last_peered":"2024-09-10T20:32:43.536861+0000","last_clean":"2024-09-10T20:32:43.536861+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:32:43.536861+0000","last_undegraded":"2024-09-10T20:32:43.536861+0000","last_fullsized":"2024-09-10T20:32:43.536861+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":9087,"log_dups_size":0,"ondisk_log_size":9087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37203968,"num_objects":9083,"num_object_clones":0,"num_object_copies":18166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9083,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7215,"num_write_kb":28860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'9003","reported_seq":9083,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:43.447710+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:32:43.447710+0000","last_peered":"2024-09-10T20:32:43.447710+0000","last_clean":"2024-09-10T20:32:43.447710+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:32:43.447710+0000","last_undegraded":"2024-09-10T20:32:43.447710+0000","last_fullsized":"2024-09-10T20:32:43.447710+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":9003,"log_dups_size":0,"ondisk_log_size":9003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36876288,"num_objects":9003,"num_object_clones":0,"num_object_copies":18006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9003,"num_write_kb":36012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9207","reported_seq":9332,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:42.152731+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:32:42.152731+0000","last_peered":"2024-09-10T20:32:42.152731+0000","last_clean":"2024-09-10T20:32:42.152731+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:32:42.152731+0000","last_undegraded":"2024-09-10T20:32:42.152731+0000","last_fullsized":"2024-09-10T20:32:42.152731+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":9207,"log_dups_size":0,"ondisk_log_size":9207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37683200,"num_objects":9200,"num_object_clones":0,"num_object_copies":18400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6203,"num_write_kb":24812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'8937","reported_seq":9073,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:42.078950+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:32:42.078950+0000","last_peered":"2024-09-10T20:32:42.078950+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:32:42.078950+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":8937,"log_dups_size":0,"ondisk_log_size":8937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36585472,"num_objects":8932,"num_object_clones":0,"num_object_copies":17864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8932,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5801,"num_write_kb":23204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5318},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9024","reported_seq":9162,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:42.037904+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:32:42.037904+0000","last_peered":"2024-09-10T20:32:42.037904+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:32:42.037904+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":9024,"log_dups_size":0,"ondisk_log_size":9024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36937728,"num_objects":9018,"num_object_clones":0,"num_object_copies":18036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5619,"num_write_kb":22476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5463},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9095","reported_seq":9219,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:41.952901+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:32:41.952901+0000","last_peered":"2024-09-10T20:32:41.952901+0000","last_clean":"2024-09-10T20:32:41.952901+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:32:41.952901+0000","last_undegraded":"2024-09-10T20:32:41.952901+0000","last_fullsized":"2024-09-10T20:32:41.952901+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":9095,"log_dups_size":0,"ondisk_log_size":9095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37244928,"num_objects":9093,"num_object_clones":0,"num_object_copies":18186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6166,"num_write_kb":24664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8961","reported_seq":9093,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:43.427939+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:32:43.427939+0000","last_peered":"2024-09-10T20:32:43.427939+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:32:43.427939+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":8961,"log_dups_size":0,"ondisk_log_size":8961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36675584,"num_objects":8954,"num_object_clones":0,"num_object_copies":17908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6954,"num_write_kb":27816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":5428}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190232+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190232+0000","last_peered":"2024-09-10T20:32:18.190232+0000","last_clean":"2024-09-10T20:32:18.190232+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190232+0000","last_undegraded":"2024-09-10T20:32:18.190232+0000","last_fullsized":"2024-09-10T20:32:18.190232+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9069","reported_seq":9177,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:42.058813+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:32:42.058813+0000","last_peered":"2024-09-10T20:32:42.058813+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:32:42.058813+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":9069,"log_dups_size":0,"ondisk_log_size":9069,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37114822,"num_objects":9063,"num_object_clones":0,"num_object_copies":18126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9063,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9067,"num_write_kb":36248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5448},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182737+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182737+0000","last_peered":"2024-09-10T20:32:18.182737+0000","last_clean":"2024-09-10T20:32:18.182737+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182737+0000","last_undegraded":"2024-09-10T20:32:18.182737+0000","last_fullsized":"2024-09-10T20:32:18.182737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9058","reported_seq":9186,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:43.469035+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:32:43.469035+0000","last_peered":"2024-09-10T20:32:43.469035+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:32:43.469035+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":9058,"log_dups_size":0,"ondisk_log_size":9058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37081088,"num_objects":9053,"num_object_clones":0,"num_object_copies":18106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9053,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6886,"num_write_kb":27544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":5420}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9098","reported_seq":9226,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:43.372037+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:32:43.372037+0000","last_peered":"2024-09-10T20:32:43.372037+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:32:43.372037+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":9098,"log_dups_size":0,"ondisk_log_size":9098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37240832,"num_objects":9092,"num_object_clones":0,"num_object_copies":18184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9092,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6884,"num_write_kb":27536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":5482}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9113","reported_seq":9245,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:42.158229+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:32:42.158229+0000","last_peered":"2024-09-10T20:32:42.158229+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:32:42.158229+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":9113,"log_dups_size":0,"ondisk_log_size":9113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37318656,"num_objects":9111,"num_object_clones":0,"num_object_copies":18222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5836,"num_write_kb":23344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5463},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9013","reported_seq":9151,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:42.098437+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:32:42.098437+0000","last_peered":"2024-09-10T20:32:42.098437+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:32:42.098437+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":9013,"log_dups_size":0,"ondisk_log_size":9013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36884480,"num_objects":9005,"num_object_clones":0,"num_object_copies":18010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5707,"num_write_kb":22828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5473},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193200+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193200+0000","last_peered":"2024-09-10T20:32:18.193200+0000","last_clean":"2024-09-10T20:32:18.193200+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193200+0000","last_undegraded":"2024-09-10T20:32:18.193200+0000","last_fullsized":"2024-09-10T20:32:18.193200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9195","reported_seq":9326,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:42.009667+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:32:42.009667+0000","last_peered":"2024-09-10T20:32:42.009667+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:32:42.009667+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":9195,"log_dups_size":0,"ondisk_log_size":9195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37646336,"num_objects":9191,"num_object_clones":0,"num_object_copies":18382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6022,"num_write_kb":24088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5478},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'8929","reported_seq":10168,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:32:42.124695+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:32:42.124695+0000","last_peered":"2024-09-10T20:32:42.124695+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:32:42.124695+0000","last_undegraded":"2024-09-10T20:32:42.124695+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":8929,"log_dups_size":0,"ondisk_log_size":8929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36524515,"num_objects":8918,"num_object_clones":0,"num_object_copies":17836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2931,"num_objects_unfound":0,"num_objects_dirty":8918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5536,"num_write_kb":22144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":537,"num_bytes_recovered":2199552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5987},{"shards":"0,2","objects":2931}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":593782185,"num_objects":144969,"num_object_clones":0,"num_object_copies":289938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2931,"num_objects_unfound":0,"num_objects_dirty":144969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109344,"num_write_kb":437356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4144,"num_bytes_recovered":16961536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1206059008,"data_stored":1205972296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145048,"ondisk_log_size":145048,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182450,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":794736,"kb_used_data":492384,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93577104,"statfs":{"total":96636764160,"available":95822954496,"internally_reserved":0,"allocated":504201216,"data_stored":503958528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,1,8,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986222,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":680704,"kb_used_data":406064,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93691136,"statfs":{"total":96636764160,"available":95939723264,"internally_reserved":0,"allocated":415809536,"data_stored":415570925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281210538},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659169,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482120,"kb_used_data":279672,"kb_used_omap":4,"kb_used_meta":202427,"kb_avail":93889720,"statfs":{"total":96636764160,"available":96143073280,"internally_reserved":0,"allocated":286384128,"data_stored":286065567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":207285934},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":504143872,"data_stored":504140259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":286093312,"data_stored":286013826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":415821824,"data_stored":415818211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:32:47.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:32:47.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'9030", 'reported_seq': 9109, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:42.132254+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:32:42.132254+0000', 'last_peered': '2024-09-10T20:32:42.132254+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:32:42.132254+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 9030, 'log_dups_size': 0, 'ondisk_log_size': 9030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36986880, 'num_objects': 9030, 'num_object_clones': 0, 'num_object_copies': 18060, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3607, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9030, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9030, 'num_write_kb': 36120, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5423}, {'shards': '0,2', 'objects': 3607}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:47.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:32:47.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'8937", 'reported_seq': 9073, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:42.078950+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:32:42.078950+0000', 'last_peered': '2024-09-10T20:32:42.078950+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:32:42.078950+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 8937, 'log_dups_size': 0, 'ondisk_log_size': 8937, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36585472, 'num_objects': 8932, 'num_object_clones': 0, 'num_object_copies': 17864, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3616, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8932, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5801, 'num_write_kb': 23204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5318}, {'shards': '0,2', 'objects': 3614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:47.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:32:47.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'9024", 'reported_seq': 9162, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:42.037904+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:32:42.037904+0000', 'last_peered': '2024-09-10T20:32:42.037904+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:32:42.037904+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 9024, 'log_dups_size': 0, 'ondisk_log_size': 9024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36937728, 'num_objects': 9018, 'num_object_clones': 0, 'num_object_copies': 18036, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3557, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9018, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5619, 'num_write_kb': 22476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5463}, {'shards': '0,2', 'objects': 3555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:47.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:32:47.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'8961", 'reported_seq': 9093, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:43.427939+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:32:43.427939+0000', 'last_peered': '2024-09-10T20:32:43.427939+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:32:43.427939+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 8961, 'log_dups_size': 0, 'ondisk_log_size': 8961, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36675584, 'num_objects': 8954, 'num_object_clones': 0, 'num_object_copies': 17908, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3528, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8954, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6954, 'num_write_kb': 27816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 5428}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:32:47.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:32:47.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'9069", 'reported_seq': 9177, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:42.058813+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:32:42.058813+0000', 'last_peered': '2024-09-10T20:32:42.058813+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:32:42.058813+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 9069, 'log_dups_size': 0, 'ondisk_log_size': 9069, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37114822, 'num_objects': 9063, 'num_object_clones': 0, 'num_object_copies': 18126, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3617, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9063, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9067, 'num_write_kb': 36248, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5448}, {'shards': '0,2', 'objects': 3615}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:47.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:32:47.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'9058", 'reported_seq': 9186, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:43.469035+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:32:43.469035+0000', 'last_peered': '2024-09-10T20:32:43.469035+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:32:43.469035+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 9058, 'log_dups_size': 0, 'ondisk_log_size': 9058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37081088, 'num_objects': 9053, 'num_object_clones': 0, 'num_object_copies': 18106, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3635, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9053, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6886, 'num_write_kb': 27544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3633}, {'shards': '2', 'objects': 5420}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:32:47.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:32:47.952 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'9098", 'reported_seq': 9226, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:43.372037+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:32:43.372037+0000', 'last_peered': '2024-09-10T20:32:43.372037+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:32:43.372037+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 9098, 'log_dups_size': 0, 'ondisk_log_size': 9098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37240832, 'num_objects': 9092, 'num_object_clones': 0, 'num_object_copies': 18184, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9092, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6884, 'num_write_kb': 27536, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3610}, {'shards': '2', 'objects': 5482}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:32:47.953 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:32:47.953 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'9113", 'reported_seq': 9245, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:42.158229+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:32:42.158229+0000', 'last_peered': '2024-09-10T20:32:42.158229+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:32:42.158229+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 9113, 'log_dups_size': 0, 'ondisk_log_size': 9113, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37318656, 'num_objects': 9111, 'num_object_clones': 0, 'num_object_copies': 18222, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3649, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9111, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5836, 'num_write_kb': 23344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5463}, {'shards': '0,2', 'objects': 3648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:47.953 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:32:47.953 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'9013", 'reported_seq': 9151, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:42.098437+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:32:42.098437+0000', 'last_peered': '2024-09-10T20:32:42.098437+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:32:42.098437+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 9013, 'log_dups_size': 0, 'ondisk_log_size': 9013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36884480, 'num_objects': 9005, 'num_object_clones': 0, 'num_object_copies': 18010, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3535, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9005, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5707, 'num_write_kb': 22828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5473}, {'shards': '0,2', 'objects': 3532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:47.953 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:32:47.954 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'9195", 'reported_seq': 9326, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:42.009667+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:32:42.009667+0000', 'last_peered': '2024-09-10T20:32:42.009667+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:32:42.009667+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 9195, 'log_dups_size': 0, 'ondisk_log_size': 9195, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37646336, 'num_objects': 9191, 'num_object_clones': 0, 'num_object_copies': 18382, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3715, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9191, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6022, 'num_write_kb': 24088, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5478}, {'shards': '0,2', 'objects': 3713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:47.954 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:32:47.954 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'8929", 'reported_seq': 10168, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:32:42.124695+0000', 'last_change': '2024-09-10T20:32:16.657652+0000', 'last_active': '2024-09-10T20:32:42.124695+0000', 'last_peered': '2024-09-10T20:32:42.124695+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:32:42.124695+0000', 'last_undegraded': '2024-09-10T20:32:42.124695+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 8929, 'log_dups_size': 0, 'ondisk_log_size': 8929, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36524515, 'num_objects': 8918, 'num_object_clones': 0, 'num_object_copies': 17836, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2931, 'num_objects_unfound': 0, 'num_objects_dirty': 8918, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5536, 'num_write_kb': 22144, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 537, 'num_bytes_recovered': 2199552, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5987}, {'shards': '0,2', 'objects': 2931}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:47.954 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:32:48.072 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout:2024-09-10T20:32:48.069299+0000 min lat: 0.00191414 max lat: 2.51074 avg lat: 0.027117 2024-09-10T20:32:48.072 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T20:32:48.072 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 200 20 147517 147497 2.88019 1.70703 0.0459614 0.027117 2024-09-10T20:32:48.646 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:48.646 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36071/289942 objects degraded (12.441%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":280,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":144971,"data_bytes":593782204,"bytes_used":2004541440,"bytes_avail":287905751040,"bytes_total":289910292480,"degraded_objects":36071,"degraded_total":289942,"degraded_ratio":0.1244076401487194,"misplaced_objects":2931,"misplaced_total":289942,"misplaced_ratio":0.010108918335391216,"recovering_objects_per_sec":16,"recovering_bytes_per_sec":65843,"recovering_keys_per_sec":0,"num_objects_recovered":193,"num_bytes_recovered":790528,"num_keys_recovered":0,"write_bytes_sec":1635164,"read_op_per_sec":0,"write_op_per_sec":399},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:32:48.672 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:32:48.673 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:32:49.072 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 201 20 147962 147942 2.87451 1.73828 0.025199 0.0271715 2024-09-10T20:32:49.267 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:49.270 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:32:49.295 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":179,"stamp":"2024-09-10T20:32:48.894390+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":603575740,"num_objects":147362,"num_object_clones":0,"num_object_copies":294724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2841,"num_objects_unfound":0,"num_objects_dirty":147362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111737,"num_write_kb":446922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4236,"num_bytes_recovered":17330195,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147442,"ondisk_log_size":147442,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1982360,"kb_used_data":1197352,"kb_used_omap":12,"kb_used_meta":784947,"kb_avail":281133160,"statfs":{"total":289910292480,"available":287880355840,"internally_reserved":0,"allocated":1226088448,"data_stored":1225288588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":803785730},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,12,8,13,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29425664,"num_objects":7184,"num_object_clones":0,"num_object_copies":14368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-283,"num_objects_unfound":0,"num_objects_dirty":7184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7184,"num_write_kb":28736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":1159168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006160"},"pg_stats":[{"pgid":"2.f","version":"45'9171","reported_seq":9251,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:47.206723+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:32:47.206723+0000","last_peered":"2024-09-10T20:32:47.206723+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:32:47.206723+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":9171,"log_dups_size":0,"ondisk_log_size":9171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37564416,"num_objects":9171,"num_object_clones":0,"num_object_copies":18342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9171,"num_write_kb":36684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5564},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9384","reported_seq":9494,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:48.518630+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:32:48.518630+0000","last_peered":"2024-09-10T20:32:48.518630+0000","last_clean":"2024-09-10T20:32:48.518630+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:32:48.518630+0000","last_undegraded":"2024-09-10T20:32:48.518630+0000","last_fullsized":"2024-09-10T20:32:48.518630+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":9384,"log_dups_size":0,"ondisk_log_size":9384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38412288,"num_objects":9378,"num_object_clones":0,"num_object_copies":18756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7570,"num_write_kb":30280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9243","reported_seq":16590,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:48.595297+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:32:48.595297+0000","last_peered":"2024-09-10T20:32:48.595297+0000","last_clean":"2024-09-10T20:32:48.595297+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:32:48.595297+0000","last_undegraded":"2024-09-10T20:32:48.595297+0000","last_fullsized":"2024-09-10T20:32:48.595297+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":9243,"log_dups_size":0,"ondisk_log_size":9243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37842944,"num_objects":9239,"num_object_clones":0,"num_object_copies":18478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7371,"num_write_kb":29484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'9160","reported_seq":9240,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:48.447367+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:32:48.447367+0000","last_peered":"2024-09-10T20:32:48.447367+0000","last_clean":"2024-09-10T20:32:48.447367+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:32:48.447367+0000","last_undegraded":"2024-09-10T20:32:48.447367+0000","last_fullsized":"2024-09-10T20:32:48.447367+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":9160,"log_dups_size":0,"ondisk_log_size":9160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37519360,"num_objects":9160,"num_object_clones":0,"num_object_copies":18320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9160,"num_write_kb":36640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9354","reported_seq":9480,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:47.220037+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:32:47.220037+0000","last_peered":"2024-09-10T20:32:47.220037+0000","last_clean":"2024-09-10T20:32:47.220037+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:32:47.220037+0000","last_undegraded":"2024-09-10T20:32:47.220037+0000","last_fullsized":"2024-09-10T20:32:47.220037+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":9354,"log_dups_size":0,"ondisk_log_size":9354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38285312,"num_objects":9347,"num_object_clones":0,"num_object_copies":18694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6350,"num_write_kb":25400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9099","reported_seq":9234,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:47.184564+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:32:47.184564+0000","last_peered":"2024-09-10T20:32:47.184564+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:32:47.184564+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":9099,"log_dups_size":0,"ondisk_log_size":9099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37249024,"num_objects":9094,"num_object_clones":0,"num_object_copies":18188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5963,"num_write_kb":23852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5480},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9172","reported_seq":9310,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:47.249540+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:32:47.249540+0000","last_peered":"2024-09-10T20:32:47.249540+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:32:47.249540+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":9172,"log_dups_size":0,"ondisk_log_size":9172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37543936,"num_objects":9166,"num_object_clones":0,"num_object_copies":18332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5767,"num_write_kb":23068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5611},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9228","reported_seq":9352,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:47.176774+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:32:47.176774+0000","last_peered":"2024-09-10T20:32:47.176774+0000","last_clean":"2024-09-10T20:32:47.176774+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:32:47.176774+0000","last_undegraded":"2024-09-10T20:32:47.176774+0000","last_fullsized":"2024-09-10T20:32:47.176774+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":9228,"log_dups_size":0,"ondisk_log_size":9228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37789696,"num_objects":9226,"num_object_clones":0,"num_object_copies":18452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9226,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6299,"num_write_kb":25196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9110","reported_seq":9239,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:48.476352+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:32:48.476352+0000","last_peered":"2024-09-10T20:32:48.476352+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:32:48.476352+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":9110,"log_dups_size":0,"ondisk_log_size":9110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37285888,"num_objects":9103,"num_object_clones":0,"num_object_copies":18206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7103,"num_write_kb":28412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":5577}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190232+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190232+0000","last_peered":"2024-09-10T20:32:18.190232+0000","last_clean":"2024-09-10T20:32:18.190232+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190232+0000","last_undegraded":"2024-09-10T20:32:18.190232+0000","last_fullsized":"2024-09-10T20:32:18.190232+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9221","reported_seq":9329,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:47.208377+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:32:47.208377+0000","last_peered":"2024-09-10T20:32:47.208377+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:32:47.208377+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":9221,"log_dups_size":0,"ondisk_log_size":9221,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37737414,"num_objects":9215,"num_object_clones":0,"num_object_copies":18430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9215,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9219,"num_write_kb":36856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5600},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182737+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182737+0000","last_peered":"2024-09-10T20:32:18.182737+0000","last_clean":"2024-09-10T20:32:18.182737+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182737+0000","last_undegraded":"2024-09-10T20:32:18.182737+0000","last_fullsized":"2024-09-10T20:32:18.182737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9191","reported_seq":9319,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:48.456521+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:32:48.456521+0000","last_peered":"2024-09-10T20:32:48.456521+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:32:48.456521+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":9191,"log_dups_size":0,"ondisk_log_size":9191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37625856,"num_objects":9186,"num_object_clones":0,"num_object_copies":18372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7019,"num_write_kb":28076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":5553}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9243","reported_seq":9371,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:48.452621+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:32:48.452621+0000","last_peered":"2024-09-10T20:32:48.452621+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:32:48.452621+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":9243,"log_dups_size":0,"ondisk_log_size":9243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37834752,"num_objects":9237,"num_object_clones":0,"num_object_copies":18474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7029,"num_write_kb":28116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":5627}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9263","reported_seq":9398,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:47.204475+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:32:47.204475+0000","last_peered":"2024-09-10T20:32:47.204475+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:32:47.204475+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":9263,"log_dups_size":0,"ondisk_log_size":9263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37933056,"num_objects":9261,"num_object_clones":0,"num_object_copies":18522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5986,"num_write_kb":23944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5613},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9181","reported_seq":9319,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:47.208381+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:32:47.208381+0000","last_peered":"2024-09-10T20:32:47.208381+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:32:47.208381+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":9181,"log_dups_size":0,"ondisk_log_size":9181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37572608,"num_objects":9173,"num_object_clones":0,"num_object_copies":18346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5875,"num_write_kb":23500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5641},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193200+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193200+0000","last_peered":"2024-09-10T20:32:18.193200+0000","last_clean":"2024-09-10T20:32:18.193200+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193200+0000","last_undegraded":"2024-09-10T20:32:18.193200+0000","last_fullsized":"2024-09-10T20:32:18.193200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9328","reported_seq":9459,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:47.158490+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:32:47.158490+0000","last_peered":"2024-09-10T20:32:47.158490+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:32:47.158490+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":9328,"log_dups_size":0,"ondisk_log_size":9328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38191104,"num_objects":9324,"num_object_clones":0,"num_object_copies":18648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6155,"num_write_kb":24620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5611},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9091","reported_seq":10510,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:32:47.254549+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:32:47.254549+0000","last_peered":"2024-09-10T20:32:47.254549+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:32:47.254549+0000","last_undegraded":"2024-09-10T20:32:47.254549+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":9091,"log_dups_size":0,"ondisk_log_size":9091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37188067,"num_objects":9080,"num_object_clones":0,"num_object_copies":18160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2841,"num_objects_unfound":0,"num_objects_dirty":9080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5698,"num_write_kb":22792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":627,"num_bytes_recovered":2568192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6239},{"shards":"0,2","objects":2841}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":603575721,"num_objects":147360,"num_object_clones":0,"num_object_copies":294720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2841,"num_objects_unfound":0,"num_objects_dirty":147360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111735,"num_write_kb":446920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4234,"num_bytes_recovered":17330176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1226031104,"data_stored":1225944392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147439,"ondisk_log_size":147439,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182451,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":810180,"kb_used_data":499508,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93561660,"statfs":{"total":96636764160,"available":95807139840,"internally_reserved":0,"allocated":511496192,"data_stored":511253504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,5,5,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986223,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":672872,"kb_used_data":410712,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93698968,"statfs":{"total":96636764160,"available":95947743232,"internally_reserved":0,"allocated":420569088,"data_stored":420330477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268431018},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,5,1,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659170,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499308,"kb_used_data":287132,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93872532,"statfs":{"total":96636764160,"available":96125472768,"internally_reserved":0,"allocated":294023168,"data_stored":293704607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":217247406},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":511602688,"data_stored":511599075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":293715968,"data_stored":293636482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":420712448,"data_stored":420708835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:32:50.072 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 202 20 148443 148423 2.86958 1.87891 0.0323987 0.0272184 2024-09-10T20:32:51.074 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 203 20 148936 148916 2.8649 1.92578 0.0323247 0.0272627 2024-09-10T20:32:52.075 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 204 20 149432 149412 2.86035 1.9375 0.0444388 0.0273066 2024-09-10T20:32:52.297 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:32:52.816 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:52.818 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:32:52.839 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":180,"stamp":"2024-09-10T20:32:50.895285+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":603575740,"num_objects":147362,"num_object_clones":0,"num_object_copies":294724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2841,"num_objects_unfound":0,"num_objects_dirty":147362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111737,"num_write_kb":446922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4236,"num_bytes_recovered":17330195,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147442,"ondisk_log_size":147442,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1982360,"kb_used_data":1197352,"kb_used_omap":12,"kb_used_meta":784947,"kb_avail":281133160,"statfs":{"total":289910292480,"available":287880355840,"internally_reserved":0,"allocated":1226088448,"data_stored":1225288588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":803785730},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,12,8,13,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19640320,"num_objects":4795,"num_object_clones":0,"num_object_copies":9590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-186,"num_objects_unfound":0,"num_objects_dirty":4795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4795,"num_write_kb":19180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":761856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006363"},"pg_stats":[{"pgid":"2.f","version":"45'9171","reported_seq":9251,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:47.206723+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:32:47.206723+0000","last_peered":"2024-09-10T20:32:47.206723+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:32:47.206723+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":9171,"log_dups_size":0,"ondisk_log_size":9171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37564416,"num_objects":9171,"num_object_clones":0,"num_object_copies":18342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9171,"num_write_kb":36684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5564},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9384","reported_seq":9494,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:48.518630+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:32:48.518630+0000","last_peered":"2024-09-10T20:32:48.518630+0000","last_clean":"2024-09-10T20:32:48.518630+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:32:48.518630+0000","last_undegraded":"2024-09-10T20:32:48.518630+0000","last_fullsized":"2024-09-10T20:32:48.518630+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":9384,"log_dups_size":0,"ondisk_log_size":9384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38412288,"num_objects":9378,"num_object_clones":0,"num_object_copies":18756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7570,"num_write_kb":30280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9243","reported_seq":16590,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:48.595297+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:32:48.595297+0000","last_peered":"2024-09-10T20:32:48.595297+0000","last_clean":"2024-09-10T20:32:48.595297+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:32:48.595297+0000","last_undegraded":"2024-09-10T20:32:48.595297+0000","last_fullsized":"2024-09-10T20:32:48.595297+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":9243,"log_dups_size":0,"ondisk_log_size":9243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37842944,"num_objects":9239,"num_object_clones":0,"num_object_copies":18478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7371,"num_write_kb":29484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'9160","reported_seq":9240,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:48.447367+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:32:48.447367+0000","last_peered":"2024-09-10T20:32:48.447367+0000","last_clean":"2024-09-10T20:32:48.447367+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:32:48.447367+0000","last_undegraded":"2024-09-10T20:32:48.447367+0000","last_fullsized":"2024-09-10T20:32:48.447367+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":9160,"log_dups_size":0,"ondisk_log_size":9160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37519360,"num_objects":9160,"num_object_clones":0,"num_object_copies":18320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9160,"num_write_kb":36640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9354","reported_seq":9480,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:47.220037+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:32:47.220037+0000","last_peered":"2024-09-10T20:32:47.220037+0000","last_clean":"2024-09-10T20:32:47.220037+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:32:47.220037+0000","last_undegraded":"2024-09-10T20:32:47.220037+0000","last_fullsized":"2024-09-10T20:32:47.220037+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":9354,"log_dups_size":0,"ondisk_log_size":9354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38285312,"num_objects":9347,"num_object_clones":0,"num_object_copies":18694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6350,"num_write_kb":25400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9099","reported_seq":9234,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:47.184564+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:32:47.184564+0000","last_peered":"2024-09-10T20:32:47.184564+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:32:47.184564+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":9099,"log_dups_size":0,"ondisk_log_size":9099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37249024,"num_objects":9094,"num_object_clones":0,"num_object_copies":18188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5963,"num_write_kb":23852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5480},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9172","reported_seq":9310,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:47.249540+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:32:47.249540+0000","last_peered":"2024-09-10T20:32:47.249540+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:32:47.249540+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":9172,"log_dups_size":0,"ondisk_log_size":9172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37543936,"num_objects":9166,"num_object_clones":0,"num_object_copies":18332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5767,"num_write_kb":23068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5611},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9228","reported_seq":9352,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:47.176774+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:32:47.176774+0000","last_peered":"2024-09-10T20:32:47.176774+0000","last_clean":"2024-09-10T20:32:47.176774+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:32:47.176774+0000","last_undegraded":"2024-09-10T20:32:47.176774+0000","last_fullsized":"2024-09-10T20:32:47.176774+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":9228,"log_dups_size":0,"ondisk_log_size":9228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37789696,"num_objects":9226,"num_object_clones":0,"num_object_copies":18452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9226,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6299,"num_write_kb":25196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9110","reported_seq":9239,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:48.476352+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:32:48.476352+0000","last_peered":"2024-09-10T20:32:48.476352+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:32:48.476352+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":9110,"log_dups_size":0,"ondisk_log_size":9110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37285888,"num_objects":9103,"num_object_clones":0,"num_object_copies":18206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7103,"num_write_kb":28412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":5577}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190232+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190232+0000","last_peered":"2024-09-10T20:32:18.190232+0000","last_clean":"2024-09-10T20:32:18.190232+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190232+0000","last_undegraded":"2024-09-10T20:32:18.190232+0000","last_fullsized":"2024-09-10T20:32:18.190232+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9221","reported_seq":9329,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:47.208377+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:32:47.208377+0000","last_peered":"2024-09-10T20:32:47.208377+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:32:47.208377+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":9221,"log_dups_size":0,"ondisk_log_size":9221,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37737414,"num_objects":9215,"num_object_clones":0,"num_object_copies":18430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9215,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9219,"num_write_kb":36856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5600},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182737+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182737+0000","last_peered":"2024-09-10T20:32:18.182737+0000","last_clean":"2024-09-10T20:32:18.182737+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182737+0000","last_undegraded":"2024-09-10T20:32:18.182737+0000","last_fullsized":"2024-09-10T20:32:18.182737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9191","reported_seq":9319,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:48.456521+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:32:48.456521+0000","last_peered":"2024-09-10T20:32:48.456521+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:32:48.456521+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":9191,"log_dups_size":0,"ondisk_log_size":9191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37625856,"num_objects":9186,"num_object_clones":0,"num_object_copies":18372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7019,"num_write_kb":28076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":5553}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9243","reported_seq":9371,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:48.452621+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:32:48.452621+0000","last_peered":"2024-09-10T20:32:48.452621+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:32:48.452621+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":9243,"log_dups_size":0,"ondisk_log_size":9243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37834752,"num_objects":9237,"num_object_clones":0,"num_object_copies":18474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7029,"num_write_kb":28116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":5627}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9263","reported_seq":9398,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:47.204475+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:32:47.204475+0000","last_peered":"2024-09-10T20:32:47.204475+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:32:47.204475+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":9263,"log_dups_size":0,"ondisk_log_size":9263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37933056,"num_objects":9261,"num_object_clones":0,"num_object_copies":18522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5986,"num_write_kb":23944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5613},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9181","reported_seq":9319,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:47.208381+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:32:47.208381+0000","last_peered":"2024-09-10T20:32:47.208381+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:32:47.208381+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":9181,"log_dups_size":0,"ondisk_log_size":9181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37572608,"num_objects":9173,"num_object_clones":0,"num_object_copies":18346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5875,"num_write_kb":23500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5641},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193200+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193200+0000","last_peered":"2024-09-10T20:32:18.193200+0000","last_clean":"2024-09-10T20:32:18.193200+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193200+0000","last_undegraded":"2024-09-10T20:32:18.193200+0000","last_fullsized":"2024-09-10T20:32:18.193200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9328","reported_seq":9459,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:47.158490+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:32:47.158490+0000","last_peered":"2024-09-10T20:32:47.158490+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:32:47.158490+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":9328,"log_dups_size":0,"ondisk_log_size":9328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38191104,"num_objects":9324,"num_object_clones":0,"num_object_copies":18648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6155,"num_write_kb":24620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5611},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9091","reported_seq":10510,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:32:47.254549+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:32:47.254549+0000","last_peered":"2024-09-10T20:32:47.254549+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:32:47.254549+0000","last_undegraded":"2024-09-10T20:32:47.254549+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":9091,"log_dups_size":0,"ondisk_log_size":9091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37188067,"num_objects":9080,"num_object_clones":0,"num_object_copies":18160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2841,"num_objects_unfound":0,"num_objects_dirty":9080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5698,"num_write_kb":22792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":627,"num_bytes_recovered":2568192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6239},{"shards":"0,2","objects":2841}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":603575721,"num_objects":147360,"num_object_clones":0,"num_object_copies":294720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2841,"num_objects_unfound":0,"num_objects_dirty":147360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111735,"num_write_kb":446920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4234,"num_bytes_recovered":17330176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1226031104,"data_stored":1225944392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147439,"ondisk_log_size":147439,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182451,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":810180,"kb_used_data":499508,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93561660,"statfs":{"total":96636764160,"available":95807139840,"internally_reserved":0,"allocated":511496192,"data_stored":511253504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,5,5,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986223,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":672872,"kb_used_data":410712,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93698968,"statfs":{"total":96636764160,"available":95947743232,"internally_reserved":0,"allocated":420569088,"data_stored":420330477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268431018},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,5,1,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659170,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499308,"kb_used_data":287132,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93872532,"statfs":{"total":96636764160,"available":96125472768,"internally_reserved":0,"allocated":294023168,"data_stored":293704607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":217247406},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":511602688,"data_stored":511599075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":293715968,"data_stored":293636482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":420712448,"data_stored":420708835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:32:52.841 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:32:53.075 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 205 20 149853 149833 2.85442 1.64453 0.0652321 0.0273611 2024-09-10T20:32:53.431 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:53.434 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:32:53.453 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":181,"stamp":"2024-09-10T20:32:52.896121+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":609711548,"num_objects":148860,"num_object_clones":0,"num_object_copies":297720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2739,"num_objects_unfound":0,"num_objects_dirty":148860,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113235,"num_write_kb":452914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4338,"num_bytes_recovered":17747987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148940,"ondisk_log_size":148940,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1976988,"kb_used_data":1204460,"kb_used_omap":12,"kb_used_meta":772467,"kb_avail":281138532,"statfs":{"total":289910292480,"available":287885856768,"internally_reserved":0,"allocated":1233367040,"data_stored":1232567180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":791006210},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,12,8,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25776128,"num_objects":6293,"num_object_clones":0,"num_object_copies":12586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-288,"num_objects_unfound":0,"num_objects_dirty":6293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6293,"num_write_kb":25172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":1179648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006279"},"pg_stats":[{"pgid":"2.f","version":"45'9320","reported_seq":9400,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:52.292838+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:32:52.292838+0000","last_peered":"2024-09-10T20:32:52.292838+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:32:52.292838+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":9320,"log_dups_size":0,"ondisk_log_size":9320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38174720,"num_objects":9320,"num_object_clones":0,"num_object_copies":18640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9320,"num_write_kb":37280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5713},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9384","reported_seq":9494,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:48.518630+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:32:48.518630+0000","last_peered":"2024-09-10T20:32:48.518630+0000","last_clean":"2024-09-10T20:32:48.518630+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:32:48.518630+0000","last_undegraded":"2024-09-10T20:32:48.518630+0000","last_fullsized":"2024-09-10T20:32:48.518630+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":9384,"log_dups_size":0,"ondisk_log_size":9384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38412288,"num_objects":9378,"num_object_clones":0,"num_object_copies":18756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7570,"num_write_kb":30280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9243","reported_seq":16590,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:48.595297+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:32:48.595297+0000","last_peered":"2024-09-10T20:32:48.595297+0000","last_clean":"2024-09-10T20:32:48.595297+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:32:48.595297+0000","last_undegraded":"2024-09-10T20:32:48.595297+0000","last_fullsized":"2024-09-10T20:32:48.595297+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":9243,"log_dups_size":0,"ondisk_log_size":9243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37842944,"num_objects":9239,"num_object_clones":0,"num_object_copies":18478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7371,"num_write_kb":29484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'9160","reported_seq":9240,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:48.447367+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:32:48.447367+0000","last_peered":"2024-09-10T20:32:48.447367+0000","last_clean":"2024-09-10T20:32:48.447367+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:32:48.447367+0000","last_undegraded":"2024-09-10T20:32:48.447367+0000","last_fullsized":"2024-09-10T20:32:48.447367+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":9160,"log_dups_size":0,"ondisk_log_size":9160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37519360,"num_objects":9160,"num_object_clones":0,"num_object_copies":18320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9160,"num_write_kb":36640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9514","reported_seq":9640,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:52.166146+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:32:52.166146+0000","last_peered":"2024-09-10T20:32:52.166146+0000","last_clean":"2024-09-10T20:32:52.166146+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:32:52.166146+0000","last_undegraded":"2024-09-10T20:32:52.166146+0000","last_fullsized":"2024-09-10T20:32:52.166146+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":9514,"log_dups_size":0,"ondisk_log_size":9514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38940672,"num_objects":9507,"num_object_clones":0,"num_object_copies":19014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6510,"num_write_kb":26040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9261","reported_seq":9396,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:52.227918+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:32:52.227918+0000","last_peered":"2024-09-10T20:32:52.227918+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:32:52.227918+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":9261,"log_dups_size":0,"ondisk_log_size":9261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37912576,"num_objects":9256,"num_object_clones":0,"num_object_copies":18512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6125,"num_write_kb":24500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5642},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9320","reported_seq":9458,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:52.105959+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:32:52.105959+0000","last_peered":"2024-09-10T20:32:52.105959+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:32:52.105959+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":9320,"log_dups_size":0,"ondisk_log_size":9320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38150144,"num_objects":9314,"num_object_clones":0,"num_object_copies":18628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5915,"num_write_kb":23660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5759},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9403","reported_seq":9527,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:52.210806+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:32:52.210806+0000","last_peered":"2024-09-10T20:32:52.210806+0000","last_clean":"2024-09-10T20:32:52.210806+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:32:52.210806+0000","last_undegraded":"2024-09-10T20:32:52.210806+0000","last_fullsized":"2024-09-10T20:32:52.210806+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":9403,"log_dups_size":0,"ondisk_log_size":9403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38506496,"num_objects":9401,"num_object_clones":0,"num_object_copies":18802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6474,"num_write_kb":25896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9110","reported_seq":9239,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:48.476352+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:32:48.476352+0000","last_peered":"2024-09-10T20:32:48.476352+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:32:48.476352+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":9110,"log_dups_size":0,"ondisk_log_size":9110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37285888,"num_objects":9103,"num_object_clones":0,"num_object_copies":18206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7103,"num_write_kb":28412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":5577}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190232+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190232+0000","last_peered":"2024-09-10T20:32:18.190232+0000","last_clean":"2024-09-10T20:32:18.190232+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190232+0000","last_undegraded":"2024-09-10T20:32:18.190232+0000","last_fullsized":"2024-09-10T20:32:18.190232+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9362","reported_seq":9466,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:52.156276+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:32:52.156276+0000","last_peered":"2024-09-10T20:32:52.156276+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:32:52.156276+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":9362,"log_dups_size":0,"ondisk_log_size":9362,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38314950,"num_objects":9356,"num_object_clones":0,"num_object_copies":18712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9360,"num_write_kb":37420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5741},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182737+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182737+0000","last_peered":"2024-09-10T20:32:18.182737+0000","last_clean":"2024-09-10T20:32:18.182737+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182737+0000","last_undegraded":"2024-09-10T20:32:18.182737+0000","last_fullsized":"2024-09-10T20:32:18.182737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9191","reported_seq":9319,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:48.456521+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:32:48.456521+0000","last_peered":"2024-09-10T20:32:48.456521+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:32:48.456521+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":9191,"log_dups_size":0,"ondisk_log_size":9191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37625856,"num_objects":9186,"num_object_clones":0,"num_object_copies":18372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7019,"num_write_kb":28076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":5553}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9243","reported_seq":9371,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:48.452621+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:32:48.452621+0000","last_peered":"2024-09-10T20:32:48.452621+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:32:48.452621+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":9243,"log_dups_size":0,"ondisk_log_size":9243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37834752,"num_objects":9237,"num_object_clones":0,"num_object_copies":18474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7029,"num_write_kb":28116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":5627}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9397","reported_seq":9532,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:52.292486+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:32:52.292486+0000","last_peered":"2024-09-10T20:32:52.292486+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:32:52.292486+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":9397,"log_dups_size":0,"ondisk_log_size":9397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38481920,"num_objects":9395,"num_object_clones":0,"num_object_copies":18790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6120,"num_write_kb":24480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5747},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9321","reported_seq":9459,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:52.284243+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:32:52.284243+0000","last_peered":"2024-09-10T20:32:52.284243+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:32:52.284243+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":9321,"log_dups_size":0,"ondisk_log_size":9321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38146048,"num_objects":9313,"num_object_clones":0,"num_object_copies":18626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6015,"num_write_kb":24060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5781},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193200+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193200+0000","last_peered":"2024-09-10T20:32:18.193200+0000","last_clean":"2024-09-10T20:32:18.193200+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193200+0000","last_undegraded":"2024-09-10T20:32:18.193200+0000","last_fullsized":"2024-09-10T20:32:18.193200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9468","reported_seq":9599,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:52.277285+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:32:52.277285+0000","last_peered":"2024-09-10T20:32:52.277285+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:32:52.277285+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":9468,"log_dups_size":0,"ondisk_log_size":9468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38764544,"num_objects":9464,"num_object_clones":0,"num_object_copies":18928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6295,"num_write_kb":25180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5751},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9240","reported_seq":10864,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:32:52.295104+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:32:52.295104+0000","last_peered":"2024-09-10T20:32:52.295104+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:32:52.295104+0000","last_undegraded":"2024-09-10T20:32:52.295104+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":9240,"log_dups_size":0,"ondisk_log_size":9240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37798371,"num_objects":9229,"num_object_clones":0,"num_object_copies":18458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2739,"num_objects_unfound":0,"num_objects_dirty":9229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5847,"num_write_kb":23388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":729,"num_bytes_recovered":2985984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6490},{"shards":"0,2","objects":2739}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":609711529,"num_objects":148858,"num_object_clones":0,"num_object_copies":297716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2739,"num_objects_unfound":0,"num_objects_dirty":148858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113233,"num_write_kb":452912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4336,"num_bytes_recovered":17747968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1233338368,"data_stored":1233251656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148937,"ondisk_log_size":148937,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182452,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":804808,"kb_used_data":506616,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93567032,"statfs":{"total":96636764160,"available":95812640768,"internally_reserved":0,"allocated":518774784,"data_stored":518532096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305327786},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986223,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":672872,"kb_used_data":410712,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93698968,"statfs":{"total":96636764160,"available":95947743232,"internally_reserved":0,"allocated":420569088,"data_stored":420330477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268431018},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,5,1,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659170,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499308,"kb_used_data":287132,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93872532,"statfs":{"total":96636764160,"available":96125472768,"internally_reserved":0,"allocated":294023168,"data_stored":293704607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":217247406},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":518909952,"data_stored":518906339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":293715968,"data_stored":293636482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":420712448,"data_stored":420708835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:32:53.454 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:32:53.454 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'9320", 'reported_seq': 9400, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:52.292838+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:32:52.292838+0000', 'last_peered': '2024-09-10T20:32:52.292838+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:32:52.292838+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 9320, 'log_dups_size': 0, 'ondisk_log_size': 9320, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38174720, 'num_objects': 9320, 'num_object_clones': 0, 'num_object_copies': 18640, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3607, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9320, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9320, 'num_write_kb': 37280, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5713}, {'shards': '0,2', 'objects': 3607}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:53.454 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:32:53.454 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'9261", 'reported_seq': 9396, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:52.227918+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:32:52.227918+0000', 'last_peered': '2024-09-10T20:32:52.227918+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:32:52.227918+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 9261, 'log_dups_size': 0, 'ondisk_log_size': 9261, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37912576, 'num_objects': 9256, 'num_object_clones': 0, 'num_object_copies': 18512, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3616, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9256, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6125, 'num_write_kb': 24500, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5642}, {'shards': '0,2', 'objects': 3614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:53.455 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:32:53.455 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'9320", 'reported_seq': 9458, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:52.105959+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:32:52.105959+0000', 'last_peered': '2024-09-10T20:32:52.105959+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:32:52.105959+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 9320, 'log_dups_size': 0, 'ondisk_log_size': 9320, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38150144, 'num_objects': 9314, 'num_object_clones': 0, 'num_object_copies': 18628, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3557, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9314, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5915, 'num_write_kb': 23660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5759}, {'shards': '0,2', 'objects': 3555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:53.455 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:32:53.455 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'9110", 'reported_seq': 9239, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:48.476352+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:32:48.476352+0000', 'last_peered': '2024-09-10T20:32:48.476352+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:32:48.476352+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 9110, 'log_dups_size': 0, 'ondisk_log_size': 9110, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37285888, 'num_objects': 9103, 'num_object_clones': 0, 'num_object_copies': 18206, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3528, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9103, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7103, 'num_write_kb': 28412, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 5577}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:32:53.455 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:32:53.455 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'9362", 'reported_seq': 9466, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:52.156276+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:32:52.156276+0000', 'last_peered': '2024-09-10T20:32:52.156276+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:32:52.156276+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 9362, 'log_dups_size': 0, 'ondisk_log_size': 9362, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38314950, 'num_objects': 9356, 'num_object_clones': 0, 'num_object_copies': 18712, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3617, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9356, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9360, 'num_write_kb': 37420, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5741}, {'shards': '0,2', 'objects': 3615}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:53.455 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:32:53.455 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'9191", 'reported_seq': 9319, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:48.456521+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:32:48.456521+0000', 'last_peered': '2024-09-10T20:32:48.456521+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:32:48.456521+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 9191, 'log_dups_size': 0, 'ondisk_log_size': 9191, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37625856, 'num_objects': 9186, 'num_object_clones': 0, 'num_object_copies': 18372, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3635, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9186, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7019, 'num_write_kb': 28076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3633}, {'shards': '2', 'objects': 5553}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:32:53.455 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:32:53.455 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'9243", 'reported_seq': 9371, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:48.452621+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:32:48.452621+0000', 'last_peered': '2024-09-10T20:32:48.452621+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:32:48.452621+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 9243, 'log_dups_size': 0, 'ondisk_log_size': 9243, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37834752, 'num_objects': 9237, 'num_object_clones': 0, 'num_object_copies': 18474, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9237, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7029, 'num_write_kb': 28116, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3610}, {'shards': '2', 'objects': 5627}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:32:53.455 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:32:53.456 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'9397", 'reported_seq': 9532, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:52.292486+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:32:52.292486+0000', 'last_peered': '2024-09-10T20:32:52.292486+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:32:52.292486+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 9397, 'log_dups_size': 0, 'ondisk_log_size': 9397, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38481920, 'num_objects': 9395, 'num_object_clones': 0, 'num_object_copies': 18790, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3649, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9395, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6120, 'num_write_kb': 24480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5747}, {'shards': '0,2', 'objects': 3648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:53.456 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:32:53.456 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'9321", 'reported_seq': 9459, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:52.284243+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:32:52.284243+0000', 'last_peered': '2024-09-10T20:32:52.284243+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:32:52.284243+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 9321, 'log_dups_size': 0, 'ondisk_log_size': 9321, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38146048, 'num_objects': 9313, 'num_object_clones': 0, 'num_object_copies': 18626, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3535, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9313, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6015, 'num_write_kb': 24060, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5781}, {'shards': '0,2', 'objects': 3532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:53.456 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:32:53.456 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'9468", 'reported_seq': 9599, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:52.277285+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:32:52.277285+0000', 'last_peered': '2024-09-10T20:32:52.277285+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:32:52.277285+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 9468, 'log_dups_size': 0, 'ondisk_log_size': 9468, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38764544, 'num_objects': 9464, 'num_object_clones': 0, 'num_object_copies': 18928, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3715, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9464, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6295, 'num_write_kb': 25180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5751}, {'shards': '0,2', 'objects': 3713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:53.456 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:32:53.456 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'9240", 'reported_seq': 10864, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:32:52.295104+0000', 'last_change': '2024-09-10T20:32:16.657652+0000', 'last_active': '2024-09-10T20:32:52.295104+0000', 'last_peered': '2024-09-10T20:32:52.295104+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:32:52.295104+0000', 'last_undegraded': '2024-09-10T20:32:52.295104+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 9240, 'log_dups_size': 0, 'ondisk_log_size': 9240, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37798371, 'num_objects': 9229, 'num_object_clones': 0, 'num_object_copies': 18458, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2739, 'num_objects_unfound': 0, 'num_objects_dirty': 9229, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5847, 'num_write_kb': 23388, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 729, 'num_bytes_recovered': 2985984, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6490}, {'shards': '0,2', 'objects': 2739}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:53.456 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:32:54.075 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 206 20 150289 150269 2.84883 1.70312 0.0265839 0.0274157 2024-09-10T20:32:54.156 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:54.156 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36071/297720 objects degraded (12.116%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":285,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":148860,"data_bytes":609711548,"bytes_used":2024435712,"bytes_avail":287885856768,"bytes_total":289910292480,"degraded_objects":36071,"degraded_total":297720,"degraded_ratio":0.12115746338841865,"misplaced_objects":2739,"misplaced_total":297720,"misplaced_ratio":0.0091999193873438124,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":98252,"recovering_keys_per_sec":0,"num_objects_recovered":288,"num_bytes_recovered":1179648,"num_keys_recovered":0,"write_bytes_sec":2146887,"read_op_per_sec":0,"write_op_per_sec":524},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:32:54.192 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:32:54.192 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:32:54.748 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:54.748 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:32:54.774 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":181,"stamp":"2024-09-10T20:32:52.896121+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":609711548,"num_objects":148860,"num_object_clones":0,"num_object_copies":297720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2739,"num_objects_unfound":0,"num_objects_dirty":148860,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113235,"num_write_kb":452914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4338,"num_bytes_recovered":17747987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148940,"ondisk_log_size":148940,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1976988,"kb_used_data":1204460,"kb_used_omap":12,"kb_used_meta":772467,"kb_avail":281138532,"statfs":{"total":289910292480,"available":287885856768,"internally_reserved":0,"allocated":1233367040,"data_stored":1232567180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":791006210},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,12,8,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25776128,"num_objects":6293,"num_object_clones":0,"num_object_copies":12586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-288,"num_objects_unfound":0,"num_objects_dirty":6293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6293,"num_write_kb":25172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":1179648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006279"},"pg_stats":[{"pgid":"2.f","version":"45'9320","reported_seq":9400,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:52.292838+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:32:52.292838+0000","last_peered":"2024-09-10T20:32:52.292838+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:32:52.292838+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":9320,"log_dups_size":0,"ondisk_log_size":9320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38174720,"num_objects":9320,"num_object_clones":0,"num_object_copies":18640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9320,"num_write_kb":37280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5713},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9384","reported_seq":9494,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:48.518630+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:32:48.518630+0000","last_peered":"2024-09-10T20:32:48.518630+0000","last_clean":"2024-09-10T20:32:48.518630+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:32:48.518630+0000","last_undegraded":"2024-09-10T20:32:48.518630+0000","last_fullsized":"2024-09-10T20:32:48.518630+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":9384,"log_dups_size":0,"ondisk_log_size":9384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38412288,"num_objects":9378,"num_object_clones":0,"num_object_copies":18756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7570,"num_write_kb":30280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9243","reported_seq":16590,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:48.595297+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:32:48.595297+0000","last_peered":"2024-09-10T20:32:48.595297+0000","last_clean":"2024-09-10T20:32:48.595297+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:32:48.595297+0000","last_undegraded":"2024-09-10T20:32:48.595297+0000","last_fullsized":"2024-09-10T20:32:48.595297+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":9243,"log_dups_size":0,"ondisk_log_size":9243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37842944,"num_objects":9239,"num_object_clones":0,"num_object_copies":18478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7371,"num_write_kb":29484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'9160","reported_seq":9240,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:48.447367+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:32:48.447367+0000","last_peered":"2024-09-10T20:32:48.447367+0000","last_clean":"2024-09-10T20:32:48.447367+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:32:48.447367+0000","last_undegraded":"2024-09-10T20:32:48.447367+0000","last_fullsized":"2024-09-10T20:32:48.447367+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":9160,"log_dups_size":0,"ondisk_log_size":9160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37519360,"num_objects":9160,"num_object_clones":0,"num_object_copies":18320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9160,"num_write_kb":36640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9514","reported_seq":9640,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:52.166146+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:32:52.166146+0000","last_peered":"2024-09-10T20:32:52.166146+0000","last_clean":"2024-09-10T20:32:52.166146+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:32:52.166146+0000","last_undegraded":"2024-09-10T20:32:52.166146+0000","last_fullsized":"2024-09-10T20:32:52.166146+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":9514,"log_dups_size":0,"ondisk_log_size":9514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38940672,"num_objects":9507,"num_object_clones":0,"num_object_copies":19014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6510,"num_write_kb":26040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9261","reported_seq":9396,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:52.227918+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:32:52.227918+0000","last_peered":"2024-09-10T20:32:52.227918+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:32:52.227918+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":9261,"log_dups_size":0,"ondisk_log_size":9261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37912576,"num_objects":9256,"num_object_clones":0,"num_object_copies":18512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6125,"num_write_kb":24500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5642},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9320","reported_seq":9458,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:52.105959+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:32:52.105959+0000","last_peered":"2024-09-10T20:32:52.105959+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:32:52.105959+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":9320,"log_dups_size":0,"ondisk_log_size":9320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38150144,"num_objects":9314,"num_object_clones":0,"num_object_copies":18628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5915,"num_write_kb":23660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5759},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9403","reported_seq":9527,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:52.210806+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:32:52.210806+0000","last_peered":"2024-09-10T20:32:52.210806+0000","last_clean":"2024-09-10T20:32:52.210806+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:32:52.210806+0000","last_undegraded":"2024-09-10T20:32:52.210806+0000","last_fullsized":"2024-09-10T20:32:52.210806+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":9403,"log_dups_size":0,"ondisk_log_size":9403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38506496,"num_objects":9401,"num_object_clones":0,"num_object_copies":18802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6474,"num_write_kb":25896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9110","reported_seq":9239,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:48.476352+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:32:48.476352+0000","last_peered":"2024-09-10T20:32:48.476352+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:32:48.476352+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":9110,"log_dups_size":0,"ondisk_log_size":9110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37285888,"num_objects":9103,"num_object_clones":0,"num_object_copies":18206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7103,"num_write_kb":28412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":5577}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190232+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190232+0000","last_peered":"2024-09-10T20:32:18.190232+0000","last_clean":"2024-09-10T20:32:18.190232+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190232+0000","last_undegraded":"2024-09-10T20:32:18.190232+0000","last_fullsized":"2024-09-10T20:32:18.190232+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9362","reported_seq":9466,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:52.156276+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:32:52.156276+0000","last_peered":"2024-09-10T20:32:52.156276+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:32:52.156276+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":9362,"log_dups_size":0,"ondisk_log_size":9362,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38314950,"num_objects":9356,"num_object_clones":0,"num_object_copies":18712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9360,"num_write_kb":37420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5741},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182737+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182737+0000","last_peered":"2024-09-10T20:32:18.182737+0000","last_clean":"2024-09-10T20:32:18.182737+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182737+0000","last_undegraded":"2024-09-10T20:32:18.182737+0000","last_fullsized":"2024-09-10T20:32:18.182737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9191","reported_seq":9319,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:48.456521+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:32:48.456521+0000","last_peered":"2024-09-10T20:32:48.456521+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:32:48.456521+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":9191,"log_dups_size":0,"ondisk_log_size":9191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37625856,"num_objects":9186,"num_object_clones":0,"num_object_copies":18372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7019,"num_write_kb":28076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":5553}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9243","reported_seq":9371,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:48.452621+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:32:48.452621+0000","last_peered":"2024-09-10T20:32:48.452621+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:32:48.452621+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":9243,"log_dups_size":0,"ondisk_log_size":9243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37834752,"num_objects":9237,"num_object_clones":0,"num_object_copies":18474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7029,"num_write_kb":28116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":5627}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9397","reported_seq":9532,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:52.292486+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:32:52.292486+0000","last_peered":"2024-09-10T20:32:52.292486+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:32:52.292486+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":9397,"log_dups_size":0,"ondisk_log_size":9397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38481920,"num_objects":9395,"num_object_clones":0,"num_object_copies":18790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6120,"num_write_kb":24480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5747},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9321","reported_seq":9459,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:52.284243+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:32:52.284243+0000","last_peered":"2024-09-10T20:32:52.284243+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:32:52.284243+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":9321,"log_dups_size":0,"ondisk_log_size":9321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38146048,"num_objects":9313,"num_object_clones":0,"num_object_copies":18626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6015,"num_write_kb":24060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5781},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193200+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193200+0000","last_peered":"2024-09-10T20:32:18.193200+0000","last_clean":"2024-09-10T20:32:18.193200+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193200+0000","last_undegraded":"2024-09-10T20:32:18.193200+0000","last_fullsized":"2024-09-10T20:32:18.193200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9468","reported_seq":9599,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:52.277285+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:32:52.277285+0000","last_peered":"2024-09-10T20:32:52.277285+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:32:52.277285+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":9468,"log_dups_size":0,"ondisk_log_size":9468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38764544,"num_objects":9464,"num_object_clones":0,"num_object_copies":18928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6295,"num_write_kb":25180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5751},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9240","reported_seq":10864,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:32:52.295104+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:32:52.295104+0000","last_peered":"2024-09-10T20:32:52.295104+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:32:52.295104+0000","last_undegraded":"2024-09-10T20:32:52.295104+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":9240,"log_dups_size":0,"ondisk_log_size":9240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37798371,"num_objects":9229,"num_object_clones":0,"num_object_copies":18458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2739,"num_objects_unfound":0,"num_objects_dirty":9229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5847,"num_write_kb":23388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":729,"num_bytes_recovered":2985984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6490},{"shards":"0,2","objects":2739}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":609711529,"num_objects":148858,"num_object_clones":0,"num_object_copies":297716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2739,"num_objects_unfound":0,"num_objects_dirty":148858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113233,"num_write_kb":452912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4336,"num_bytes_recovered":17747968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1233338368,"data_stored":1233251656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148937,"ondisk_log_size":148937,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182452,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":804808,"kb_used_data":506616,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93567032,"statfs":{"total":96636764160,"available":95812640768,"internally_reserved":0,"allocated":518774784,"data_stored":518532096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305327786},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986223,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":672872,"kb_used_data":410712,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93698968,"statfs":{"total":96636764160,"available":95947743232,"internally_reserved":0,"allocated":420569088,"data_stored":420330477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268431018},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,5,1,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659170,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499308,"kb_used_data":287132,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93872532,"statfs":{"total":96636764160,"available":96125472768,"internally_reserved":0,"allocated":294023168,"data_stored":293704607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":217247406},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":518909952,"data_stored":518906339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":293715968,"data_stored":293636482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":420712448,"data_stored":420708835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:32:55.075 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 207 20 150742 150722 2.84362 1.76953 0.0474445 0.0274664 2024-09-10T20:32:56.075 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 208 20 151236 151216 2.83923 1.92969 0.0356023 0.0275088 2024-09-10T20:32:57.075 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 209 20 151709 151689 2.83448 1.84766 0.0561308 0.0275545 2024-09-10T20:32:57.776 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:32:58.075 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 210 20 152167 152147 2.8295 1.78906 0.0298128 0.0276027 2024-09-10T20:32:58.324 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:58.326 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:32:58.350 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":183,"stamp":"2024-09-10T20:32:56.897809+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":613246396,"num_objects":149723,"num_object_clones":0,"num_object_copies":299446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2739,"num_objects_unfound":0,"num_objects_dirty":149723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114098,"num_write_kb":456366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4338,"num_bytes_recovered":17747987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149803,"ondisk_log_size":149803,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1997568,"kb_used_data":1216712,"kb_used_omap":12,"kb_used_meta":780787,"kb_avail":281117952,"statfs":{"total":289910292480,"available":287864782848,"internally_reserved":0,"allocated":1245913088,"data_stored":1245113228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":799525890},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,1,7,13,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19464192,"num_objects":4752,"num_object_clones":0,"num_object_copies":9504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-192,"num_objects_unfound":0,"num_objects_dirty":4752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4752,"num_write_kb":19008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":786432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005098"},"pg_stats":[{"pgid":"2.f","version":"45'9320","reported_seq":9400,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:52.292838+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:32:52.292838+0000","last_peered":"2024-09-10T20:32:52.292838+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:32:52.292838+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":9320,"log_dups_size":0,"ondisk_log_size":9320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38174720,"num_objects":9320,"num_object_clones":0,"num_object_copies":18640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9320,"num_write_kb":37280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5713},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9530","reported_seq":9640,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:53.484237+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:32:53.484237+0000","last_peered":"2024-09-10T20:32:53.484237+0000","last_clean":"2024-09-10T20:32:53.484237+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:32:53.484237+0000","last_undegraded":"2024-09-10T20:32:53.484237+0000","last_fullsized":"2024-09-10T20:32:53.484237+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":9530,"log_dups_size":0,"ondisk_log_size":9530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39010304,"num_objects":9524,"num_object_clones":0,"num_object_copies":19048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7716,"num_write_kb":30864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9387","reported_seq":16734,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:53.393839+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:32:53.393839+0000","last_peered":"2024-09-10T20:32:53.393839+0000","last_clean":"2024-09-10T20:32:53.393839+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:32:53.393839+0000","last_undegraded":"2024-09-10T20:32:53.393839+0000","last_fullsized":"2024-09-10T20:32:53.393839+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":9387,"log_dups_size":0,"ondisk_log_size":9387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38432768,"num_objects":9383,"num_object_clones":0,"num_object_copies":18766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9383,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7515,"num_write_kb":30060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'9292","reported_seq":9372,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:53.396613+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:32:53.396613+0000","last_peered":"2024-09-10T20:32:53.396613+0000","last_clean":"2024-09-10T20:32:53.396613+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:32:53.396613+0000","last_undegraded":"2024-09-10T20:32:53.396613+0000","last_fullsized":"2024-09-10T20:32:53.396613+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":9292,"log_dups_size":0,"ondisk_log_size":9292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38060032,"num_objects":9292,"num_object_clones":0,"num_object_copies":18584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9292,"num_write_kb":37168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9514","reported_seq":9640,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:52.166146+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:32:52.166146+0000","last_peered":"2024-09-10T20:32:52.166146+0000","last_clean":"2024-09-10T20:32:52.166146+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:32:52.166146+0000","last_undegraded":"2024-09-10T20:32:52.166146+0000","last_fullsized":"2024-09-10T20:32:52.166146+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":9514,"log_dups_size":0,"ondisk_log_size":9514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38940672,"num_objects":9507,"num_object_clones":0,"num_object_copies":19014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6510,"num_write_kb":26040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9261","reported_seq":9396,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:52.227918+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:32:52.227918+0000","last_peered":"2024-09-10T20:32:52.227918+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:32:52.227918+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":9261,"log_dups_size":0,"ondisk_log_size":9261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37912576,"num_objects":9256,"num_object_clones":0,"num_object_copies":18512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6125,"num_write_kb":24500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5642},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9320","reported_seq":9458,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:52.105959+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:32:52.105959+0000","last_peered":"2024-09-10T20:32:52.105959+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:32:52.105959+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":9320,"log_dups_size":0,"ondisk_log_size":9320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38150144,"num_objects":9314,"num_object_clones":0,"num_object_copies":18628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5915,"num_write_kb":23660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5759},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9403","reported_seq":9527,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:52.210806+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:32:52.210806+0000","last_peered":"2024-09-10T20:32:52.210806+0000","last_clean":"2024-09-10T20:32:52.210806+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:32:52.210806+0000","last_undegraded":"2024-09-10T20:32:52.210806+0000","last_fullsized":"2024-09-10T20:32:52.210806+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":9403,"log_dups_size":0,"ondisk_log_size":9403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38506496,"num_objects":9401,"num_object_clones":0,"num_object_copies":18802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6474,"num_write_kb":25896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9266","reported_seq":9398,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:53.483109+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:32:53.483109+0000","last_peered":"2024-09-10T20:32:53.483109+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:32:53.483109+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":9266,"log_dups_size":0,"ondisk_log_size":9266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37924864,"num_objects":9259,"num_object_clones":0,"num_object_copies":18518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7259,"num_write_kb":29036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":5733}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190232+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190232+0000","last_peered":"2024-09-10T20:32:18.190232+0000","last_clean":"2024-09-10T20:32:18.190232+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190232+0000","last_undegraded":"2024-09-10T20:32:18.190232+0000","last_fullsized":"2024-09-10T20:32:18.190232+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9362","reported_seq":9466,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:52.156276+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:32:52.156276+0000","last_peered":"2024-09-10T20:32:52.156276+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:32:52.156276+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":9362,"log_dups_size":0,"ondisk_log_size":9362,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38314950,"num_objects":9356,"num_object_clones":0,"num_object_copies":18712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9360,"num_write_kb":37420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5741},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182737+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182737+0000","last_peered":"2024-09-10T20:32:18.182737+0000","last_clean":"2024-09-10T20:32:18.182737+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182737+0000","last_undegraded":"2024-09-10T20:32:18.182737+0000","last_fullsized":"2024-09-10T20:32:18.182737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9327","reported_seq":9455,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:53.480499+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:32:53.480499+0000","last_peered":"2024-09-10T20:32:53.480499+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:32:53.480499+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":9327,"log_dups_size":0,"ondisk_log_size":9327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38182912,"num_objects":9322,"num_object_clones":0,"num_object_copies":18644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7155,"num_write_kb":28620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":5689}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9392","reported_seq":9520,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:53.451753+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:32:53.451753+0000","last_peered":"2024-09-10T20:32:53.451753+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:32:53.451753+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":9392,"log_dups_size":0,"ondisk_log_size":9392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38445056,"num_objects":9386,"num_object_clones":0,"num_object_copies":18772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7178,"num_write_kb":28712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":5776}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9397","reported_seq":9532,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:52.292486+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:32:52.292486+0000","last_peered":"2024-09-10T20:32:52.292486+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:32:52.292486+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":9397,"log_dups_size":0,"ondisk_log_size":9397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38481920,"num_objects":9395,"num_object_clones":0,"num_object_copies":18790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6120,"num_write_kb":24480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5747},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9321","reported_seq":9459,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:52.284243+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:32:52.284243+0000","last_peered":"2024-09-10T20:32:52.284243+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:32:52.284243+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":9321,"log_dups_size":0,"ondisk_log_size":9321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38146048,"num_objects":9313,"num_object_clones":0,"num_object_copies":18626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6015,"num_write_kb":24060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5781},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193200+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193200+0000","last_peered":"2024-09-10T20:32:18.193200+0000","last_clean":"2024-09-10T20:32:18.193200+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193200+0000","last_undegraded":"2024-09-10T20:32:18.193200+0000","last_fullsized":"2024-09-10T20:32:18.193200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9468","reported_seq":9599,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:52.277285+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:32:52.277285+0000","last_peered":"2024-09-10T20:32:52.277285+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:32:52.277285+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":9468,"log_dups_size":0,"ondisk_log_size":9468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38764544,"num_objects":9464,"num_object_clones":0,"num_object_copies":18928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6295,"num_write_kb":25180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5751},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9240","reported_seq":10864,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:32:52.295104+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:32:52.295104+0000","last_peered":"2024-09-10T20:32:52.295104+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:32:52.295104+0000","last_undegraded":"2024-09-10T20:32:52.295104+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":9240,"log_dups_size":0,"ondisk_log_size":9240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37798371,"num_objects":9229,"num_object_clones":0,"num_object_copies":18458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2739,"num_objects_unfound":0,"num_objects_dirty":9229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5847,"num_write_kb":23388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":729,"num_bytes_recovered":2985984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6490},{"shards":"0,2","objects":2739}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":613246377,"num_objects":149721,"num_object_clones":0,"num_object_copies":299442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2739,"num_objects_unfound":0,"num_objects_dirty":149721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114096,"num_write_kb":456364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4336,"num_bytes_recovered":17747968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1245786112,"data_stored":1245699400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149800,"ondisk_log_size":149800,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182452,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":804808,"kb_used_data":506616,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93567032,"statfs":{"total":96636764160,"available":95812640768,"internally_reserved":0,"allocated":518774784,"data_stored":518532096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305327786},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986224,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":681900,"kb_used_data":415572,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93689940,"statfs":{"total":96636764160,"available":95938498560,"internally_reserved":0,"allocated":425545728,"data_stored":425307117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272690858},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659171,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":510860,"kb_used_data":294524,"kb_used_omap":4,"kb_used_meta":216315,"kb_avail":93860980,"statfs":{"total":96636764160,"available":96113643520,"internally_reserved":0,"allocated":301592576,"data_stored":301274015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":221507246},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,0,2,4,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":518909952,"data_stored":518906339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":301363200,"data_stored":301283714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":425512960,"data_stored":425509347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:32:58.351 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:32:58.947 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:58.948 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:32:58.971 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":184,"stamp":"2024-09-10T20:32:58.898438+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":622659004,"num_objects":152021,"num_object_clones":0,"num_object_copies":304042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2635,"num_objects_unfound":0,"num_objects_dirty":152021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116396,"num_write_kb":465558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4442,"num_bytes_recovered":18173971,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152101,"ondisk_log_size":152101,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2046948,"kb_used_data":1235572,"kb_used_omap":12,"kb_used_meta":811315,"kb_avail":281068572,"statfs":{"total":289910292480,"available":287814217728,"internally_reserved":0,"allocated":1265225728,"data_stored":1264425868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":830786562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,5,2,6,8,8,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28876800,"num_objects":7050,"num_object_clones":0,"num_object_copies":14100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-296,"num_objects_unfound":0,"num_objects_dirty":7050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7050,"num_write_kb":28200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":1212416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004903"},"pg_stats":[{"pgid":"2.f","version":"45'9471","reported_seq":9549,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:57.321314+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:32:57.321314+0000","last_peered":"2024-09-10T20:32:57.321314+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:32:57.321314+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":9471,"log_dups_size":0,"ondisk_log_size":9471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38793216,"num_objects":9471,"num_object_clones":0,"num_object_copies":18942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9471,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9471,"num_write_kb":37884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5864},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9677","reported_seq":9787,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:58.442031+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:32:58.442031+0000","last_peered":"2024-09-10T20:32:58.442031+0000","last_clean":"2024-09-10T20:32:58.442031+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:32:58.442031+0000","last_undegraded":"2024-09-10T20:32:58.442031+0000","last_fullsized":"2024-09-10T20:32:58.442031+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":9677,"log_dups_size":0,"ondisk_log_size":9677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39612416,"num_objects":9671,"num_object_clones":0,"num_object_copies":19342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7863,"num_write_kb":31452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9536","reported_seq":16883,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:58.637197+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:32:58.637197+0000","last_peered":"2024-09-10T20:32:58.637197+0000","last_clean":"2024-09-10T20:32:58.637197+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:32:58.637197+0000","last_undegraded":"2024-09-10T20:32:58.637197+0000","last_fullsized":"2024-09-10T20:32:58.637197+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":9536,"log_dups_size":0,"ondisk_log_size":9536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39043072,"num_objects":9532,"num_object_clones":0,"num_object_copies":19064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7664,"num_write_kb":30656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'9444","reported_seq":9522,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:58.540927+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:32:58.540927+0000","last_peered":"2024-09-10T20:32:58.540927+0000","last_clean":"2024-09-10T20:32:58.540927+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:32:58.540927+0000","last_undegraded":"2024-09-10T20:32:58.540927+0000","last_fullsized":"2024-09-10T20:32:58.540927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":9444,"log_dups_size":0,"ondisk_log_size":9444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38682624,"num_objects":9444,"num_object_clones":0,"num_object_copies":18888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9444,"num_write_kb":37776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9658","reported_seq":9784,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:57.291966+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:32:57.291966+0000","last_peered":"2024-09-10T20:32:57.291966+0000","last_clean":"2024-09-10T20:32:57.291966+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:32:57.291966+0000","last_undegraded":"2024-09-10T20:32:57.291966+0000","last_fullsized":"2024-09-10T20:32:57.291966+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":9658,"log_dups_size":0,"ondisk_log_size":9658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39530496,"num_objects":9651,"num_object_clones":0,"num_object_copies":19302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6654,"num_write_kb":26616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9411","reported_seq":9547,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:57.249254+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:32:57.249254+0000","last_peered":"2024-09-10T20:32:57.249254+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:32:57.249254+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":9411,"log_dups_size":0,"ondisk_log_size":9411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38526976,"num_objects":9406,"num_object_clones":0,"num_object_copies":18812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6275,"num_write_kb":25100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5792},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9479","reported_seq":9617,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:57.312075+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:32:57.312075+0000","last_peered":"2024-09-10T20:32:57.312075+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:32:57.312075+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":9479,"log_dups_size":0,"ondisk_log_size":9479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38801408,"num_objects":9473,"num_object_clones":0,"num_object_copies":18946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6074,"num_write_kb":24296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5918},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9550","reported_seq":9674,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:57.320006+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:32:57.320006+0000","last_peered":"2024-09-10T20:32:57.320006+0000","last_clean":"2024-09-10T20:32:57.320006+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:32:57.320006+0000","last_undegraded":"2024-09-10T20:32:57.320006+0000","last_fullsized":"2024-09-10T20:32:57.320006+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":9550,"log_dups_size":0,"ondisk_log_size":9550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39108608,"num_objects":9548,"num_object_clones":0,"num_object_copies":19096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6621,"num_write_kb":26484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9390","reported_seq":9522,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:58.481789+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:32:58.481789+0000","last_peered":"2024-09-10T20:32:58.481789+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:32:58.481789+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":9390,"log_dups_size":0,"ondisk_log_size":9390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38432768,"num_objects":9383,"num_object_clones":0,"num_object_copies":18766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9383,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7383,"num_write_kb":29532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":5857}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190232+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190232+0000","last_peered":"2024-09-10T20:32:18.190232+0000","last_clean":"2024-09-10T20:32:18.190232+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190232+0000","last_undegraded":"2024-09-10T20:32:18.190232+0000","last_fullsized":"2024-09-10T20:32:18.190232+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9502","reported_seq":9610,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:57.332790+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:32:57.332790+0000","last_peered":"2024-09-10T20:32:57.332790+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:32:57.332790+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":9502,"log_dups_size":0,"ondisk_log_size":9502,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38888390,"num_objects":9496,"num_object_clones":0,"num_object_copies":18992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9500,"num_write_kb":37980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5881},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182737+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182737+0000","last_peered":"2024-09-10T20:32:18.182737+0000","last_clean":"2024-09-10T20:32:18.182737+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182737+0000","last_undegraded":"2024-09-10T20:32:18.182737+0000","last_fullsized":"2024-09-10T20:32:18.182737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9470","reported_seq":9598,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:58.489486+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:32:58.489486+0000","last_peered":"2024-09-10T20:32:58.489486+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:32:58.489486+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":9470,"log_dups_size":0,"ondisk_log_size":9470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38768640,"num_objects":9465,"num_object_clones":0,"num_object_copies":18930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7298,"num_write_kb":29192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":5832}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9514","reported_seq":9642,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:58.507929+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:32:58.507929+0000","last_peered":"2024-09-10T20:32:58.507929+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:32:58.507929+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":9514,"log_dups_size":0,"ondisk_log_size":9514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38944768,"num_objects":9508,"num_object_clones":0,"num_object_copies":19016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7300,"num_write_kb":29200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":5898}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9551","reported_seq":9686,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:57.281598+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:32:57.281598+0000","last_peered":"2024-09-10T20:32:57.281598+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:32:57.281598+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":9551,"log_dups_size":0,"ondisk_log_size":9551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39112704,"num_objects":9549,"num_object_clones":0,"num_object_copies":19098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6274,"num_write_kb":25096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5901},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9450","reported_seq":9588,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:57.335520+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:32:57.335520+0000","last_peered":"2024-09-10T20:32:57.335520+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:32:57.335520+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":9450,"log_dups_size":0,"ondisk_log_size":9450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38674432,"num_objects":9442,"num_object_clones":0,"num_object_copies":18884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6144,"num_write_kb":24576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5910},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193200+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193200+0000","last_peered":"2024-09-10T20:32:18.193200+0000","last_clean":"2024-09-10T20:32:18.193200+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193200+0000","last_undegraded":"2024-09-10T20:32:18.193200+0000","last_fullsized":"2024-09-10T20:32:18.193200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9617","reported_seq":9748,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:57.211475+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:32:57.211475+0000","last_peered":"2024-09-10T20:32:57.211475+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:32:57.211475+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":9617,"log_dups_size":0,"ondisk_log_size":9617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39374848,"num_objects":9613,"num_object_clones":0,"num_object_copies":19226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6444,"num_write_kb":25776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5900},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9378","reported_seq":11210,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:32:57.326802+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:32:57.326802+0000","last_peered":"2024-09-10T20:32:57.326802+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:32:57.326802+0000","last_undegraded":"2024-09-10T20:32:57.326802+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":9378,"log_dups_size":0,"ondisk_log_size":9378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38363619,"num_objects":9367,"num_object_clones":0,"num_object_copies":18734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2635,"num_objects_unfound":0,"num_objects_dirty":9367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5985,"num_write_kb":23940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":833,"num_bytes_recovered":3411968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6732},{"shards":"0,2","objects":2635}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":622658985,"num_objects":152019,"num_object_clones":0,"num_object_copies":304038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2635,"num_objects_unfound":0,"num_objects_dirty":152019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116394,"num_write_kb":465556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4440,"num_bytes_recovered":18173952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1265156096,"data_stored":1265069384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152098,"ondisk_log_size":152098,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182453,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":815948,"kb_used_data":513596,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93555892,"statfs":{"total":96636764160,"available":95801233408,"internally_reserved":0,"allocated":525922304,"data_stored":525679616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,1,3,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986225,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":690604,"kb_used_data":420124,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93681236,"statfs":{"total":96636764160,"available":95929585664,"internally_reserved":0,"allocated":430206976,"data_stored":429968365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276950698},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659172,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":540396,"kb_used_data":301852,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93831444,"statfs":{"total":96636764160,"available":96083398656,"internally_reserved":0,"allocated":309096448,"data_stored":308777887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":244248238},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,0,4,5,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":526143488,"data_stored":526139875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":308862976,"data_stored":308783490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":430149632,"data_stored":430146019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:32:58.972 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:32:58.972 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'9471", 'reported_seq': 9549, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:57.321314+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:32:57.321314+0000', 'last_peered': '2024-09-10T20:32:57.321314+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:32:57.321314+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 9471, 'log_dups_size': 0, 'ondisk_log_size': 9471, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38793216, 'num_objects': 9471, 'num_object_clones': 0, 'num_object_copies': 18942, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3607, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9471, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9471, 'num_write_kb': 37884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5864}, {'shards': '0,2', 'objects': 3607}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:58.972 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:32:58.972 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'9411", 'reported_seq': 9547, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:57.249254+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:32:57.249254+0000', 'last_peered': '2024-09-10T20:32:57.249254+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:32:57.249254+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 9411, 'log_dups_size': 0, 'ondisk_log_size': 9411, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38526976, 'num_objects': 9406, 'num_object_clones': 0, 'num_object_copies': 18812, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3616, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9406, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6275, 'num_write_kb': 25100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5792}, {'shards': '0,2', 'objects': 3614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:58.972 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:32:58.973 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'9479", 'reported_seq': 9617, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:57.312075+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:32:57.312075+0000', 'last_peered': '2024-09-10T20:32:57.312075+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:32:57.312075+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 9479, 'log_dups_size': 0, 'ondisk_log_size': 9479, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38801408, 'num_objects': 9473, 'num_object_clones': 0, 'num_object_copies': 18946, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3557, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9473, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6074, 'num_write_kb': 24296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5918}, {'shards': '0,2', 'objects': 3555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:58.973 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:32:58.973 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'9390", 'reported_seq': 9522, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:58.481789+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:32:58.481789+0000', 'last_peered': '2024-09-10T20:32:58.481789+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:32:58.481789+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 9390, 'log_dups_size': 0, 'ondisk_log_size': 9390, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38432768, 'num_objects': 9383, 'num_object_clones': 0, 'num_object_copies': 18766, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3528, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9383, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7383, 'num_write_kb': 29532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 5857}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:32:58.973 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:32:58.973 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'9502", 'reported_seq': 9610, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:57.332790+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:32:57.332790+0000', 'last_peered': '2024-09-10T20:32:57.332790+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:32:57.332790+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 9502, 'log_dups_size': 0, 'ondisk_log_size': 9502, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38888390, 'num_objects': 9496, 'num_object_clones': 0, 'num_object_copies': 18992, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3617, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9496, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9500, 'num_write_kb': 37980, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5881}, {'shards': '0,2', 'objects': 3615}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:58.973 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:32:58.973 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'9470", 'reported_seq': 9598, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:58.489486+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:32:58.489486+0000', 'last_peered': '2024-09-10T20:32:58.489486+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:32:58.489486+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 9470, 'log_dups_size': 0, 'ondisk_log_size': 9470, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38768640, 'num_objects': 9465, 'num_object_clones': 0, 'num_object_copies': 18930, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3635, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9465, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7298, 'num_write_kb': 29192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3633}, {'shards': '2', 'objects': 5832}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:32:58.973 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:32:58.973 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'9514", 'reported_seq': 9642, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:58.507929+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:32:58.507929+0000', 'last_peered': '2024-09-10T20:32:58.507929+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:32:58.507929+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 9514, 'log_dups_size': 0, 'ondisk_log_size': 9514, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38944768, 'num_objects': 9508, 'num_object_clones': 0, 'num_object_copies': 19016, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9508, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7300, 'num_write_kb': 29200, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3610}, {'shards': '2', 'objects': 5898}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:32:58.973 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:32:58.973 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'9551", 'reported_seq': 9686, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:57.281598+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:32:57.281598+0000', 'last_peered': '2024-09-10T20:32:57.281598+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:32:57.281598+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 9551, 'log_dups_size': 0, 'ondisk_log_size': 9551, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39112704, 'num_objects': 9549, 'num_object_clones': 0, 'num_object_copies': 19098, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3649, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9549, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6274, 'num_write_kb': 25096, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5901}, {'shards': '0,2', 'objects': 3648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:58.974 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:32:58.974 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'9450", 'reported_seq': 9588, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:57.335520+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:32:57.335520+0000', 'last_peered': '2024-09-10T20:32:57.335520+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:32:57.335520+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 9450, 'log_dups_size': 0, 'ondisk_log_size': 9450, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38674432, 'num_objects': 9442, 'num_object_clones': 0, 'num_object_copies': 18884, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3535, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9442, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6144, 'num_write_kb': 24576, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5910}, {'shards': '0,2', 'objects': 3532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:58.974 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:32:58.974 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'9617", 'reported_seq': 9748, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:57.211475+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:32:57.211475+0000', 'last_peered': '2024-09-10T20:32:57.211475+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:32:57.211475+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 9617, 'log_dups_size': 0, 'ondisk_log_size': 9617, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39374848, 'num_objects': 9613, 'num_object_clones': 0, 'num_object_copies': 19226, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3715, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9613, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6444, 'num_write_kb': 25776, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5900}, {'shards': '0,2', 'objects': 3713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:58.974 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:32:58.974 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'9378", 'reported_seq': 11210, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:32:57.326802+0000', 'last_change': '2024-09-10T20:32:16.657652+0000', 'last_active': '2024-09-10T20:32:57.326802+0000', 'last_peered': '2024-09-10T20:32:57.326802+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:32:57.326802+0000', 'last_undegraded': '2024-09-10T20:32:57.326802+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 9378, 'log_dups_size': 0, 'ondisk_log_size': 9378, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38363619, 'num_objects': 9367, 'num_object_clones': 0, 'num_object_copies': 18734, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2635, 'num_objects_unfound': 0, 'num_objects_dirty': 9367, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5985, 'num_write_kb': 23940, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 833, 'num_bytes_recovered': 3411968, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6732}, {'shards': '0,2', 'objects': 2635}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:32:58.974 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:32:59.075 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 211 20 152593 152573 2.82398 1.66406 0.0293697 0.0276548 2024-09-10T20:32:59.644 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:32:59.644 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36071/304042 objects degraded (11.864%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":291,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":152021,"data_bytes":622659004,"bytes_used":2096074752,"bytes_avail":287814217728,"bytes_total":289910292480,"degraded_objects":36071,"degraded_total":304042,"degraded_ratio":0.11863821445721316,"misplaced_objects":2635,"misplaced_total":304042,"misplaced_ratio":0.0086665658034087399,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":100993,"recovering_keys_per_sec":0,"num_objects_recovered":296,"num_bytes_recovered":1212416,"num_keys_recovered":0,"write_bytes_sec":2405417,"read_op_per_sec":0,"write_op_per_sec":587},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:32:59.670 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:32:59.671 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:33:00.075 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 212 20 153006 152986 2.81827 1.61328 0.0681177 0.0277115 2024-09-10T20:33:00.218 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:00.220 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:33:00.245 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":184,"stamp":"2024-09-10T20:32:58.898438+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":622659004,"num_objects":152021,"num_object_clones":0,"num_object_copies":304042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2635,"num_objects_unfound":0,"num_objects_dirty":152021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116396,"num_write_kb":465558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4442,"num_bytes_recovered":18173971,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152101,"ondisk_log_size":152101,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2046948,"kb_used_data":1235572,"kb_used_omap":12,"kb_used_meta":811315,"kb_avail":281068572,"statfs":{"total":289910292480,"available":287814217728,"internally_reserved":0,"allocated":1265225728,"data_stored":1264425868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":830786562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,5,2,6,8,8,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28876800,"num_objects":7050,"num_object_clones":0,"num_object_copies":14100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-296,"num_objects_unfound":0,"num_objects_dirty":7050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7050,"num_write_kb":28200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":1212416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004903"},"pg_stats":[{"pgid":"2.f","version":"45'9471","reported_seq":9549,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:57.321314+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:32:57.321314+0000","last_peered":"2024-09-10T20:32:57.321314+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:32:57.321314+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":9471,"log_dups_size":0,"ondisk_log_size":9471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38793216,"num_objects":9471,"num_object_clones":0,"num_object_copies":18942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9471,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9471,"num_write_kb":37884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5864},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9677","reported_seq":9787,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:58.442031+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:32:58.442031+0000","last_peered":"2024-09-10T20:32:58.442031+0000","last_clean":"2024-09-10T20:32:58.442031+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:32:58.442031+0000","last_undegraded":"2024-09-10T20:32:58.442031+0000","last_fullsized":"2024-09-10T20:32:58.442031+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":9677,"log_dups_size":0,"ondisk_log_size":9677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39612416,"num_objects":9671,"num_object_clones":0,"num_object_copies":19342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7863,"num_write_kb":31452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9536","reported_seq":16883,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:58.637197+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:32:58.637197+0000","last_peered":"2024-09-10T20:32:58.637197+0000","last_clean":"2024-09-10T20:32:58.637197+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:32:58.637197+0000","last_undegraded":"2024-09-10T20:32:58.637197+0000","last_fullsized":"2024-09-10T20:32:58.637197+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":9536,"log_dups_size":0,"ondisk_log_size":9536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39043072,"num_objects":9532,"num_object_clones":0,"num_object_copies":19064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7664,"num_write_kb":30656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'9444","reported_seq":9522,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:58.540927+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:32:58.540927+0000","last_peered":"2024-09-10T20:32:58.540927+0000","last_clean":"2024-09-10T20:32:58.540927+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:32:58.540927+0000","last_undegraded":"2024-09-10T20:32:58.540927+0000","last_fullsized":"2024-09-10T20:32:58.540927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":9444,"log_dups_size":0,"ondisk_log_size":9444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38682624,"num_objects":9444,"num_object_clones":0,"num_object_copies":18888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9444,"num_write_kb":37776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9658","reported_seq":9784,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:57.291966+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:32:57.291966+0000","last_peered":"2024-09-10T20:32:57.291966+0000","last_clean":"2024-09-10T20:32:57.291966+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:32:57.291966+0000","last_undegraded":"2024-09-10T20:32:57.291966+0000","last_fullsized":"2024-09-10T20:32:57.291966+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":9658,"log_dups_size":0,"ondisk_log_size":9658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39530496,"num_objects":9651,"num_object_clones":0,"num_object_copies":19302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6654,"num_write_kb":26616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9411","reported_seq":9547,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:57.249254+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:32:57.249254+0000","last_peered":"2024-09-10T20:32:57.249254+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:32:57.249254+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":9411,"log_dups_size":0,"ondisk_log_size":9411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38526976,"num_objects":9406,"num_object_clones":0,"num_object_copies":18812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6275,"num_write_kb":25100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5792},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9479","reported_seq":9617,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:57.312075+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:32:57.312075+0000","last_peered":"2024-09-10T20:32:57.312075+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:32:57.312075+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":9479,"log_dups_size":0,"ondisk_log_size":9479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38801408,"num_objects":9473,"num_object_clones":0,"num_object_copies":18946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6074,"num_write_kb":24296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5918},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9550","reported_seq":9674,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:57.320006+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:32:57.320006+0000","last_peered":"2024-09-10T20:32:57.320006+0000","last_clean":"2024-09-10T20:32:57.320006+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:32:57.320006+0000","last_undegraded":"2024-09-10T20:32:57.320006+0000","last_fullsized":"2024-09-10T20:32:57.320006+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":9550,"log_dups_size":0,"ondisk_log_size":9550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39108608,"num_objects":9548,"num_object_clones":0,"num_object_copies":19096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6621,"num_write_kb":26484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9390","reported_seq":9522,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:58.481789+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:32:58.481789+0000","last_peered":"2024-09-10T20:32:58.481789+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:32:58.481789+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":9390,"log_dups_size":0,"ondisk_log_size":9390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38432768,"num_objects":9383,"num_object_clones":0,"num_object_copies":18766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9383,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7383,"num_write_kb":29532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":5857}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190232+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190232+0000","last_peered":"2024-09-10T20:32:18.190232+0000","last_clean":"2024-09-10T20:32:18.190232+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190232+0000","last_undegraded":"2024-09-10T20:32:18.190232+0000","last_fullsized":"2024-09-10T20:32:18.190232+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9502","reported_seq":9610,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:57.332790+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:32:57.332790+0000","last_peered":"2024-09-10T20:32:57.332790+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:32:57.332790+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":9502,"log_dups_size":0,"ondisk_log_size":9502,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38888390,"num_objects":9496,"num_object_clones":0,"num_object_copies":18992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9500,"num_write_kb":37980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5881},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182737+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182737+0000","last_peered":"2024-09-10T20:32:18.182737+0000","last_clean":"2024-09-10T20:32:18.182737+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182737+0000","last_undegraded":"2024-09-10T20:32:18.182737+0000","last_fullsized":"2024-09-10T20:32:18.182737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9470","reported_seq":9598,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:58.489486+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:32:58.489486+0000","last_peered":"2024-09-10T20:32:58.489486+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:32:58.489486+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":9470,"log_dups_size":0,"ondisk_log_size":9470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38768640,"num_objects":9465,"num_object_clones":0,"num_object_copies":18930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7298,"num_write_kb":29192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":5832}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9514","reported_seq":9642,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:58.507929+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:32:58.507929+0000","last_peered":"2024-09-10T20:32:58.507929+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:32:58.507929+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":9514,"log_dups_size":0,"ondisk_log_size":9514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38944768,"num_objects":9508,"num_object_clones":0,"num_object_copies":19016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7300,"num_write_kb":29200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":5898}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9551","reported_seq":9686,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:57.281598+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:32:57.281598+0000","last_peered":"2024-09-10T20:32:57.281598+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:32:57.281598+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":9551,"log_dups_size":0,"ondisk_log_size":9551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39112704,"num_objects":9549,"num_object_clones":0,"num_object_copies":19098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6274,"num_write_kb":25096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5901},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9450","reported_seq":9588,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:57.335520+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:32:57.335520+0000","last_peered":"2024-09-10T20:32:57.335520+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:32:57.335520+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":9450,"log_dups_size":0,"ondisk_log_size":9450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38674432,"num_objects":9442,"num_object_clones":0,"num_object_copies":18884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6144,"num_write_kb":24576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5910},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193200+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193200+0000","last_peered":"2024-09-10T20:32:18.193200+0000","last_clean":"2024-09-10T20:32:18.193200+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193200+0000","last_undegraded":"2024-09-10T20:32:18.193200+0000","last_fullsized":"2024-09-10T20:32:18.193200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9617","reported_seq":9748,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:57.211475+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:32:57.211475+0000","last_peered":"2024-09-10T20:32:57.211475+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:32:57.211475+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":9617,"log_dups_size":0,"ondisk_log_size":9617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39374848,"num_objects":9613,"num_object_clones":0,"num_object_copies":19226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6444,"num_write_kb":25776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5900},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9378","reported_seq":11210,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:32:57.326802+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:32:57.326802+0000","last_peered":"2024-09-10T20:32:57.326802+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:32:57.326802+0000","last_undegraded":"2024-09-10T20:32:57.326802+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":9378,"log_dups_size":0,"ondisk_log_size":9378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38363619,"num_objects":9367,"num_object_clones":0,"num_object_copies":18734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2635,"num_objects_unfound":0,"num_objects_dirty":9367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5985,"num_write_kb":23940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":833,"num_bytes_recovered":3411968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6732},{"shards":"0,2","objects":2635}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":622658985,"num_objects":152019,"num_object_clones":0,"num_object_copies":304038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2635,"num_objects_unfound":0,"num_objects_dirty":152019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116394,"num_write_kb":465556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4440,"num_bytes_recovered":18173952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1265156096,"data_stored":1265069384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152098,"ondisk_log_size":152098,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182453,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":815948,"kb_used_data":513596,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93555892,"statfs":{"total":96636764160,"available":95801233408,"internally_reserved":0,"allocated":525922304,"data_stored":525679616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,1,3,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986225,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":690604,"kb_used_data":420124,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93681236,"statfs":{"total":96636764160,"available":95929585664,"internally_reserved":0,"allocated":430206976,"data_stored":429968365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276950698},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659172,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":540396,"kb_used_data":301852,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93831444,"statfs":{"total":96636764160,"available":96083398656,"internally_reserved":0,"allocated":309096448,"data_stored":308777887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":244248238},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,0,4,5,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":526143488,"data_stored":526139875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":308862976,"data_stored":308783490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":430149632,"data_stored":430146019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:33:01.075 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 213 20 153475 153455 2.81364 1.83203 0.0214752 0.027759 2024-09-10T20:33:02.075 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 214 20 153947 153927 2.80911 1.84375 0.0229971 0.0278045 2024-09-10T20:33:03.075 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 215 20 154422 154402 2.80467 1.85547 0.00998203 0.027847 2024-09-10T20:33:03.247 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:33:03.798 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:03.801 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:33:03.829 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":186,"stamp":"2024-09-10T20:33:02.899754+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":628446652,"num_objects":153434,"num_object_clones":0,"num_object_copies":306868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2533,"num_objects_unfound":0,"num_objects_dirty":153434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":117809,"num_write_kb":471210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4544,"num_bytes_recovered":18591763,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153514,"ondisk_log_size":153514,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2057756,"kb_used_data":1242220,"kb_used_omap":12,"kb_used_meta":815475,"kb_avail":281057764,"statfs":{"total":289910292480,"available":287803150336,"internally_reserved":0,"allocated":1272033280,"data_stored":1271233420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":835046402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,4,1,7,10,9,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24870912,"num_objects":6072,"num_object_clones":0,"num_object_copies":12144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-308,"num_objects_unfound":0,"num_objects_dirty":6072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6072,"num_write_kb":24288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":1261568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004468"},"pg_stats":[{"pgid":"2.f","version":"45'9621","reported_seq":9698,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:02.399958+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:33:02.399958+0000","last_peered":"2024-09-10T20:33:02.399958+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:33:02.399958+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":9621,"log_dups_size":0,"ondisk_log_size":9621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39407616,"num_objects":9621,"num_object_clones":0,"num_object_copies":19242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9621,"num_write_kb":38484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6014},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9677","reported_seq":9787,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:58.442031+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:32:58.442031+0000","last_peered":"2024-09-10T20:32:58.442031+0000","last_clean":"2024-09-10T20:32:58.442031+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:32:58.442031+0000","last_undegraded":"2024-09-10T20:32:58.442031+0000","last_fullsized":"2024-09-10T20:32:58.442031+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":9677,"log_dups_size":0,"ondisk_log_size":9677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39612416,"num_objects":9671,"num_object_clones":0,"num_object_copies":19342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7863,"num_write_kb":31452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9536","reported_seq":16883,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:58.637197+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:32:58.637197+0000","last_peered":"2024-09-10T20:32:58.637197+0000","last_clean":"2024-09-10T20:32:58.637197+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:32:58.637197+0000","last_undegraded":"2024-09-10T20:32:58.637197+0000","last_fullsized":"2024-09-10T20:32:58.637197+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":9536,"log_dups_size":0,"ondisk_log_size":9536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39043072,"num_objects":9532,"num_object_clones":0,"num_object_copies":19064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7664,"num_write_kb":30656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'9444","reported_seq":9522,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:58.540927+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:32:58.540927+0000","last_peered":"2024-09-10T20:32:58.540927+0000","last_clean":"2024-09-10T20:32:58.540927+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:32:58.540927+0000","last_undegraded":"2024-09-10T20:32:58.540927+0000","last_fullsized":"2024-09-10T20:32:58.540927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":9444,"log_dups_size":0,"ondisk_log_size":9444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38682624,"num_objects":9444,"num_object_clones":0,"num_object_copies":18888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9444,"num_write_kb":37776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9793","reported_seq":9919,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:02.317652+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:33:02.317652+0000","last_peered":"2024-09-10T20:33:02.317652+0000","last_clean":"2024-09-10T20:33:02.317652+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:33:02.317652+0000","last_undegraded":"2024-09-10T20:33:02.317652+0000","last_fullsized":"2024-09-10T20:33:02.317652+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":9793,"log_dups_size":0,"ondisk_log_size":9793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40083456,"num_objects":9786,"num_object_clones":0,"num_object_copies":19572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6789,"num_write_kb":27156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9563","reported_seq":9699,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:02.285674+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:33:02.285674+0000","last_peered":"2024-09-10T20:33:02.285674+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:33:02.285674+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":9563,"log_dups_size":0,"ondisk_log_size":9563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39149568,"num_objects":9558,"num_object_clones":0,"num_object_copies":19116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6427,"num_write_kb":25708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5944},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9594","reported_seq":9732,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:02.336632+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:33:02.336632+0000","last_peered":"2024-09-10T20:33:02.336632+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:33:02.336632+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":9594,"log_dups_size":0,"ondisk_log_size":9594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39272448,"num_objects":9588,"num_object_clones":0,"num_object_copies":19176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6189,"num_write_kb":24756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6033},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9672","reported_seq":9796,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:02.298893+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:33:02.298893+0000","last_peered":"2024-09-10T20:33:02.298893+0000","last_clean":"2024-09-10T20:33:02.298893+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:33:02.298893+0000","last_undegraded":"2024-09-10T20:33:02.298893+0000","last_fullsized":"2024-09-10T20:33:02.298893+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":9672,"log_dups_size":0,"ondisk_log_size":9672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39608320,"num_objects":9670,"num_object_clones":0,"num_object_copies":19340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6743,"num_write_kb":26972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9390","reported_seq":9522,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:58.481789+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:32:58.481789+0000","last_peered":"2024-09-10T20:32:58.481789+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:32:58.481789+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":9390,"log_dups_size":0,"ondisk_log_size":9390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38432768,"num_objects":9383,"num_object_clones":0,"num_object_copies":18766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9383,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7383,"num_write_kb":29532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":5857}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190232+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190232+0000","last_peered":"2024-09-10T20:32:18.190232+0000","last_clean":"2024-09-10T20:32:18.190232+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190232+0000","last_undegraded":"2024-09-10T20:32:18.190232+0000","last_fullsized":"2024-09-10T20:32:18.190232+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9657","reported_seq":9765,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:02.378001+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:33:02.378001+0000","last_peered":"2024-09-10T20:33:02.378001+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:33:02.378001+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":9657,"log_dups_size":0,"ondisk_log_size":9657,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39523270,"num_objects":9651,"num_object_clones":0,"num_object_copies":19302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9655,"num_write_kb":38600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6036},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182737+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182737+0000","last_peered":"2024-09-10T20:32:18.182737+0000","last_clean":"2024-09-10T20:32:18.182737+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182737+0000","last_undegraded":"2024-09-10T20:32:18.182737+0000","last_fullsized":"2024-09-10T20:32:18.182737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9470","reported_seq":9598,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:58.489486+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:32:58.489486+0000","last_peered":"2024-09-10T20:32:58.489486+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:32:58.489486+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":9470,"log_dups_size":0,"ondisk_log_size":9470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38768640,"num_objects":9465,"num_object_clones":0,"num_object_copies":18930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7298,"num_write_kb":29192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":5832}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9514","reported_seq":9642,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:58.507929+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:32:58.507929+0000","last_peered":"2024-09-10T20:32:58.507929+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:32:58.507929+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":9514,"log_dups_size":0,"ondisk_log_size":9514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38944768,"num_objects":9508,"num_object_clones":0,"num_object_copies":19016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7300,"num_write_kb":29200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":5898}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9683","reported_seq":9817,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:02.404580+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:33:02.404580+0000","last_peered":"2024-09-10T20:33:02.404580+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:33:02.404580+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":9683,"log_dups_size":0,"ondisk_log_size":9683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39653376,"num_objects":9681,"num_object_clones":0,"num_object_copies":19362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6406,"num_write_kb":25624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6033},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9609","reported_seq":9747,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:02.395456+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:33:02.395456+0000","last_peered":"2024-09-10T20:33:02.395456+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:33:02.395456+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":9609,"log_dups_size":0,"ondisk_log_size":9609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39325696,"num_objects":9601,"num_object_clones":0,"num_object_copies":19202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6303,"num_write_kb":25212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6069},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193200+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193200+0000","last_peered":"2024-09-10T20:32:18.193200+0000","last_clean":"2024-09-10T20:32:18.193200+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193200+0000","last_undegraded":"2024-09-10T20:32:18.193200+0000","last_fullsized":"2024-09-10T20:32:18.193200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9759","reported_seq":9890,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:02.422577+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:33:02.422577+0000","last_peered":"2024-09-10T20:33:02.422577+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:33:02.422577+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":9759,"log_dups_size":0,"ondisk_log_size":9759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39956480,"num_objects":9755,"num_object_clones":0,"num_object_copies":19510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6586,"num_write_kb":26344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6042},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9529","reported_seq":11564,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:33:02.434324+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:33:02.434324+0000","last_peered":"2024-09-10T20:33:02.434324+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:33:02.434324+0000","last_undegraded":"2024-09-10T20:33:02.434324+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":9529,"log_dups_size":0,"ondisk_log_size":9529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38982115,"num_objects":9518,"num_object_clones":0,"num_object_copies":19036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2533,"num_objects_unfound":0,"num_objects_dirty":9518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6136,"num_write_kb":24544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":935,"num_bytes_recovered":3829760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6985},{"shards":"0,2","objects":2533}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":628446633,"num_objects":153432,"num_object_clones":0,"num_object_copies":306864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2533,"num_objects_unfound":0,"num_objects_dirty":153432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":117807,"num_write_kb":471208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4542,"num_bytes_recovered":18591744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1272094720,"data_stored":1272008008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153511,"ondisk_log_size":153511,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182454,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":826756,"kb_used_data":520244,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93545084,"statfs":{"total":96636764160,"available":95790166016,"internally_reserved":0,"allocated":532729856,"data_stored":532487168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,2,5,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986225,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":690604,"kb_used_data":420124,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93681236,"statfs":{"total":96636764160,"available":95929585664,"internally_reserved":0,"allocated":430206976,"data_stored":429968365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276950698},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659172,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":540396,"kb_used_data":301852,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93831444,"statfs":{"total":96636764160,"available":96083398656,"internally_reserved":0,"allocated":309096448,"data_stored":308777887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":244248238},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,0,4,5,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":533082112,"data_stored":533078499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":308862976,"data_stored":308783490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":430149632,"data_stored":430146019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:33:03.830 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:33:04.076 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 216 19 154845 154826 2.79935 1.65625 0.0900884 0.0278999 2024-09-10T20:33:04.382 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:04.385 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:33:04.412 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":186,"stamp":"2024-09-10T20:33:02.899754+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":628446652,"num_objects":153434,"num_object_clones":0,"num_object_copies":306868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2533,"num_objects_unfound":0,"num_objects_dirty":153434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":117809,"num_write_kb":471210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4544,"num_bytes_recovered":18591763,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153514,"ondisk_log_size":153514,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2057756,"kb_used_data":1242220,"kb_used_omap":12,"kb_used_meta":815475,"kb_avail":281057764,"statfs":{"total":289910292480,"available":287803150336,"internally_reserved":0,"allocated":1272033280,"data_stored":1271233420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":835046402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,4,1,7,10,9,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24870912,"num_objects":6072,"num_object_clones":0,"num_object_copies":12144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-308,"num_objects_unfound":0,"num_objects_dirty":6072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6072,"num_write_kb":24288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":1261568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004468"},"pg_stats":[{"pgid":"2.f","version":"45'9621","reported_seq":9698,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:02.399958+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:33:02.399958+0000","last_peered":"2024-09-10T20:33:02.399958+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:33:02.399958+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":9621,"log_dups_size":0,"ondisk_log_size":9621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39407616,"num_objects":9621,"num_object_clones":0,"num_object_copies":19242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9621,"num_write_kb":38484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6014},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9677","reported_seq":9787,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:58.442031+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:32:58.442031+0000","last_peered":"2024-09-10T20:32:58.442031+0000","last_clean":"2024-09-10T20:32:58.442031+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:32:58.442031+0000","last_undegraded":"2024-09-10T20:32:58.442031+0000","last_fullsized":"2024-09-10T20:32:58.442031+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":9677,"log_dups_size":0,"ondisk_log_size":9677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39612416,"num_objects":9671,"num_object_clones":0,"num_object_copies":19342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7863,"num_write_kb":31452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9536","reported_seq":16883,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:58.637197+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:32:58.637197+0000","last_peered":"2024-09-10T20:32:58.637197+0000","last_clean":"2024-09-10T20:32:58.637197+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:32:58.637197+0000","last_undegraded":"2024-09-10T20:32:58.637197+0000","last_fullsized":"2024-09-10T20:32:58.637197+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":9536,"log_dups_size":0,"ondisk_log_size":9536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39043072,"num_objects":9532,"num_object_clones":0,"num_object_copies":19064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7664,"num_write_kb":30656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'9444","reported_seq":9522,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:58.540927+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:32:58.540927+0000","last_peered":"2024-09-10T20:32:58.540927+0000","last_clean":"2024-09-10T20:32:58.540927+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:32:58.540927+0000","last_undegraded":"2024-09-10T20:32:58.540927+0000","last_fullsized":"2024-09-10T20:32:58.540927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":9444,"log_dups_size":0,"ondisk_log_size":9444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38682624,"num_objects":9444,"num_object_clones":0,"num_object_copies":18888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9444,"num_write_kb":37776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9793","reported_seq":9919,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:02.317652+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:33:02.317652+0000","last_peered":"2024-09-10T20:33:02.317652+0000","last_clean":"2024-09-10T20:33:02.317652+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:33:02.317652+0000","last_undegraded":"2024-09-10T20:33:02.317652+0000","last_fullsized":"2024-09-10T20:33:02.317652+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":9793,"log_dups_size":0,"ondisk_log_size":9793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40083456,"num_objects":9786,"num_object_clones":0,"num_object_copies":19572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6789,"num_write_kb":27156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9563","reported_seq":9699,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:02.285674+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:33:02.285674+0000","last_peered":"2024-09-10T20:33:02.285674+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:33:02.285674+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":9563,"log_dups_size":0,"ondisk_log_size":9563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39149568,"num_objects":9558,"num_object_clones":0,"num_object_copies":19116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6427,"num_write_kb":25708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5944},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9594","reported_seq":9732,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:02.336632+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:33:02.336632+0000","last_peered":"2024-09-10T20:33:02.336632+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:33:02.336632+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":9594,"log_dups_size":0,"ondisk_log_size":9594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39272448,"num_objects":9588,"num_object_clones":0,"num_object_copies":19176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6189,"num_write_kb":24756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6033},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9672","reported_seq":9796,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:02.298893+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:33:02.298893+0000","last_peered":"2024-09-10T20:33:02.298893+0000","last_clean":"2024-09-10T20:33:02.298893+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:33:02.298893+0000","last_undegraded":"2024-09-10T20:33:02.298893+0000","last_fullsized":"2024-09-10T20:33:02.298893+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":9672,"log_dups_size":0,"ondisk_log_size":9672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39608320,"num_objects":9670,"num_object_clones":0,"num_object_copies":19340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6743,"num_write_kb":26972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9390","reported_seq":9522,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:58.481789+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:32:58.481789+0000","last_peered":"2024-09-10T20:32:58.481789+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:32:58.481789+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":9390,"log_dups_size":0,"ondisk_log_size":9390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38432768,"num_objects":9383,"num_object_clones":0,"num_object_copies":18766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9383,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7383,"num_write_kb":29532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":5857}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190232+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190232+0000","last_peered":"2024-09-10T20:32:18.190232+0000","last_clean":"2024-09-10T20:32:18.190232+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190232+0000","last_undegraded":"2024-09-10T20:32:18.190232+0000","last_fullsized":"2024-09-10T20:32:18.190232+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9657","reported_seq":9765,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:02.378001+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:33:02.378001+0000","last_peered":"2024-09-10T20:33:02.378001+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:33:02.378001+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":9657,"log_dups_size":0,"ondisk_log_size":9657,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39523270,"num_objects":9651,"num_object_clones":0,"num_object_copies":19302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9655,"num_write_kb":38600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6036},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182737+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182737+0000","last_peered":"2024-09-10T20:32:18.182737+0000","last_clean":"2024-09-10T20:32:18.182737+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182737+0000","last_undegraded":"2024-09-10T20:32:18.182737+0000","last_fullsized":"2024-09-10T20:32:18.182737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9470","reported_seq":9598,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:58.489486+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:32:58.489486+0000","last_peered":"2024-09-10T20:32:58.489486+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:32:58.489486+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":9470,"log_dups_size":0,"ondisk_log_size":9470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38768640,"num_objects":9465,"num_object_clones":0,"num_object_copies":18930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7298,"num_write_kb":29192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":5832}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9514","reported_seq":9642,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:32:58.507929+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:32:58.507929+0000","last_peered":"2024-09-10T20:32:58.507929+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:32:58.507929+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":9514,"log_dups_size":0,"ondisk_log_size":9514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38944768,"num_objects":9508,"num_object_clones":0,"num_object_copies":19016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7300,"num_write_kb":29200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":5898}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9683","reported_seq":9817,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:02.404580+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:33:02.404580+0000","last_peered":"2024-09-10T20:33:02.404580+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:33:02.404580+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":9683,"log_dups_size":0,"ondisk_log_size":9683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39653376,"num_objects":9681,"num_object_clones":0,"num_object_copies":19362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6406,"num_write_kb":25624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6033},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9609","reported_seq":9747,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:02.395456+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:33:02.395456+0000","last_peered":"2024-09-10T20:33:02.395456+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:33:02.395456+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":9609,"log_dups_size":0,"ondisk_log_size":9609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39325696,"num_objects":9601,"num_object_clones":0,"num_object_copies":19202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6303,"num_write_kb":25212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6069},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193200+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193200+0000","last_peered":"2024-09-10T20:32:18.193200+0000","last_clean":"2024-09-10T20:32:18.193200+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193200+0000","last_undegraded":"2024-09-10T20:32:18.193200+0000","last_fullsized":"2024-09-10T20:32:18.193200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9759","reported_seq":9890,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:02.422577+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:33:02.422577+0000","last_peered":"2024-09-10T20:33:02.422577+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:33:02.422577+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":9759,"log_dups_size":0,"ondisk_log_size":9759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39956480,"num_objects":9755,"num_object_clones":0,"num_object_copies":19510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6586,"num_write_kb":26344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6042},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9529","reported_seq":11564,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:33:02.434324+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:33:02.434324+0000","last_peered":"2024-09-10T20:33:02.434324+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:33:02.434324+0000","last_undegraded":"2024-09-10T20:33:02.434324+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":9529,"log_dups_size":0,"ondisk_log_size":9529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38982115,"num_objects":9518,"num_object_clones":0,"num_object_copies":19036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2533,"num_objects_unfound":0,"num_objects_dirty":9518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6136,"num_write_kb":24544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":935,"num_bytes_recovered":3829760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6985},{"shards":"0,2","objects":2533}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":628446633,"num_objects":153432,"num_object_clones":0,"num_object_copies":306864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2533,"num_objects_unfound":0,"num_objects_dirty":153432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":117807,"num_write_kb":471208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4542,"num_bytes_recovered":18591744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1272094720,"data_stored":1272008008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153511,"ondisk_log_size":153511,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182454,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":826756,"kb_used_data":520244,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93545084,"statfs":{"total":96636764160,"available":95790166016,"internally_reserved":0,"allocated":532729856,"data_stored":532487168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,2,5,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986225,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":690604,"kb_used_data":420124,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93681236,"statfs":{"total":96636764160,"available":95929585664,"internally_reserved":0,"allocated":430206976,"data_stored":429968365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276950698},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659172,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":540396,"kb_used_data":301852,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93831444,"statfs":{"total":96636764160,"available":96083398656,"internally_reserved":0,"allocated":309096448,"data_stored":308777887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":244248238},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,0,4,5,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":533082112,"data_stored":533078499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":308862976,"data_stored":308783490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":430149632,"data_stored":430146019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:33:04.414 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:33:04.414 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'9621", 'reported_seq': 9698, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:02.399958+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:33:02.399958+0000', 'last_peered': '2024-09-10T20:33:02.399958+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:33:02.399958+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 9621, 'log_dups_size': 0, 'ondisk_log_size': 9621, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39407616, 'num_objects': 9621, 'num_object_clones': 0, 'num_object_copies': 19242, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3607, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9621, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9621, 'num_write_kb': 38484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6014}, {'shards': '0,2', 'objects': 3607}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:04.414 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:33:04.414 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'9563", 'reported_seq': 9699, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:02.285674+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:33:02.285674+0000', 'last_peered': '2024-09-10T20:33:02.285674+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:33:02.285674+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 9563, 'log_dups_size': 0, 'ondisk_log_size': 9563, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39149568, 'num_objects': 9558, 'num_object_clones': 0, 'num_object_copies': 19116, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3616, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9558, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6427, 'num_write_kb': 25708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5944}, {'shards': '0,2', 'objects': 3614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:04.414 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:33:04.414 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'9594", 'reported_seq': 9732, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:02.336632+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:33:02.336632+0000', 'last_peered': '2024-09-10T20:33:02.336632+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:33:02.336632+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 9594, 'log_dups_size': 0, 'ondisk_log_size': 9594, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39272448, 'num_objects': 9588, 'num_object_clones': 0, 'num_object_copies': 19176, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3557, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9588, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6189, 'num_write_kb': 24756, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6033}, {'shards': '0,2', 'objects': 3555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:04.415 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:33:04.415 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'9390", 'reported_seq': 9522, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:58.481789+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:32:58.481789+0000', 'last_peered': '2024-09-10T20:32:58.481789+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:32:58.481789+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 9390, 'log_dups_size': 0, 'ondisk_log_size': 9390, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38432768, 'num_objects': 9383, 'num_object_clones': 0, 'num_object_copies': 18766, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3528, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9383, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7383, 'num_write_kb': 29532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 5857}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:33:04.415 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:33:04.415 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'9657", 'reported_seq': 9765, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:02.378001+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:33:02.378001+0000', 'last_peered': '2024-09-10T20:33:02.378001+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:33:02.378001+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 9657, 'log_dups_size': 0, 'ondisk_log_size': 9657, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39523270, 'num_objects': 9651, 'num_object_clones': 0, 'num_object_copies': 19302, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3617, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9651, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9655, 'num_write_kb': 38600, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6036}, {'shards': '0,2', 'objects': 3615}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:04.415 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:33:04.415 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'9470", 'reported_seq': 9598, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:58.489486+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:32:58.489486+0000', 'last_peered': '2024-09-10T20:32:58.489486+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:32:58.489486+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 9470, 'log_dups_size': 0, 'ondisk_log_size': 9470, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38768640, 'num_objects': 9465, 'num_object_clones': 0, 'num_object_copies': 18930, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3635, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9465, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7298, 'num_write_kb': 29192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3633}, {'shards': '2', 'objects': 5832}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:33:04.415 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:33:04.415 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'9514", 'reported_seq': 9642, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:32:58.507929+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:32:58.507929+0000', 'last_peered': '2024-09-10T20:32:58.507929+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:32:58.507929+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 9514, 'log_dups_size': 0, 'ondisk_log_size': 9514, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38944768, 'num_objects': 9508, 'num_object_clones': 0, 'num_object_copies': 19016, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9508, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7300, 'num_write_kb': 29200, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3610}, {'shards': '2', 'objects': 5898}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:33:04.415 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:33:04.415 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'9683", 'reported_seq': 9817, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:02.404580+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:33:02.404580+0000', 'last_peered': '2024-09-10T20:33:02.404580+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:33:02.404580+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 9683, 'log_dups_size': 0, 'ondisk_log_size': 9683, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39653376, 'num_objects': 9681, 'num_object_clones': 0, 'num_object_copies': 19362, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3649, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9681, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6406, 'num_write_kb': 25624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6033}, {'shards': '0,2', 'objects': 3648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:04.415 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:33:04.416 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'9609", 'reported_seq': 9747, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:02.395456+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:33:02.395456+0000', 'last_peered': '2024-09-10T20:33:02.395456+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:33:02.395456+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 9609, 'log_dups_size': 0, 'ondisk_log_size': 9609, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39325696, 'num_objects': 9601, 'num_object_clones': 0, 'num_object_copies': 19202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3535, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9601, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6303, 'num_write_kb': 25212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6069}, {'shards': '0,2', 'objects': 3532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:04.416 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:33:04.416 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'9759", 'reported_seq': 9890, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:02.422577+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:33:02.422577+0000', 'last_peered': '2024-09-10T20:33:02.422577+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:33:02.422577+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 9759, 'log_dups_size': 0, 'ondisk_log_size': 9759, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39956480, 'num_objects': 9755, 'num_object_clones': 0, 'num_object_copies': 19510, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3715, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9755, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6586, 'num_write_kb': 26344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6042}, {'shards': '0,2', 'objects': 3713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:04.416 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:33:04.416 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'9529", 'reported_seq': 11564, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:33:02.434324+0000', 'last_change': '2024-09-10T20:32:16.657652+0000', 'last_active': '2024-09-10T20:33:02.434324+0000', 'last_peered': '2024-09-10T20:33:02.434324+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:33:02.434324+0000', 'last_undegraded': '2024-09-10T20:33:02.434324+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 9529, 'log_dups_size': 0, 'ondisk_log_size': 9529, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38982115, 'num_objects': 9518, 'num_object_clones': 0, 'num_object_copies': 19036, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2533, 'num_objects_unfound': 0, 'num_objects_dirty': 9518, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6136, 'num_write_kb': 24544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 935, 'num_bytes_recovered': 3829760, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6985}, {'shards': '0,2', 'objects': 2533}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:04.416 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:33:05.076 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 217 20 155269 155249 2.79407 1.65234 0.0522128 0.0279534 2024-09-10T20:33:05.126 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:05.126 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36071/306868 objects degraded (11.755%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":296,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":153434,"data_bytes":628446652,"bytes_used":2107142144,"bytes_avail":287803150336,"bytes_total":289910292480,"degraded_objects":36071,"degraded_total":306868,"degraded_ratio":0.11754565480923394,"misplaced_objects":2533,"misplaced_total":306868,"misplaced_ratio":0.0082543634396548358,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":105091,"recovering_keys_per_sec":0,"num_objects_recovered":308,"num_bytes_recovered":1261568,"num_keys_recovered":0,"write_bytes_sec":2071804,"read_op_per_sec":0,"write_op_per_sec":505},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:33:05.154 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:33:05.155 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:33:05.738 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:05.740 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:33:05.769 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":187,"stamp":"2024-09-10T20:33:04.900309+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":632059324,"num_objects":154316,"num_object_clones":0,"num_object_copies":308632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2533,"num_objects_unfound":0,"num_objects_dirty":154316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":118691,"num_write_kb":474738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4544,"num_bytes_recovered":18591763,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154396,"ondisk_log_size":154396,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2061152,"kb_used_data":1253936,"kb_used_omap":12,"kb_used_meta":807155,"kb_avail":281054368,"statfs":{"total":289910292480,"available":287799672832,"internally_reserved":0,"allocated":1284030464,"data_stored":1283230604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":826526722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,5,5,5,12,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22347776,"num_objects":5456,"num_object_clones":0,"num_object_copies":10912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-206,"num_objects_unfound":0,"num_objects_dirty":5456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5456,"num_write_kb":21824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":843776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004188"},"pg_stats":[{"pgid":"2.f","version":"45'9621","reported_seq":9698,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:02.399958+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:33:02.399958+0000","last_peered":"2024-09-10T20:33:02.399958+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:33:02.399958+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":9621,"log_dups_size":0,"ondisk_log_size":9621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39407616,"num_objects":9621,"num_object_clones":0,"num_object_copies":19242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9621,"num_write_kb":38484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6014},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9830","reported_seq":9940,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:03.452061+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:33:03.452061+0000","last_peered":"2024-09-10T20:33:03.452061+0000","last_clean":"2024-09-10T20:33:03.452061+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:33:03.452061+0000","last_undegraded":"2024-09-10T20:33:03.452061+0000","last_fullsized":"2024-09-10T20:33:03.452061+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":9830,"log_dups_size":0,"ondisk_log_size":9830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40239104,"num_objects":9824,"num_object_clones":0,"num_object_copies":19648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8016,"num_write_kb":32064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9681","reported_seq":17028,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:03.812598+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:33:03.812598+0000","last_peered":"2024-09-10T20:33:03.812598+0000","last_clean":"2024-09-10T20:33:03.812598+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:33:03.812598+0000","last_undegraded":"2024-09-10T20:33:03.812598+0000","last_fullsized":"2024-09-10T20:33:03.812598+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":9681,"log_dups_size":0,"ondisk_log_size":9681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39636992,"num_objects":9677,"num_object_clones":0,"num_object_copies":19354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7809,"num_write_kb":31236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'9587","reported_seq":9667,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:03.387811+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:33:03.387811+0000","last_peered":"2024-09-10T20:33:03.387811+0000","last_clean":"2024-09-10T20:33:03.387811+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:33:03.387811+0000","last_undegraded":"2024-09-10T20:33:03.387811+0000","last_fullsized":"2024-09-10T20:33:03.387811+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":9587,"log_dups_size":0,"ondisk_log_size":9587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39268352,"num_objects":9587,"num_object_clones":0,"num_object_copies":19174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9587,"num_write_kb":38348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9793","reported_seq":9919,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:02.317652+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:33:02.317652+0000","last_peered":"2024-09-10T20:33:02.317652+0000","last_clean":"2024-09-10T20:33:02.317652+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:33:02.317652+0000","last_undegraded":"2024-09-10T20:33:02.317652+0000","last_fullsized":"2024-09-10T20:33:02.317652+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":9793,"log_dups_size":0,"ondisk_log_size":9793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40083456,"num_objects":9786,"num_object_clones":0,"num_object_copies":19572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6789,"num_write_kb":27156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9563","reported_seq":9699,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:02.285674+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:33:02.285674+0000","last_peered":"2024-09-10T20:33:02.285674+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:33:02.285674+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":9563,"log_dups_size":0,"ondisk_log_size":9563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39149568,"num_objects":9558,"num_object_clones":0,"num_object_copies":19116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6427,"num_write_kb":25708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5944},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9594","reported_seq":9732,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:02.336632+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:33:02.336632+0000","last_peered":"2024-09-10T20:33:02.336632+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:33:02.336632+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":9594,"log_dups_size":0,"ondisk_log_size":9594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39272448,"num_objects":9588,"num_object_clones":0,"num_object_copies":19176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6189,"num_write_kb":24756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6033},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9672","reported_seq":9796,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:02.298893+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:33:02.298893+0000","last_peered":"2024-09-10T20:33:02.298893+0000","last_clean":"2024-09-10T20:33:02.298893+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:33:02.298893+0000","last_undegraded":"2024-09-10T20:33:02.298893+0000","last_fullsized":"2024-09-10T20:33:02.298893+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":9672,"log_dups_size":0,"ondisk_log_size":9672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39608320,"num_objects":9670,"num_object_clones":0,"num_object_copies":19340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6743,"num_write_kb":26972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9529","reported_seq":9661,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:03.528989+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:33:03.528989+0000","last_peered":"2024-09-10T20:33:03.528989+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:33:03.528989+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":9529,"log_dups_size":0,"ondisk_log_size":9529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39002112,"num_objects":9522,"num_object_clones":0,"num_object_copies":19044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7522,"num_write_kb":30088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":5996}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190232+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190232+0000","last_peered":"2024-09-10T20:32:18.190232+0000","last_clean":"2024-09-10T20:32:18.190232+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190232+0000","last_undegraded":"2024-09-10T20:32:18.190232+0000","last_fullsized":"2024-09-10T20:32:18.190232+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9657","reported_seq":9765,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:02.378001+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:33:02.378001+0000","last_peered":"2024-09-10T20:33:02.378001+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:33:02.378001+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":9657,"log_dups_size":0,"ondisk_log_size":9657,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39523270,"num_objects":9651,"num_object_clones":0,"num_object_copies":19302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9655,"num_write_kb":38600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6036},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182737+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182737+0000","last_peered":"2024-09-10T20:32:18.182737+0000","last_clean":"2024-09-10T20:32:18.182737+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182737+0000","last_undegraded":"2024-09-10T20:32:18.182737+0000","last_fullsized":"2024-09-10T20:32:18.182737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9630","reported_seq":9758,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:03.442848+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:33:03.442848+0000","last_peered":"2024-09-10T20:33:03.442848+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:33:03.442848+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":9630,"log_dups_size":0,"ondisk_log_size":9630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39424000,"num_objects":9625,"num_object_clones":0,"num_object_copies":19250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7458,"num_write_kb":29832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":5992}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9656","reported_seq":9783,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:03.553452+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:33:03.553452+0000","last_peered":"2024-09-10T20:33:03.553452+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:33:03.553452+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":9656,"log_dups_size":0,"ondisk_log_size":9656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39526400,"num_objects":9650,"num_object_clones":0,"num_object_copies":19300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7442,"num_write_kb":29768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":6040}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9683","reported_seq":9817,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:02.404580+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:33:02.404580+0000","last_peered":"2024-09-10T20:33:02.404580+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:33:02.404580+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":9683,"log_dups_size":0,"ondisk_log_size":9683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39653376,"num_objects":9681,"num_object_clones":0,"num_object_copies":19362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6406,"num_write_kb":25624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6033},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9609","reported_seq":9747,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:02.395456+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:33:02.395456+0000","last_peered":"2024-09-10T20:33:02.395456+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:33:02.395456+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":9609,"log_dups_size":0,"ondisk_log_size":9609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39325696,"num_objects":9601,"num_object_clones":0,"num_object_copies":19202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6303,"num_write_kb":25212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6069},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193200+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193200+0000","last_peered":"2024-09-10T20:32:18.193200+0000","last_clean":"2024-09-10T20:32:18.193200+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193200+0000","last_undegraded":"2024-09-10T20:32:18.193200+0000","last_fullsized":"2024-09-10T20:32:18.193200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9759","reported_seq":9890,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:02.422577+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:33:02.422577+0000","last_peered":"2024-09-10T20:33:02.422577+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:33:02.422577+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":9759,"log_dups_size":0,"ondisk_log_size":9759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39956480,"num_objects":9755,"num_object_clones":0,"num_object_copies":19510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6586,"num_write_kb":26344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6042},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9529","reported_seq":11564,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:33:02.434324+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:33:02.434324+0000","last_peered":"2024-09-10T20:33:02.434324+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:33:02.434324+0000","last_undegraded":"2024-09-10T20:33:02.434324+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":9529,"log_dups_size":0,"ondisk_log_size":9529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38982115,"num_objects":9518,"num_object_clones":0,"num_object_copies":19036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2533,"num_objects_unfound":0,"num_objects_dirty":9518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6136,"num_write_kb":24544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":935,"num_bytes_recovered":3829760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6985},{"shards":"0,2","objects":2533}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":632059305,"num_objects":154314,"num_object_clones":0,"num_object_copies":308628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2533,"num_objects_unfound":0,"num_objects_dirty":154314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":118689,"num_write_kb":474736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4542,"num_bytes_recovered":18591744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1284272128,"data_stored":1284185416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154393,"ondisk_log_size":154393,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182454,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":826756,"kb_used_data":520244,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93545084,"statfs":{"total":96636764160,"available":95790166016,"internally_reserved":0,"allocated":532729856,"data_stored":532487168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,2,5,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986226,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":699340,"kb_used_data":424700,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93672500,"statfs":{"total":96636764160,"available":95920640000,"internally_reserved":0,"allocated":434892800,"data_stored":434654189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281210538},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,1,6,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659173,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535056,"kb_used_data":308992,"kb_used_omap":4,"kb_used_meta":226043,"kb_avail":93836784,"statfs":{"total":96636764160,"available":96088866816,"internally_reserved":0,"allocated":316407808,"data_stored":316089247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":231468718},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":533082112,"data_stored":533078499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":316411904,"data_stored":316332418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":434778112,"data_stored":434774499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:33:06.077 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 218 20 155709 155689 2.78912 1.71875 0.0817939 0.0280029 2024-09-10T20:33:07.077 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 219 20 156193 156173 2.78502 1.89062 0.0213142 0.0280431 2024-09-10T20:33:08.077 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout:2024-09-10T20:33:08.074383+0000 min lat: 0.00191414 max lat: 2.51074 avg lat: 0.0280869 2024-09-10T20:33:08.077 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T20:33:08.077 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 220 20 156674 156654 2.7809 1.87891 0.02444 0.0280869 2024-09-10T20:33:08.772 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:33:09.077 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 221 20 157113 157093 2.77608 1.71484 0.0269958 0.0281331 2024-09-10T20:33:09.315 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:09.321 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:33:09.336 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":189,"stamp":"2024-09-10T20:33:08.902060+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":641279420,"num_objects":156567,"num_object_clones":0,"num_object_copies":313134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2429,"num_objects_unfound":0,"num_objects_dirty":156567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120942,"num_write_kb":483742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4648,"num_bytes_recovered":19017747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156647,"ondisk_log_size":156647,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2093376,"kb_used_data":1272272,"kb_used_omap":12,"kb_used_meta":821043,"kb_avail":281022144,"statfs":{"total":289910292480,"available":287766675456,"internally_reserved":0,"allocated":1302806528,"data_stored":1302006668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":840748034},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,4,5,14,7,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28033024,"num_objects":6844,"num_object_clones":0,"num_object_copies":13688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-310,"num_objects_unfound":0,"num_objects_dirty":6844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6844,"num_write_kb":27376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":1269760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004250"},"pg_stats":[{"pgid":"2.f","version":"45'9761","reported_seq":9840,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:07.497818+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:33:07.497818+0000","last_peered":"2024-09-10T20:33:07.497818+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:33:07.497818+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":9761,"log_dups_size":0,"ondisk_log_size":9761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39981056,"num_objects":9761,"num_object_clones":0,"num_object_copies":19522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9761,"num_write_kb":39044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6154},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9945","reported_seq":10055,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:08.375942+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:33:08.375942+0000","last_peered":"2024-09-10T20:33:08.375942+0000","last_clean":"2024-09-10T20:33:08.375942+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:33:08.375942+0000","last_undegraded":"2024-09-10T20:33:08.375942+0000","last_fullsized":"2024-09-10T20:33:08.375942+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":9945,"log_dups_size":0,"ondisk_log_size":9945,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40710144,"num_objects":9939,"num_object_clones":0,"num_object_copies":19878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9939,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8131,"num_write_kb":32524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9813","reported_seq":17160,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:08.821760+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:33:08.821760+0000","last_peered":"2024-09-10T20:33:08.821760+0000","last_clean":"2024-09-10T20:33:08.821760+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:33:08.821760+0000","last_undegraded":"2024-09-10T20:33:08.821760+0000","last_fullsized":"2024-09-10T20:33:08.821760+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":9813,"log_dups_size":0,"ondisk_log_size":9813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40177664,"num_objects":9809,"num_object_clones":0,"num_object_copies":19618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7941,"num_write_kb":31764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'9755","reported_seq":9835,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:08.470829+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:33:08.470829+0000","last_peered":"2024-09-10T20:33:08.470829+0000","last_clean":"2024-09-10T20:33:08.470829+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:33:08.470829+0000","last_undegraded":"2024-09-10T20:33:08.470829+0000","last_fullsized":"2024-09-10T20:33:08.470829+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":9755,"log_dups_size":0,"ondisk_log_size":9755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39956480,"num_objects":9755,"num_object_clones":0,"num_object_copies":19510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9755,"num_write_kb":39020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9931","reported_seq":10057,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:07.509551+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:33:07.509551+0000","last_peered":"2024-09-10T20:33:07.509551+0000","last_clean":"2024-09-10T20:33:07.509551+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:33:07.509551+0000","last_undegraded":"2024-09-10T20:33:07.509551+0000","last_fullsized":"2024-09-10T20:33:07.509551+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":9931,"log_dups_size":0,"ondisk_log_size":9931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40648704,"num_objects":9924,"num_object_clones":0,"num_object_copies":19848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6927,"num_write_kb":27708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9708","reported_seq":9844,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:07.461682+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:33:07.461682+0000","last_peered":"2024-09-10T20:33:07.461682+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:33:07.461682+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":9708,"log_dups_size":0,"ondisk_log_size":9708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39743488,"num_objects":9703,"num_object_clones":0,"num_object_copies":19406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6572,"num_write_kb":26288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6089},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9743","reported_seq":9879,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:07.464508+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:33:07.464508+0000","last_peered":"2024-09-10T20:33:07.464508+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:33:07.464508+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":9743,"log_dups_size":0,"ondisk_log_size":9743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39882752,"num_objects":9737,"num_object_clones":0,"num_object_copies":19474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6338,"num_write_kb":25352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6182},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9803","reported_seq":9927,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:07.384885+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:33:07.384885+0000","last_peered":"2024-09-10T20:33:07.384885+0000","last_clean":"2024-09-10T20:33:07.384885+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:33:07.384885+0000","last_undegraded":"2024-09-10T20:33:07.384885+0000","last_fullsized":"2024-09-10T20:33:07.384885+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":9803,"log_dups_size":0,"ondisk_log_size":9803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40144896,"num_objects":9801,"num_object_clones":0,"num_object_copies":19602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9801,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6874,"num_write_kb":27496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9667","reported_seq":9799,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:08.222109+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:33:08.222109+0000","last_peered":"2024-09-10T20:33:08.222109+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:33:08.222109+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":9667,"log_dups_size":0,"ondisk_log_size":9667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39567360,"num_objects":9660,"num_object_clones":0,"num_object_copies":19320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9660,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7660,"num_write_kb":30640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":6134}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190232+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190232+0000","last_peered":"2024-09-10T20:32:18.190232+0000","last_clean":"2024-09-10T20:32:18.190232+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190232+0000","last_undegraded":"2024-09-10T20:32:18.190232+0000","last_fullsized":"2024-09-10T20:32:18.190232+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9803","reported_seq":9909,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:07.531828+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:33:07.531828+0000","last_peered":"2024-09-10T20:33:07.531828+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:33:07.531828+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":9803,"log_dups_size":0,"ondisk_log_size":9803,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40121286,"num_objects":9797,"num_object_clones":0,"num_object_copies":19594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9801,"num_write_kb":39184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6182},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9769","reported_seq":9897,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:08.468763+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:33:08.468763+0000","last_peered":"2024-09-10T20:33:08.468763+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:33:08.468763+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":9769,"log_dups_size":0,"ondisk_log_size":9769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39993344,"num_objects":9764,"num_object_clones":0,"num_object_copies":19528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7597,"num_write_kb":30388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":6131}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9787","reported_seq":9914,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:08.553944+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:33:08.553944+0000","last_peered":"2024-09-10T20:33:08.553944+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:33:08.553944+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":9787,"log_dups_size":0,"ondisk_log_size":9787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40062976,"num_objects":9781,"num_object_clones":0,"num_object_copies":19562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7573,"num_write_kb":30292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":6171}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9820","reported_seq":9955,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:07.535985+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:33:07.535985+0000","last_peered":"2024-09-10T20:33:07.535985+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:33:07.535985+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":9820,"log_dups_size":0,"ondisk_log_size":9820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40214528,"num_objects":9818,"num_object_clones":0,"num_object_copies":19636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9818,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6543,"num_write_kb":26172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6170},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9739","reported_seq":9877,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:07.460145+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:33:07.460145+0000","last_peered":"2024-09-10T20:33:07.460145+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:33:07.460145+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":9739,"log_dups_size":0,"ondisk_log_size":9739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39858176,"num_objects":9731,"num_object_clones":0,"num_object_copies":19462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6433,"num_write_kb":25732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6199},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193200+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193200+0000","last_peered":"2024-09-10T20:32:18.193200+0000","last_clean":"2024-09-10T20:32:18.193200+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193200+0000","last_undegraded":"2024-09-10T20:32:18.193200+0000","last_fullsized":"2024-09-10T20:32:18.193200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9930","reported_seq":10061,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:07.439633+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:33:07.439633+0000","last_peered":"2024-09-10T20:33:07.439633+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:33:07.439633+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":9930,"log_dups_size":0,"ondisk_log_size":9930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40656896,"num_objects":9926,"num_object_clones":0,"num_object_copies":19852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6757,"num_write_kb":27028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6213},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9670","reported_seq":11914,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:33:07.531761+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:33:07.531761+0000","last_peered":"2024-09-10T20:33:07.531761+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:33:07.531761+0000","last_undegraded":"2024-09-10T20:33:07.531761+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":9670,"log_dups_size":0,"ondisk_log_size":9670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39559651,"num_objects":9659,"num_object_clones":0,"num_object_copies":19318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2429,"num_objects_unfound":0,"num_objects_dirty":9659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6277,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1039,"num_bytes_recovered":4255744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7230},{"shards":"0,2","objects":2429}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":641279401,"num_objects":156565,"num_object_clones":0,"num_object_copies":313130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2429,"num_objects_unfound":0,"num_objects_dirty":156565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120940,"num_write_kb":483740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4646,"num_bytes_recovered":19017728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1303392256,"data_stored":1303305544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156644,"ondisk_log_size":156644,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182455,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":855688,"kb_used_data":526968,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93516152,"statfs":{"total":96636764160,"available":95760539648,"internally_reserved":0,"allocated":539615232,"data_stored":539372544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336588458},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,2,5,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986227,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":691312,"kb_used_data":429152,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93680528,"statfs":{"total":96636764160,"available":95928860672,"internally_reserved":0,"allocated":439451648,"data_stored":439213037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268431018},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659174,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546376,"kb_used_data":316152,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93825464,"statfs":{"total":96636764160,"available":96077275136,"internally_reserved":0,"allocated":323739648,"data_stored":323421087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":235728558},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,5,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":540233728,"data_stored":540230115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":323915776,"data_stored":323836290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":439242752,"data_stored":439239139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:33:09.338 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:33:09.884 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:09.887 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:33:09.915 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":189,"stamp":"2024-09-10T20:33:08.902060+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":641279420,"num_objects":156567,"num_object_clones":0,"num_object_copies":313134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2429,"num_objects_unfound":0,"num_objects_dirty":156567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120942,"num_write_kb":483742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4648,"num_bytes_recovered":19017747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156647,"ondisk_log_size":156647,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2093376,"kb_used_data":1272272,"kb_used_omap":12,"kb_used_meta":821043,"kb_avail":281022144,"statfs":{"total":289910292480,"available":287766675456,"internally_reserved":0,"allocated":1302806528,"data_stored":1302006668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":840748034},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,4,5,14,7,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28033024,"num_objects":6844,"num_object_clones":0,"num_object_copies":13688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-310,"num_objects_unfound":0,"num_objects_dirty":6844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6844,"num_write_kb":27376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":1269760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004250"},"pg_stats":[{"pgid":"2.f","version":"45'9761","reported_seq":9840,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:07.497818+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:33:07.497818+0000","last_peered":"2024-09-10T20:33:07.497818+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:33:07.497818+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":9761,"log_dups_size":0,"ondisk_log_size":9761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39981056,"num_objects":9761,"num_object_clones":0,"num_object_copies":19522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9761,"num_write_kb":39044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6154},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9945","reported_seq":10055,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:08.375942+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:33:08.375942+0000","last_peered":"2024-09-10T20:33:08.375942+0000","last_clean":"2024-09-10T20:33:08.375942+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:33:08.375942+0000","last_undegraded":"2024-09-10T20:33:08.375942+0000","last_fullsized":"2024-09-10T20:33:08.375942+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":9945,"log_dups_size":0,"ondisk_log_size":9945,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40710144,"num_objects":9939,"num_object_clones":0,"num_object_copies":19878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9939,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8131,"num_write_kb":32524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9813","reported_seq":17160,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:08.821760+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:33:08.821760+0000","last_peered":"2024-09-10T20:33:08.821760+0000","last_clean":"2024-09-10T20:33:08.821760+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:33:08.821760+0000","last_undegraded":"2024-09-10T20:33:08.821760+0000","last_fullsized":"2024-09-10T20:33:08.821760+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":9813,"log_dups_size":0,"ondisk_log_size":9813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40177664,"num_objects":9809,"num_object_clones":0,"num_object_copies":19618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7941,"num_write_kb":31764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'9755","reported_seq":9835,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:08.470829+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:33:08.470829+0000","last_peered":"2024-09-10T20:33:08.470829+0000","last_clean":"2024-09-10T20:33:08.470829+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:33:08.470829+0000","last_undegraded":"2024-09-10T20:33:08.470829+0000","last_fullsized":"2024-09-10T20:33:08.470829+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":9755,"log_dups_size":0,"ondisk_log_size":9755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39956480,"num_objects":9755,"num_object_clones":0,"num_object_copies":19510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9755,"num_write_kb":39020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9931","reported_seq":10057,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:07.509551+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:33:07.509551+0000","last_peered":"2024-09-10T20:33:07.509551+0000","last_clean":"2024-09-10T20:33:07.509551+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:33:07.509551+0000","last_undegraded":"2024-09-10T20:33:07.509551+0000","last_fullsized":"2024-09-10T20:33:07.509551+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":9931,"log_dups_size":0,"ondisk_log_size":9931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40648704,"num_objects":9924,"num_object_clones":0,"num_object_copies":19848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6927,"num_write_kb":27708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9708","reported_seq":9844,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:07.461682+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:33:07.461682+0000","last_peered":"2024-09-10T20:33:07.461682+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:33:07.461682+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":9708,"log_dups_size":0,"ondisk_log_size":9708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39743488,"num_objects":9703,"num_object_clones":0,"num_object_copies":19406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6572,"num_write_kb":26288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6089},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9743","reported_seq":9879,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:07.464508+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:33:07.464508+0000","last_peered":"2024-09-10T20:33:07.464508+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:33:07.464508+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":9743,"log_dups_size":0,"ondisk_log_size":9743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39882752,"num_objects":9737,"num_object_clones":0,"num_object_copies":19474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6338,"num_write_kb":25352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6182},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9803","reported_seq":9927,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:07.384885+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:33:07.384885+0000","last_peered":"2024-09-10T20:33:07.384885+0000","last_clean":"2024-09-10T20:33:07.384885+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:33:07.384885+0000","last_undegraded":"2024-09-10T20:33:07.384885+0000","last_fullsized":"2024-09-10T20:33:07.384885+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":9803,"log_dups_size":0,"ondisk_log_size":9803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40144896,"num_objects":9801,"num_object_clones":0,"num_object_copies":19602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9801,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6874,"num_write_kb":27496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9667","reported_seq":9799,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:08.222109+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:33:08.222109+0000","last_peered":"2024-09-10T20:33:08.222109+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:33:08.222109+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":9667,"log_dups_size":0,"ondisk_log_size":9667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39567360,"num_objects":9660,"num_object_clones":0,"num_object_copies":19320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9660,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7660,"num_write_kb":30640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":6134}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190232+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190232+0000","last_peered":"2024-09-10T20:32:18.190232+0000","last_clean":"2024-09-10T20:32:18.190232+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190232+0000","last_undegraded":"2024-09-10T20:32:18.190232+0000","last_fullsized":"2024-09-10T20:32:18.190232+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9803","reported_seq":9909,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:07.531828+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:33:07.531828+0000","last_peered":"2024-09-10T20:33:07.531828+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:33:07.531828+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":9803,"log_dups_size":0,"ondisk_log_size":9803,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40121286,"num_objects":9797,"num_object_clones":0,"num_object_copies":19594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9801,"num_write_kb":39184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6182},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9769","reported_seq":9897,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:08.468763+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:33:08.468763+0000","last_peered":"2024-09-10T20:33:08.468763+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:33:08.468763+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":9769,"log_dups_size":0,"ondisk_log_size":9769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39993344,"num_objects":9764,"num_object_clones":0,"num_object_copies":19528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7597,"num_write_kb":30388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":6131}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9787","reported_seq":9914,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:08.553944+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:33:08.553944+0000","last_peered":"2024-09-10T20:33:08.553944+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:33:08.553944+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":9787,"log_dups_size":0,"ondisk_log_size":9787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40062976,"num_objects":9781,"num_object_clones":0,"num_object_copies":19562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7573,"num_write_kb":30292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":6171}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9820","reported_seq":9955,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:07.535985+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:33:07.535985+0000","last_peered":"2024-09-10T20:33:07.535985+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:33:07.535985+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":9820,"log_dups_size":0,"ondisk_log_size":9820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40214528,"num_objects":9818,"num_object_clones":0,"num_object_copies":19636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9818,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6543,"num_write_kb":26172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6170},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9739","reported_seq":9877,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:07.460145+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:33:07.460145+0000","last_peered":"2024-09-10T20:33:07.460145+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:33:07.460145+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":9739,"log_dups_size":0,"ondisk_log_size":9739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39858176,"num_objects":9731,"num_object_clones":0,"num_object_copies":19462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6433,"num_write_kb":25732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6199},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193200+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193200+0000","last_peered":"2024-09-10T20:32:18.193200+0000","last_clean":"2024-09-10T20:32:18.193200+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193200+0000","last_undegraded":"2024-09-10T20:32:18.193200+0000","last_fullsized":"2024-09-10T20:32:18.193200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9930","reported_seq":10061,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:07.439633+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:33:07.439633+0000","last_peered":"2024-09-10T20:33:07.439633+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:33:07.439633+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":9930,"log_dups_size":0,"ondisk_log_size":9930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40656896,"num_objects":9926,"num_object_clones":0,"num_object_copies":19852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6757,"num_write_kb":27028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6213},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9670","reported_seq":11914,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:33:07.531761+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:33:07.531761+0000","last_peered":"2024-09-10T20:33:07.531761+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:33:07.531761+0000","last_undegraded":"2024-09-10T20:33:07.531761+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":9670,"log_dups_size":0,"ondisk_log_size":9670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39559651,"num_objects":9659,"num_object_clones":0,"num_object_copies":19318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2429,"num_objects_unfound":0,"num_objects_dirty":9659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6277,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1039,"num_bytes_recovered":4255744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7230},{"shards":"0,2","objects":2429}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":641279401,"num_objects":156565,"num_object_clones":0,"num_object_copies":313130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2429,"num_objects_unfound":0,"num_objects_dirty":156565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120940,"num_write_kb":483740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4646,"num_bytes_recovered":19017728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1303392256,"data_stored":1303305544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156644,"ondisk_log_size":156644,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182455,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":855688,"kb_used_data":526968,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93516152,"statfs":{"total":96636764160,"available":95760539648,"internally_reserved":0,"allocated":539615232,"data_stored":539372544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336588458},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,2,5,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986227,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":691312,"kb_used_data":429152,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93680528,"statfs":{"total":96636764160,"available":95928860672,"internally_reserved":0,"allocated":439451648,"data_stored":439213037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268431018},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659174,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546376,"kb_used_data":316152,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93825464,"statfs":{"total":96636764160,"available":96077275136,"internally_reserved":0,"allocated":323739648,"data_stored":323421087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":235728558},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,5,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":540233728,"data_stored":540230115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":323915776,"data_stored":323836290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":439242752,"data_stored":439239139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:33:09.917 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:33:09.917 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'9761", 'reported_seq': 9840, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:07.497818+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:33:07.497818+0000', 'last_peered': '2024-09-10T20:33:07.497818+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:33:07.497818+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 9761, 'log_dups_size': 0, 'ondisk_log_size': 9761, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39981056, 'num_objects': 9761, 'num_object_clones': 0, 'num_object_copies': 19522, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3607, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9761, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9761, 'num_write_kb': 39044, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6154}, {'shards': '0,2', 'objects': 3607}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:09.917 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:33:09.917 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'9708", 'reported_seq': 9844, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:07.461682+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:33:07.461682+0000', 'last_peered': '2024-09-10T20:33:07.461682+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:33:07.461682+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 9708, 'log_dups_size': 0, 'ondisk_log_size': 9708, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39743488, 'num_objects': 9703, 'num_object_clones': 0, 'num_object_copies': 19406, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3616, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9703, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6572, 'num_write_kb': 26288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6089}, {'shards': '0,2', 'objects': 3614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:09.917 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:33:09.917 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'9743", 'reported_seq': 9879, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:07.464508+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:33:07.464508+0000', 'last_peered': '2024-09-10T20:33:07.464508+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:33:07.464508+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 9743, 'log_dups_size': 0, 'ondisk_log_size': 9743, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39882752, 'num_objects': 9737, 'num_object_clones': 0, 'num_object_copies': 19474, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3557, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9737, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6338, 'num_write_kb': 25352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6182}, {'shards': '0,2', 'objects': 3555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:09.917 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:33:09.918 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'9667", 'reported_seq': 9799, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:08.222109+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:33:08.222109+0000', 'last_peered': '2024-09-10T20:33:08.222109+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:33:08.222109+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 9667, 'log_dups_size': 0, 'ondisk_log_size': 9667, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39567360, 'num_objects': 9660, 'num_object_clones': 0, 'num_object_copies': 19320, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3528, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9660, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7660, 'num_write_kb': 30640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 6134}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:33:09.918 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:33:09.918 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'9803", 'reported_seq': 9909, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:07.531828+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:33:07.531828+0000', 'last_peered': '2024-09-10T20:33:07.531828+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:33:07.531828+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 9803, 'log_dups_size': 0, 'ondisk_log_size': 9803, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40121286, 'num_objects': 9797, 'num_object_clones': 0, 'num_object_copies': 19594, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3617, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9797, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9801, 'num_write_kb': 39184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6182}, {'shards': '0,2', 'objects': 3615}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:09.918 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:33:09.918 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'9769", 'reported_seq': 9897, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:08.468763+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:33:08.468763+0000', 'last_peered': '2024-09-10T20:33:08.468763+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:33:08.468763+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 9769, 'log_dups_size': 0, 'ondisk_log_size': 9769, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39993344, 'num_objects': 9764, 'num_object_clones': 0, 'num_object_copies': 19528, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3635, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9764, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7597, 'num_write_kb': 30388, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3633}, {'shards': '2', 'objects': 6131}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:33:09.918 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:33:09.918 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'9787", 'reported_seq': 9914, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:08.553944+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:33:08.553944+0000', 'last_peered': '2024-09-10T20:33:08.553944+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:33:08.553944+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 9787, 'log_dups_size': 0, 'ondisk_log_size': 9787, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40062976, 'num_objects': 9781, 'num_object_clones': 0, 'num_object_copies': 19562, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9781, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7573, 'num_write_kb': 30292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3610}, {'shards': '2', 'objects': 6171}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:33:09.918 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:33:09.918 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'9820", 'reported_seq': 9955, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:07.535985+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:33:07.535985+0000', 'last_peered': '2024-09-10T20:33:07.535985+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:33:07.535985+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 9820, 'log_dups_size': 0, 'ondisk_log_size': 9820, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40214528, 'num_objects': 9818, 'num_object_clones': 0, 'num_object_copies': 19636, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3649, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9818, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6543, 'num_write_kb': 26172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6170}, {'shards': '0,2', 'objects': 3648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:09.919 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:33:09.919 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'9739", 'reported_seq': 9877, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:07.460145+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:33:07.460145+0000', 'last_peered': '2024-09-10T20:33:07.460145+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:33:07.460145+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 9739, 'log_dups_size': 0, 'ondisk_log_size': 9739, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39858176, 'num_objects': 9731, 'num_object_clones': 0, 'num_object_copies': 19462, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3535, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9731, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6433, 'num_write_kb': 25732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6199}, {'shards': '0,2', 'objects': 3532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:09.919 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:33:09.919 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'9930", 'reported_seq': 10061, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:07.439633+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:33:07.439633+0000', 'last_peered': '2024-09-10T20:33:07.439633+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:33:07.439633+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 9930, 'log_dups_size': 0, 'ondisk_log_size': 9930, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40656896, 'num_objects': 9926, 'num_object_clones': 0, 'num_object_copies': 19852, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3715, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9926, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6757, 'num_write_kb': 27028, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6213}, {'shards': '0,2', 'objects': 3713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:09.919 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:33:09.919 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'9670", 'reported_seq': 11914, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:33:07.531761+0000', 'last_change': '2024-09-10T20:32:16.657652+0000', 'last_active': '2024-09-10T20:33:07.531761+0000', 'last_peered': '2024-09-10T20:33:07.531761+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:33:07.531761+0000', 'last_undegraded': '2024-09-10T20:33:07.531761+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 9670, 'log_dups_size': 0, 'ondisk_log_size': 9670, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39559651, 'num_objects': 9659, 'num_object_clones': 0, 'num_object_copies': 19318, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2429, 'num_objects_unfound': 0, 'num_objects_dirty': 9659, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6277, 'num_write_kb': 25108, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1039, 'num_bytes_recovered': 4255744, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7230}, {'shards': '0,2', 'objects': 2429}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:09.919 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:33:10.077 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 222 20 157538 157518 2.77105 1.66016 0.0512519 0.028186 2024-09-10T20:33:10.585 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:10.585 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36071/313134 objects degraded (11.519%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":302,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":156567,"data_bytes":641279420,"bytes_used":2143617024,"bytes_avail":287766675456,"bytes_total":289910292480,"degraded_objects":36071,"degraded_total":313134,"degraded_ratio":0.11519349543645852,"misplaced_objects":2429,"misplaced_total":313134,"misplaced_ratio":0.0077570624716575024,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":105775,"recovering_keys_per_sec":0,"num_objects_recovered":310,"num_bytes_recovered":1269760,"num_keys_recovered":0,"write_bytes_sec":2335258,"read_op_per_sec":0,"write_op_per_sec":570},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:33:10.609 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:33:10.610 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:33:11.077 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 223 20 157947 157927 2.76579 1.59766 0.0311955 0.0282386 2024-09-10T20:33:11.158 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:11.159 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:33:11.183 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":190,"stamp":"2024-09-10T20:33:10.902875+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":641279420,"num_objects":156567,"num_object_clones":0,"num_object_copies":313134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2429,"num_objects_unfound":0,"num_objects_dirty":156567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120942,"num_write_kb":483742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4648,"num_bytes_recovered":19017747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156647,"ondisk_log_size":156647,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2093376,"kb_used_data":1272272,"kb_used_omap":12,"kb_used_meta":821043,"kb_avail":281022144,"statfs":{"total":289910292480,"available":287766675456,"internally_reserved":0,"allocated":1302806528,"data_stored":1302006668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":840748034},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,4,5,14,7,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18620416,"num_objects":4546,"num_object_clones":0,"num_object_copies":9092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-206,"num_objects_unfound":0,"num_objects_dirty":4546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4546,"num_write_kb":18184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":843776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004436"},"pg_stats":[{"pgid":"2.f","version":"45'9761","reported_seq":9840,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:07.497818+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:33:07.497818+0000","last_peered":"2024-09-10T20:33:07.497818+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:33:07.497818+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":9761,"log_dups_size":0,"ondisk_log_size":9761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39981056,"num_objects":9761,"num_object_clones":0,"num_object_copies":19522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9761,"num_write_kb":39044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6154},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9945","reported_seq":10055,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:08.375942+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:33:08.375942+0000","last_peered":"2024-09-10T20:33:08.375942+0000","last_clean":"2024-09-10T20:33:08.375942+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:33:08.375942+0000","last_undegraded":"2024-09-10T20:33:08.375942+0000","last_fullsized":"2024-09-10T20:33:08.375942+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":9945,"log_dups_size":0,"ondisk_log_size":9945,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40710144,"num_objects":9939,"num_object_clones":0,"num_object_copies":19878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9939,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8131,"num_write_kb":32524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9813","reported_seq":17160,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:08.821760+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:33:08.821760+0000","last_peered":"2024-09-10T20:33:08.821760+0000","last_clean":"2024-09-10T20:33:08.821760+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:33:08.821760+0000","last_undegraded":"2024-09-10T20:33:08.821760+0000","last_fullsized":"2024-09-10T20:33:08.821760+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":9813,"log_dups_size":0,"ondisk_log_size":9813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40177664,"num_objects":9809,"num_object_clones":0,"num_object_copies":19618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7941,"num_write_kb":31764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'9755","reported_seq":9835,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:08.470829+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:33:08.470829+0000","last_peered":"2024-09-10T20:33:08.470829+0000","last_clean":"2024-09-10T20:33:08.470829+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:33:08.470829+0000","last_undegraded":"2024-09-10T20:33:08.470829+0000","last_fullsized":"2024-09-10T20:33:08.470829+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":9755,"log_dups_size":0,"ondisk_log_size":9755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39956480,"num_objects":9755,"num_object_clones":0,"num_object_copies":19510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9755,"num_write_kb":39020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9931","reported_seq":10057,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:07.509551+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:33:07.509551+0000","last_peered":"2024-09-10T20:33:07.509551+0000","last_clean":"2024-09-10T20:33:07.509551+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:33:07.509551+0000","last_undegraded":"2024-09-10T20:33:07.509551+0000","last_fullsized":"2024-09-10T20:33:07.509551+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":9931,"log_dups_size":0,"ondisk_log_size":9931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40648704,"num_objects":9924,"num_object_clones":0,"num_object_copies":19848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6927,"num_write_kb":27708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9708","reported_seq":9844,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:07.461682+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:33:07.461682+0000","last_peered":"2024-09-10T20:33:07.461682+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:33:07.461682+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":9708,"log_dups_size":0,"ondisk_log_size":9708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39743488,"num_objects":9703,"num_object_clones":0,"num_object_copies":19406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6572,"num_write_kb":26288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6089},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9743","reported_seq":9879,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:07.464508+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:33:07.464508+0000","last_peered":"2024-09-10T20:33:07.464508+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:33:07.464508+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":9743,"log_dups_size":0,"ondisk_log_size":9743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39882752,"num_objects":9737,"num_object_clones":0,"num_object_copies":19474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6338,"num_write_kb":25352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6182},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9803","reported_seq":9927,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:07.384885+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:33:07.384885+0000","last_peered":"2024-09-10T20:33:07.384885+0000","last_clean":"2024-09-10T20:33:07.384885+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:33:07.384885+0000","last_undegraded":"2024-09-10T20:33:07.384885+0000","last_fullsized":"2024-09-10T20:33:07.384885+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":9803,"log_dups_size":0,"ondisk_log_size":9803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40144896,"num_objects":9801,"num_object_clones":0,"num_object_copies":19602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9801,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6874,"num_write_kb":27496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9667","reported_seq":9799,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:08.222109+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:33:08.222109+0000","last_peered":"2024-09-10T20:33:08.222109+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:33:08.222109+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":9667,"log_dups_size":0,"ondisk_log_size":9667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39567360,"num_objects":9660,"num_object_clones":0,"num_object_copies":19320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9660,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7660,"num_write_kb":30640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":6134}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190232+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190232+0000","last_peered":"2024-09-10T20:32:18.190232+0000","last_clean":"2024-09-10T20:32:18.190232+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190232+0000","last_undegraded":"2024-09-10T20:32:18.190232+0000","last_fullsized":"2024-09-10T20:32:18.190232+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9803","reported_seq":9909,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:07.531828+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:33:07.531828+0000","last_peered":"2024-09-10T20:33:07.531828+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:33:07.531828+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":9803,"log_dups_size":0,"ondisk_log_size":9803,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40121286,"num_objects":9797,"num_object_clones":0,"num_object_copies":19594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9801,"num_write_kb":39184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6182},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9769","reported_seq":9897,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:08.468763+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:33:08.468763+0000","last_peered":"2024-09-10T20:33:08.468763+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:33:08.468763+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":9769,"log_dups_size":0,"ondisk_log_size":9769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39993344,"num_objects":9764,"num_object_clones":0,"num_object_copies":19528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7597,"num_write_kb":30388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":6131}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9787","reported_seq":9914,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:08.553944+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:33:08.553944+0000","last_peered":"2024-09-10T20:33:08.553944+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:33:08.553944+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":9787,"log_dups_size":0,"ondisk_log_size":9787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40062976,"num_objects":9781,"num_object_clones":0,"num_object_copies":19562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7573,"num_write_kb":30292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":6171}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9820","reported_seq":9955,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:07.535985+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:33:07.535985+0000","last_peered":"2024-09-10T20:33:07.535985+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:33:07.535985+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":9820,"log_dups_size":0,"ondisk_log_size":9820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40214528,"num_objects":9818,"num_object_clones":0,"num_object_copies":19636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9818,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6543,"num_write_kb":26172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6170},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9739","reported_seq":9877,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:07.460145+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:33:07.460145+0000","last_peered":"2024-09-10T20:33:07.460145+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:33:07.460145+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":9739,"log_dups_size":0,"ondisk_log_size":9739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39858176,"num_objects":9731,"num_object_clones":0,"num_object_copies":19462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6433,"num_write_kb":25732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6199},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193200+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193200+0000","last_peered":"2024-09-10T20:32:18.193200+0000","last_clean":"2024-09-10T20:32:18.193200+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193200+0000","last_undegraded":"2024-09-10T20:32:18.193200+0000","last_fullsized":"2024-09-10T20:32:18.193200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9930","reported_seq":10061,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:07.439633+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:33:07.439633+0000","last_peered":"2024-09-10T20:33:07.439633+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:33:07.439633+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":9930,"log_dups_size":0,"ondisk_log_size":9930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40656896,"num_objects":9926,"num_object_clones":0,"num_object_copies":19852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6757,"num_write_kb":27028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6213},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9670","reported_seq":11914,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:33:07.531761+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:33:07.531761+0000","last_peered":"2024-09-10T20:33:07.531761+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:33:07.531761+0000","last_undegraded":"2024-09-10T20:33:07.531761+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":9670,"log_dups_size":0,"ondisk_log_size":9670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39559651,"num_objects":9659,"num_object_clones":0,"num_object_copies":19318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2429,"num_objects_unfound":0,"num_objects_dirty":9659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6277,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1039,"num_bytes_recovered":4255744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7230},{"shards":"0,2","objects":2429}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":641279401,"num_objects":156565,"num_object_clones":0,"num_object_copies":313130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2429,"num_objects_unfound":0,"num_objects_dirty":156565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120940,"num_write_kb":483740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4646,"num_bytes_recovered":19017728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1303392256,"data_stored":1303305544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156644,"ondisk_log_size":156644,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182455,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":855688,"kb_used_data":526968,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93516152,"statfs":{"total":96636764160,"available":95760539648,"internally_reserved":0,"allocated":539615232,"data_stored":539372544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336588458},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,2,5,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986227,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":691312,"kb_used_data":429152,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93680528,"statfs":{"total":96636764160,"available":95928860672,"internally_reserved":0,"allocated":439451648,"data_stored":439213037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268431018},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659174,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546376,"kb_used_data":316152,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93825464,"statfs":{"total":96636764160,"available":96077275136,"internally_reserved":0,"allocated":323739648,"data_stored":323421087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":235728558},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,5,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":540233728,"data_stored":540230115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":323915776,"data_stored":323836290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":439242752,"data_stored":439239139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:33:12.077 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 224 20 158417 158397 2.76164 1.83594 0.0703233 0.0282829 2024-09-10T20:33:13.077 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 225 20 158897 158877 2.7577 1.875 0.0594182 0.0283221 2024-09-10T20:33:14.077 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 226 20 159372 159352 2.7537 1.85547 0.0296278 0.0283632 2024-09-10T20:33:14.185 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:33:14.740 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:14.743 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:33:14.768 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2024-09-10T20:33:12.903679+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647263676,"num_objects":158028,"num_object_clones":0,"num_object_copies":316056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2324,"num_objects_unfound":0,"num_objects_dirty":158028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":122403,"num_write_kb":489586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4753,"num_bytes_recovered":19447827,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158108,"ondisk_log_size":158108,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2087808,"kb_used_data":1279184,"kb_used_omap":12,"kb_used_meta":808563,"kb_avail":281027712,"statfs":{"total":289910292480,"available":287772377088,"internally_reserved":0,"allocated":1309884416,"data_stored":1309084556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":827968514},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,4,6,10,10,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24604672,"num_objects":6007,"num_object_clones":0,"num_object_copies":12014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-311,"num_objects_unfound":0,"num_objects_dirty":6007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6007,"num_write_kb":24028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":1273856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004555"},"pg_stats":[{"pgid":"2.f","version":"45'9903","reported_seq":9983,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:12.527042+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:33:12.527042+0000","last_peered":"2024-09-10T20:33:12.527042+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:33:12.527042+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":9903,"log_dups_size":0,"ondisk_log_size":9903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40562688,"num_objects":9903,"num_object_clones":0,"num_object_copies":19806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9903,"num_write_kb":39612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6296},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9945","reported_seq":10055,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:08.375942+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:33:08.375942+0000","last_peered":"2024-09-10T20:33:08.375942+0000","last_clean":"2024-09-10T20:33:08.375942+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:33:08.375942+0000","last_undegraded":"2024-09-10T20:33:08.375942+0000","last_fullsized":"2024-09-10T20:33:08.375942+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":9945,"log_dups_size":0,"ondisk_log_size":9945,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40710144,"num_objects":9939,"num_object_clones":0,"num_object_copies":19878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9939,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8131,"num_write_kb":32524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9813","reported_seq":17160,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:08.821760+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:33:08.821760+0000","last_peered":"2024-09-10T20:33:08.821760+0000","last_clean":"2024-09-10T20:33:08.821760+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:33:08.821760+0000","last_undegraded":"2024-09-10T20:33:08.821760+0000","last_fullsized":"2024-09-10T20:33:08.821760+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":9813,"log_dups_size":0,"ondisk_log_size":9813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40177664,"num_objects":9809,"num_object_clones":0,"num_object_copies":19618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7941,"num_write_kb":31764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'9755","reported_seq":9835,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:08.470829+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:33:08.470829+0000","last_peered":"2024-09-10T20:33:08.470829+0000","last_clean":"2024-09-10T20:33:08.470829+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:33:08.470829+0000","last_undegraded":"2024-09-10T20:33:08.470829+0000","last_fullsized":"2024-09-10T20:33:08.470829+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":9755,"log_dups_size":0,"ondisk_log_size":9755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39956480,"num_objects":9755,"num_object_clones":0,"num_object_copies":19510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9755,"num_write_kb":39020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10088","reported_seq":10214,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:12.567499+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:33:12.567499+0000","last_peered":"2024-09-10T20:33:12.567499+0000","last_clean":"2024-09-10T20:33:12.567499+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:33:12.567499+0000","last_undegraded":"2024-09-10T20:33:12.567499+0000","last_fullsized":"2024-09-10T20:33:12.567499+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41291776,"num_objects":10081,"num_object_clones":0,"num_object_copies":20162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7084,"num_write_kb":28336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9850","reported_seq":9984,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:12.554453+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:33:12.554453+0000","last_peered":"2024-09-10T20:33:12.554453+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:33:12.554453+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":9850,"log_dups_size":0,"ondisk_log_size":9850,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40325120,"num_objects":9845,"num_object_clones":0,"num_object_copies":19690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6714,"num_write_kb":26856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6231},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9893","reported_seq":10029,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:12.588359+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:33:12.588359+0000","last_peered":"2024-09-10T20:33:12.588359+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:33:12.588359+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":9893,"log_dups_size":0,"ondisk_log_size":9893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40497152,"num_objects":9887,"num_object_clones":0,"num_object_copies":19774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6488,"num_write_kb":25952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6332},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9936","reported_seq":10060,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:12.403021+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:33:12.403021+0000","last_peered":"2024-09-10T20:33:12.403021+0000","last_clean":"2024-09-10T20:33:12.403021+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:33:12.403021+0000","last_undegraded":"2024-09-10T20:33:12.403021+0000","last_fullsized":"2024-09-10T20:33:12.403021+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":9936,"log_dups_size":0,"ondisk_log_size":9936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40689664,"num_objects":9934,"num_object_clones":0,"num_object_copies":19868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7007,"num_write_kb":28028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9667","reported_seq":9799,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:08.222109+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:33:08.222109+0000","last_peered":"2024-09-10T20:33:08.222109+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:33:08.222109+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":9667,"log_dups_size":0,"ondisk_log_size":9667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39567360,"num_objects":9660,"num_object_clones":0,"num_object_copies":19320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9660,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7660,"num_write_kb":30640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":6134}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190232+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190232+0000","last_peered":"2024-09-10T20:32:18.190232+0000","last_clean":"2024-09-10T20:32:18.190232+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190232+0000","last_undegraded":"2024-09-10T20:32:18.190232+0000","last_fullsized":"2024-09-10T20:32:18.190232+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9944","reported_seq":10052,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:12.589939+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:33:12.589939+0000","last_peered":"2024-09-10T20:33:12.589939+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:33:12.589939+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":9944,"log_dups_size":0,"ondisk_log_size":9944,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40698822,"num_objects":9938,"num_object_clones":0,"num_object_copies":19876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9942,"num_write_kb":39748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6323},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9769","reported_seq":9897,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:08.468763+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:33:08.468763+0000","last_peered":"2024-09-10T20:33:08.468763+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:33:08.468763+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":9769,"log_dups_size":0,"ondisk_log_size":9769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39993344,"num_objects":9764,"num_object_clones":0,"num_object_copies":19528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7597,"num_write_kb":30388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":6131}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9787","reported_seq":9914,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:08.553944+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:33:08.553944+0000","last_peered":"2024-09-10T20:33:08.553944+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:33:08.553944+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":9787,"log_dups_size":0,"ondisk_log_size":9787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40062976,"num_objects":9781,"num_object_clones":0,"num_object_copies":19562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7573,"num_write_kb":30292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":6171}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9960","reported_seq":10095,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:12.508248+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:33:12.508248+0000","last_peered":"2024-09-10T20:33:12.508248+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:33:12.508248+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":9960,"log_dups_size":0,"ondisk_log_size":9960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40787968,"num_objects":9958,"num_object_clones":0,"num_object_copies":19916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6683,"num_write_kb":26732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6310},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9898","reported_seq":10036,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:12.462644+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:33:12.462644+0000","last_peered":"2024-09-10T20:33:12.462644+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:33:12.462644+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":9898,"log_dups_size":0,"ondisk_log_size":9898,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40509440,"num_objects":9890,"num_object_clones":0,"num_object_copies":19780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6592,"num_write_kb":26368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6358},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193200+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193200+0000","last_peered":"2024-09-10T20:32:18.193200+0000","last_clean":"2024-09-10T20:32:18.193200+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193200+0000","last_undegraded":"2024-09-10T20:32:18.193200+0000","last_fullsized":"2024-09-10T20:32:18.193200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10079","reported_seq":10210,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:12.453112+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:33:12.453112+0000","last_peered":"2024-09-10T20:33:12.453112+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:33:12.453112+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41267200,"num_objects":10075,"num_object_clones":0,"num_object_copies":20150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6906,"num_write_kb":27624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6362},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9818","reported_seq":12272,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:33:12.591837+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:33:12.591837+0000","last_peered":"2024-09-10T20:33:12.591837+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:33:12.591837+0000","last_undegraded":"2024-09-10T20:33:12.591837+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":9818,"log_dups_size":0,"ondisk_log_size":9818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40165859,"num_objects":9807,"num_object_clones":0,"num_object_copies":19614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2324,"num_objects_unfound":0,"num_objects_dirty":9807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6425,"num_write_kb":25700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1144,"num_bytes_recovered":4685824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7483},{"shards":"0,2","objects":2324}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":647263657,"num_objects":158026,"num_object_clones":0,"num_object_copies":316052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2324,"num_objects_unfound":0,"num_objects_dirty":158026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":122401,"num_write_kb":489584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4751,"num_bytes_recovered":19447808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1310425088,"data_stored":1310338376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158105,"ondisk_log_size":158105,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182456,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":850120,"kb_used_data":533880,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93521720,"statfs":{"total":96636764160,"available":95766241280,"internally_reserved":0,"allocated":546693120,"data_stored":546450432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323808938},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,3,1,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986227,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":691312,"kb_used_data":429152,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93680528,"statfs":{"total":96636764160,"available":95928860672,"internally_reserved":0,"allocated":439451648,"data_stored":439213037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268431018},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659174,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546376,"kb_used_data":316152,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93825464,"statfs":{"total":96636764160,"available":96077275136,"internally_reserved":0,"allocated":323739648,"data_stored":323421087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":235728558},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,5,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":547266560,"data_stored":547262947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":323915776,"data_stored":323836290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":439242752,"data_stored":439239139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:33:14.770 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:33:15.078 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 227 20 159795 159775 2.74885 1.65234 0.0893712 0.0284137 2024-09-10T20:33:15.305 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:15.308 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:33:15.335 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":192,"stamp":"2024-09-10T20:33:14.904453+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650716604,"num_objects":158871,"num_object_clones":0,"num_object_copies":317742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2324,"num_objects_unfound":0,"num_objects_dirty":158871,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123246,"num_write_kb":492958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4753,"num_bytes_recovered":19447827,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158951,"ondisk_log_size":158951,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2107616,"kb_used_data":1290672,"kb_used_omap":12,"kb_used_meta":816883,"kb_avail":281007904,"statfs":{"total":289910292480,"available":287752093696,"internally_reserved":0,"allocated":1321648128,"data_stored":1320848268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":836488194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,3,6,6,11,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22269952,"num_objects":5437,"num_object_clones":0,"num_object_copies":10874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-209,"num_objects_unfound":0,"num_objects_dirty":5437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5437,"num_write_kb":21748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":856064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004698"},"pg_stats":[{"pgid":"2.f","version":"45'9903","reported_seq":9983,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:12.527042+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:33:12.527042+0000","last_peered":"2024-09-10T20:33:12.527042+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:33:12.527042+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":9903,"log_dups_size":0,"ondisk_log_size":9903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40562688,"num_objects":9903,"num_object_clones":0,"num_object_copies":19806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9903,"num_write_kb":39612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6296},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10095","reported_seq":10205,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:13.529056+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:33:13.529056+0000","last_peered":"2024-09-10T20:33:13.529056+0000","last_clean":"2024-09-10T20:33:13.529056+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:33:13.529056+0000","last_undegraded":"2024-09-10T20:33:13.529056+0000","last_fullsized":"2024-09-10T20:33:13.529056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41324544,"num_objects":10089,"num_object_clones":0,"num_object_copies":20178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8281,"num_write_kb":33124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9952","reported_seq":17299,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:13.928290+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:33:13.928290+0000","last_peered":"2024-09-10T20:33:13.928290+0000","last_clean":"2024-09-10T20:33:13.928290+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:33:13.928290+0000","last_undegraded":"2024-09-10T20:33:13.928290+0000","last_fullsized":"2024-09-10T20:33:13.928290+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":9952,"log_dups_size":0,"ondisk_log_size":9952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40747008,"num_objects":9948,"num_object_clones":0,"num_object_copies":19896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8080,"num_write_kb":32320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'9900","reported_seq":9980,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:13.479797+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:33:13.479797+0000","last_peered":"2024-09-10T20:33:13.479797+0000","last_clean":"2024-09-10T20:33:13.479797+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:33:13.479797+0000","last_undegraded":"2024-09-10T20:33:13.479797+0000","last_fullsized":"2024-09-10T20:33:13.479797+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":9900,"log_dups_size":0,"ondisk_log_size":9900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40550400,"num_objects":9900,"num_object_clones":0,"num_object_copies":19800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9900,"num_write_kb":39600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10088","reported_seq":10214,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:12.567499+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:33:12.567499+0000","last_peered":"2024-09-10T20:33:12.567499+0000","last_clean":"2024-09-10T20:33:12.567499+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:33:12.567499+0000","last_undegraded":"2024-09-10T20:33:12.567499+0000","last_fullsized":"2024-09-10T20:33:12.567499+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41291776,"num_objects":10081,"num_object_clones":0,"num_object_copies":20162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7084,"num_write_kb":28336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9850","reported_seq":9984,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:12.554453+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:33:12.554453+0000","last_peered":"2024-09-10T20:33:12.554453+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:33:12.554453+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":9850,"log_dups_size":0,"ondisk_log_size":9850,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40325120,"num_objects":9845,"num_object_clones":0,"num_object_copies":19690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6714,"num_write_kb":26856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6231},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9893","reported_seq":10029,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:12.588359+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:33:12.588359+0000","last_peered":"2024-09-10T20:33:12.588359+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:33:12.588359+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":9893,"log_dups_size":0,"ondisk_log_size":9893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40497152,"num_objects":9887,"num_object_clones":0,"num_object_copies":19774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6488,"num_write_kb":25952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6332},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9936","reported_seq":10060,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:12.403021+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:33:12.403021+0000","last_peered":"2024-09-10T20:33:12.403021+0000","last_clean":"2024-09-10T20:33:12.403021+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:33:12.403021+0000","last_undegraded":"2024-09-10T20:33:12.403021+0000","last_fullsized":"2024-09-10T20:33:12.403021+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":9936,"log_dups_size":0,"ondisk_log_size":9936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40689664,"num_objects":9934,"num_object_clones":0,"num_object_copies":19868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7007,"num_write_kb":28028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9800","reported_seq":9932,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:13.514796+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:33:13.514796+0000","last_peered":"2024-09-10T20:33:13.514796+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:33:13.514796+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":9800,"log_dups_size":0,"ondisk_log_size":9800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40112128,"num_objects":9793,"num_object_clones":0,"num_object_copies":19586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7793,"num_write_kb":31172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":6267}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190232+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190232+0000","last_peered":"2024-09-10T20:32:18.190232+0000","last_clean":"2024-09-10T20:32:18.190232+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190232+0000","last_undegraded":"2024-09-10T20:32:18.190232+0000","last_fullsized":"2024-09-10T20:32:18.190232+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9944","reported_seq":10052,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:12.589939+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:33:12.589939+0000","last_peered":"2024-09-10T20:33:12.589939+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:33:12.589939+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":9944,"log_dups_size":0,"ondisk_log_size":9944,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40698822,"num_objects":9938,"num_object_clones":0,"num_object_copies":19876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9942,"num_write_kb":39748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6323},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9910","reported_seq":10038,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:13.552063+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:33:13.552063+0000","last_peered":"2024-09-10T20:33:13.552063+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:33:13.552063+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":9910,"log_dups_size":0,"ondisk_log_size":9910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40570880,"num_objects":9905,"num_object_clones":0,"num_object_copies":19810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7738,"num_write_kb":30952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":6272}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9922","reported_seq":10048,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:13.536842+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:33:13.536842+0000","last_peered":"2024-09-10T20:33:13.536842+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:33:13.536842+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":9922,"log_dups_size":0,"ondisk_log_size":9922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40615936,"num_objects":9916,"num_object_clones":0,"num_object_copies":19832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7708,"num_write_kb":30832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":6306}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9960","reported_seq":10095,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:12.508248+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:33:12.508248+0000","last_peered":"2024-09-10T20:33:12.508248+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:33:12.508248+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":9960,"log_dups_size":0,"ondisk_log_size":9960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40787968,"num_objects":9958,"num_object_clones":0,"num_object_copies":19916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6683,"num_write_kb":26732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6310},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9898","reported_seq":10036,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:12.462644+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:33:12.462644+0000","last_peered":"2024-09-10T20:33:12.462644+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:33:12.462644+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":9898,"log_dups_size":0,"ondisk_log_size":9898,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40509440,"num_objects":9890,"num_object_clones":0,"num_object_copies":19780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6592,"num_write_kb":26368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6358},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193200+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193200+0000","last_peered":"2024-09-10T20:32:18.193200+0000","last_clean":"2024-09-10T20:32:18.193200+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193200+0000","last_undegraded":"2024-09-10T20:32:18.193200+0000","last_fullsized":"2024-09-10T20:32:18.193200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10079","reported_seq":10210,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:12.453112+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:33:12.453112+0000","last_peered":"2024-09-10T20:33:12.453112+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:33:12.453112+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41267200,"num_objects":10075,"num_object_clones":0,"num_object_copies":20150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6906,"num_write_kb":27624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6362},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9818","reported_seq":12272,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:33:12.591837+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:33:12.591837+0000","last_peered":"2024-09-10T20:33:12.591837+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:33:12.591837+0000","last_undegraded":"2024-09-10T20:33:12.591837+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":9818,"log_dups_size":0,"ondisk_log_size":9818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40165859,"num_objects":9807,"num_object_clones":0,"num_object_copies":19614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2324,"num_objects_unfound":0,"num_objects_dirty":9807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6425,"num_write_kb":25700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1144,"num_bytes_recovered":4685824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7483},{"shards":"0,2","objects":2324}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":650716585,"num_objects":158869,"num_object_clones":0,"num_object_copies":317738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2324,"num_objects_unfound":0,"num_objects_dirty":158869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123244,"num_write_kb":492956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4751,"num_bytes_recovered":19447808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1322389504,"data_stored":1322302792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158948,"ondisk_log_size":158948,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182456,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":850120,"kb_used_data":533880,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93521720,"statfs":{"total":96636764160,"available":95766241280,"internally_reserved":0,"allocated":546693120,"data_stored":546450432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323808938},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,3,1,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986228,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":699880,"kb_used_data":433560,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93671960,"statfs":{"total":96636764160,"available":95920087040,"internally_reserved":0,"allocated":443965440,"data_stored":443726829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272690858},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659175,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":557616,"kb_used_data":323232,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93814224,"statfs":{"total":96636764160,"available":96065765376,"internally_reserved":0,"allocated":330989568,"data_stored":330671007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":239988398},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":547266560,"data_stored":547262947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":331214848,"data_stored":331135362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":443908096,"data_stored":443904483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:33:15.336 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:33:15.336 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'9903", 'reported_seq': 9983, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:12.527042+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:33:12.527042+0000', 'last_peered': '2024-09-10T20:33:12.527042+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:33:12.527042+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 9903, 'log_dups_size': 0, 'ondisk_log_size': 9903, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40562688, 'num_objects': 9903, 'num_object_clones': 0, 'num_object_copies': 19806, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3607, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9903, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9903, 'num_write_kb': 39612, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6296}, {'shards': '0,2', 'objects': 3607}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:15.336 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:33:15.337 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'9850", 'reported_seq': 9984, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:12.554453+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:33:12.554453+0000', 'last_peered': '2024-09-10T20:33:12.554453+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:33:12.554453+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 9850, 'log_dups_size': 0, 'ondisk_log_size': 9850, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40325120, 'num_objects': 9845, 'num_object_clones': 0, 'num_object_copies': 19690, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3616, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9845, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6714, 'num_write_kb': 26856, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6231}, {'shards': '0,2', 'objects': 3614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:15.337 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:33:15.337 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'9893", 'reported_seq': 10029, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:12.588359+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:33:12.588359+0000', 'last_peered': '2024-09-10T20:33:12.588359+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:33:12.588359+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 9893, 'log_dups_size': 0, 'ondisk_log_size': 9893, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40497152, 'num_objects': 9887, 'num_object_clones': 0, 'num_object_copies': 19774, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3557, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9887, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6488, 'num_write_kb': 25952, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6332}, {'shards': '0,2', 'objects': 3555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:15.337 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:33:15.337 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'9800", 'reported_seq': 9932, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:13.514796+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:33:13.514796+0000', 'last_peered': '2024-09-10T20:33:13.514796+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:33:13.514796+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 9800, 'log_dups_size': 0, 'ondisk_log_size': 9800, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40112128, 'num_objects': 9793, 'num_object_clones': 0, 'num_object_copies': 19586, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3528, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9793, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7793, 'num_write_kb': 31172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 6267}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:33:15.337 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:33:15.337 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'9944", 'reported_seq': 10052, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:12.589939+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:33:12.589939+0000', 'last_peered': '2024-09-10T20:33:12.589939+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:33:12.589939+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 9944, 'log_dups_size': 0, 'ondisk_log_size': 9944, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40698822, 'num_objects': 9938, 'num_object_clones': 0, 'num_object_copies': 19876, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3617, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9938, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9942, 'num_write_kb': 39748, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6323}, {'shards': '0,2', 'objects': 3615}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:15.337 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:33:15.337 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'9910", 'reported_seq': 10038, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:13.552063+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:33:13.552063+0000', 'last_peered': '2024-09-10T20:33:13.552063+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:33:13.552063+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 9910, 'log_dups_size': 0, 'ondisk_log_size': 9910, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40570880, 'num_objects': 9905, 'num_object_clones': 0, 'num_object_copies': 19810, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3635, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9905, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7738, 'num_write_kb': 30952, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3633}, {'shards': '2', 'objects': 6272}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:33:15.338 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:33:15.338 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'9922", 'reported_seq': 10048, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:13.536842+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:33:13.536842+0000', 'last_peered': '2024-09-10T20:33:13.536842+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:33:13.536842+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 9922, 'log_dups_size': 0, 'ondisk_log_size': 9922, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40615936, 'num_objects': 9916, 'num_object_clones': 0, 'num_object_copies': 19832, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9916, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7708, 'num_write_kb': 30832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3610}, {'shards': '2', 'objects': 6306}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:33:15.338 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:33:15.338 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'9960", 'reported_seq': 10095, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:12.508248+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:33:12.508248+0000', 'last_peered': '2024-09-10T20:33:12.508248+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:33:12.508248+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 9960, 'log_dups_size': 0, 'ondisk_log_size': 9960, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40787968, 'num_objects': 9958, 'num_object_clones': 0, 'num_object_copies': 19916, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3649, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9958, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6683, 'num_write_kb': 26732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6310}, {'shards': '0,2', 'objects': 3648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:15.338 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:33:15.338 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'9898", 'reported_seq': 10036, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:12.462644+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:33:12.462644+0000', 'last_peered': '2024-09-10T20:33:12.462644+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:33:12.462644+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 9898, 'log_dups_size': 0, 'ondisk_log_size': 9898, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40509440, 'num_objects': 9890, 'num_object_clones': 0, 'num_object_copies': 19780, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3535, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9890, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6592, 'num_write_kb': 26368, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6358}, {'shards': '0,2', 'objects': 3532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:15.338 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:33:15.338 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'10079", 'reported_seq': 10210, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:12.453112+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:33:12.453112+0000', 'last_peered': '2024-09-10T20:33:12.453112+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:33:12.453112+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41267200, 'num_objects': 10075, 'num_object_clones': 0, 'num_object_copies': 20150, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3715, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10075, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6906, 'num_write_kb': 27624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6362}, {'shards': '0,2', 'objects': 3713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:15.338 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:33:15.338 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'9818", 'reported_seq': 12272, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:33:12.591837+0000', 'last_change': '2024-09-10T20:32:16.657652+0000', 'last_active': '2024-09-10T20:33:12.591837+0000', 'last_peered': '2024-09-10T20:33:12.591837+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:33:12.591837+0000', 'last_undegraded': '2024-09-10T20:33:12.591837+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 9818, 'log_dups_size': 0, 'ondisk_log_size': 9818, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40165859, 'num_objects': 9807, 'num_object_clones': 0, 'num_object_copies': 19614, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2324, 'num_objects_unfound': 0, 'num_objects_dirty': 9807, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6425, 'num_write_kb': 25700, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1144, 'num_bytes_recovered': 4685824, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7483}, {'shards': '0,2', 'objects': 2324}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:15.339 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:33:16.025 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:16.025 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36071/317742 objects degraded (11.352%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":307,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":158871,"data_bytes":650716604,"bytes_used":2158198784,"bytes_avail":287752093696,"bytes_total":289910292480,"degraded_objects":36071,"degraded_total":317742,"degraded_ratio":0.11352292111209722,"misplaced_objects":2324,"misplaced_total":317742,"misplaced_ratio":0.0073141101900283875,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":71310,"recovering_keys_per_sec":0,"num_objects_recovered":209,"num_bytes_recovered":856064,"num_keys_recovered":0,"write_bytes_sec":1855102,"read_op_per_sec":0,"write_op_per_sec":452},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:33:16.050 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:33:16.050 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:33:16.078 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 228 20 160202 160182 2.74377 1.58984 0.0221329 0.0284653 2024-09-10T20:33:16.614 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:16.617 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:33:16.641 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":192,"stamp":"2024-09-10T20:33:14.904453+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650716604,"num_objects":158871,"num_object_clones":0,"num_object_copies":317742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2324,"num_objects_unfound":0,"num_objects_dirty":158871,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123246,"num_write_kb":492958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4753,"num_bytes_recovered":19447827,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158951,"ondisk_log_size":158951,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2107616,"kb_used_data":1290672,"kb_used_omap":12,"kb_used_meta":816883,"kb_avail":281007904,"statfs":{"total":289910292480,"available":287752093696,"internally_reserved":0,"allocated":1321648128,"data_stored":1320848268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":836488194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,3,6,6,11,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22269952,"num_objects":5437,"num_object_clones":0,"num_object_copies":10874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-209,"num_objects_unfound":0,"num_objects_dirty":5437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5437,"num_write_kb":21748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":856064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004698"},"pg_stats":[{"pgid":"2.f","version":"45'9903","reported_seq":9983,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:12.527042+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:33:12.527042+0000","last_peered":"2024-09-10T20:33:12.527042+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:33:12.527042+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":9903,"log_dups_size":0,"ondisk_log_size":9903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40562688,"num_objects":9903,"num_object_clones":0,"num_object_copies":19806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9903,"num_write_kb":39612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6296},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10095","reported_seq":10205,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:13.529056+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:33:13.529056+0000","last_peered":"2024-09-10T20:33:13.529056+0000","last_clean":"2024-09-10T20:33:13.529056+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:33:13.529056+0000","last_undegraded":"2024-09-10T20:33:13.529056+0000","last_fullsized":"2024-09-10T20:33:13.529056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41324544,"num_objects":10089,"num_object_clones":0,"num_object_copies":20178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8281,"num_write_kb":33124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9952","reported_seq":17299,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:13.928290+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:33:13.928290+0000","last_peered":"2024-09-10T20:33:13.928290+0000","last_clean":"2024-09-10T20:33:13.928290+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:33:13.928290+0000","last_undegraded":"2024-09-10T20:33:13.928290+0000","last_fullsized":"2024-09-10T20:33:13.928290+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":9952,"log_dups_size":0,"ondisk_log_size":9952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40747008,"num_objects":9948,"num_object_clones":0,"num_object_copies":19896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8080,"num_write_kb":32320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'9900","reported_seq":9980,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:13.479797+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:33:13.479797+0000","last_peered":"2024-09-10T20:33:13.479797+0000","last_clean":"2024-09-10T20:33:13.479797+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:33:13.479797+0000","last_undegraded":"2024-09-10T20:33:13.479797+0000","last_fullsized":"2024-09-10T20:33:13.479797+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":9900,"log_dups_size":0,"ondisk_log_size":9900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40550400,"num_objects":9900,"num_object_clones":0,"num_object_copies":19800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9900,"num_write_kb":39600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10088","reported_seq":10214,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:12.567499+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:33:12.567499+0000","last_peered":"2024-09-10T20:33:12.567499+0000","last_clean":"2024-09-10T20:33:12.567499+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:33:12.567499+0000","last_undegraded":"2024-09-10T20:33:12.567499+0000","last_fullsized":"2024-09-10T20:33:12.567499+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41291776,"num_objects":10081,"num_object_clones":0,"num_object_copies":20162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7084,"num_write_kb":28336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9850","reported_seq":9984,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:12.554453+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:33:12.554453+0000","last_peered":"2024-09-10T20:33:12.554453+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:33:12.554453+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":9850,"log_dups_size":0,"ondisk_log_size":9850,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40325120,"num_objects":9845,"num_object_clones":0,"num_object_copies":19690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6714,"num_write_kb":26856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6231},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9893","reported_seq":10029,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:12.588359+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:33:12.588359+0000","last_peered":"2024-09-10T20:33:12.588359+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:33:12.588359+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":9893,"log_dups_size":0,"ondisk_log_size":9893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40497152,"num_objects":9887,"num_object_clones":0,"num_object_copies":19774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6488,"num_write_kb":25952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6332},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9936","reported_seq":10060,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:12.403021+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:33:12.403021+0000","last_peered":"2024-09-10T20:33:12.403021+0000","last_clean":"2024-09-10T20:33:12.403021+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:33:12.403021+0000","last_undegraded":"2024-09-10T20:33:12.403021+0000","last_fullsized":"2024-09-10T20:33:12.403021+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":9936,"log_dups_size":0,"ondisk_log_size":9936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40689664,"num_objects":9934,"num_object_clones":0,"num_object_copies":19868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7007,"num_write_kb":28028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9800","reported_seq":9932,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:13.514796+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:33:13.514796+0000","last_peered":"2024-09-10T20:33:13.514796+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:33:13.514796+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":9800,"log_dups_size":0,"ondisk_log_size":9800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40112128,"num_objects":9793,"num_object_clones":0,"num_object_copies":19586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7793,"num_write_kb":31172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":6267}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190232+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190232+0000","last_peered":"2024-09-10T20:32:18.190232+0000","last_clean":"2024-09-10T20:32:18.190232+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190232+0000","last_undegraded":"2024-09-10T20:32:18.190232+0000","last_fullsized":"2024-09-10T20:32:18.190232+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9944","reported_seq":10052,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:12.589939+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:33:12.589939+0000","last_peered":"2024-09-10T20:33:12.589939+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:33:12.589939+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":9944,"log_dups_size":0,"ondisk_log_size":9944,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40698822,"num_objects":9938,"num_object_clones":0,"num_object_copies":19876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9942,"num_write_kb":39748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6323},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9910","reported_seq":10038,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:13.552063+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:33:13.552063+0000","last_peered":"2024-09-10T20:33:13.552063+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:33:13.552063+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":9910,"log_dups_size":0,"ondisk_log_size":9910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40570880,"num_objects":9905,"num_object_clones":0,"num_object_copies":19810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7738,"num_write_kb":30952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":6272}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9922","reported_seq":10048,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:13.536842+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:33:13.536842+0000","last_peered":"2024-09-10T20:33:13.536842+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:33:13.536842+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":9922,"log_dups_size":0,"ondisk_log_size":9922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40615936,"num_objects":9916,"num_object_clones":0,"num_object_copies":19832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7708,"num_write_kb":30832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":6306}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9960","reported_seq":10095,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:12.508248+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:33:12.508248+0000","last_peered":"2024-09-10T20:33:12.508248+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:33:12.508248+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":9960,"log_dups_size":0,"ondisk_log_size":9960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40787968,"num_objects":9958,"num_object_clones":0,"num_object_copies":19916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6683,"num_write_kb":26732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6310},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9898","reported_seq":10036,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:12.462644+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:33:12.462644+0000","last_peered":"2024-09-10T20:33:12.462644+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:33:12.462644+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":9898,"log_dups_size":0,"ondisk_log_size":9898,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40509440,"num_objects":9890,"num_object_clones":0,"num_object_copies":19780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6592,"num_write_kb":26368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6358},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193200+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193200+0000","last_peered":"2024-09-10T20:32:18.193200+0000","last_clean":"2024-09-10T20:32:18.193200+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193200+0000","last_undegraded":"2024-09-10T20:32:18.193200+0000","last_fullsized":"2024-09-10T20:32:18.193200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10079","reported_seq":10210,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:12.453112+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:33:12.453112+0000","last_peered":"2024-09-10T20:33:12.453112+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:33:12.453112+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41267200,"num_objects":10075,"num_object_clones":0,"num_object_copies":20150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6906,"num_write_kb":27624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6362},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9818","reported_seq":12272,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:33:12.591837+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:33:12.591837+0000","last_peered":"2024-09-10T20:33:12.591837+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:33:12.591837+0000","last_undegraded":"2024-09-10T20:33:12.591837+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":9818,"log_dups_size":0,"ondisk_log_size":9818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40165859,"num_objects":9807,"num_object_clones":0,"num_object_copies":19614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2324,"num_objects_unfound":0,"num_objects_dirty":9807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6425,"num_write_kb":25700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1144,"num_bytes_recovered":4685824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7483},{"shards":"0,2","objects":2324}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":650716585,"num_objects":158869,"num_object_clones":0,"num_object_copies":317738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2324,"num_objects_unfound":0,"num_objects_dirty":158869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123244,"num_write_kb":492956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4751,"num_bytes_recovered":19447808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1322389504,"data_stored":1322302792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158948,"ondisk_log_size":158948,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182456,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":850120,"kb_used_data":533880,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93521720,"statfs":{"total":96636764160,"available":95766241280,"internally_reserved":0,"allocated":546693120,"data_stored":546450432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323808938},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,3,1,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986228,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":699880,"kb_used_data":433560,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93671960,"statfs":{"total":96636764160,"available":95920087040,"internally_reserved":0,"allocated":443965440,"data_stored":443726829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272690858},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659175,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":557616,"kb_used_data":323232,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93814224,"statfs":{"total":96636764160,"available":96065765376,"internally_reserved":0,"allocated":330989568,"data_stored":330671007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":239988398},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":547266560,"data_stored":547262947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":331214848,"data_stored":331135362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":443908096,"data_stored":443904483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:33:17.078 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 229 20 160645 160625 2.73934 1.73047 0.0142544 0.028512 2024-09-10T20:33:18.078 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 230 20 161117 161097 2.73545 1.84375 0.0516242 0.0285524 2024-09-10T20:33:19.080 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 231 20 161594 161574 2.73167 1.86328 0.0143435 0.0285918 2024-09-10T20:33:19.644 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:33:20.078 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 232 20 162034 162014 2.72731 1.71875 0.0376198 0.0286362 2024-09-10T20:33:20.188 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:20.188 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:33:20.215 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":194,"stamp":"2024-09-10T20:33:18.906241+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659207612,"num_objects":160944,"num_object_clones":0,"num_object_copies":321888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2229,"num_objects_unfound":0,"num_objects_dirty":160944,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":125319,"num_write_kb":501250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4848,"num_bytes_recovered":19836947,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160324,"ondisk_log_size":160324,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2127092,"kb_used_data":1301828,"kb_used_omap":12,"kb_used_meta":825203,"kb_avail":280988428,"statfs":{"total":289910292480,"available":287732150272,"internally_reserved":0,"allocated":1333071872,"data_stored":1332272012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":845007874},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,6,7,9,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27148288,"num_objects":6628,"num_object_clones":0,"num_object_copies":13256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-304,"num_objects_unfound":0,"num_objects_dirty":6628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6628,"num_write_kb":26512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":1245184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004993"},"pg_stats":[{"pgid":"2.f","version":"45'10051","reported_seq":10131,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:17.602976+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:33:17.602976+0000","last_peered":"2024-09-10T20:33:17.602976+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:33:17.602976+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41168896,"num_objects":10051,"num_object_clones":0,"num_object_copies":20102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10051,"num_write_kb":40204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6444},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10232","reported_seq":10342,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:18.529041+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:33:18.529041+0000","last_peered":"2024-09-10T20:33:18.529041+0000","last_clean":"2024-09-10T20:33:18.529041+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:33:18.529041+0000","last_undegraded":"2024-09-10T20:33:18.529041+0000","last_fullsized":"2024-09-10T20:33:18.529041+0000","mapping_epoch":35,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41885696,"num_objects":10226,"num_object_clones":0,"num_object_copies":20452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10226,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8418,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9952","reported_seq":17299,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:13.928290+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:33:13.928290+0000","last_peered":"2024-09-10T20:33:13.928290+0000","last_clean":"2024-09-10T20:33:13.928290+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:33:13.928290+0000","last_undegraded":"2024-09-10T20:33:13.928290+0000","last_fullsized":"2024-09-10T20:33:13.928290+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":9952,"log_dups_size":0,"ondisk_log_size":9952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40747008,"num_objects":9948,"num_object_clones":0,"num_object_copies":19896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8080,"num_write_kb":32320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10030","reported_seq":10108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:18.513480+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:33:18.513480+0000","last_peered":"2024-09-10T20:33:18.513480+0000","last_clean":"2024-09-10T20:33:18.513480+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:33:18.513480+0000","last_undegraded":"2024-09-10T20:33:18.513480+0000","last_fullsized":"2024-09-10T20:33:18.513480+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41082880,"num_objects":10030,"num_object_clones":0,"num_object_copies":20060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10030,"num_write_kb":40120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10243","reported_seq":10368,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:17.584129+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:33:17.584129+0000","last_peered":"2024-09-10T20:33:17.584129+0000","last_clean":"2024-09-10T20:33:17.584129+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:33:17.584129+0000","last_undegraded":"2024-09-10T20:33:17.584129+0000","last_fullsized":"2024-09-10T20:33:17.584129+0000","mapping_epoch":34,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41926656,"num_objects":10236,"num_object_clones":0,"num_object_copies":20472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7239,"num_write_kb":28956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9968","reported_seq":10104,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:17.561024+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:33:17.561024+0000","last_peered":"2024-09-10T20:33:17.561024+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:33:17.561024+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":9968,"log_dups_size":0,"ondisk_log_size":9968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40808448,"num_objects":9963,"num_object_clones":0,"num_object_copies":19926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6832,"num_write_kb":27328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6349},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10023","reported_seq":10161,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:17.543785+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:33:17.543785+0000","last_peered":"2024-09-10T20:33:17.543785+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:33:17.543785+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41029632,"num_objects":10017,"num_object_clones":0,"num_object_copies":20034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6618,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6462},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10082","reported_seq":10207,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:17.620785+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:33:17.620785+0000","last_peered":"2024-09-10T20:33:17.620785+0000","last_clean":"2024-09-10T20:33:17.620785+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:33:17.620785+0000","last_undegraded":"2024-09-10T20:33:17.620785+0000","last_fullsized":"2024-09-10T20:33:17.620785+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41287680,"num_objects":10080,"num_object_clones":0,"num_object_copies":20160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7153,"num_write_kb":28612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9949","reported_seq":10081,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:18.513005+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:33:18.513005+0000","last_peered":"2024-09-10T20:33:18.513005+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:33:18.513005+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":9949,"log_dups_size":0,"ondisk_log_size":9949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40722432,"num_objects":9942,"num_object_clones":0,"num_object_copies":19884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7942,"num_write_kb":31768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":6416}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190232+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190232+0000","last_peered":"2024-09-10T20:32:18.190232+0000","last_clean":"2024-09-10T20:32:18.190232+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190232+0000","last_undegraded":"2024-09-10T20:32:18.190232+0000","last_fullsized":"2024-09-10T20:32:18.190232+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10087","reported_seq":10196,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:17.566604+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:33:17.566604+0000","last_peered":"2024-09-10T20:33:17.566604+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:33:17.566604+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41284550,"num_objects":10081,"num_object_clones":0,"num_object_copies":20162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10085,"num_write_kb":40320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6466},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10038","reported_seq":10166,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:18.593011+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:33:18.593011+0000","last_peered":"2024-09-10T20:33:18.593011+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:33:18.593011+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41095168,"num_objects":10033,"num_object_clones":0,"num_object_copies":20066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7866,"num_write_kb":31464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":6400}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10074","reported_seq":10203,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:18.546848+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:33:18.546848+0000","last_peered":"2024-09-10T20:33:18.546848+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:33:18.546848+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41238528,"num_objects":10068,"num_object_clones":0,"num_object_copies":20136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7860,"num_write_kb":31440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":6458}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10107","reported_seq":10242,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:17.520839+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:33:17.520839+0000","last_peered":"2024-09-10T20:33:17.520839+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:33:17.520839+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41390080,"num_objects":10105,"num_object_clones":0,"num_object_copies":20210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6830,"num_write_kb":27320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6457},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10001","reported_seq":10138,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:17.599350+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:33:17.599350+0000","last_peered":"2024-09-10T20:33:17.599350+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:33:17.599350+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40931328,"num_objects":9993,"num_object_clones":0,"num_object_copies":19986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6695,"num_write_kb":26780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6461},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193200+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193200+0000","last_peered":"2024-09-10T20:32:18.193200+0000","last_clean":"2024-09-10T20:32:18.193200+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193200+0000","last_undegraded":"2024-09-10T20:32:18.193200+0000","last_fullsized":"2024-09-10T20:32:18.193200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10212","reported_seq":10341,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:17.594202+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:33:17.594202+0000","last_peered":"2024-09-10T20:33:17.594202+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:33:17.594202+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41811968,"num_objects":10208,"num_object_clones":0,"num_object_copies":20416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7039,"num_write_kb":28156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6495},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9972","reported_seq":12616,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:33:17.621183+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:33:17.621183+0000","last_peered":"2024-09-10T20:33:17.621183+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:33:17.621183+0000","last_undegraded":"2024-09-10T20:33:17.621183+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":9972,"log_dups_size":0,"ondisk_log_size":9972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40796643,"num_objects":9961,"num_object_clones":0,"num_object_copies":19922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2229,"num_objects_unfound":0,"num_objects_dirty":9961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6579,"num_write_kb":26316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1239,"num_bytes_recovered":5074944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7732},{"shards":"0,2","objects":2229}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":659207593,"num_objects":160942,"num_object_clones":0,"num_object_copies":321884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2229,"num_objects_unfound":0,"num_objects_dirty":160942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":125317,"num_write_kb":501248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4846,"num_bytes_recovered":19836928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1333796864,"data_stored":1333710152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160321,"ondisk_log_size":160321,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182457,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":860912,"kb_used_data":540512,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93510928,"statfs":{"total":96636764160,"available":95755190272,"internally_reserved":0,"allocated":553484288,"data_stored":553241600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328068778},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986229,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":708564,"kb_used_data":438084,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93663276,"statfs":{"total":96636764160,"available":95911194624,"internally_reserved":0,"allocated":448598016,"data_stored":448359405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276950698},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659175,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":557616,"kb_used_data":323232,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93814224,"statfs":{"total":96636764160,"available":96065765376,"internally_reserved":0,"allocated":330989568,"data_stored":330671007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":239988398},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":554041344,"data_stored":554037731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":331214848,"data_stored":331135362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":448540672,"data_stored":448537059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:33:20.216 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:33:20.774 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:20.777 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:33:20.803 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":194,"stamp":"2024-09-10T20:33:18.906241+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659207612,"num_objects":160944,"num_object_clones":0,"num_object_copies":321888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2229,"num_objects_unfound":0,"num_objects_dirty":160944,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":125319,"num_write_kb":501250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4848,"num_bytes_recovered":19836947,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160324,"ondisk_log_size":160324,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2127092,"kb_used_data":1301828,"kb_used_omap":12,"kb_used_meta":825203,"kb_avail":280988428,"statfs":{"total":289910292480,"available":287732150272,"internally_reserved":0,"allocated":1333071872,"data_stored":1332272012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":845007874},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,6,7,9,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27148288,"num_objects":6628,"num_object_clones":0,"num_object_copies":13256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-304,"num_objects_unfound":0,"num_objects_dirty":6628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6628,"num_write_kb":26512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":1245184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004993"},"pg_stats":[{"pgid":"2.f","version":"45'10051","reported_seq":10131,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:17.602976+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:33:17.602976+0000","last_peered":"2024-09-10T20:33:17.602976+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:33:17.602976+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41168896,"num_objects":10051,"num_object_clones":0,"num_object_copies":20102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10051,"num_write_kb":40204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6444},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10232","reported_seq":10342,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:18.529041+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:33:18.529041+0000","last_peered":"2024-09-10T20:33:18.529041+0000","last_clean":"2024-09-10T20:33:18.529041+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:33:18.529041+0000","last_undegraded":"2024-09-10T20:33:18.529041+0000","last_fullsized":"2024-09-10T20:33:18.529041+0000","mapping_epoch":35,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41885696,"num_objects":10226,"num_object_clones":0,"num_object_copies":20452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10226,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8418,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9952","reported_seq":17299,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:13.928290+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:33:13.928290+0000","last_peered":"2024-09-10T20:33:13.928290+0000","last_clean":"2024-09-10T20:33:13.928290+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:33:13.928290+0000","last_undegraded":"2024-09-10T20:33:13.928290+0000","last_fullsized":"2024-09-10T20:33:13.928290+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":9952,"log_dups_size":0,"ondisk_log_size":9952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40747008,"num_objects":9948,"num_object_clones":0,"num_object_copies":19896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8080,"num_write_kb":32320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10030","reported_seq":10108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:18.513480+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:33:18.513480+0000","last_peered":"2024-09-10T20:33:18.513480+0000","last_clean":"2024-09-10T20:33:18.513480+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:33:18.513480+0000","last_undegraded":"2024-09-10T20:33:18.513480+0000","last_fullsized":"2024-09-10T20:33:18.513480+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41082880,"num_objects":10030,"num_object_clones":0,"num_object_copies":20060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10030,"num_write_kb":40120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10243","reported_seq":10368,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:17.584129+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:33:17.584129+0000","last_peered":"2024-09-10T20:33:17.584129+0000","last_clean":"2024-09-10T20:33:17.584129+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:33:17.584129+0000","last_undegraded":"2024-09-10T20:33:17.584129+0000","last_fullsized":"2024-09-10T20:33:17.584129+0000","mapping_epoch":34,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41926656,"num_objects":10236,"num_object_clones":0,"num_object_copies":20472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7239,"num_write_kb":28956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9968","reported_seq":10104,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:17.561024+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:33:17.561024+0000","last_peered":"2024-09-10T20:33:17.561024+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:33:17.561024+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":9968,"log_dups_size":0,"ondisk_log_size":9968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40808448,"num_objects":9963,"num_object_clones":0,"num_object_copies":19926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6832,"num_write_kb":27328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6349},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10023","reported_seq":10161,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:17.543785+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:33:17.543785+0000","last_peered":"2024-09-10T20:33:17.543785+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:33:17.543785+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41029632,"num_objects":10017,"num_object_clones":0,"num_object_copies":20034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6618,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6462},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10082","reported_seq":10207,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:17.620785+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:33:17.620785+0000","last_peered":"2024-09-10T20:33:17.620785+0000","last_clean":"2024-09-10T20:33:17.620785+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:33:17.620785+0000","last_undegraded":"2024-09-10T20:33:17.620785+0000","last_fullsized":"2024-09-10T20:33:17.620785+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41287680,"num_objects":10080,"num_object_clones":0,"num_object_copies":20160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7153,"num_write_kb":28612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9949","reported_seq":10081,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:18.513005+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:33:18.513005+0000","last_peered":"2024-09-10T20:33:18.513005+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:33:18.513005+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":9949,"log_dups_size":0,"ondisk_log_size":9949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40722432,"num_objects":9942,"num_object_clones":0,"num_object_copies":19884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7942,"num_write_kb":31768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":6416}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190232+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190232+0000","last_peered":"2024-09-10T20:32:18.190232+0000","last_clean":"2024-09-10T20:32:18.190232+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190232+0000","last_undegraded":"2024-09-10T20:32:18.190232+0000","last_fullsized":"2024-09-10T20:32:18.190232+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10087","reported_seq":10196,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:17.566604+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:33:17.566604+0000","last_peered":"2024-09-10T20:33:17.566604+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:33:17.566604+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41284550,"num_objects":10081,"num_object_clones":0,"num_object_copies":20162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10085,"num_write_kb":40320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6466},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10038","reported_seq":10166,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:18.593011+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:33:18.593011+0000","last_peered":"2024-09-10T20:33:18.593011+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:33:18.593011+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41095168,"num_objects":10033,"num_object_clones":0,"num_object_copies":20066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7866,"num_write_kb":31464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":6400}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10074","reported_seq":10203,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:18.546848+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:33:18.546848+0000","last_peered":"2024-09-10T20:33:18.546848+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:33:18.546848+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41238528,"num_objects":10068,"num_object_clones":0,"num_object_copies":20136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7860,"num_write_kb":31440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":6458}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10107","reported_seq":10242,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:17.520839+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:33:17.520839+0000","last_peered":"2024-09-10T20:33:17.520839+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:33:17.520839+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41390080,"num_objects":10105,"num_object_clones":0,"num_object_copies":20210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6830,"num_write_kb":27320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6457},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10001","reported_seq":10138,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:17.599350+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:33:17.599350+0000","last_peered":"2024-09-10T20:33:17.599350+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:33:17.599350+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40931328,"num_objects":9993,"num_object_clones":0,"num_object_copies":19986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6695,"num_write_kb":26780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6461},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193200+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193200+0000","last_peered":"2024-09-10T20:32:18.193200+0000","last_clean":"2024-09-10T20:32:18.193200+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193200+0000","last_undegraded":"2024-09-10T20:32:18.193200+0000","last_fullsized":"2024-09-10T20:32:18.193200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10212","reported_seq":10341,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:17.594202+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:33:17.594202+0000","last_peered":"2024-09-10T20:33:17.594202+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:33:17.594202+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41811968,"num_objects":10208,"num_object_clones":0,"num_object_copies":20416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7039,"num_write_kb":28156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6495},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9972","reported_seq":12616,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:33:17.621183+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:33:17.621183+0000","last_peered":"2024-09-10T20:33:17.621183+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:33:17.621183+0000","last_undegraded":"2024-09-10T20:33:17.621183+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":9972,"log_dups_size":0,"ondisk_log_size":9972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40796643,"num_objects":9961,"num_object_clones":0,"num_object_copies":19922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2229,"num_objects_unfound":0,"num_objects_dirty":9961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6579,"num_write_kb":26316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1239,"num_bytes_recovered":5074944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7732},{"shards":"0,2","objects":2229}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":659207593,"num_objects":160942,"num_object_clones":0,"num_object_copies":321884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2229,"num_objects_unfound":0,"num_objects_dirty":160942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":125317,"num_write_kb":501248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4846,"num_bytes_recovered":19836928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1333796864,"data_stored":1333710152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160321,"ondisk_log_size":160321,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182457,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":860912,"kb_used_data":540512,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93510928,"statfs":{"total":96636764160,"available":95755190272,"internally_reserved":0,"allocated":553484288,"data_stored":553241600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328068778},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986229,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":708564,"kb_used_data":438084,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93663276,"statfs":{"total":96636764160,"available":95911194624,"internally_reserved":0,"allocated":448598016,"data_stored":448359405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276950698},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659175,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":557616,"kb_used_data":323232,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93814224,"statfs":{"total":96636764160,"available":96065765376,"internally_reserved":0,"allocated":330989568,"data_stored":330671007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":239988398},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":554041344,"data_stored":554037731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":331214848,"data_stored":331135362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":448540672,"data_stored":448537059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:33:20.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:33:20.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'10051", 'reported_seq': 10131, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:17.602976+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:33:17.602976+0000', 'last_peered': '2024-09-10T20:33:17.602976+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:33:17.602976+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41168896, 'num_objects': 10051, 'num_object_clones': 0, 'num_object_copies': 20102, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3607, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10051, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10051, 'num_write_kb': 40204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6444}, {'shards': '0,2', 'objects': 3607}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:20.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:33:20.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'9968", 'reported_seq': 10104, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:17.561024+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:33:17.561024+0000', 'last_peered': '2024-09-10T20:33:17.561024+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:33:17.561024+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 9968, 'log_dups_size': 0, 'ondisk_log_size': 9968, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40808448, 'num_objects': 9963, 'num_object_clones': 0, 'num_object_copies': 19926, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3616, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9963, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6832, 'num_write_kb': 27328, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6349}, {'shards': '0,2', 'objects': 3614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:20.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:33:20.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'10023", 'reported_seq': 10161, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:17.543785+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:33:17.543785+0000', 'last_peered': '2024-09-10T20:33:17.543785+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:33:17.543785+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41029632, 'num_objects': 10017, 'num_object_clones': 0, 'num_object_copies': 20034, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3557, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10017, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6618, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6462}, {'shards': '0,2', 'objects': 3555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:20.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:33:20.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'9949", 'reported_seq': 10081, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:18.513005+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:33:18.513005+0000', 'last_peered': '2024-09-10T20:33:18.513005+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:33:18.513005+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 9949, 'log_dups_size': 0, 'ondisk_log_size': 9949, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40722432, 'num_objects': 9942, 'num_object_clones': 0, 'num_object_copies': 19884, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3528, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9942, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7942, 'num_write_kb': 31768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 6416}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:33:20.806 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:33:20.806 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'10087", 'reported_seq': 10196, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:17.566604+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:33:17.566604+0000', 'last_peered': '2024-09-10T20:33:17.566604+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:33:17.566604+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41284550, 'num_objects': 10081, 'num_object_clones': 0, 'num_object_copies': 20162, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3617, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10081, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10085, 'num_write_kb': 40320, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6466}, {'shards': '0,2', 'objects': 3615}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:20.806 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:33:20.806 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'10038", 'reported_seq': 10166, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:18.593011+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:33:18.593011+0000', 'last_peered': '2024-09-10T20:33:18.593011+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:33:18.593011+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41095168, 'num_objects': 10033, 'num_object_clones': 0, 'num_object_copies': 20066, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3635, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10033, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7866, 'num_write_kb': 31464, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3633}, {'shards': '2', 'objects': 6400}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:33:20.806 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:33:20.806 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'10074", 'reported_seq': 10203, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:18.546848+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:33:18.546848+0000', 'last_peered': '2024-09-10T20:33:18.546848+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:33:18.546848+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41238528, 'num_objects': 10068, 'num_object_clones': 0, 'num_object_copies': 20136, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10068, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7860, 'num_write_kb': 31440, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3610}, {'shards': '2', 'objects': 6458}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:33:20.806 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:33:20.806 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'10107", 'reported_seq': 10242, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:17.520839+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:33:17.520839+0000', 'last_peered': '2024-09-10T20:33:17.520839+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:33:17.520839+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41390080, 'num_objects': 10105, 'num_object_clones': 0, 'num_object_copies': 20210, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3649, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10105, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6830, 'num_write_kb': 27320, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6457}, {'shards': '0,2', 'objects': 3648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:20.806 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:33:20.806 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'10001", 'reported_seq': 10138, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:17.599350+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:33:17.599350+0000', 'last_peered': '2024-09-10T20:33:17.599350+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:33:17.599350+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40931328, 'num_objects': 9993, 'num_object_clones': 0, 'num_object_copies': 19986, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3535, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9993, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6695, 'num_write_kb': 26780, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6461}, {'shards': '0,2', 'objects': 3532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:20.806 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:33:20.807 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'10212", 'reported_seq': 10341, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:17.594202+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:33:17.594202+0000', 'last_peered': '2024-09-10T20:33:17.594202+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:33:17.594202+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41811968, 'num_objects': 10208, 'num_object_clones': 0, 'num_object_copies': 20416, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3715, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10208, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7039, 'num_write_kb': 28156, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6495}, {'shards': '0,2', 'objects': 3713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:20.807 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:33:20.807 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'9972", 'reported_seq': 12616, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:33:17.621183+0000', 'last_change': '2024-09-10T20:32:16.657652+0000', 'last_active': '2024-09-10T20:33:17.621183+0000', 'last_peered': '2024-09-10T20:33:17.621183+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:33:17.621183+0000', 'last_undegraded': '2024-09-10T20:33:17.621183+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 9972, 'log_dups_size': 0, 'ondisk_log_size': 9972, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40796643, 'num_objects': 9961, 'num_object_clones': 0, 'num_object_copies': 19922, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2229, 'num_objects_unfound': 0, 'num_objects_dirty': 9961, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6579, 'num_write_kb': 26316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1239, 'num_bytes_recovered': 5074944, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7732}, {'shards': '0,2', 'objects': 2229}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:20.807 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:33:21.078 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 233 20 162455 162435 2.72266 1.64453 0.0409983 0.0286881 2024-09-10T20:33:21.480 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:21.481 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36071/322154 objects degraded (11.197%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":313,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":161077,"data_bytes":659752380,"bytes_used":2193850368,"bytes_avail":287716442112,"bytes_total":289910292480,"degraded_objects":36071,"degraded_total":322154,"degraded_ratio":0.11196818912693927,"misplaced_objects":2229,"misplaced_total":322154,"misplaced_ratio":0.0069190511370338407,"recovering_objects_per_sec":16,"recovering_bytes_per_sec":68237,"recovering_keys_per_sec":0,"num_objects_recovered":200,"num_bytes_recovered":819200,"num_keys_recovered":0,"write_bytes_sec":1538765,"read_op_per_sec":0,"write_op_per_sec":375},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:33:21.505 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:33:21.505 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:33:22.069 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:22.069 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:33:22.079 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 234 19 162859 162840 2.71778 1.58203 0.0765487 0.0287385 2024-09-10T20:33:22.091 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2024-09-10T20:33:20.907112+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659752380,"num_objects":161077,"num_object_clones":0,"num_object_copies":322154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2229,"num_objects_unfound":0,"num_objects_dirty":161077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":125452,"num_write_kb":501782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4848,"num_bytes_recovered":19836947,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160457,"ondisk_log_size":160457,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2142432,"kb_used_data":1308848,"kb_used_omap":12,"kb_used_meta":833523,"kb_avail":280973088,"statfs":{"total":289910292480,"available":287716442112,"internally_reserved":0,"allocated":1340260352,"data_stored":1339460492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":853527554},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,7,8,9,4,3,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18472960,"num_objects":4510,"num_object_clones":0,"num_object_copies":9020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-200,"num_objects_unfound":0,"num_objects_dirty":4510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4510,"num_write_kb":18040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":200,"num_bytes_recovered":819200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005052"},"pg_stats":[{"pgid":"2.f","version":"45'10051","reported_seq":10131,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:17.602976+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:33:17.602976+0000","last_peered":"2024-09-10T20:33:17.602976+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:33:17.602976+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41168896,"num_objects":10051,"num_object_clones":0,"num_object_copies":20102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10051,"num_write_kb":40204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6444},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10232","reported_seq":10342,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:18.529041+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:33:18.529041+0000","last_peered":"2024-09-10T20:33:18.529041+0000","last_clean":"2024-09-10T20:33:18.529041+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:33:18.529041+0000","last_undegraded":"2024-09-10T20:33:18.529041+0000","last_fullsized":"2024-09-10T20:33:18.529041+0000","mapping_epoch":35,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41885696,"num_objects":10226,"num_object_clones":0,"num_object_copies":20452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10226,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8418,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10085","reported_seq":17432,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:18.943039+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:33:18.943039+0000","last_peered":"2024-09-10T20:33:18.943039+0000","last_clean":"2024-09-10T20:33:18.943039+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:33:18.943039+0000","last_undegraded":"2024-09-10T20:33:18.943039+0000","last_fullsized":"2024-09-10T20:33:18.943039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41291776,"num_objects":10081,"num_object_clones":0,"num_object_copies":20162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8213,"num_write_kb":32852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10030","reported_seq":10108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:18.513480+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:33:18.513480+0000","last_peered":"2024-09-10T20:33:18.513480+0000","last_clean":"2024-09-10T20:33:18.513480+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:33:18.513480+0000","last_undegraded":"2024-09-10T20:33:18.513480+0000","last_fullsized":"2024-09-10T20:33:18.513480+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41082880,"num_objects":10030,"num_object_clones":0,"num_object_copies":20060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10030,"num_write_kb":40120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10243","reported_seq":10368,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:17.584129+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:33:17.584129+0000","last_peered":"2024-09-10T20:33:17.584129+0000","last_clean":"2024-09-10T20:33:17.584129+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:33:17.584129+0000","last_undegraded":"2024-09-10T20:33:17.584129+0000","last_fullsized":"2024-09-10T20:33:17.584129+0000","mapping_epoch":34,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41926656,"num_objects":10236,"num_object_clones":0,"num_object_copies":20472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7239,"num_write_kb":28956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9968","reported_seq":10104,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:17.561024+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:33:17.561024+0000","last_peered":"2024-09-10T20:33:17.561024+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:33:17.561024+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":9968,"log_dups_size":0,"ondisk_log_size":9968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40808448,"num_objects":9963,"num_object_clones":0,"num_object_copies":19926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6832,"num_write_kb":27328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6349},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10023","reported_seq":10161,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:17.543785+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:33:17.543785+0000","last_peered":"2024-09-10T20:33:17.543785+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:33:17.543785+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41029632,"num_objects":10017,"num_object_clones":0,"num_object_copies":20034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6618,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6462},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10082","reported_seq":10207,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:17.620785+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:33:17.620785+0000","last_peered":"2024-09-10T20:33:17.620785+0000","last_clean":"2024-09-10T20:33:17.620785+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:33:17.620785+0000","last_undegraded":"2024-09-10T20:33:17.620785+0000","last_fullsized":"2024-09-10T20:33:17.620785+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41287680,"num_objects":10080,"num_object_clones":0,"num_object_copies":20160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7153,"num_write_kb":28612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9949","reported_seq":10081,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:18.513005+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:33:18.513005+0000","last_peered":"2024-09-10T20:33:18.513005+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:33:18.513005+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":9949,"log_dups_size":0,"ondisk_log_size":9949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40722432,"num_objects":9942,"num_object_clones":0,"num_object_copies":19884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7942,"num_write_kb":31768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":6416}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190232+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190232+0000","last_peered":"2024-09-10T20:32:18.190232+0000","last_clean":"2024-09-10T20:32:18.190232+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190232+0000","last_undegraded":"2024-09-10T20:32:18.190232+0000","last_fullsized":"2024-09-10T20:32:18.190232+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10087","reported_seq":10196,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:17.566604+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:33:17.566604+0000","last_peered":"2024-09-10T20:33:17.566604+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:33:17.566604+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41284550,"num_objects":10081,"num_object_clones":0,"num_object_copies":20162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10085,"num_write_kb":40320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6466},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10038","reported_seq":10166,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:18.593011+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:33:18.593011+0000","last_peered":"2024-09-10T20:33:18.593011+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:33:18.593011+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41095168,"num_objects":10033,"num_object_clones":0,"num_object_copies":20066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7866,"num_write_kb":31464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":6400}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10074","reported_seq":10203,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:18.546848+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:33:18.546848+0000","last_peered":"2024-09-10T20:33:18.546848+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:33:18.546848+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41238528,"num_objects":10068,"num_object_clones":0,"num_object_copies":20136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7860,"num_write_kb":31440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":6458}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10107","reported_seq":10242,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:17.520839+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:33:17.520839+0000","last_peered":"2024-09-10T20:33:17.520839+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:33:17.520839+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41390080,"num_objects":10105,"num_object_clones":0,"num_object_copies":20210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6830,"num_write_kb":27320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6457},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10001","reported_seq":10138,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:17.599350+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:33:17.599350+0000","last_peered":"2024-09-10T20:33:17.599350+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:33:17.599350+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40931328,"num_objects":9993,"num_object_clones":0,"num_object_copies":19986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6695,"num_write_kb":26780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6461},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193200+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193200+0000","last_peered":"2024-09-10T20:32:18.193200+0000","last_clean":"2024-09-10T20:32:18.193200+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193200+0000","last_undegraded":"2024-09-10T20:32:18.193200+0000","last_fullsized":"2024-09-10T20:32:18.193200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10212","reported_seq":10341,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:17.594202+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:33:17.594202+0000","last_peered":"2024-09-10T20:33:17.594202+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:33:17.594202+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41811968,"num_objects":10208,"num_object_clones":0,"num_object_copies":20416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7039,"num_write_kb":28156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6495},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9972","reported_seq":12616,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:33:17.621183+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:33:17.621183+0000","last_peered":"2024-09-10T20:33:17.621183+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:33:17.621183+0000","last_undegraded":"2024-09-10T20:33:17.621183+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":9972,"log_dups_size":0,"ondisk_log_size":9972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40796643,"num_objects":9961,"num_object_clones":0,"num_object_copies":19922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2229,"num_objects_unfound":0,"num_objects_dirty":9961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6579,"num_write_kb":26316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1239,"num_bytes_recovered":5074944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7732},{"shards":"0,2","objects":2229}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":659752361,"num_objects":161075,"num_object_clones":0,"num_object_copies":322150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2229,"num_objects_unfound":0,"num_objects_dirty":161075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":125450,"num_write_kb":501780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4846,"num_bytes_recovered":19836928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1340981248,"data_stored":1340894536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160454,"ondisk_log_size":160454,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182457,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":860912,"kb_used_data":540512,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93510928,"statfs":{"total":96636764160,"available":95755190272,"internally_reserved":0,"allocated":553484288,"data_stored":553241600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328068778},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986229,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":708564,"kb_used_data":438084,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93663276,"statfs":{"total":96636764160,"available":95911194624,"internally_reserved":0,"allocated":448598016,"data_stored":448359405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276950698},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659176,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572956,"kb_used_data":330252,"kb_used_omap":4,"kb_used_meta":242683,"kb_avail":93798884,"statfs":{"total":96636764160,"available":96050057216,"internally_reserved":0,"allocated":338178048,"data_stored":337859487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":248508078},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,2,2,3,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":554041344,"data_stored":554037731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":338399232,"data_stored":338319746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":448540672,"data_stored":448537059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:33:23.079 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 235 20 163335 163315 2.71411 1.85547 0.0318457 0.028778 2024-09-10T20:33:24.079 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 236 20 163797 163777 2.71026 1.80469 0.0295971 0.0288184 2024-09-10T20:33:25.079 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 237 20 164262 164242 2.70649 1.81641 0.0419077 0.028859 2024-09-10T20:33:25.092 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:33:25.653 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:25.653 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:33:25.677 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":197,"stamp":"2024-09-10T20:33:24.908771+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668943804,"num_objects":163321,"num_object_clones":0,"num_object_copies":326642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2118,"num_objects_unfound":0,"num_objects_dirty":163321,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127696,"num_write_kb":510758,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4959,"num_bytes_recovered":20291603,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160801,"ondisk_log_size":160801,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2174384,"kb_used_data":1326912,"kb_used_omap":12,"kb_used_meta":847411,"kb_avail":280941136,"statfs":{"total":289910292480,"available":287683723264,"internally_reserved":0,"allocated":1358757888,"data_stored":1357958028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":867748866},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,5,10,18,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21680128,"num_objects":5293,"num_object_clones":0,"num_object_copies":10586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-206,"num_objects_unfound":0,"num_objects_dirty":5293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5293,"num_write_kb":21172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":843776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005092"},"pg_stats":[{"pgid":"2.f","version":"45'10176","reported_seq":10256,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:22.598144+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:33:22.598144+0000","last_peered":"2024-09-10T20:33:22.598144+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:33:22.598144+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41680896,"num_objects":10176,"num_object_clones":0,"num_object_copies":20352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10176,"num_write_kb":40704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6569},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10367","reported_seq":10476,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:23.587083+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:33:23.587083+0000","last_peered":"2024-09-10T20:33:23.587083+0000","last_clean":"2024-09-10T20:33:23.587083+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:33:23.587083+0000","last_undegraded":"2024-09-10T20:33:23.587083+0000","last_fullsized":"2024-09-10T20:33:23.587083+0000","mapping_epoch":35,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42438656,"num_objects":10361,"num_object_clones":0,"num_object_copies":20722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8553,"num_write_kb":34212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10248","reported_seq":17595,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:23.958153+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:33:23.958153+0000","last_peered":"2024-09-10T20:33:23.958153+0000","last_clean":"2024-09-10T20:33:23.958153+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:33:23.958153+0000","last_undegraded":"2024-09-10T20:33:23.958153+0000","last_fullsized":"2024-09-10T20:33:23.958153+0000","mapping_epoch":44,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41959424,"num_objects":10244,"num_object_clones":0,"num_object_copies":20488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8376,"num_write_kb":33504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10171","reported_seq":10251,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:23.499942+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:33:23.499942+0000","last_peered":"2024-09-10T20:33:23.499942+0000","last_clean":"2024-09-10T20:33:23.499942+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:33:23.499942+0000","last_undegraded":"2024-09-10T20:33:23.499942+0000","last_fullsized":"2024-09-10T20:33:23.499942+0000","mapping_epoch":35,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41660416,"num_objects":10171,"num_object_clones":0,"num_object_copies":20342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10171,"num_write_kb":40684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10379","reported_seq":10506,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:22.597304+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:33:22.597304+0000","last_peered":"2024-09-10T20:33:22.597304+0000","last_clean":"2024-09-10T20:33:22.597304+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:33:22.597304+0000","last_undegraded":"2024-09-10T20:33:22.597304+0000","last_fullsized":"2024-09-10T20:33:22.597304+0000","mapping_epoch":34,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42483712,"num_objects":10372,"num_object_clones":0,"num_object_copies":20744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7375,"num_write_kb":29500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10104","reported_seq":10238,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:22.634834+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:33:22.634834+0000","last_peered":"2024-09-10T20:33:22.634834+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:33:22.634834+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41365504,"num_objects":10099,"num_object_clones":0,"num_object_copies":20198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10099,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6968,"num_write_kb":27872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6485},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10157","reported_seq":10291,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:22.634819+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:33:22.634819+0000","last_peered":"2024-09-10T20:33:22.634819+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:33:22.634819+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41578496,"num_objects":10151,"num_object_clones":0,"num_object_copies":20302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10151,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6752,"num_write_kb":27008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6596},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10226","reported_seq":10351,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:22.533671+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:33:22.533671+0000","last_peered":"2024-09-10T20:33:22.533671+0000","last_clean":"2024-09-10T20:33:22.533671+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:33:22.533671+0000","last_undegraded":"2024-09-10T20:33:22.533671+0000","last_fullsized":"2024-09-10T20:33:22.533671+0000","mapping_epoch":34,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41877504,"num_objects":10224,"num_object_clones":0,"num_object_copies":20448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7297,"num_write_kb":29188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10081","reported_seq":10213,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:23.604559+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:33:23.604559+0000","last_peered":"2024-09-10T20:33:23.604559+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:33:23.604559+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41263104,"num_objects":10074,"num_object_clones":0,"num_object_copies":20148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8074,"num_write_kb":32296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":6548}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190273+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190273+0000","last_peered":"2024-09-10T20:32:18.190273+0000","last_clean":"2024-09-10T20:32:18.190273+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190273+0000","last_undegraded":"2024-09-10T20:32:18.190273+0000","last_fullsized":"2024-09-10T20:32:18.190273+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10220","reported_seq":10329,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:22.413000+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:33:22.413000+0000","last_peered":"2024-09-10T20:33:22.413000+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:33:22.413000+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41829318,"num_objects":10214,"num_object_clones":0,"num_object_copies":20428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10218,"num_write_kb":40852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6599},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10162","reported_seq":10291,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:23.552465+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:33:23.552465+0000","last_peered":"2024-09-10T20:33:23.552465+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:33:23.552465+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41603072,"num_objects":10157,"num_object_clones":0,"num_object_copies":20314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7990,"num_write_kb":31960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":6524}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10227","reported_seq":10355,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:23.584876+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:33:23.584876+0000","last_peered":"2024-09-10T20:33:23.584876+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:33:23.584876+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41865216,"num_objects":10221,"num_object_clones":0,"num_object_copies":20442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8013,"num_write_kb":32052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":6611}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10260","reported_seq":10394,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:22.673507+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:33:22.673507+0000","last_peered":"2024-09-10T20:33:22.673507+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:33:22.673507+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42016768,"num_objects":10258,"num_object_clones":0,"num_object_copies":20516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6983,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6610},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10142","reported_seq":10281,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:22.699438+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:33:22.699438+0000","last_peered":"2024-09-10T20:33:22.699438+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:33:22.699438+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41508864,"num_objects":10134,"num_object_clones":0,"num_object_copies":20268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6836,"num_write_kb":27344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6602},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193230+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193230+0000","last_peered":"2024-09-10T20:32:18.193230+0000","last_clean":"2024-09-10T20:32:18.193230+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193230+0000","last_undegraded":"2024-09-10T20:32:18.193230+0000","last_fullsized":"2024-09-10T20:32:18.193230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10355","reported_seq":10487,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:22.649029+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:33:22.649029+0000","last_peered":"2024-09-10T20:33:22.649029+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:33:22.649029+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42397696,"num_objects":10351,"num_object_clones":0,"num_object_copies":20702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7182,"num_write_kb":28728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6638},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10123","reported_seq":12987,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:33:22.698704+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:33:22.698704+0000","last_peered":"2024-09-10T20:33:22.698704+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:33:22.698704+0000","last_undegraded":"2024-09-10T20:33:22.698704+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41415139,"num_objects":10112,"num_object_clones":0,"num_object_copies":20224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2118,"num_objects_unfound":0,"num_objects_dirty":10112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6730,"num_write_kb":26920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1350,"num_bytes_recovered":5529600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7994},{"shards":"0,2","objects":2118}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":668943785,"num_objects":163319,"num_object_clones":0,"num_object_copies":326638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2118,"num_objects_unfound":0,"num_objects_dirty":163319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127694,"num_write_kb":510756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4957,"num_bytes_recovered":20291584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1359785984,"data_stored":1359699272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160798,"ondisk_log_size":160798,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182458,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":893888,"kb_used_data":547120,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93477952,"statfs":{"total":96636764160,"available":95721422848,"internally_reserved":0,"allocated":560250880,"data_stored":560008192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,9,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986230,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":712988,"kb_used_data":442508,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93658852,"statfs":{"total":96636764160,"available":95906664448,"internally_reserved":0,"allocated":453128192,"data_stored":452889581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276950698},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,4,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659177,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":567508,"kb_used_data":337284,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93804332,"statfs":{"total":96636764160,"available":96055635968,"internally_reserved":0,"allocated":345378816,"data_stored":345060255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":235728558},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,3,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":560922624,"data_stored":560919011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":345747456,"data_stored":345667970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":453115904,"data_stored":453112291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:33:25.679 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:33:26.079 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 238 20 164675 164655 2.70189 1.61328 0.0189662 0.0289083 2024-09-10T20:33:26.215 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:26.215 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:33:26.243 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":197,"stamp":"2024-09-10T20:33:24.908771+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668943804,"num_objects":163321,"num_object_clones":0,"num_object_copies":326642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2118,"num_objects_unfound":0,"num_objects_dirty":163321,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127696,"num_write_kb":510758,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4959,"num_bytes_recovered":20291603,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160801,"ondisk_log_size":160801,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2174384,"kb_used_data":1326912,"kb_used_omap":12,"kb_used_meta":847411,"kb_avail":280941136,"statfs":{"total":289910292480,"available":287683723264,"internally_reserved":0,"allocated":1358757888,"data_stored":1357958028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":867748866},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,5,10,18,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21680128,"num_objects":5293,"num_object_clones":0,"num_object_copies":10586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-206,"num_objects_unfound":0,"num_objects_dirty":5293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5293,"num_write_kb":21172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":843776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005092"},"pg_stats":[{"pgid":"2.f","version":"45'10176","reported_seq":10256,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:22.598144+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:33:22.598144+0000","last_peered":"2024-09-10T20:33:22.598144+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:33:22.598144+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41680896,"num_objects":10176,"num_object_clones":0,"num_object_copies":20352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10176,"num_write_kb":40704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6569},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10367","reported_seq":10476,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:23.587083+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:33:23.587083+0000","last_peered":"2024-09-10T20:33:23.587083+0000","last_clean":"2024-09-10T20:33:23.587083+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:33:23.587083+0000","last_undegraded":"2024-09-10T20:33:23.587083+0000","last_fullsized":"2024-09-10T20:33:23.587083+0000","mapping_epoch":35,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42438656,"num_objects":10361,"num_object_clones":0,"num_object_copies":20722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8553,"num_write_kb":34212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10248","reported_seq":17595,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:23.958153+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:33:23.958153+0000","last_peered":"2024-09-10T20:33:23.958153+0000","last_clean":"2024-09-10T20:33:23.958153+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:33:23.958153+0000","last_undegraded":"2024-09-10T20:33:23.958153+0000","last_fullsized":"2024-09-10T20:33:23.958153+0000","mapping_epoch":44,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41959424,"num_objects":10244,"num_object_clones":0,"num_object_copies":20488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8376,"num_write_kb":33504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10171","reported_seq":10251,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:23.499942+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:33:23.499942+0000","last_peered":"2024-09-10T20:33:23.499942+0000","last_clean":"2024-09-10T20:33:23.499942+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:33:23.499942+0000","last_undegraded":"2024-09-10T20:33:23.499942+0000","last_fullsized":"2024-09-10T20:33:23.499942+0000","mapping_epoch":35,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41660416,"num_objects":10171,"num_object_clones":0,"num_object_copies":20342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10171,"num_write_kb":40684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10379","reported_seq":10506,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:22.597304+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:33:22.597304+0000","last_peered":"2024-09-10T20:33:22.597304+0000","last_clean":"2024-09-10T20:33:22.597304+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:33:22.597304+0000","last_undegraded":"2024-09-10T20:33:22.597304+0000","last_fullsized":"2024-09-10T20:33:22.597304+0000","mapping_epoch":34,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42483712,"num_objects":10372,"num_object_clones":0,"num_object_copies":20744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7375,"num_write_kb":29500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10104","reported_seq":10238,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:22.634834+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:33:22.634834+0000","last_peered":"2024-09-10T20:33:22.634834+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:33:22.634834+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41365504,"num_objects":10099,"num_object_clones":0,"num_object_copies":20198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10099,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6968,"num_write_kb":27872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6485},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10157","reported_seq":10291,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:22.634819+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:33:22.634819+0000","last_peered":"2024-09-10T20:33:22.634819+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:33:22.634819+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41578496,"num_objects":10151,"num_object_clones":0,"num_object_copies":20302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10151,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6752,"num_write_kb":27008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6596},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10226","reported_seq":10351,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:22.533671+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:33:22.533671+0000","last_peered":"2024-09-10T20:33:22.533671+0000","last_clean":"2024-09-10T20:33:22.533671+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:33:22.533671+0000","last_undegraded":"2024-09-10T20:33:22.533671+0000","last_fullsized":"2024-09-10T20:33:22.533671+0000","mapping_epoch":34,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41877504,"num_objects":10224,"num_object_clones":0,"num_object_copies":20448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7297,"num_write_kb":29188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10081","reported_seq":10213,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:23.604559+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:33:23.604559+0000","last_peered":"2024-09-10T20:33:23.604559+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:33:23.604559+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41263104,"num_objects":10074,"num_object_clones":0,"num_object_copies":20148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8074,"num_write_kb":32296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":6548}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190273+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190273+0000","last_peered":"2024-09-10T20:32:18.190273+0000","last_clean":"2024-09-10T20:32:18.190273+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190273+0000","last_undegraded":"2024-09-10T20:32:18.190273+0000","last_fullsized":"2024-09-10T20:32:18.190273+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10220","reported_seq":10329,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:22.413000+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:33:22.413000+0000","last_peered":"2024-09-10T20:33:22.413000+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:33:22.413000+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41829318,"num_objects":10214,"num_object_clones":0,"num_object_copies":20428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10218,"num_write_kb":40852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6599},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10162","reported_seq":10291,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:23.552465+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:33:23.552465+0000","last_peered":"2024-09-10T20:33:23.552465+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:33:23.552465+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41603072,"num_objects":10157,"num_object_clones":0,"num_object_copies":20314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7990,"num_write_kb":31960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":6524}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10227","reported_seq":10355,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:23.584876+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:33:23.584876+0000","last_peered":"2024-09-10T20:33:23.584876+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:33:23.584876+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41865216,"num_objects":10221,"num_object_clones":0,"num_object_copies":20442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8013,"num_write_kb":32052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":6611}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10260","reported_seq":10394,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:22.673507+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:33:22.673507+0000","last_peered":"2024-09-10T20:33:22.673507+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:33:22.673507+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42016768,"num_objects":10258,"num_object_clones":0,"num_object_copies":20516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6983,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6610},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10142","reported_seq":10281,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:22.699438+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:33:22.699438+0000","last_peered":"2024-09-10T20:33:22.699438+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:33:22.699438+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41508864,"num_objects":10134,"num_object_clones":0,"num_object_copies":20268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6836,"num_write_kb":27344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6602},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193230+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193230+0000","last_peered":"2024-09-10T20:32:18.193230+0000","last_clean":"2024-09-10T20:32:18.193230+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193230+0000","last_undegraded":"2024-09-10T20:32:18.193230+0000","last_fullsized":"2024-09-10T20:32:18.193230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10355","reported_seq":10487,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:22.649029+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:33:22.649029+0000","last_peered":"2024-09-10T20:33:22.649029+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:33:22.649029+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42397696,"num_objects":10351,"num_object_clones":0,"num_object_copies":20702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7182,"num_write_kb":28728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6638},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10123","reported_seq":12987,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:33:22.698704+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:33:22.698704+0000","last_peered":"2024-09-10T20:33:22.698704+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:33:22.698704+0000","last_undegraded":"2024-09-10T20:33:22.698704+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41415139,"num_objects":10112,"num_object_clones":0,"num_object_copies":20224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2118,"num_objects_unfound":0,"num_objects_dirty":10112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6730,"num_write_kb":26920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1350,"num_bytes_recovered":5529600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7994},{"shards":"0,2","objects":2118}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":668943785,"num_objects":163319,"num_object_clones":0,"num_object_copies":326638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2118,"num_objects_unfound":0,"num_objects_dirty":163319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127694,"num_write_kb":510756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4957,"num_bytes_recovered":20291584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1359785984,"data_stored":1359699272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160798,"ondisk_log_size":160798,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182458,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":893888,"kb_used_data":547120,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93477952,"statfs":{"total":96636764160,"available":95721422848,"internally_reserved":0,"allocated":560250880,"data_stored":560008192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,9,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986230,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":712988,"kb_used_data":442508,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93658852,"statfs":{"total":96636764160,"available":95906664448,"internally_reserved":0,"allocated":453128192,"data_stored":452889581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276950698},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,4,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659177,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":567508,"kb_used_data":337284,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93804332,"statfs":{"total":96636764160,"available":96055635968,"internally_reserved":0,"allocated":345378816,"data_stored":345060255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":235728558},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,3,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":560922624,"data_stored":560919011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":345747456,"data_stored":345667970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":453115904,"data_stored":453112291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:33:26.245 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:33:26.245 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'10176", 'reported_seq': 10256, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:22.598144+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:33:22.598144+0000', 'last_peered': '2024-09-10T20:33:22.598144+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:33:22.598144+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41680896, 'num_objects': 10176, 'num_object_clones': 0, 'num_object_copies': 20352, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3607, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10176, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10176, 'num_write_kb': 40704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6569}, {'shards': '0,2', 'objects': 3607}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:26.245 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:33:26.245 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'10104", 'reported_seq': 10238, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:22.634834+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:33:22.634834+0000', 'last_peered': '2024-09-10T20:33:22.634834+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:33:22.634834+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41365504, 'num_objects': 10099, 'num_object_clones': 0, 'num_object_copies': 20198, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3616, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10099, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6968, 'num_write_kb': 27872, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6485}, {'shards': '0,2', 'objects': 3614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:26.245 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:33:26.245 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'10157", 'reported_seq': 10291, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:22.634819+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:33:22.634819+0000', 'last_peered': '2024-09-10T20:33:22.634819+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:33:22.634819+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41578496, 'num_objects': 10151, 'num_object_clones': 0, 'num_object_copies': 20302, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3557, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10151, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6752, 'num_write_kb': 27008, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6596}, {'shards': '0,2', 'objects': 3555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:26.245 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:33:26.245 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'10081", 'reported_seq': 10213, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:23.604559+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:33:23.604559+0000', 'last_peered': '2024-09-10T20:33:23.604559+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:33:23.604559+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41263104, 'num_objects': 10074, 'num_object_clones': 0, 'num_object_copies': 20148, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3528, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10074, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8074, 'num_write_kb': 32296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 6548}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:33:26.245 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:33:26.246 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'10220", 'reported_seq': 10329, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:22.413000+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:33:22.413000+0000', 'last_peered': '2024-09-10T20:33:22.413000+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:33:22.413000+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41829318, 'num_objects': 10214, 'num_object_clones': 0, 'num_object_copies': 20428, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3617, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10214, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10218, 'num_write_kb': 40852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6599}, {'shards': '0,2', 'objects': 3615}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:26.246 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:33:26.246 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'10162", 'reported_seq': 10291, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:23.552465+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:33:23.552465+0000', 'last_peered': '2024-09-10T20:33:23.552465+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:33:23.552465+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41603072, 'num_objects': 10157, 'num_object_clones': 0, 'num_object_copies': 20314, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3635, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10157, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7990, 'num_write_kb': 31960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3633}, {'shards': '2', 'objects': 6524}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:33:26.246 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:33:26.246 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'10227", 'reported_seq': 10355, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:23.584876+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:33:23.584876+0000', 'last_peered': '2024-09-10T20:33:23.584876+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:33:23.584876+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41865216, 'num_objects': 10221, 'num_object_clones': 0, 'num_object_copies': 20442, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10221, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8013, 'num_write_kb': 32052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3610}, {'shards': '2', 'objects': 6611}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:33:26.246 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:33:26.246 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'10260", 'reported_seq': 10394, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:22.673507+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:33:22.673507+0000', 'last_peered': '2024-09-10T20:33:22.673507+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:33:22.673507+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42016768, 'num_objects': 10258, 'num_object_clones': 0, 'num_object_copies': 20516, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3649, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10258, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6983, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6610}, {'shards': '0,2', 'objects': 3648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:26.246 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:33:26.246 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'10142", 'reported_seq': 10281, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:22.699438+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:33:22.699438+0000', 'last_peered': '2024-09-10T20:33:22.699438+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:33:22.699438+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41508864, 'num_objects': 10134, 'num_object_clones': 0, 'num_object_copies': 20268, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3535, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10134, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6836, 'num_write_kb': 27344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6602}, {'shards': '0,2', 'objects': 3532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:26.246 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:33:26.247 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'10355", 'reported_seq': 10487, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:22.649029+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:33:22.649029+0000', 'last_peered': '2024-09-10T20:33:22.649029+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:33:22.649029+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42397696, 'num_objects': 10351, 'num_object_clones': 0, 'num_object_copies': 20702, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3715, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10351, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7182, 'num_write_kb': 28728, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6638}, {'shards': '0,2', 'objects': 3713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:26.247 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:33:26.247 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'10123", 'reported_seq': 12987, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:33:22.698704+0000', 'last_change': '2024-09-10T20:32:16.657652+0000', 'last_active': '2024-09-10T20:33:22.698704+0000', 'last_peered': '2024-09-10T20:33:22.698704+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:33:22.698704+0000', 'last_undegraded': '2024-09-10T20:33:22.698704+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41415139, 'num_objects': 10112, 'num_object_clones': 0, 'num_object_copies': 20224, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2118, 'num_objects_unfound': 0, 'num_objects_dirty': 10112, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6730, 'num_write_kb': 26920, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1350, 'num_bytes_recovered': 5529600, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7994}, {'shards': '0,2', 'objects': 2118}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:26.247 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:33:26.960 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:26.961 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36071/326642 objects degraded (11.043%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":318,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":163321,"data_bytes":668943804,"bytes_used":2226569216,"bytes_avail":287683723264,"bytes_total":289910292480,"degraded_objects":36071,"degraded_total":326642,"degraded_ratio":0.11042976714568242,"misplaced_objects":2118,"misplaced_total":326642,"misplaced_ratio":0.0064841630898659682,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":70284,"recovering_keys_per_sec":0,"num_objects_recovered":206,"num_bytes_recovered":843776,"num_keys_recovered":0,"write_bytes_sec":1805910,"read_op_per_sec":0,"write_op_per_sec":440},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:33:26.983 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:33:26.983 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:33:27.079 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout: 239 20 165099 165079 2.69752 1.65625 0.038274 0.0289534 2024-09-10T20:33:27.560 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:27.560 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:33:27.583 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":198,"stamp":"2024-09-10T20:33:26.909628+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668943804,"num_objects":163321,"num_object_clones":0,"num_object_copies":326642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2118,"num_objects_unfound":0,"num_objects_dirty":163321,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127696,"num_write_kb":510758,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4959,"num_bytes_recovered":20291603,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160801,"ondisk_log_size":160801,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2174384,"kb_used_data":1326912,"kb_used_omap":12,"kb_used_meta":847411,"kb_avail":280941136,"statfs":{"total":289910292480,"available":287683723264,"internally_reserved":0,"allocated":1358757888,"data_stored":1357958028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":867748866},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,5,10,18,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18227200,"num_objects":4450,"num_object_clones":0,"num_object_copies":8900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-206,"num_objects_unfound":0,"num_objects_dirty":4450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4450,"num_write_kb":17800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":843776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005174"},"pg_stats":[{"pgid":"2.f","version":"45'10176","reported_seq":10256,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:22.598144+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:33:22.598144+0000","last_peered":"2024-09-10T20:33:22.598144+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:33:22.598144+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41680896,"num_objects":10176,"num_object_clones":0,"num_object_copies":20352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10176,"num_write_kb":40704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6569},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10367","reported_seq":10476,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:23.587083+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:33:23.587083+0000","last_peered":"2024-09-10T20:33:23.587083+0000","last_clean":"2024-09-10T20:33:23.587083+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:33:23.587083+0000","last_undegraded":"2024-09-10T20:33:23.587083+0000","last_fullsized":"2024-09-10T20:33:23.587083+0000","mapping_epoch":35,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42438656,"num_objects":10361,"num_object_clones":0,"num_object_copies":20722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8553,"num_write_kb":34212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10248","reported_seq":17595,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:23.958153+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:33:23.958153+0000","last_peered":"2024-09-10T20:33:23.958153+0000","last_clean":"2024-09-10T20:33:23.958153+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:33:23.958153+0000","last_undegraded":"2024-09-10T20:33:23.958153+0000","last_fullsized":"2024-09-10T20:33:23.958153+0000","mapping_epoch":44,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41959424,"num_objects":10244,"num_object_clones":0,"num_object_copies":20488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8376,"num_write_kb":33504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10171","reported_seq":10251,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:23.499942+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:33:23.499942+0000","last_peered":"2024-09-10T20:33:23.499942+0000","last_clean":"2024-09-10T20:33:23.499942+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:33:23.499942+0000","last_undegraded":"2024-09-10T20:33:23.499942+0000","last_fullsized":"2024-09-10T20:33:23.499942+0000","mapping_epoch":35,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41660416,"num_objects":10171,"num_object_clones":0,"num_object_copies":20342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10171,"num_write_kb":40684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10379","reported_seq":10506,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:22.597304+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:33:22.597304+0000","last_peered":"2024-09-10T20:33:22.597304+0000","last_clean":"2024-09-10T20:33:22.597304+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:33:22.597304+0000","last_undegraded":"2024-09-10T20:33:22.597304+0000","last_fullsized":"2024-09-10T20:33:22.597304+0000","mapping_epoch":34,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42483712,"num_objects":10372,"num_object_clones":0,"num_object_copies":20744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7375,"num_write_kb":29500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10104","reported_seq":10238,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:22.634834+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:33:22.634834+0000","last_peered":"2024-09-10T20:33:22.634834+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:33:22.634834+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41365504,"num_objects":10099,"num_object_clones":0,"num_object_copies":20198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10099,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6968,"num_write_kb":27872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6485},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10157","reported_seq":10291,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:22.634819+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:33:22.634819+0000","last_peered":"2024-09-10T20:33:22.634819+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:33:22.634819+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41578496,"num_objects":10151,"num_object_clones":0,"num_object_copies":20302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10151,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6752,"num_write_kb":27008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6596},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10226","reported_seq":10351,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:22.533671+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:33:22.533671+0000","last_peered":"2024-09-10T20:33:22.533671+0000","last_clean":"2024-09-10T20:33:22.533671+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:33:22.533671+0000","last_undegraded":"2024-09-10T20:33:22.533671+0000","last_fullsized":"2024-09-10T20:33:22.533671+0000","mapping_epoch":34,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41877504,"num_objects":10224,"num_object_clones":0,"num_object_copies":20448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7297,"num_write_kb":29188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10081","reported_seq":10213,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:23.604559+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:33:23.604559+0000","last_peered":"2024-09-10T20:33:23.604559+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:33:23.604559+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41263104,"num_objects":10074,"num_object_clones":0,"num_object_copies":20148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8074,"num_write_kb":32296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":6548}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190273+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190273+0000","last_peered":"2024-09-10T20:32:18.190273+0000","last_clean":"2024-09-10T20:32:18.190273+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190273+0000","last_undegraded":"2024-09-10T20:32:18.190273+0000","last_fullsized":"2024-09-10T20:32:18.190273+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10220","reported_seq":10329,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:22.413000+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:33:22.413000+0000","last_peered":"2024-09-10T20:33:22.413000+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:33:22.413000+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41829318,"num_objects":10214,"num_object_clones":0,"num_object_copies":20428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10218,"num_write_kb":40852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6599},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10162","reported_seq":10291,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:23.552465+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:33:23.552465+0000","last_peered":"2024-09-10T20:33:23.552465+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:33:23.552465+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41603072,"num_objects":10157,"num_object_clones":0,"num_object_copies":20314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7990,"num_write_kb":31960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3633},{"shards":"2","objects":6524}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10227","reported_seq":10355,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:23.584876+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:33:23.584876+0000","last_peered":"2024-09-10T20:33:23.584876+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:33:23.584876+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41865216,"num_objects":10221,"num_object_clones":0,"num_object_copies":20442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8013,"num_write_kb":32052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":6611}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10260","reported_seq":10394,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:22.673507+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:33:22.673507+0000","last_peered":"2024-09-10T20:33:22.673507+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:33:22.673507+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42016768,"num_objects":10258,"num_object_clones":0,"num_object_copies":20516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6983,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6610},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10142","reported_seq":10281,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:22.699438+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:33:22.699438+0000","last_peered":"2024-09-10T20:33:22.699438+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:33:22.699438+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41508864,"num_objects":10134,"num_object_clones":0,"num_object_copies":20268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6836,"num_write_kb":27344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6602},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193230+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193230+0000","last_peered":"2024-09-10T20:32:18.193230+0000","last_clean":"2024-09-10T20:32:18.193230+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193230+0000","last_undegraded":"2024-09-10T20:32:18.193230+0000","last_fullsized":"2024-09-10T20:32:18.193230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10355","reported_seq":10487,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:22.649029+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:33:22.649029+0000","last_peered":"2024-09-10T20:33:22.649029+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:33:22.649029+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42397696,"num_objects":10351,"num_object_clones":0,"num_object_copies":20702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7182,"num_write_kb":28728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6638},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10123","reported_seq":12987,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:33:22.698704+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:33:22.698704+0000","last_peered":"2024-09-10T20:33:22.698704+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:33:22.698704+0000","last_undegraded":"2024-09-10T20:33:22.698704+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41415139,"num_objects":10112,"num_object_clones":0,"num_object_copies":20224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2118,"num_objects_unfound":0,"num_objects_dirty":10112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6730,"num_write_kb":26920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1350,"num_bytes_recovered":5529600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7994},{"shards":"0,2","objects":2118}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":668943785,"num_objects":163319,"num_object_clones":0,"num_object_copies":326638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2118,"num_objects_unfound":0,"num_objects_dirty":163319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127694,"num_write_kb":510756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4957,"num_bytes_recovered":20291584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1359785984,"data_stored":1359699272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160798,"ondisk_log_size":160798,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182458,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":893888,"kb_used_data":547120,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93477952,"statfs":{"total":96636764160,"available":95721422848,"internally_reserved":0,"allocated":560250880,"data_stored":560008192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,9,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986230,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":712988,"kb_used_data":442508,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93658852,"statfs":{"total":96636764160,"available":95906664448,"internally_reserved":0,"allocated":453128192,"data_stored":452889581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276950698},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,4,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659177,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":567508,"kb_used_data":337284,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93804332,"statfs":{"total":96636764160,"available":96055635968,"internally_reserved":0,"allocated":345378816,"data_stored":345060255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":235728558},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,3,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":560922624,"data_stored":560919011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":345747456,"data_stored":345667970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":453115904,"data_stored":453112291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:33:28.079 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout:Total time run: 240.028 2024-09-10T20:33:28.638 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout:Total writes made: 165517 2024-09-10T20:33:28.639 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout:Write size: 4096 2024-09-10T20:33:28.639 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout:Object size: 4096 2024-09-10T20:33:28.639 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout:Bandwidth (MB/sec): 2.69365 2024-09-10T20:33:28.639 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout:Stddev Bandwidth: 1.87168 2024-09-10T20:33:28.639 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout:Max bandwidth (MB/sec): 12.4883 2024-09-10T20:33:28.639 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout:Min bandwidth (MB/sec): 0 2024-09-10T20:33:28.639 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout:Average IOPS: 689 2024-09-10T20:33:28.639 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout:Stddev IOPS: 479.155 2024-09-10T20:33:28.639 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout:Max IOPS: 3197 2024-09-10T20:33:28.639 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout:Min IOPS: 0 2024-09-10T20:33:28.639 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout:Average Latency(s): 0.0289982 2024-09-10T20:33:28.639 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout:Stddev Latency(s): 0.0415122 2024-09-10T20:33:28.640 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout:Max latency(s): 2.51074 2024-09-10T20:33:28.640 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout:Min latency(s): 0.00191414 2024-09-10T20:33:28.640 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout:Cleaning up (deleting benchmark objects) 2024-09-10T20:33:30.586 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:33:31.140 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:31.141 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:33:31.164 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":200,"stamp":"2024-09-10T20:33:30.911274+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":677094868,"num_objects":165312,"num_object_clones":0,"num_object_copies":330624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2010,"num_objects_unfound":0,"num_objects_dirty":165312,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":129935,"num_write_kb":519215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5067,"num_bytes_recovered":20733971,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160840,"ondisk_log_size":160840,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2204144,"kb_used_data":1344184,"kb_used_omap":12,"kb_used_meta":859891,"kb_avail":280911376,"statfs":{"total":289910292480,"available":287653249024,"internally_reserved":0,"allocated":1376444416,"data_stored":1375636412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":880528386},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,2,6,7,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17887256,"num_objects":4368,"num_object_clones":0,"num_object_copies":8736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-219,"num_objects_unfound":0,"num_objects_dirty":4368,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4616,"num_write_kb":17965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":219,"num_bytes_recovered":897024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005033"},"pg_stats":[{"pgid":"2.f","version":"45'10320","reported_seq":10400,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:27.738603+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:33:27.738603+0000","last_peered":"2024-09-10T20:33:27.738603+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:33:27.738603+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42270720,"num_objects":10320,"num_object_clones":0,"num_object_copies":20640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10320,"num_write_kb":41280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6713},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10489","reported_seq":10598,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:28.618028+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:33:28.618028+0000","last_peered":"2024-09-10T20:33:28.618028+0000","last_clean":"2024-09-10T20:33:28.618028+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:33:28.618028+0000","last_undegraded":"2024-09-10T20:33:28.618028+0000","last_fullsized":"2024-09-10T20:33:28.618028+0000","mapping_epoch":35,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42815488,"num_objects":10453,"num_object_clones":0,"num_object_copies":20906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8675,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10395","reported_seq":17742,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:29.009860+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:33:29.009860+0000","last_peered":"2024-09-10T20:33:29.009860+0000","last_clean":"2024-09-10T20:33:29.009860+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:33:29.009860+0000","last_undegraded":"2024-09-10T20:33:29.009860+0000","last_fullsized":"2024-09-10T20:33:29.009860+0000","mapping_epoch":44,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42270744,"num_objects":10321,"num_object_clones":0,"num_object_copies":20642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10321,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8523,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10315","reported_seq":10395,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:28.455775+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:33:28.455775+0000","last_peered":"2024-09-10T20:33:28.455775+0000","last_clean":"2024-09-10T20:33:28.455775+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:33:28.455775+0000","last_undegraded":"2024-09-10T20:33:28.455775+0000","last_fullsized":"2024-09-10T20:33:28.455775+0000","mapping_epoch":35,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42143744,"num_objects":10289,"num_object_clones":0,"num_object_copies":20578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10315,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10524","reported_seq":10651,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:27.690824+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:33:27.690824+0000","last_peered":"2024-09-10T20:33:27.690824+0000","last_clean":"2024-09-10T20:33:27.690824+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:33:27.690824+0000","last_undegraded":"2024-09-10T20:33:27.690824+0000","last_fullsized":"2024-09-10T20:33:27.690824+0000","mapping_epoch":34,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43077632,"num_objects":10517,"num_object_clones":0,"num_object_copies":21034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7520,"num_write_kb":30080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10233","reported_seq":10369,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:27.740761+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:33:27.740761+0000","last_peered":"2024-09-10T20:33:27.740761+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:33:27.740761+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41893888,"num_objects":10228,"num_object_clones":0,"num_object_copies":20456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7097,"num_write_kb":28388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6614},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10300","reported_seq":10438,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:27.663962+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:33:27.663962+0000","last_peered":"2024-09-10T20:33:27.663962+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:33:27.663962+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42164224,"num_objects":10294,"num_object_clones":0,"num_object_copies":20588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6895,"num_write_kb":27580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6739},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10359","reported_seq":10484,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:27.690555+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:33:27.690555+0000","last_peered":"2024-09-10T20:33:27.690555+0000","last_clean":"2024-09-10T20:33:27.690555+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:33:27.690555+0000","last_undegraded":"2024-09-10T20:33:27.690555+0000","last_fullsized":"2024-09-10T20:33:27.690555+0000","mapping_epoch":34,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42422272,"num_objects":10357,"num_object_clones":0,"num_object_copies":20714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7430,"num_write_kb":29720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10246","reported_seq":10378,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:28.586550+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:33:28.586550+0000","last_peered":"2024-09-10T20:33:28.586550+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:33:28.586550+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41750528,"num_objects":10193,"num_object_clones":0,"num_object_copies":20386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8239,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3502},{"shards":"2","objects":6667}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190273+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190273+0000","last_peered":"2024-09-10T20:32:18.190273+0000","last_clean":"2024-09-10T20:32:18.190273+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190273+0000","last_undegraded":"2024-09-10T20:32:18.190273+0000","last_fullsized":"2024-09-10T20:32:18.190273+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10359","reported_seq":10468,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:27.636708+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:33:27.636708+0000","last_peered":"2024-09-10T20:33:27.636708+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:33:27.636708+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42398662,"num_objects":10353,"num_object_clones":0,"num_object_copies":20706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10357,"num_write_kb":41408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6738},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10305","reported_seq":10434,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:28.612945+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:33:28.612945+0000","last_peered":"2024-09-10T20:33:28.612945+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:33:28.612945+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42024960,"num_objects":10260,"num_object_clones":0,"num_object_copies":20520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8133,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3613},{"shards":"2","objects":6627}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10353","reported_seq":10482,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:28.583821+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:33:28.583821+0000","last_peered":"2024-09-10T20:33:28.583821+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:33:28.583821+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42233856,"num_objects":10311,"num_object_clones":0,"num_object_copies":20622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8139,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3592},{"shards":"2","objects":6701}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10410","reported_seq":10545,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:27.662270+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:33:27.662270+0000","last_peered":"2024-09-10T20:33:27.662270+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:33:27.662270+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42631168,"num_objects":10408,"num_object_clones":0,"num_object_copies":20816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7133,"num_write_kb":28532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6760},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10284","reported_seq":10423,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:27.727075+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:33:27.727075+0000","last_peered":"2024-09-10T20:33:27.727075+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:33:27.727075+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42090496,"num_objects":10276,"num_object_clones":0,"num_object_copies":20552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6978,"num_write_kb":27912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6744},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193230+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193230+0000","last_peered":"2024-09-10T20:32:18.193230+0000","last_clean":"2024-09-10T20:32:18.193230+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193230+0000","last_undegraded":"2024-09-10T20:32:18.193230+0000","last_fullsized":"2024-09-10T20:32:18.193230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10488","reported_seq":10620,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:27.705509+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:33:27.705509+0000","last_peered":"2024-09-10T20:33:27.705509+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:33:27.705509+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42942464,"num_objects":10484,"num_object_clones":0,"num_object_copies":20968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7315,"num_write_kb":29260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6771},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10257","reported_seq":13339,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:33:27.741365+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:33:27.741365+0000","last_peered":"2024-09-10T20:33:27.741365+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:33:27.741365+0000","last_undegraded":"2024-09-10T20:33:27.741365+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41964003,"num_objects":10246,"num_object_clones":0,"num_object_copies":20492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2010,"num_objects_unfound":0,"num_objects_dirty":10246,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6864,"num_write_kb":27456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1458,"num_bytes_recovered":5971968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8236},{"shards":"0,2","objects":2010}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":677094849,"num_objects":165310,"num_object_clones":0,"num_object_copies":330620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2010,"num_objects_unfound":0,"num_objects_dirty":165310,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":129933,"num_write_kb":519213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5065,"num_bytes_recovered":20733952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1375686656,"data_stored":1375591800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160837,"ondisk_log_size":160837,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182459,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":904812,"kb_used_data":553884,"kb_used_omap":4,"kb_used_meta":350907,"kb_avail":93467028,"statfs":{"total":96636764160,"available":95710236672,"internally_reserved":0,"allocated":567177216,"data_stored":566934528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":359329450},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,5,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986231,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":721332,"kb_used_data":446684,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93650508,"statfs":{"total":96636764160,"available":95898120192,"internally_reserved":0,"allocated":457404416,"data_stored":457161733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281210538},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,2,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659178,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":578000,"kb_used_data":343616,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93793840,"statfs":{"total":96636764160,"available":96044892160,"internally_reserved":0,"allocated":351862784,"data_stored":351540151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":239988398},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":567787520,"data_stored":567783907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":351371264,"data_stored":351287706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":456527872,"data_stored":456520187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:33:31.166 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:33:31.727 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:31.728 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:33:31.749 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":200,"stamp":"2024-09-10T20:33:30.911274+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":677094868,"num_objects":165312,"num_object_clones":0,"num_object_copies":330624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2010,"num_objects_unfound":0,"num_objects_dirty":165312,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":129935,"num_write_kb":519215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5067,"num_bytes_recovered":20733971,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160840,"ondisk_log_size":160840,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2204144,"kb_used_data":1344184,"kb_used_omap":12,"kb_used_meta":859891,"kb_avail":280911376,"statfs":{"total":289910292480,"available":287653249024,"internally_reserved":0,"allocated":1376444416,"data_stored":1375636412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":880528386},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,2,6,7,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17887256,"num_objects":4368,"num_object_clones":0,"num_object_copies":8736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-219,"num_objects_unfound":0,"num_objects_dirty":4368,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4616,"num_write_kb":17965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":219,"num_bytes_recovered":897024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005033"},"pg_stats":[{"pgid":"2.f","version":"45'10320","reported_seq":10400,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:27.738603+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:33:27.738603+0000","last_peered":"2024-09-10T20:33:27.738603+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:33:27.738603+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42270720,"num_objects":10320,"num_object_clones":0,"num_object_copies":20640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10320,"num_write_kb":41280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6713},{"shards":"0,2","objects":3607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10489","reported_seq":10598,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:28.618028+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:33:28.618028+0000","last_peered":"2024-09-10T20:33:28.618028+0000","last_clean":"2024-09-10T20:33:28.618028+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:33:28.618028+0000","last_undegraded":"2024-09-10T20:33:28.618028+0000","last_fullsized":"2024-09-10T20:33:28.618028+0000","mapping_epoch":35,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42815488,"num_objects":10453,"num_object_clones":0,"num_object_copies":20906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8675,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10395","reported_seq":17742,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:29.009860+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:33:29.009860+0000","last_peered":"2024-09-10T20:33:29.009860+0000","last_clean":"2024-09-10T20:33:29.009860+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:33:29.009860+0000","last_undegraded":"2024-09-10T20:33:29.009860+0000","last_fullsized":"2024-09-10T20:33:29.009860+0000","mapping_epoch":44,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42270744,"num_objects":10321,"num_object_clones":0,"num_object_copies":20642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10321,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8523,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10315","reported_seq":10395,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:28.455775+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:33:28.455775+0000","last_peered":"2024-09-10T20:33:28.455775+0000","last_clean":"2024-09-10T20:33:28.455775+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:33:28.455775+0000","last_undegraded":"2024-09-10T20:33:28.455775+0000","last_fullsized":"2024-09-10T20:33:28.455775+0000","mapping_epoch":35,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42143744,"num_objects":10289,"num_object_clones":0,"num_object_copies":20578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10315,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10524","reported_seq":10651,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:27.690824+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:33:27.690824+0000","last_peered":"2024-09-10T20:33:27.690824+0000","last_clean":"2024-09-10T20:33:27.690824+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:33:27.690824+0000","last_undegraded":"2024-09-10T20:33:27.690824+0000","last_fullsized":"2024-09-10T20:33:27.690824+0000","mapping_epoch":34,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43077632,"num_objects":10517,"num_object_clones":0,"num_object_copies":21034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7520,"num_write_kb":30080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10233","reported_seq":10369,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:27.740761+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:33:27.740761+0000","last_peered":"2024-09-10T20:33:27.740761+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:33:27.740761+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41893888,"num_objects":10228,"num_object_clones":0,"num_object_copies":20456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7097,"num_write_kb":28388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6614},{"shards":"0,2","objects":3614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10300","reported_seq":10438,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:27.663962+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:33:27.663962+0000","last_peered":"2024-09-10T20:33:27.663962+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:33:27.663962+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42164224,"num_objects":10294,"num_object_clones":0,"num_object_copies":20588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6895,"num_write_kb":27580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6739},{"shards":"0,2","objects":3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10359","reported_seq":10484,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:27.690555+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:33:27.690555+0000","last_peered":"2024-09-10T20:33:27.690555+0000","last_clean":"2024-09-10T20:33:27.690555+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:33:27.690555+0000","last_undegraded":"2024-09-10T20:33:27.690555+0000","last_fullsized":"2024-09-10T20:33:27.690555+0000","mapping_epoch":34,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42422272,"num_objects":10357,"num_object_clones":0,"num_object_copies":20714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7430,"num_write_kb":29720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10246","reported_seq":10378,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:28.586550+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:33:28.586550+0000","last_peered":"2024-09-10T20:33:28.586550+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:33:28.586550+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41750528,"num_objects":10193,"num_object_clones":0,"num_object_copies":20386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8239,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3502},{"shards":"2","objects":6667}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190273+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190273+0000","last_peered":"2024-09-10T20:32:18.190273+0000","last_clean":"2024-09-10T20:32:18.190273+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190273+0000","last_undegraded":"2024-09-10T20:32:18.190273+0000","last_fullsized":"2024-09-10T20:32:18.190273+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10359","reported_seq":10468,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:27.636708+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:33:27.636708+0000","last_peered":"2024-09-10T20:33:27.636708+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:33:27.636708+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42398662,"num_objects":10353,"num_object_clones":0,"num_object_copies":20706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10357,"num_write_kb":41408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6738},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10305","reported_seq":10434,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:28.612945+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:33:28.612945+0000","last_peered":"2024-09-10T20:33:28.612945+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:33:28.612945+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42024960,"num_objects":10260,"num_object_clones":0,"num_object_copies":20520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8133,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3613},{"shards":"2","objects":6627}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10353","reported_seq":10482,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:28.583821+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:33:28.583821+0000","last_peered":"2024-09-10T20:33:28.583821+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:33:28.583821+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42233856,"num_objects":10311,"num_object_clones":0,"num_object_copies":20622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8139,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3592},{"shards":"2","objects":6701}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10410","reported_seq":10545,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:27.662270+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:33:27.662270+0000","last_peered":"2024-09-10T20:33:27.662270+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:33:27.662270+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42631168,"num_objects":10408,"num_object_clones":0,"num_object_copies":20816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7133,"num_write_kb":28532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6760},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10284","reported_seq":10423,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:27.727075+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:33:27.727075+0000","last_peered":"2024-09-10T20:33:27.727075+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:33:27.727075+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42090496,"num_objects":10276,"num_object_clones":0,"num_object_copies":20552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6978,"num_write_kb":27912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6744},{"shards":"0,2","objects":3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193230+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193230+0000","last_peered":"2024-09-10T20:32:18.193230+0000","last_clean":"2024-09-10T20:32:18.193230+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193230+0000","last_undegraded":"2024-09-10T20:32:18.193230+0000","last_fullsized":"2024-09-10T20:32:18.193230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10488","reported_seq":10620,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:27.705509+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:33:27.705509+0000","last_peered":"2024-09-10T20:33:27.705509+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:33:27.705509+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42942464,"num_objects":10484,"num_object_clones":0,"num_object_copies":20968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7315,"num_write_kb":29260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6771},{"shards":"0,2","objects":3713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10257","reported_seq":13339,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:33:27.741365+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:33:27.741365+0000","last_peered":"2024-09-10T20:33:27.741365+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:33:27.741365+0000","last_undegraded":"2024-09-10T20:33:27.741365+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41964003,"num_objects":10246,"num_object_clones":0,"num_object_copies":20492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2010,"num_objects_unfound":0,"num_objects_dirty":10246,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6864,"num_write_kb":27456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1458,"num_bytes_recovered":5971968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8236},{"shards":"0,2","objects":2010}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":677094849,"num_objects":165310,"num_object_clones":0,"num_object_copies":330620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36071,"num_objects_misplaced":2010,"num_objects_unfound":0,"num_objects_dirty":165310,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":129933,"num_write_kb":519213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5065,"num_bytes_recovered":20733952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1375686656,"data_stored":1375591800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160837,"ondisk_log_size":160837,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182459,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":904812,"kb_used_data":553884,"kb_used_omap":4,"kb_used_meta":350907,"kb_avail":93467028,"statfs":{"total":96636764160,"available":95710236672,"internally_reserved":0,"allocated":567177216,"data_stored":566934528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":359329450},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,5,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986231,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":721332,"kb_used_data":446684,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93650508,"statfs":{"total":96636764160,"available":95898120192,"internally_reserved":0,"allocated":457404416,"data_stored":457161733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281210538},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,2,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659178,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":578000,"kb_used_data":343616,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93793840,"statfs":{"total":96636764160,"available":96044892160,"internally_reserved":0,"allocated":351862784,"data_stored":351540151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":239988398},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":567787520,"data_stored":567783907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":351371264,"data_stored":351287706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":456527872,"data_stored":456520187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:33:31.750 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:33:31.750 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'10320", 'reported_seq': 10400, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:27.738603+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:33:27.738603+0000', 'last_peered': '2024-09-10T20:33:27.738603+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:33:27.738603+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42270720, 'num_objects': 10320, 'num_object_clones': 0, 'num_object_copies': 20640, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3607, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10320, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10320, 'num_write_kb': 41280, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6713}, {'shards': '0,2', 'objects': 3607}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:31.750 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:33:31.750 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'10233", 'reported_seq': 10369, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:27.740761+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:33:27.740761+0000', 'last_peered': '2024-09-10T20:33:27.740761+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:33:27.740761+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41893888, 'num_objects': 10228, 'num_object_clones': 0, 'num_object_copies': 20456, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3616, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10228, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7097, 'num_write_kb': 28388, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6614}, {'shards': '0,2', 'objects': 3614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:31.750 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:33:31.750 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'10300", 'reported_seq': 10438, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:27.663962+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:33:27.663962+0000', 'last_peered': '2024-09-10T20:33:27.663962+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:33:27.663962+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42164224, 'num_objects': 10294, 'num_object_clones': 0, 'num_object_copies': 20588, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3557, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10294, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6895, 'num_write_kb': 27580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6739}, {'shards': '0,2', 'objects': 3555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:31.751 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:33:31.751 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'10246", 'reported_seq': 10378, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:28.586550+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:33:28.586550+0000', 'last_peered': '2024-09-10T20:33:28.586550+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:33:28.586550+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41750528, 'num_objects': 10193, 'num_object_clones': 0, 'num_object_copies': 20386, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3528, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10193, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8239, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3502}, {'shards': '2', 'objects': 6667}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:33:31.751 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:33:31.751 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'10359", 'reported_seq': 10468, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:27.636708+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:33:27.636708+0000', 'last_peered': '2024-09-10T20:33:27.636708+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:33:27.636708+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42398662, 'num_objects': 10353, 'num_object_clones': 0, 'num_object_copies': 20706, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3617, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10353, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10357, 'num_write_kb': 41408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6738}, {'shards': '0,2', 'objects': 3615}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:31.751 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:33:31.751 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'10305", 'reported_seq': 10434, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:28.612945+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:33:28.612945+0000', 'last_peered': '2024-09-10T20:33:28.612945+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:33:28.612945+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42024960, 'num_objects': 10260, 'num_object_clones': 0, 'num_object_copies': 20520, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3635, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10260, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8133, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3613}, {'shards': '2', 'objects': 6627}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:33:31.751 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:33:31.751 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'10353", 'reported_seq': 10482, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:28.583821+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:33:28.583821+0000', 'last_peered': '2024-09-10T20:33:28.583821+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:33:28.583821+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42233856, 'num_objects': 10311, 'num_object_clones': 0, 'num_object_copies': 20622, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10311, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8139, 'num_write_kb': 32484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3592}, {'shards': '2', 'objects': 6701}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:33:31.751 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:33:31.751 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'10410", 'reported_seq': 10545, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:27.662270+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:33:27.662270+0000', 'last_peered': '2024-09-10T20:33:27.662270+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:33:27.662270+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42631168, 'num_objects': 10408, 'num_object_clones': 0, 'num_object_copies': 20816, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3649, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10408, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7133, 'num_write_kb': 28532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6760}, {'shards': '0,2', 'objects': 3648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:31.751 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:33:31.752 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'10284", 'reported_seq': 10423, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:27.727075+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:33:27.727075+0000', 'last_peered': '2024-09-10T20:33:27.727075+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:33:27.727075+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42090496, 'num_objects': 10276, 'num_object_clones': 0, 'num_object_copies': 20552, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3535, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10276, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6978, 'num_write_kb': 27912, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6744}, {'shards': '0,2', 'objects': 3532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:31.752 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:33:31.752 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'10488", 'reported_seq': 10620, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:27.705509+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:33:27.705509+0000', 'last_peered': '2024-09-10T20:33:27.705509+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:33:27.705509+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42942464, 'num_objects': 10484, 'num_object_clones': 0, 'num_object_copies': 20968, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3715, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10484, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7315, 'num_write_kb': 29260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6771}, {'shards': '0,2', 'objects': 3713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:31.752 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:33:31.752 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'10257", 'reported_seq': 13339, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:33:27.741365+0000', 'last_change': '2024-09-10T20:32:16.657652+0000', 'last_active': '2024-09-10T20:33:27.741365+0000', 'last_peered': '2024-09-10T20:33:27.741365+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:33:27.741365+0000', 'last_undegraded': '2024-09-10T20:33:27.741365+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41964003, 'num_objects': 10246, 'num_object_clones': 0, 'num_object_copies': 20492, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2010, 'num_objects_unfound': 0, 'num_objects_dirty': 10246, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6864, 'num_write_kb': 27456, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1458, 'num_bytes_recovered': 5971968, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 8236}, {'shards': '0,2', 'objects': 2010}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:31.753 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:33:32.420 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:32.420 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36071/330624 objects degraded (10.910%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":324,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":165312,"data_bytes":677094868,"bytes_used":2257043456,"bytes_avail":287653249024,"bytes_total":289910292480,"degraded_objects":36071,"degraded_total":330624,"degraded_ratio":0.10909976287262872,"misplaced_objects":2010,"misplaced_total":330624,"misplaced_ratio":0.0060794134727061551,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":74720,"recovering_keys_per_sec":0,"num_objects_recovered":219,"num_bytes_recovered":897024,"num_keys_recovered":0,"read_bytes_sec":85,"write_bytes_sec":1532370,"read_op_per_sec":0,"write_op_per_sec":384},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:33:32.455 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:33:32.455 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:33:33.006 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:33.006 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:33:33.039 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":201,"stamp":"2024-09-10T20:33:32.912047+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":672196052,"num_objects":164116,"num_object_clones":0,"num_object_copies":328232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36048,"num_objects_misplaced":1861,"num_objects_unfound":0,"num_objects_dirty":164116,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":131305,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5239,"num_bytes_recovered":21344275,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160910,"ondisk_log_size":160910,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2187712,"kb_used_data":1340220,"kb_used_omap":12,"kb_used_meta":847411,"kb_avail":280927808,"statfs":{"total":289910292480,"available":287670075392,"internally_reserved":0,"allocated":1372385280,"data_stored":1371577276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":867748866},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,1,4,3,2,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":12443672,"num_objects":3039,"num_object_clones":0,"num_object_copies":6078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-23,"num_objects_misplaced":-368,"num_objects_unfound":0,"num_objects_dirty":3039,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5853,"num_write_kb":17781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":391,"num_bytes_recovered":1507328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004934"},"pg_stats":[{"pgid":"2.f","version":"45'10458","reported_seq":10547,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:32.719634+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:33:32.719634+0000","last_peered":"2024-09-10T20:33:32.719634+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:33:32.719634+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41779200,"num_objects":10200,"num_object_clones":0,"num_object_copies":20400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3603,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10458,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6597},{"shards":"0,2","objects":3478}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10489","reported_seq":10598,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:28.618028+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:33:28.618028+0000","last_peered":"2024-09-10T20:33:28.618028+0000","last_clean":"2024-09-10T20:33:28.618028+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:33:28.618028+0000","last_undegraded":"2024-09-10T20:33:28.618028+0000","last_fullsized":"2024-09-10T20:33:28.618028+0000","mapping_epoch":35,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42815488,"num_objects":10453,"num_object_clones":0,"num_object_copies":20906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8675,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10395","reported_seq":17742,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:29.009860+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:33:29.009860+0000","last_peered":"2024-09-10T20:33:29.009860+0000","last_clean":"2024-09-10T20:33:29.009860+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:33:29.009860+0000","last_undegraded":"2024-09-10T20:33:29.009860+0000","last_fullsized":"2024-09-10T20:33:29.009860+0000","mapping_epoch":44,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42270744,"num_objects":10321,"num_object_clones":0,"num_object_copies":20642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10321,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8523,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10315","reported_seq":10395,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:28.455775+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:33:28.455775+0000","last_peered":"2024-09-10T20:33:28.455775+0000","last_clean":"2024-09-10T20:33:28.455775+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:33:28.455775+0000","last_undegraded":"2024-09-10T20:33:28.455775+0000","last_fullsized":"2024-09-10T20:33:28.455775+0000","mapping_epoch":35,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42143744,"num_objects":10289,"num_object_clones":0,"num_object_copies":20578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10315,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10673","reported_seq":10800,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:32.646865+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:33:32.646865+0000","last_peered":"2024-09-10T20:33:32.646865+0000","last_clean":"2024-09-10T20:33:32.646865+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:33:32.646865+0000","last_undegraded":"2024-09-10T20:33:32.646865+0000","last_fullsized":"2024-09-10T20:33:32.646865+0000","mapping_epoch":34,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42582016,"num_objects":10396,"num_object_clones":0,"num_object_copies":20792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7669,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10360","reported_seq":10503,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:32.721379+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:33:32.721379+0000","last_peered":"2024-09-10T20:33:32.721379+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:33:32.721379+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41414656,"num_objects":10111,"num_object_clones":0,"num_object_copies":20222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3613,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7224,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6500},{"shards":"0,2","objects":3492}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10453","reported_seq":10603,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:32.746344+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:33:32.746344+0000","last_peered":"2024-09-10T20:33:32.746344+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:33:32.746344+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41594880,"num_objects":10155,"num_object_clones":0,"num_object_copies":20310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3551,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7048,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6606},{"shards":"0,2","objects":3409}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10473","reported_seq":10598,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:32.781928+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:33:32.781928+0000","last_peered":"2024-09-10T20:33:32.781928+0000","last_clean":"2024-09-10T20:33:32.781928+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:33:32.781928+0000","last_undegraded":"2024-09-10T20:33:32.781928+0000","last_fullsized":"2024-09-10T20:33:32.781928+0000","mapping_epoch":34,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42037248,"num_objects":10263,"num_object_clones":0,"num_object_copies":20526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7544,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10246","reported_seq":10378,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:28.586550+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:33:28.586550+0000","last_peered":"2024-09-10T20:33:28.586550+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:33:28.586550+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41750528,"num_objects":10193,"num_object_clones":0,"num_object_copies":20386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8239,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3502},{"shards":"2","objects":6667}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190273+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190273+0000","last_peered":"2024-09-10T20:32:18.190273+0000","last_clean":"2024-09-10T20:32:18.190273+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190273+0000","last_undegraded":"2024-09-10T20:32:18.190273+0000","last_fullsized":"2024-09-10T20:32:18.190273+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10493","reported_seq":10606,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:32.684868+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:33:32.684868+0000","last_peered":"2024-09-10T20:33:32.684868+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:33:32.684868+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41931718,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3615,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10491,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6626},{"shards":"0,2","objects":3491}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10305","reported_seq":10434,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:28.612945+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:33:28.612945+0000","last_peered":"2024-09-10T20:33:28.612945+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:33:28.612945+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42024960,"num_objects":10260,"num_object_clones":0,"num_object_copies":20520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8133,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3613},{"shards":"2","objects":6627}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10353","reported_seq":10482,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:28.583821+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:33:28.583821+0000","last_peered":"2024-09-10T20:33:28.583821+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:33:28.583821+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42233856,"num_objects":10311,"num_object_clones":0,"num_object_copies":20622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8139,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3592},{"shards":"2","objects":6701}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10559","reported_seq":10698,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:32.753811+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:33:32.753811+0000","last_peered":"2024-09-10T20:33:32.753811+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:33:32.753811+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42070016,"num_objects":10271,"num_object_clones":0,"num_object_copies":20542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3647,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7282,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6625},{"shards":"0,2","objects":3505}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10428","reported_seq":10570,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:32.805970+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:33:32.805970+0000","last_peered":"2024-09-10T20:33:32.805970+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:33:32.805970+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41598976,"num_objects":10156,"num_object_clones":0,"num_object_copies":20312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3533,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7122,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6626},{"shards":"0,2","objects":3400}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193230+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193230+0000","last_peered":"2024-09-10T20:32:18.193230+0000","last_clean":"2024-09-10T20:32:18.193230+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193230+0000","last_undegraded":"2024-09-10T20:32:18.193230+0000","last_fullsized":"2024-09-10T20:32:18.193230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10626","reported_seq":10766,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:32.768444+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:33:32.768444+0000","last_peered":"2024-09-10T20:33:32.768444+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:33:32.768444+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42434560,"num_objects":10360,"num_object_clones":0,"num_object_copies":20720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3711,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7453,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6651},{"shards":"0,2","objects":3582}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10381","reported_seq":13762,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:33:32.804775+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:33:32.804775+0000","last_peered":"2024-09-10T20:33:32.804775+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:33:32.804775+0000","last_undegraded":"2024-09-10T20:33:32.804775+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41513443,"num_objects":10136,"num_object_clones":0,"num_object_copies":20272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1861,"num_objects_unfound":0,"num_objects_dirty":10136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6988,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1607,"num_bytes_recovered":6582272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8275},{"shards":"0,2","objects":1861}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":672196033,"num_objects":164114,"num_object_clones":0,"num_object_copies":328228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36048,"num_objects_misplaced":1861,"num_objects_unfound":0,"num_objects_dirty":164114,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":131303,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5237,"num_bytes_recovered":21344256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1369583616,"data_stored":1369488760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160907,"ondisk_log_size":160907,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182460,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":888380,"kb_used_data":549920,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93483460,"statfs":{"total":96636764160,"available":95727063040,"internally_reserved":0,"allocated":563118080,"data_stored":562875392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,1,0,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986231,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":721332,"kb_used_data":446684,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93650508,"statfs":{"total":96636764160,"available":95898120192,"internally_reserved":0,"allocated":457404416,"data_stored":457161733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281210538},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,2,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659178,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":578000,"kb_used_data":343616,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93793840,"statfs":{"total":96636764160,"available":96044892160,"internally_reserved":0,"allocated":351862784,"data_stored":351540151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":239988398},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":561684480,"data_stored":561680867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":351371264,"data_stored":351287706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":456527872,"data_stored":456520187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:33:36.041 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:33:36.580 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:36.581 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:33:36.608 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":202,"stamp":"2024-09-10T20:33:34.912749+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668784084,"num_objects":163283,"num_object_clones":0,"num_object_copies":326566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36046,"num_objects_misplaced":1861,"num_objects_unfound":0,"num_objects_dirty":163283,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":132138,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5241,"num_bytes_recovered":21344275,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160943,"ondisk_log_size":160943,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2192740,"kb_used_data":1335528,"kb_used_omap":12,"kb_used_meta":857139,"kb_avail":280922780,"statfs":{"total":289910292480,"available":287664926720,"internally_reserved":0,"allocated":1367580672,"data_stored":1366772668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":877710338},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,5,9,9,0,5,0,0,0,0,2],"upper_bound":8192},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3313688,"num_objects":810,"num_object_clones":0,"num_object_copies":1620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-25,"num_objects_misplaced":-257,"num_objects_unfound":0,"num_objects_dirty":810,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5290,"num_write_kb":12197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":1052672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004811"},"pg_stats":[{"pgid":"2.f","version":"45'10458","reported_seq":10547,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:32.719634+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:33:32.719634+0000","last_peered":"2024-09-10T20:33:32.719634+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:33:32.719634+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41779200,"num_objects":10200,"num_object_clones":0,"num_object_copies":20400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3603,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10458,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6597},{"shards":"0,2","objects":3478}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10641","reported_seq":10751,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:33.524088+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:33:33.524088+0000","last_peered":"2024-09-10T20:33:33.524088+0000","last_clean":"2024-09-10T20:33:33.524088+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:33:33.524088+0000","last_undegraded":"2024-09-10T20:33:33.524088+0000","last_fullsized":"2024-09-10T20:33:33.524088+0000","mapping_epoch":35,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42192896,"num_objects":10301,"num_object_clones":0,"num_object_copies":20602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10301,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8827,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10521","reported_seq":17869,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:34.079826+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:33:34.079826+0000","last_peered":"2024-09-10T20:33:34.079826+0000","last_clean":"2024-09-10T20:33:34.079826+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:33:34.079826+0000","last_undegraded":"2024-09-10T20:33:34.079826+0000","last_fullsized":"2024-09-10T20:33:34.079826+0000","mapping_epoch":44,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41754648,"num_objects":10195,"num_object_clones":0,"num_object_copies":20390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10195,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8649,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10464","reported_seq":10544,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:33.544968+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:33:33.544968+0000","last_peered":"2024-09-10T20:33:33.544968+0000","last_clean":"2024-09-10T20:33:33.544968+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:33:33.544968+0000","last_undegraded":"2024-09-10T20:33:33.544968+0000","last_fullsized":"2024-09-10T20:33:33.544968+0000","mapping_epoch":35,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41533440,"num_objects":10140,"num_object_clones":0,"num_object_copies":20280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10464,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10673","reported_seq":10800,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:32.646865+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:33:32.646865+0000","last_peered":"2024-09-10T20:33:32.646865+0000","last_clean":"2024-09-10T20:33:32.646865+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:33:32.646865+0000","last_undegraded":"2024-09-10T20:33:32.646865+0000","last_fullsized":"2024-09-10T20:33:32.646865+0000","mapping_epoch":34,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42582016,"num_objects":10396,"num_object_clones":0,"num_object_copies":20792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7669,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10360","reported_seq":10503,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:32.721379+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:33:32.721379+0000","last_peered":"2024-09-10T20:33:32.721379+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:33:32.721379+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41414656,"num_objects":10111,"num_object_clones":0,"num_object_copies":20222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3613,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7224,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6500},{"shards":"0,2","objects":3492}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10453","reported_seq":10603,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:32.746344+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:33:32.746344+0000","last_peered":"2024-09-10T20:33:32.746344+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:33:32.746344+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41594880,"num_objects":10155,"num_object_clones":0,"num_object_copies":20310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3551,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7048,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6606},{"shards":"0,2","objects":3409}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10473","reported_seq":10598,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:32.781928+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:33:32.781928+0000","last_peered":"2024-09-10T20:33:32.781928+0000","last_clean":"2024-09-10T20:33:32.781928+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:33:32.781928+0000","last_undegraded":"2024-09-10T20:33:32.781928+0000","last_fullsized":"2024-09-10T20:33:32.781928+0000","mapping_epoch":34,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42037248,"num_objects":10263,"num_object_clones":0,"num_object_copies":20526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7544,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10382","reported_seq":10515,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:33.589925+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:33:33.589925+0000","last_peered":"2024-09-10T20:33:33.589925+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:33:33.589925+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41193472,"num_objects":10057,"num_object_clones":0,"num_object_copies":20114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8375,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3366},{"shards":"2","objects":6532}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190273+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190273+0000","last_peered":"2024-09-10T20:32:18.190273+0000","last_clean":"2024-09-10T20:32:18.190273+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190273+0000","last_undegraded":"2024-09-10T20:32:18.190273+0000","last_fullsized":"2024-09-10T20:32:18.190273+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10493","reported_seq":10606,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:32.684868+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:33:32.684868+0000","last_peered":"2024-09-10T20:33:32.684868+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:33:32.684868+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41931718,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3615,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10491,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6626},{"shards":"0,2","objects":3491}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10434","reported_seq":10564,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:33.611550+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:33:33.611550+0000","last_peered":"2024-09-10T20:33:33.611550+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:33:33.611550+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41496576,"num_objects":10131,"num_object_clones":0,"num_object_copies":20262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3634,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8262,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3484},{"shards":"2","objects":6499}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10494","reported_seq":10623,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:33.433470+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:33:33.433470+0000","last_peered":"2024-09-10T20:33:33.433470+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:33:33.433470+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41656320,"num_objects":10170,"num_object_clones":0,"num_object_copies":20340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8280,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3451},{"shards":"2","objects":6560}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10559","reported_seq":10698,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:32.753811+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:33:32.753811+0000","last_peered":"2024-09-10T20:33:32.753811+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:33:32.753811+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42070016,"num_objects":10271,"num_object_clones":0,"num_object_copies":20542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3647,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7282,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6625},{"shards":"0,2","objects":3505}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10428","reported_seq":10570,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:32.805970+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:33:32.805970+0000","last_peered":"2024-09-10T20:33:32.805970+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:33:32.805970+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41598976,"num_objects":10156,"num_object_clones":0,"num_object_copies":20312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3533,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7122,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6626},{"shards":"0,2","objects":3400}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193230+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193230+0000","last_peered":"2024-09-10T20:32:18.193230+0000","last_clean":"2024-09-10T20:32:18.193230+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193230+0000","last_undegraded":"2024-09-10T20:32:18.193230+0000","last_fullsized":"2024-09-10T20:32:18.193230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10626","reported_seq":10766,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:32.768444+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:33:32.768444+0000","last_peered":"2024-09-10T20:33:32.768444+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:33:32.768444+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42434560,"num_objects":10360,"num_object_clones":0,"num_object_copies":20720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3711,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7453,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6651},{"shards":"0,2","objects":3582}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10381","reported_seq":13762,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:33:32.804775+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:33:32.804775+0000","last_peered":"2024-09-10T20:33:32.804775+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:33:32.804775+0000","last_undegraded":"2024-09-10T20:33:32.804775+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41513443,"num_objects":10136,"num_object_clones":0,"num_object_copies":20272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1861,"num_objects_unfound":0,"num_objects_dirty":10136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6988,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1607,"num_bytes_recovered":6582272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8275},{"shards":"0,2","objects":1861}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":668784065,"num_objects":163281,"num_object_clones":0,"num_object_copies":326562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36046,"num_objects_misplaced":1861,"num_objects_unfound":0,"num_objects_dirty":163281,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":132136,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5239,"num_bytes_recovered":21344256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1364729856,"data_stored":1364635000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160940,"ondisk_log_size":160940,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182460,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":888380,"kb_used_data":549920,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93483460,"statfs":{"total":96636764160,"available":95727063040,"internally_reserved":0,"allocated":563118080,"data_stored":562875392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,1,0,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986232,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":704568,"kb_used_data":442408,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93667272,"statfs":{"total":96636764160,"available":95915286528,"internally_reserved":0,"allocated":453025792,"data_stored":452783109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268431018},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,2,3,0,0,0,0,0,0,2],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659179,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":599792,"kb_used_data":343200,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93772048,"statfs":{"total":96636764160,"available":96022577152,"internally_reserved":0,"allocated":351436800,"data_stored":351114167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":262729390},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,5,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":561684480,"data_stored":561680867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":351002624,"data_stored":350919066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":452042752,"data_stored":452035067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:33:36.609 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:33:37.176 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:37.177 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:33:37.205 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":203,"stamp":"2024-09-10T20:33:36.913290+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668784084,"num_objects":163283,"num_object_clones":0,"num_object_copies":326566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36046,"num_objects_misplaced":1861,"num_objects_unfound":0,"num_objects_dirty":163283,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":132138,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5241,"num_bytes_recovered":21344275,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160943,"ondisk_log_size":160943,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2192740,"kb_used_data":1335528,"kb_used_omap":12,"kb_used_meta":857139,"kb_avail":280922780,"statfs":{"total":289910292480,"available":287664926720,"internally_reserved":0,"allocated":1367580672,"data_stored":1366772668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":877710338},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,5,9,9,0,5,0,0,0,0,2],"upper_bound":8192},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-159720,"num_objects":-38,"num_object_clones":0,"num_object_copies":-76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-25,"num_objects_misplaced":-257,"num_objects_unfound":0,"num_objects_dirty":-38,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4442,"num_write_kb":8805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":1052672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004518"},"pg_stats":[{"pgid":"2.f","version":"45'10458","reported_seq":10547,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:32.719634+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:33:32.719634+0000","last_peered":"2024-09-10T20:33:32.719634+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:33:32.719634+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41779200,"num_objects":10200,"num_object_clones":0,"num_object_copies":20400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3603,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10458,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6597},{"shards":"0,2","objects":3478}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10641","reported_seq":10751,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:33.524088+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:33:33.524088+0000","last_peered":"2024-09-10T20:33:33.524088+0000","last_clean":"2024-09-10T20:33:33.524088+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:33:33.524088+0000","last_undegraded":"2024-09-10T20:33:33.524088+0000","last_fullsized":"2024-09-10T20:33:33.524088+0000","mapping_epoch":35,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42192896,"num_objects":10301,"num_object_clones":0,"num_object_copies":20602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10301,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8827,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10521","reported_seq":17869,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:34.079826+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:33:34.079826+0000","last_peered":"2024-09-10T20:33:34.079826+0000","last_clean":"2024-09-10T20:33:34.079826+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:33:34.079826+0000","last_undegraded":"2024-09-10T20:33:34.079826+0000","last_fullsized":"2024-09-10T20:33:34.079826+0000","mapping_epoch":44,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41754648,"num_objects":10195,"num_object_clones":0,"num_object_copies":20390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10195,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8649,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10464","reported_seq":10544,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:33.544968+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:33:33.544968+0000","last_peered":"2024-09-10T20:33:33.544968+0000","last_clean":"2024-09-10T20:33:33.544968+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:33:33.544968+0000","last_undegraded":"2024-09-10T20:33:33.544968+0000","last_fullsized":"2024-09-10T20:33:33.544968+0000","mapping_epoch":35,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41533440,"num_objects":10140,"num_object_clones":0,"num_object_copies":20280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10464,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10673","reported_seq":10800,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:32.646865+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:33:32.646865+0000","last_peered":"2024-09-10T20:33:32.646865+0000","last_clean":"2024-09-10T20:33:32.646865+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:33:32.646865+0000","last_undegraded":"2024-09-10T20:33:32.646865+0000","last_fullsized":"2024-09-10T20:33:32.646865+0000","mapping_epoch":34,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42582016,"num_objects":10396,"num_object_clones":0,"num_object_copies":20792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7669,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10360","reported_seq":10503,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:32.721379+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:33:32.721379+0000","last_peered":"2024-09-10T20:33:32.721379+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:33:32.721379+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41414656,"num_objects":10111,"num_object_clones":0,"num_object_copies":20222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3613,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7224,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6500},{"shards":"0,2","objects":3492}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10453","reported_seq":10603,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:32.746344+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:33:32.746344+0000","last_peered":"2024-09-10T20:33:32.746344+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:33:32.746344+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41594880,"num_objects":10155,"num_object_clones":0,"num_object_copies":20310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3551,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7048,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6606},{"shards":"0,2","objects":3409}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10473","reported_seq":10598,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:32.781928+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:33:32.781928+0000","last_peered":"2024-09-10T20:33:32.781928+0000","last_clean":"2024-09-10T20:33:32.781928+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:33:32.781928+0000","last_undegraded":"2024-09-10T20:33:32.781928+0000","last_fullsized":"2024-09-10T20:33:32.781928+0000","mapping_epoch":34,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42037248,"num_objects":10263,"num_object_clones":0,"num_object_copies":20526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7544,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10382","reported_seq":10515,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:33.589925+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:33:33.589925+0000","last_peered":"2024-09-10T20:33:33.589925+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:33:33.589925+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41193472,"num_objects":10057,"num_object_clones":0,"num_object_copies":20114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8375,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3366},{"shards":"2","objects":6532}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190273+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190273+0000","last_peered":"2024-09-10T20:32:18.190273+0000","last_clean":"2024-09-10T20:32:18.190273+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190273+0000","last_undegraded":"2024-09-10T20:32:18.190273+0000","last_fullsized":"2024-09-10T20:32:18.190273+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10493","reported_seq":10606,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:32.684868+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:33:32.684868+0000","last_peered":"2024-09-10T20:33:32.684868+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:33:32.684868+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41931718,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3615,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10491,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6626},{"shards":"0,2","objects":3491}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10434","reported_seq":10564,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:33.611550+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:33:33.611550+0000","last_peered":"2024-09-10T20:33:33.611550+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:33:33.611550+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41496576,"num_objects":10131,"num_object_clones":0,"num_object_copies":20262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3634,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8262,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3484},{"shards":"2","objects":6499}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10494","reported_seq":10623,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:33.433470+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:33:33.433470+0000","last_peered":"2024-09-10T20:33:33.433470+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:33:33.433470+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41656320,"num_objects":10170,"num_object_clones":0,"num_object_copies":20340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8280,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3451},{"shards":"2","objects":6560}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10559","reported_seq":10698,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:32.753811+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:33:32.753811+0000","last_peered":"2024-09-10T20:33:32.753811+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:33:32.753811+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42070016,"num_objects":10271,"num_object_clones":0,"num_object_copies":20542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3647,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7282,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6625},{"shards":"0,2","objects":3505}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10428","reported_seq":10570,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:32.805970+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:33:32.805970+0000","last_peered":"2024-09-10T20:33:32.805970+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:33:32.805970+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41598976,"num_objects":10156,"num_object_clones":0,"num_object_copies":20312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3533,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7122,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6626},{"shards":"0,2","objects":3400}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193230+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193230+0000","last_peered":"2024-09-10T20:32:18.193230+0000","last_clean":"2024-09-10T20:32:18.193230+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193230+0000","last_undegraded":"2024-09-10T20:32:18.193230+0000","last_fullsized":"2024-09-10T20:32:18.193230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10626","reported_seq":10766,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:32.768444+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:33:32.768444+0000","last_peered":"2024-09-10T20:33:32.768444+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:33:32.768444+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42434560,"num_objects":10360,"num_object_clones":0,"num_object_copies":20720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3711,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7453,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6651},{"shards":"0,2","objects":3582}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10381","reported_seq":13762,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:33:32.804775+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:33:32.804775+0000","last_peered":"2024-09-10T20:33:32.804775+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:33:32.804775+0000","last_undegraded":"2024-09-10T20:33:32.804775+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41513443,"num_objects":10136,"num_object_clones":0,"num_object_copies":20272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1861,"num_objects_unfound":0,"num_objects_dirty":10136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6988,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1607,"num_bytes_recovered":6582272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8275},{"shards":"0,2","objects":1861}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":668784065,"num_objects":163281,"num_object_clones":0,"num_object_copies":326562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36046,"num_objects_misplaced":1861,"num_objects_unfound":0,"num_objects_dirty":163281,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":132136,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5239,"num_bytes_recovered":21344256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1364729856,"data_stored":1364635000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160940,"ondisk_log_size":160940,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182460,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":888380,"kb_used_data":549920,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93483460,"statfs":{"total":96636764160,"available":95727063040,"internally_reserved":0,"allocated":563118080,"data_stored":562875392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,1,0,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986232,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":704568,"kb_used_data":442408,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93667272,"statfs":{"total":96636764160,"available":95915286528,"internally_reserved":0,"allocated":453025792,"data_stored":452783109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268431018},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,2,3,0,0,0,0,0,0,2],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659179,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":599792,"kb_used_data":343200,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93772048,"statfs":{"total":96636764160,"available":96022577152,"internally_reserved":0,"allocated":351436800,"data_stored":351114167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":262729390},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,5,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":561684480,"data_stored":561680867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":351002624,"data_stored":350919066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":452042752,"data_stored":452035067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:33:37.207 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:33:37.207 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'10458", 'reported_seq': 10547, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:32.719634+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:33:32.719634+0000', 'last_peered': '2024-09-10T20:33:32.719634+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:33:32.719634+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41779200, 'num_objects': 10200, 'num_object_clones': 0, 'num_object_copies': 20400, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3603, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10200, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10458, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 5, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6597}, {'shards': '0,2', 'objects': 3478}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:37.207 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:33:37.207 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'10360", 'reported_seq': 10503, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:32.721379+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:33:32.721379+0000', 'last_peered': '2024-09-10T20:33:32.721379+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:33:32.721379+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41414656, 'num_objects': 10111, 'num_object_clones': 0, 'num_object_copies': 20222, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3613, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10111, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7224, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6500}, {'shards': '0,2', 'objects': 3492}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:37.207 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:33:37.207 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'10453", 'reported_seq': 10603, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:32.746344+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:33:32.746344+0000', 'last_peered': '2024-09-10T20:33:32.746344+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:33:32.746344+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41594880, 'num_objects': 10155, 'num_object_clones': 0, 'num_object_copies': 20310, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3551, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10155, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7048, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 6, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6606}, {'shards': '0,2', 'objects': 3409}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:37.207 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:33:37.207 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'10382", 'reported_seq': 10515, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:33.589925+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:33:33.589925+0000', 'last_peered': '2024-09-10T20:33:33.589925+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:33:33.589925+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41193472, 'num_objects': 10057, 'num_object_clones': 0, 'num_object_copies': 20114, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10057, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8375, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3366}, {'shards': '2', 'objects': 6532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:33:37.207 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:33:37.207 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'10493", 'reported_seq': 10606, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:32.684868+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:33:32.684868+0000', 'last_peered': '2024-09-10T20:33:32.684868+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:33:32.684868+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41931718, 'num_objects': 10239, 'num_object_clones': 0, 'num_object_copies': 20478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3615, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10239, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10491, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6626}, {'shards': '0,2', 'objects': 3491}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:37.208 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:33:37.208 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'10434", 'reported_seq': 10564, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:33.611550+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:33:33.611550+0000', 'last_peered': '2024-09-10T20:33:33.611550+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:33:33.611550+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41496576, 'num_objects': 10131, 'num_object_clones': 0, 'num_object_copies': 20262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3634, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8262, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3484}, {'shards': '2', 'objects': 6499}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:33:37.208 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:33:37.208 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'10494", 'reported_seq': 10623, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:33.433470+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:33:33.433470+0000', 'last_peered': '2024-09-10T20:33:33.433470+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:33:33.433470+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41656320, 'num_objects': 10170, 'num_object_clones': 0, 'num_object_copies': 20340, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10170, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8280, 'num_write_kb': 32484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3451}, {'shards': '2', 'objects': 6560}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:33:37.208 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:33:37.208 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'10559", 'reported_seq': 10698, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:32.753811+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:33:32.753811+0000', 'last_peered': '2024-09-10T20:33:32.753811+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:33:32.753811+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42070016, 'num_objects': 10271, 'num_object_clones': 0, 'num_object_copies': 20542, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3647, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10271, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7282, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6625}, {'shards': '0,2', 'objects': 3505}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:37.208 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:33:37.208 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'10428", 'reported_seq': 10570, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:32.805970+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:33:32.805970+0000', 'last_peered': '2024-09-10T20:33:32.805970+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:33:32.805970+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41598976, 'num_objects': 10156, 'num_object_clones': 0, 'num_object_copies': 20312, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3533, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10156, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7122, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6626}, {'shards': '0,2', 'objects': 3400}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:37.208 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:33:37.209 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'10626", 'reported_seq': 10766, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:32.768444+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:33:32.768444+0000', 'last_peered': '2024-09-10T20:33:32.768444+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:33:32.768444+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "36'600", 'ondisk_log_start': "36'600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42434560, 'num_objects': 10360, 'num_object_clones': 0, 'num_object_copies': 20720, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3711, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10360, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7453, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6651}, {'shards': '0,2', 'objects': 3582}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:37.209 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:33:37.209 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'10381", 'reported_seq': 13762, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:33:32.804775+0000', 'last_change': '2024-09-10T20:32:16.657652+0000', 'last_active': '2024-09-10T20:33:32.804775+0000', 'last_peered': '2024-09-10T20:33:32.804775+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:33:32.804775+0000', 'last_undegraded': '2024-09-10T20:33:32.804775+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41513443, 'num_objects': 10136, 'num_object_clones': 0, 'num_object_copies': 20272, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1861, 'num_objects_unfound': 0, 'num_objects_dirty': 10136, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6988, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1607, 'num_bytes_recovered': 6582272, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 8275}, {'shards': '0,2', 'objects': 1861}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:37.209 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:33:37.904 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:37.904 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36046/326566 objects degraded (11.038%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":329,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":163283,"data_bytes":668784084,"bytes_used":2245365760,"bytes_avail":287664926720,"bytes_total":289910292480,"degraded_objects":36046,"degraded_total":326566,"degraded_ratio":0.11037891268533773,"misplaced_objects":1861,"misplaced_total":326566,"misplaced_ratio":0.0056986949039397857,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":87689,"recovering_keys_per_sec":0,"num_objects_recovered":282,"num_bytes_recovered":1052672,"num_keys_recovered":0,"read_bytes_sec":85,"write_bytes_sec":751077,"read_op_per_sec":0,"write_op_per_sec":370},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:33:37.929 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:33:37.929 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:33:38.498 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:38.499 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:33:38.520 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":203,"stamp":"2024-09-10T20:33:36.913290+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668784084,"num_objects":163283,"num_object_clones":0,"num_object_copies":326566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36046,"num_objects_misplaced":1861,"num_objects_unfound":0,"num_objects_dirty":163283,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":132138,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5241,"num_bytes_recovered":21344275,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160943,"ondisk_log_size":160943,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2192740,"kb_used_data":1335528,"kb_used_omap":12,"kb_used_meta":857139,"kb_avail":280922780,"statfs":{"total":289910292480,"available":287664926720,"internally_reserved":0,"allocated":1367580672,"data_stored":1366772668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":877710338},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,5,9,9,0,5,0,0,0,0,2],"upper_bound":8192},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-159720,"num_objects":-38,"num_object_clones":0,"num_object_copies":-76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-25,"num_objects_misplaced":-257,"num_objects_unfound":0,"num_objects_dirty":-38,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4442,"num_write_kb":8805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":1052672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004518"},"pg_stats":[{"pgid":"2.f","version":"45'10458","reported_seq":10547,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:32.719634+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:33:32.719634+0000","last_peered":"2024-09-10T20:33:32.719634+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:33:32.719634+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41779200,"num_objects":10200,"num_object_clones":0,"num_object_copies":20400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3603,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10458,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6597},{"shards":"0,2","objects":3478}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10641","reported_seq":10751,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:33.524088+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:33:33.524088+0000","last_peered":"2024-09-10T20:33:33.524088+0000","last_clean":"2024-09-10T20:33:33.524088+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:33:33.524088+0000","last_undegraded":"2024-09-10T20:33:33.524088+0000","last_fullsized":"2024-09-10T20:33:33.524088+0000","mapping_epoch":35,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42192896,"num_objects":10301,"num_object_clones":0,"num_object_copies":20602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10301,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8827,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10521","reported_seq":17869,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:34.079826+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:33:34.079826+0000","last_peered":"2024-09-10T20:33:34.079826+0000","last_clean":"2024-09-10T20:33:34.079826+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:33:34.079826+0000","last_undegraded":"2024-09-10T20:33:34.079826+0000","last_fullsized":"2024-09-10T20:33:34.079826+0000","mapping_epoch":44,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41754648,"num_objects":10195,"num_object_clones":0,"num_object_copies":20390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10195,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8649,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10464","reported_seq":10544,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:33.544968+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:33:33.544968+0000","last_peered":"2024-09-10T20:33:33.544968+0000","last_clean":"2024-09-10T20:33:33.544968+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:33:33.544968+0000","last_undegraded":"2024-09-10T20:33:33.544968+0000","last_fullsized":"2024-09-10T20:33:33.544968+0000","mapping_epoch":35,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41533440,"num_objects":10140,"num_object_clones":0,"num_object_copies":20280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10464,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10673","reported_seq":10800,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:32.646865+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:33:32.646865+0000","last_peered":"2024-09-10T20:33:32.646865+0000","last_clean":"2024-09-10T20:33:32.646865+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:33:32.646865+0000","last_undegraded":"2024-09-10T20:33:32.646865+0000","last_fullsized":"2024-09-10T20:33:32.646865+0000","mapping_epoch":34,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42582016,"num_objects":10396,"num_object_clones":0,"num_object_copies":20792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7669,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10360","reported_seq":10503,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:32.721379+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:33:32.721379+0000","last_peered":"2024-09-10T20:33:32.721379+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:33:32.721379+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41414656,"num_objects":10111,"num_object_clones":0,"num_object_copies":20222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3613,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7224,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6500},{"shards":"0,2","objects":3492}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10453","reported_seq":10603,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:32.746344+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:33:32.746344+0000","last_peered":"2024-09-10T20:33:32.746344+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:33:32.746344+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41594880,"num_objects":10155,"num_object_clones":0,"num_object_copies":20310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3551,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7048,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6606},{"shards":"0,2","objects":3409}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10473","reported_seq":10598,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:32.781928+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:33:32.781928+0000","last_peered":"2024-09-10T20:33:32.781928+0000","last_clean":"2024-09-10T20:33:32.781928+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:33:32.781928+0000","last_undegraded":"2024-09-10T20:33:32.781928+0000","last_fullsized":"2024-09-10T20:33:32.781928+0000","mapping_epoch":34,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42037248,"num_objects":10263,"num_object_clones":0,"num_object_copies":20526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7544,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10382","reported_seq":10515,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:33.589925+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:33:33.589925+0000","last_peered":"2024-09-10T20:33:33.589925+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:33:33.589925+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41193472,"num_objects":10057,"num_object_clones":0,"num_object_copies":20114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8375,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3366},{"shards":"2","objects":6532}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190273+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190273+0000","last_peered":"2024-09-10T20:32:18.190273+0000","last_clean":"2024-09-10T20:32:18.190273+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190273+0000","last_undegraded":"2024-09-10T20:32:18.190273+0000","last_fullsized":"2024-09-10T20:32:18.190273+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10493","reported_seq":10606,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:32.684868+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:33:32.684868+0000","last_peered":"2024-09-10T20:33:32.684868+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:33:32.684868+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41931718,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3615,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10491,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6626},{"shards":"0,2","objects":3491}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10434","reported_seq":10564,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:33.611550+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:33:33.611550+0000","last_peered":"2024-09-10T20:33:33.611550+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:33:33.611550+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41496576,"num_objects":10131,"num_object_clones":0,"num_object_copies":20262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3634,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8262,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3484},{"shards":"2","objects":6499}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10494","reported_seq":10623,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:33.433470+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:33:33.433470+0000","last_peered":"2024-09-10T20:33:33.433470+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:33:33.433470+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41656320,"num_objects":10170,"num_object_clones":0,"num_object_copies":20340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8280,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3451},{"shards":"2","objects":6560}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10559","reported_seq":10698,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:32.753811+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:33:32.753811+0000","last_peered":"2024-09-10T20:33:32.753811+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:33:32.753811+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42070016,"num_objects":10271,"num_object_clones":0,"num_object_copies":20542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3647,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7282,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6625},{"shards":"0,2","objects":3505}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10428","reported_seq":10570,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:32.805970+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:33:32.805970+0000","last_peered":"2024-09-10T20:33:32.805970+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:33:32.805970+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41598976,"num_objects":10156,"num_object_clones":0,"num_object_copies":20312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3533,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7122,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6626},{"shards":"0,2","objects":3400}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193230+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193230+0000","last_peered":"2024-09-10T20:32:18.193230+0000","last_clean":"2024-09-10T20:32:18.193230+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193230+0000","last_undegraded":"2024-09-10T20:32:18.193230+0000","last_fullsized":"2024-09-10T20:32:18.193230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10626","reported_seq":10766,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:32.768444+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:33:32.768444+0000","last_peered":"2024-09-10T20:33:32.768444+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:33:32.768444+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42434560,"num_objects":10360,"num_object_clones":0,"num_object_copies":20720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3711,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7453,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6651},{"shards":"0,2","objects":3582}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10381","reported_seq":13762,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:33:32.804775+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:33:32.804775+0000","last_peered":"2024-09-10T20:33:32.804775+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:33:32.804775+0000","last_undegraded":"2024-09-10T20:33:32.804775+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41513443,"num_objects":10136,"num_object_clones":0,"num_object_copies":20272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1861,"num_objects_unfound":0,"num_objects_dirty":10136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6988,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1607,"num_bytes_recovered":6582272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8275},{"shards":"0,2","objects":1861}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":668784065,"num_objects":163281,"num_object_clones":0,"num_object_copies":326562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36046,"num_objects_misplaced":1861,"num_objects_unfound":0,"num_objects_dirty":163281,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":132136,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5239,"num_bytes_recovered":21344256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1364729856,"data_stored":1364635000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160940,"ondisk_log_size":160940,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182460,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":888380,"kb_used_data":549920,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93483460,"statfs":{"total":96636764160,"available":95727063040,"internally_reserved":0,"allocated":563118080,"data_stored":562875392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,1,0,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986232,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":704568,"kb_used_data":442408,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93667272,"statfs":{"total":96636764160,"available":95915286528,"internally_reserved":0,"allocated":453025792,"data_stored":452783109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268431018},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,2,3,0,0,0,0,0,0,2],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659179,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":599792,"kb_used_data":343200,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93772048,"statfs":{"total":96636764160,"available":96022577152,"internally_reserved":0,"allocated":351436800,"data_stored":351114167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":262729390},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,5,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":561684480,"data_stored":561680867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":351002624,"data_stored":350919066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":452042752,"data_stored":452035067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:33:41.523 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:33:41.956 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:41.956 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:33:41.973 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":205,"stamp":"2024-09-10T20:33:40.914732+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659596756,"num_objects":161040,"num_object_clones":0,"num_object_copies":322080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36031,"num_objects_misplaced":1704,"num_objects_unfound":0,"num_objects_dirty":161040,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":134381,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5413,"num_bytes_recovered":21987347,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160686,"ondisk_log_size":160686,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2199004,"kb_used_data":1323748,"kb_used_omap":12,"kb_used_meta":875187,"kb_avail":280916516,"statfs":{"total":289910292480,"available":287658512384,"internally_reserved":0,"allocated":1355517952,"data_stored":1354709948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":896191490},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,3,2,7,11,1,0,6,0,0,0,0,2],"upper_bound":16384},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17186792,"num_objects":-4195,"num_object_clones":0,"num_object_copies":-8390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-40,"num_objects_misplaced":-306,"num_objects_unfound":0,"num_objects_dirty":-4195,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4593,"num_write_kb":793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":346,"num_bytes_recovered":1253376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004417"},"pg_stats":[{"pgid":"2.f","version":"45'10602","reported_seq":10693,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:37.801198+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:33:37.801198+0000","last_peered":"2024-09-10T20:33:37.801198+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:33:37.801198+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41189376,"num_objects":10056,"num_object_clones":0,"num_object_copies":20112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10056,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10602,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6454},{"shards":"0,2","objects":3334}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10783","reported_seq":10893,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:38.427614+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:33:38.427614+0000","last_peered":"2024-09-10T20:33:38.427614+0000","last_clean":"2024-09-10T20:33:38.427614+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:33:38.427614+0000","last_undegraded":"2024-09-10T20:33:38.427614+0000","last_fullsized":"2024-09-10T20:33:38.427614+0000","mapping_epoch":35,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41611264,"num_objects":10159,"num_object_clones":0,"num_object_copies":20318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8969,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10673","reported_seq":18021,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:39.116249+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:33:39.116249+0000","last_peered":"2024-09-10T20:33:39.116249+0000","last_clean":"2024-09-10T20:33:39.116249+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:33:39.116249+0000","last_undegraded":"2024-09-10T20:33:39.116249+0000","last_fullsized":"2024-09-10T20:33:39.116249+0000","mapping_epoch":44,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41132056,"num_objects":10043,"num_object_clones":0,"num_object_copies":20086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10043,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8801,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10603","reported_seq":10683,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:38.543083+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:33:38.543083+0000","last_peered":"2024-09-10T20:33:38.543083+0000","last_clean":"2024-09-10T20:33:38.543083+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:33:38.543083+0000","last_undegraded":"2024-09-10T20:33:38.543083+0000","last_fullsized":"2024-09-10T20:33:38.543083+0000","mapping_epoch":35,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40964096,"num_objects":10001,"num_object_clones":0,"num_object_copies":20002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10603,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10807","reported_seq":10934,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:37.800895+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:33:37.800895+0000","last_peered":"2024-09-10T20:33:37.800895+0000","last_clean":"2024-09-10T20:33:37.800895+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:33:37.800895+0000","last_undegraded":"2024-09-10T20:33:37.800895+0000","last_fullsized":"2024-09-10T20:33:37.800895+0000","mapping_epoch":34,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42033152,"num_objects":10262,"num_object_clones":0,"num_object_copies":20524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7803,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10489","reported_seq":10634,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:37.799613+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:33:37.799613+0000","last_peered":"2024-09-10T20:33:37.799613+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:33:37.799613+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40886272,"num_objects":9982,"num_object_clones":0,"num_object_copies":19964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7353,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6372},{"shards":"0,2","objects":3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10594","reported_seq":10754,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:37.813749+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:33:37.813749+0000","last_peered":"2024-09-10T20:33:37.813749+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:33:37.813749+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41017344,"num_objects":10014,"num_object_clones":0,"num_object_copies":20028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7189,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6470},{"shards":"0,2","objects":3268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10618","reported_seq":10743,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:37.673412+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:33:37.673412+0000","last_peered":"2024-09-10T20:33:37.673412+0000","last_clean":"2024-09-10T20:33:37.673412+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:33:37.673412+0000","last_undegraded":"2024-09-10T20:33:37.673412+0000","last_fullsized":"2024-09-10T20:33:37.673412+0000","mapping_epoch":34,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41443328,"num_objects":10118,"num_object_clones":0,"num_object_copies":20236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7689,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10518","reported_seq":10651,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:38.584999+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:33:38.584999+0000","last_peered":"2024-09-10T20:33:38.584999+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:33:38.584999+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40636416,"num_objects":9921,"num_object_clones":0,"num_object_copies":19842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8511,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3230},{"shards":"2","objects":6396}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190273+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190273+0000","last_peered":"2024-09-10T20:32:18.190273+0000","last_clean":"2024-09-10T20:32:18.190273+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190273+0000","last_undegraded":"2024-09-10T20:32:18.190273+0000","last_fullsized":"2024-09-10T20:32:18.190273+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10626","reported_seq":10743,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:37.727389+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:33:37.727389+0000","last_peered":"2024-09-10T20:33:37.727389+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:33:37.727389+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41386950,"num_objects":10106,"num_object_clones":0,"num_object_copies":20212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3613,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10624,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6495},{"shards":"0,2","objects":3358}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10561","reported_seq":10692,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:38.606202+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:33:38.606202+0000","last_peered":"2024-09-10T20:33:38.606202+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:33:38.606202+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40976384,"num_objects":10004,"num_object_clones":0,"num_object_copies":20008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3634,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8389,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3357},{"shards":"2","objects":6372}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10641","reported_seq":10770,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:38.621593+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:33:38.621593+0000","last_peered":"2024-09-10T20:33:38.621593+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:33:38.621593+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41054208,"num_objects":10023,"num_object_clones":0,"num_object_copies":20046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10023,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8427,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3304},{"shards":"2","objects":6413}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10710","reported_seq":10855,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:37.820941+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:33:37.820941+0000","last_peered":"2024-09-10T20:33:37.820941+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:33:37.820941+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41451520,"num_objects":10120,"num_object_clones":0,"num_object_copies":20240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3644,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7433,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6477},{"shards":"0,2","objects":3354}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10576","reported_seq":10721,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:37.768533+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:33:37.768533+0000","last_peered":"2024-09-10T20:33:37.768533+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:33:37.768533+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40992768,"num_objects":10008,"num_object_clones":0,"num_object_copies":20016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3532,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7270,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6479},{"shards":"0,2","objects":3252}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193230+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193230+0000","last_peered":"2024-09-10T20:32:18.193230+0000","last_clean":"2024-09-10T20:32:18.193230+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193230+0000","last_undegraded":"2024-09-10T20:32:18.193230+0000","last_fullsized":"2024-09-10T20:32:18.193230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10781","reported_seq":10925,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:37.727413+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:33:37.727413+0000","last_peered":"2024-09-10T20:33:37.727413+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:33:37.727413+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41799680,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3709,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7608,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6498},{"shards":"0,2","objects":3427}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10501","reported_seq":14196,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:33:37.826872+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:33:37.826872+0000","last_peered":"2024-09-10T20:33:37.826872+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:33:37.826872+0000","last_undegraded":"2024-09-10T20:33:37.826872+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41021923,"num_objects":10016,"num_object_clones":0,"num_object_copies":20032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1704,"num_objects_unfound":0,"num_objects_dirty":10016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7108,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1764,"num_bytes_recovered":7225344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8312},{"shards":"0,2","objects":1704}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":659596737,"num_objects":161038,"num_object_clones":0,"num_object_copies":322076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36031,"num_objects_misplaced":1704,"num_objects_unfound":0,"num_objects_dirty":161038,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":134379,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5411,"num_bytes_recovered":21987328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1352863744,"data_stored":1352768888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160683,"ondisk_log_size":160683,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182461,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":885700,"kb_used_data":543088,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93486140,"statfs":{"total":96636764160,"available":95729807360,"internally_reserved":0,"allocated":556122112,"data_stored":555879424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,4,3,1,0,6],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986233,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":704232,"kb_used_data":437908,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93667608,"statfs":{"total":96636764160,"available":95915630592,"internally_reserved":0,"allocated":448417792,"data_stored":448175109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272690858},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,0,2,2,0,0,0,0,0,0,0,2],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659180,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609072,"kb_used_data":342752,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93762768,"statfs":{"total":96636764160,"available":96013074432,"internally_reserved":0,"allocated":350978048,"data_stored":350655415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":272690862},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":554786816,"data_stored":554783203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":350543872,"data_stored":350460314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":447533056,"data_stored":447525371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:33:41.974 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:33:42.549 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:42.553 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:33:42.573 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":205,"stamp":"2024-09-10T20:33:40.914732+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659596756,"num_objects":161040,"num_object_clones":0,"num_object_copies":322080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36031,"num_objects_misplaced":1704,"num_objects_unfound":0,"num_objects_dirty":161040,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":134381,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5413,"num_bytes_recovered":21987347,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160686,"ondisk_log_size":160686,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2199004,"kb_used_data":1323748,"kb_used_omap":12,"kb_used_meta":875187,"kb_avail":280916516,"statfs":{"total":289910292480,"available":287658512384,"internally_reserved":0,"allocated":1355517952,"data_stored":1354709948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":896191490},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,3,2,7,11,1,0,6,0,0,0,0,2],"upper_bound":16384},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17186792,"num_objects":-4195,"num_object_clones":0,"num_object_copies":-8390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-40,"num_objects_misplaced":-306,"num_objects_unfound":0,"num_objects_dirty":-4195,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4593,"num_write_kb":793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":346,"num_bytes_recovered":1253376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004417"},"pg_stats":[{"pgid":"2.f","version":"45'10602","reported_seq":10693,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:37.801198+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:33:37.801198+0000","last_peered":"2024-09-10T20:33:37.801198+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:33:37.801198+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41189376,"num_objects":10056,"num_object_clones":0,"num_object_copies":20112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10056,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10602,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6454},{"shards":"0,2","objects":3334}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10783","reported_seq":10893,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:38.427614+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:33:38.427614+0000","last_peered":"2024-09-10T20:33:38.427614+0000","last_clean":"2024-09-10T20:33:38.427614+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:33:38.427614+0000","last_undegraded":"2024-09-10T20:33:38.427614+0000","last_fullsized":"2024-09-10T20:33:38.427614+0000","mapping_epoch":35,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41611264,"num_objects":10159,"num_object_clones":0,"num_object_copies":20318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8969,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10673","reported_seq":18021,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:39.116249+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:33:39.116249+0000","last_peered":"2024-09-10T20:33:39.116249+0000","last_clean":"2024-09-10T20:33:39.116249+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:33:39.116249+0000","last_undegraded":"2024-09-10T20:33:39.116249+0000","last_fullsized":"2024-09-10T20:33:39.116249+0000","mapping_epoch":44,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41132056,"num_objects":10043,"num_object_clones":0,"num_object_copies":20086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10043,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8801,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10603","reported_seq":10683,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:38.543083+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:33:38.543083+0000","last_peered":"2024-09-10T20:33:38.543083+0000","last_clean":"2024-09-10T20:33:38.543083+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:33:38.543083+0000","last_undegraded":"2024-09-10T20:33:38.543083+0000","last_fullsized":"2024-09-10T20:33:38.543083+0000","mapping_epoch":35,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40964096,"num_objects":10001,"num_object_clones":0,"num_object_copies":20002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10603,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10807","reported_seq":10934,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:37.800895+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:33:37.800895+0000","last_peered":"2024-09-10T20:33:37.800895+0000","last_clean":"2024-09-10T20:33:37.800895+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:33:37.800895+0000","last_undegraded":"2024-09-10T20:33:37.800895+0000","last_fullsized":"2024-09-10T20:33:37.800895+0000","mapping_epoch":34,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42033152,"num_objects":10262,"num_object_clones":0,"num_object_copies":20524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7803,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10489","reported_seq":10634,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:37.799613+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:33:37.799613+0000","last_peered":"2024-09-10T20:33:37.799613+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:33:37.799613+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40886272,"num_objects":9982,"num_object_clones":0,"num_object_copies":19964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7353,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6372},{"shards":"0,2","objects":3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10594","reported_seq":10754,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:37.813749+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:33:37.813749+0000","last_peered":"2024-09-10T20:33:37.813749+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:33:37.813749+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41017344,"num_objects":10014,"num_object_clones":0,"num_object_copies":20028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7189,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6470},{"shards":"0,2","objects":3268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10618","reported_seq":10743,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:37.673412+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:33:37.673412+0000","last_peered":"2024-09-10T20:33:37.673412+0000","last_clean":"2024-09-10T20:33:37.673412+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:33:37.673412+0000","last_undegraded":"2024-09-10T20:33:37.673412+0000","last_fullsized":"2024-09-10T20:33:37.673412+0000","mapping_epoch":34,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41443328,"num_objects":10118,"num_object_clones":0,"num_object_copies":20236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7689,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10518","reported_seq":10651,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:38.584999+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:33:38.584999+0000","last_peered":"2024-09-10T20:33:38.584999+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:33:38.584999+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40636416,"num_objects":9921,"num_object_clones":0,"num_object_copies":19842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8511,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3230},{"shards":"2","objects":6396}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190273+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190273+0000","last_peered":"2024-09-10T20:32:18.190273+0000","last_clean":"2024-09-10T20:32:18.190273+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190273+0000","last_undegraded":"2024-09-10T20:32:18.190273+0000","last_fullsized":"2024-09-10T20:32:18.190273+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10626","reported_seq":10743,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:37.727389+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:33:37.727389+0000","last_peered":"2024-09-10T20:33:37.727389+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:33:37.727389+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41386950,"num_objects":10106,"num_object_clones":0,"num_object_copies":20212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3613,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10624,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6495},{"shards":"0,2","objects":3358}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10561","reported_seq":10692,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:38.606202+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:33:38.606202+0000","last_peered":"2024-09-10T20:33:38.606202+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:33:38.606202+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40976384,"num_objects":10004,"num_object_clones":0,"num_object_copies":20008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3634,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8389,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3357},{"shards":"2","objects":6372}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10641","reported_seq":10770,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:38.621593+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:33:38.621593+0000","last_peered":"2024-09-10T20:33:38.621593+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:33:38.621593+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41054208,"num_objects":10023,"num_object_clones":0,"num_object_copies":20046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10023,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8427,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3304},{"shards":"2","objects":6413}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10710","reported_seq":10855,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:37.820941+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:33:37.820941+0000","last_peered":"2024-09-10T20:33:37.820941+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:33:37.820941+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41451520,"num_objects":10120,"num_object_clones":0,"num_object_copies":20240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3644,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7433,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6477},{"shards":"0,2","objects":3354}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10576","reported_seq":10721,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:37.768533+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:33:37.768533+0000","last_peered":"2024-09-10T20:33:37.768533+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:33:37.768533+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40992768,"num_objects":10008,"num_object_clones":0,"num_object_copies":20016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3532,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7270,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6479},{"shards":"0,2","objects":3252}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193230+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193230+0000","last_peered":"2024-09-10T20:32:18.193230+0000","last_clean":"2024-09-10T20:32:18.193230+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193230+0000","last_undegraded":"2024-09-10T20:32:18.193230+0000","last_fullsized":"2024-09-10T20:32:18.193230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10781","reported_seq":10925,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:37.727413+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:33:37.727413+0000","last_peered":"2024-09-10T20:33:37.727413+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:33:37.727413+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41799680,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3709,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7608,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6498},{"shards":"0,2","objects":3427}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10501","reported_seq":14196,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:33:37.826872+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:33:37.826872+0000","last_peered":"2024-09-10T20:33:37.826872+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:33:37.826872+0000","last_undegraded":"2024-09-10T20:33:37.826872+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41021923,"num_objects":10016,"num_object_clones":0,"num_object_copies":20032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1704,"num_objects_unfound":0,"num_objects_dirty":10016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7108,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1764,"num_bytes_recovered":7225344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8312},{"shards":"0,2","objects":1704}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":659596737,"num_objects":161038,"num_object_clones":0,"num_object_copies":322076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36031,"num_objects_misplaced":1704,"num_objects_unfound":0,"num_objects_dirty":161038,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":134379,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5411,"num_bytes_recovered":21987328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1352863744,"data_stored":1352768888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160683,"ondisk_log_size":160683,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182461,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":885700,"kb_used_data":543088,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93486140,"statfs":{"total":96636764160,"available":95729807360,"internally_reserved":0,"allocated":556122112,"data_stored":555879424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,4,3,1,0,6],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986233,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":704232,"kb_used_data":437908,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93667608,"statfs":{"total":96636764160,"available":95915630592,"internally_reserved":0,"allocated":448417792,"data_stored":448175109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272690858},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,0,2,2,0,0,0,0,0,0,0,2],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659180,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609072,"kb_used_data":342752,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93762768,"statfs":{"total":96636764160,"available":96013074432,"internally_reserved":0,"allocated":350978048,"data_stored":350655415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":272690862},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":554786816,"data_stored":554783203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":350543872,"data_stored":350460314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":447533056,"data_stored":447525371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:33:42.574 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:33:42.575 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'10602", 'reported_seq': 10693, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:37.801198+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:33:37.801198+0000', 'last_peered': '2024-09-10T20:33:37.801198+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:33:37.801198+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "36'600", 'ondisk_log_start': "36'600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41189376, 'num_objects': 10056, 'num_object_clones': 0, 'num_object_copies': 20112, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3602, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10056, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10602, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 6, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6454}, {'shards': '0,2', 'objects': 3334}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:42.575 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:33:42.575 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'10489", 'reported_seq': 10634, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:37.799613+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:33:37.799613+0000', 'last_peered': '2024-09-10T20:33:37.799613+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:33:37.799613+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40886272, 'num_objects': 9982, 'num_object_clones': 0, 'num_object_copies': 19964, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9982, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7353, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6372}, {'shards': '0,2', 'objects': 3363}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:42.575 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:33:42.575 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'10594", 'reported_seq': 10754, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:37.813749+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:33:37.813749+0000', 'last_peered': '2024-09-10T20:33:37.813749+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:33:37.813749+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41017344, 'num_objects': 10014, 'num_object_clones': 0, 'num_object_copies': 20028, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10014, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7189, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 11, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6470}, {'shards': '0,2', 'objects': 3268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:42.575 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:33:42.575 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'10518", 'reported_seq': 10651, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:38.584999+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:33:38.584999+0000', 'last_peered': '2024-09-10T20:33:38.584999+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:33:38.584999+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40636416, 'num_objects': 9921, 'num_object_clones': 0, 'num_object_copies': 19842, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9921, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8511, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3230}, {'shards': '2', 'objects': 6396}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:33:42.575 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:33:42.575 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'10626", 'reported_seq': 10743, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:37.727389+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:33:37.727389+0000', 'last_peered': '2024-09-10T20:33:37.727389+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:33:37.727389+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "36'600", 'ondisk_log_start': "36'600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41386950, 'num_objects': 10106, 'num_object_clones': 0, 'num_object_copies': 20212, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3613, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10106, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10624, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 6, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6495}, {'shards': '0,2', 'objects': 3358}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:42.576 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:33:42.576 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'10561", 'reported_seq': 10692, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:38.606202+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:33:38.606202+0000', 'last_peered': '2024-09-10T20:33:38.606202+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:33:38.606202+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40976384, 'num_objects': 10004, 'num_object_clones': 0, 'num_object_copies': 20008, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3634, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10004, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8389, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3357}, {'shards': '2', 'objects': 6372}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:33:42.576 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:33:42.576 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'10641", 'reported_seq': 10770, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:38.621593+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:33:38.621593+0000', 'last_peered': '2024-09-10T20:33:38.621593+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:33:38.621593+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "36'600", 'ondisk_log_start': "36'600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41054208, 'num_objects': 10023, 'num_object_clones': 0, 'num_object_copies': 20046, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10023, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8427, 'num_write_kb': 32484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3304}, {'shards': '2', 'objects': 6413}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:33:42.576 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:33:42.576 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'10710", 'reported_seq': 10855, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:37.820941+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:33:37.820941+0000', 'last_peered': '2024-09-10T20:33:37.820941+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:33:37.820941+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41451520, 'num_objects': 10120, 'num_object_clones': 0, 'num_object_copies': 20240, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3644, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10120, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7433, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 5, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6477}, {'shards': '0,2', 'objects': 3354}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:42.576 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:33:42.576 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'10576", 'reported_seq': 10721, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:37.768533+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:33:37.768533+0000', 'last_peered': '2024-09-10T20:33:37.768533+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:33:37.768533+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40992768, 'num_objects': 10008, 'num_object_clones': 0, 'num_object_copies': 20016, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3532, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10008, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7270, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6479}, {'shards': '0,2', 'objects': 3252}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:42.576 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:33:42.576 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'10781", 'reported_seq': 10925, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:37.727413+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:33:37.727413+0000', 'last_peered': '2024-09-10T20:33:37.727413+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:33:37.727413+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41799680, 'num_objects': 10205, 'num_object_clones': 0, 'num_object_copies': 20410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3709, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10205, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7608, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 6, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6498}, {'shards': '0,2', 'objects': 3427}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:42.576 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:33:42.576 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'10501", 'reported_seq': 14196, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:33:37.826872+0000', 'last_change': '2024-09-10T20:32:16.657652+0000', 'last_active': '2024-09-10T20:33:37.826872+0000', 'last_peered': '2024-09-10T20:33:37.826872+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:33:37.826872+0000', 'last_undegraded': '2024-09-10T20:33:37.826872+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41021923, 'num_objects': 10016, 'num_object_clones': 0, 'num_object_copies': 20032, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1704, 'num_objects_unfound': 0, 'num_objects_dirty': 10016, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7108, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1764, 'num_bytes_recovered': 7225344, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 8312}, {'shards': '0,2', 'objects': 1704}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:42.577 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:33:43.259 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:43.260 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36031/322080 objects degraded (11.187%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":334,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":161040,"data_bytes":659596756,"bytes_used":2251780096,"bytes_avail":287658512384,"bytes_total":289910292480,"degraded_objects":36031,"degraded_total":322080,"degraded_ratio":0.11186972180824639,"misplaced_objects":1704,"misplaced_total":322080,"misplaced_ratio":0.0052906110283159466,"recovering_objects_per_sec":28,"recovering_bytes_per_sec":104409,"recovering_keys_per_sec":0,"num_objects_recovered":346,"num_bytes_recovered":1253376,"num_keys_recovered":0,"read_bytes_sec":85,"write_bytes_sec":67644,"read_op_per_sec":0,"write_op_per_sec":382},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:33:43.283 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:33:43.284 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:33:43.843 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:43.847 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:33:43.875 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":206,"stamp":"2024-09-10T20:33:42.915633+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654144980,"num_objects":159709,"num_object_clones":0,"num_object_copies":319418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36002,"num_objects_misplaced":1530,"num_objects_unfound":0,"num_objects_dirty":159709,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":135712,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5616,"num_bytes_recovered":22700051,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160617,"ondisk_log_size":160617,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2201032,"kb_used_data":1317460,"kb_used_omap":12,"kb_used_meta":883507,"kb_avail":280914488,"statfs":{"total":289910292480,"available":287656435712,"internally_reserved":0,"allocated":1349079040,"data_stored":1348271036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":904711170},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,7,3,8,11,7,0,0,0,0,0,0,2],"upper_bound":16384},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22949888,"num_objects":-5603,"num_object_clones":0,"num_object_copies":-11206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-69,"num_objects_misplaced":-480,"num_objects_unfound":0,"num_objects_dirty":-5603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5777,"num_write_kb":348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":549,"num_bytes_recovered":1966080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004358"},"pg_stats":[{"pgid":"2.f","version":"45'10726","reported_seq":10821,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:42.709985+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:33:42.709985+0000","last_peered":"2024-09-10T20:33:42.709985+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:33:42.709985+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40681472,"num_objects":9932,"num_object_clones":0,"num_object_copies":19864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3600,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9932,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10726,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6332},{"shards":"0,2","objects":3210}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10783","reported_seq":10893,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:38.427614+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:33:38.427614+0000","last_peered":"2024-09-10T20:33:38.427614+0000","last_clean":"2024-09-10T20:33:38.427614+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:33:38.427614+0000","last_undegraded":"2024-09-10T20:33:38.427614+0000","last_fullsized":"2024-09-10T20:33:38.427614+0000","mapping_epoch":35,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41611264,"num_objects":10159,"num_object_clones":0,"num_object_copies":20318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8969,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10673","reported_seq":18021,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:39.116249+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:33:39.116249+0000","last_peered":"2024-09-10T20:33:39.116249+0000","last_clean":"2024-09-10T20:33:39.116249+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:33:39.116249+0000","last_undegraded":"2024-09-10T20:33:39.116249+0000","last_fullsized":"2024-09-10T20:33:39.116249+0000","mapping_epoch":44,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41132056,"num_objects":10043,"num_object_clones":0,"num_object_copies":20086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10043,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8801,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10603","reported_seq":10683,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:38.543083+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:33:38.543083+0000","last_peered":"2024-09-10T20:33:38.543083+0000","last_clean":"2024-09-10T20:33:38.543083+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:33:38.543083+0000","last_undegraded":"2024-09-10T20:33:38.543083+0000","last_fullsized":"2024-09-10T20:33:38.543083+0000","mapping_epoch":35,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40964096,"num_objects":10001,"num_object_clones":0,"num_object_copies":20002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10603,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10946","reported_seq":11073,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:42.794807+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:33:42.794807+0000","last_peered":"2024-09-10T20:33:42.794807+0000","last_clean":"2024-09-10T20:33:42.794807+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:33:42.794807+0000","last_undegraded":"2024-09-10T20:33:42.794807+0000","last_fullsized":"2024-09-10T20:33:42.794807+0000","mapping_epoch":34,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41463808,"num_objects":10123,"num_object_clones":0,"num_object_copies":20246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7942,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10620","reported_seq":10765,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:42.816471+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:33:42.816471+0000","last_peered":"2024-09-10T20:33:42.816471+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:33:42.816471+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40349696,"num_objects":9851,"num_object_clones":0,"num_object_copies":19702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3611,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7484,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6242},{"shards":"0,2","objects":3232}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10725","reported_seq":10898,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:42.827835+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:33:42.827835+0000","last_peered":"2024-09-10T20:33:42.827835+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:33:42.827835+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40480768,"num_objects":9883,"num_object_clones":0,"num_object_copies":19766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7320,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6347},{"shards":"0,2","objects":3137}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10759","reported_seq":10884,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:42.812633+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:33:42.812633+0000","last_peered":"2024-09-10T20:33:42.812633+0000","last_clean":"2024-09-10T20:33:42.812633+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:33:42.812633+0000","last_undegraded":"2024-09-10T20:33:42.812633+0000","last_fullsized":"2024-09-10T20:33:42.812633+0000","mapping_epoch":34,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40865792,"num_objects":9977,"num_object_clones":0,"num_object_copies":19954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7830,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10518","reported_seq":10651,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:38.584999+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:33:38.584999+0000","last_peered":"2024-09-10T20:33:38.584999+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:33:38.584999+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40636416,"num_objects":9921,"num_object_clones":0,"num_object_copies":19842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8511,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3230},{"shards":"2","objects":6396}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190273+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190273+0000","last_peered":"2024-09-10T20:32:18.190273+0000","last_clean":"2024-09-10T20:32:18.190273+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190273+0000","last_undegraded":"2024-09-10T20:32:18.190273+0000","last_fullsized":"2024-09-10T20:32:18.190273+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10777","reported_seq":10904,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:42.800810+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:33:42.800810+0000","last_peered":"2024-09-10T20:33:42.800810+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:33:42.800810+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40768454,"num_objects":9955,"num_object_clones":0,"num_object_copies":19910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10775,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6349},{"shards":"0,2","objects":3207}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10561","reported_seq":10692,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:38.606202+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:33:38.606202+0000","last_peered":"2024-09-10T20:33:38.606202+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:33:38.606202+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40976384,"num_objects":10004,"num_object_clones":0,"num_object_copies":20008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3634,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8389,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3357},{"shards":"2","objects":6372}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10641","reported_seq":10770,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:38.621593+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:33:38.621593+0000","last_peered":"2024-09-10T20:33:38.621593+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:33:38.621593+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41054208,"num_objects":10023,"num_object_clones":0,"num_object_copies":20046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10023,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8427,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3304},{"shards":"2","objects":6413}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10851","reported_seq":11006,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:42.819531+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:33:42.819531+0000","last_peered":"2024-09-10T20:33:42.819531+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:33:42.819531+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40873984,"num_objects":9979,"num_object_clones":0,"num_object_copies":19958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3639,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7574,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6341},{"shards":"0,2","objects":3213}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10705","reported_seq":10864,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:42.846664+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:33:42.846664+0000","last_peered":"2024-09-10T20:33:42.846664+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:33:42.846664+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40464384,"num_objects":9879,"num_object_clones":0,"num_object_copies":19758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3525,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9879,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7399,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6357},{"shards":"0,2","objects":3123}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193230+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193230+0000","last_peered":"2024-09-10T20:32:18.193230+0000","last_clean":"2024-09-10T20:32:18.193230+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193230+0000","last_undegraded":"2024-09-10T20:32:18.193230+0000","last_fullsized":"2024-09-10T20:32:18.193230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10907","reported_seq":11053,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:42.738760+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:33:42.738760+0000","last_peered":"2024-09-10T20:33:42.738760+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:33:42.738760+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41283584,"num_objects":10079,"num_object_clones":0,"num_object_copies":20158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3708,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7734,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6373},{"shards":"0,2","objects":3301}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10619","reported_seq":14661,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:33:42.839223+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:33:42.839223+0000","last_peered":"2024-09-10T20:33:42.839223+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:33:42.839223+0000","last_undegraded":"2024-09-10T20:33:42.839223+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40538595,"num_objects":9898,"num_object_clones":0,"num_object_copies":19796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1530,"num_objects_unfound":0,"num_objects_dirty":9898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7226,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1938,"num_bytes_recovered":7938048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8368},{"shards":"0,2","objects":1530}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":654144961,"num_objects":159707,"num_object_clones":0,"num_object_copies":319414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36002,"num_objects_misplaced":1530,"num_objects_unfound":0,"num_objects_dirty":159707,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":135710,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5614,"num_bytes_recovered":22700032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1346306048,"data_stored":1346211192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160614,"ondisk_log_size":160614,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182462,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":887728,"kb_used_data":536800,"kb_used_omap":4,"kb_used_meta":350907,"kb_avail":93484112,"statfs":{"total":96636764160,"available":95727730688,"internally_reserved":0,"allocated":549683200,"data_stored":549440512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":359329450},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,4,1,5,3,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986233,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":704232,"kb_used_data":437908,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93667608,"statfs":{"total":96636764160,"available":95915630592,"internally_reserved":0,"allocated":448417792,"data_stored":448175109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272690858},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,0,2,2,0,0,0,0,0,0,0,2],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659180,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609072,"kb_used_data":342752,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93762768,"statfs":{"total":96636764160,"available":96013074432,"internally_reserved":0,"allocated":350978048,"data_stored":350655415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":272690862},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":548229120,"data_stored":548225507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":350543872,"data_stored":350460314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":447533056,"data_stored":447525371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:33:46.877 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:33:47.465 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:47.465 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:33:47.500 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":208,"stamp":"2024-09-10T20:33:46.917063+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650782164,"num_objects":158888,"num_object_clones":0,"num_object_copies":317776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35998,"num_objects_misplaced":1530,"num_objects_unfound":0,"num_objects_dirty":158888,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":136533,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5620,"num_bytes_recovered":22700051,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160638,"ondisk_log_size":160638,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2204584,"kb_used_data":1312692,"kb_used_omap":12,"kb_used_meta":891827,"kb_avail":280910936,"statfs":{"total":289910292480,"available":287652798464,"internally_reserved":0,"allocated":1344196608,"data_stored":1343388604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":913230850},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,10,1,10,5,7,0,0,0,0,0,0,2],"upper_bound":16384},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18001920,"num_objects":-4395,"num_object_clones":0,"num_object_copies":-8790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-48,"num_objects_misplaced":-331,"num_objects_unfound":0,"num_objects_dirty":-4395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4395,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":379,"num_bytes_recovered":1355776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004314"},"pg_stats":[{"pgid":"2.f","version":"45'10726","reported_seq":10821,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:42.709985+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:33:42.709985+0000","last_peered":"2024-09-10T20:33:42.709985+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:33:42.709985+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40681472,"num_objects":9932,"num_object_clones":0,"num_object_copies":19864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3600,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9932,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10726,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6332},{"shards":"0,2","objects":3210}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10914","reported_seq":11023,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:43.584851+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:33:43.584851+0000","last_peered":"2024-09-10T20:33:43.584851+0000","last_clean":"2024-09-10T20:33:43.584851+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:33:43.584851+0000","last_undegraded":"2024-09-10T20:33:43.584851+0000","last_fullsized":"2024-09-10T20:33:43.584851+0000","mapping_epoch":35,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41074688,"num_objects":10028,"num_object_clones":0,"num_object_copies":20056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9100,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10814","reported_seq":18161,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:44.143866+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:33:44.143866+0000","last_peered":"2024-09-10T20:33:44.143866+0000","last_clean":"2024-09-10T20:33:44.143866+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:33:44.143866+0000","last_undegraded":"2024-09-10T20:33:44.143866+0000","last_fullsized":"2024-09-10T20:33:44.143866+0000","mapping_epoch":44,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40554520,"num_objects":9902,"num_object_clones":0,"num_object_copies":19804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9902,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8942,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10745","reported_seq":10826,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:43.584150+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:33:43.584150+0000","last_peered":"2024-09-10T20:33:43.584150+0000","last_clean":"2024-09-10T20:33:43.584150+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:33:43.584150+0000","last_undegraded":"2024-09-10T20:33:43.584150+0000","last_fullsized":"2024-09-10T20:33:43.584150+0000","mapping_epoch":35,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40382464,"num_objects":9859,"num_object_clones":0,"num_object_copies":19718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10745,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10946","reported_seq":11073,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:42.794807+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:33:42.794807+0000","last_peered":"2024-09-10T20:33:42.794807+0000","last_clean":"2024-09-10T20:33:42.794807+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:33:42.794807+0000","last_undegraded":"2024-09-10T20:33:42.794807+0000","last_fullsized":"2024-09-10T20:33:42.794807+0000","mapping_epoch":34,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41463808,"num_objects":10123,"num_object_clones":0,"num_object_copies":20246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7942,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10620","reported_seq":10765,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:42.816471+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:33:42.816471+0000","last_peered":"2024-09-10T20:33:42.816471+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:33:42.816471+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40349696,"num_objects":9851,"num_object_clones":0,"num_object_copies":19702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3611,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7484,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6242},{"shards":"0,2","objects":3232}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10725","reported_seq":10898,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:42.827835+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:33:42.827835+0000","last_peered":"2024-09-10T20:33:42.827835+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:33:42.827835+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40480768,"num_objects":9883,"num_object_clones":0,"num_object_copies":19766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7320,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6347},{"shards":"0,2","objects":3137}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10759","reported_seq":10884,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:42.812633+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:33:42.812633+0000","last_peered":"2024-09-10T20:33:42.812633+0000","last_clean":"2024-09-10T20:33:42.812633+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:33:42.812633+0000","last_undegraded":"2024-09-10T20:33:42.812633+0000","last_fullsized":"2024-09-10T20:33:42.812633+0000","mapping_epoch":34,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40865792,"num_objects":9977,"num_object_clones":0,"num_object_copies":19954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7830,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10667","reported_seq":10803,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:43.583031+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:33:43.583031+0000","last_peered":"2024-09-10T20:33:43.583031+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:33:43.583031+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40026112,"num_objects":9772,"num_object_clones":0,"num_object_copies":19544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8660,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3081},{"shards":"2","objects":6248}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190273+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190273+0000","last_peered":"2024-09-10T20:32:18.190273+0000","last_clean":"2024-09-10T20:32:18.190273+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190273+0000","last_undegraded":"2024-09-10T20:32:18.190273+0000","last_fullsized":"2024-09-10T20:32:18.190273+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10777","reported_seq":10904,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:42.800810+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:33:42.800810+0000","last_peered":"2024-09-10T20:33:42.800810+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:33:42.800810+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40768454,"num_objects":9955,"num_object_clones":0,"num_object_copies":19910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10775,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6349},{"shards":"0,2","objects":3207}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10689","reported_seq":10821,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:43.619910+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:33:43.619910+0000","last_peered":"2024-09-10T20:33:43.619910+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:33:43.619910+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40452096,"num_objects":9876,"num_object_clones":0,"num_object_copies":19752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3633,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8517,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3229},{"shards":"2","objects":6245}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10771","reported_seq":10904,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:43.554084+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:33:43.554084+0000","last_peered":"2024-09-10T20:33:43.554084+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:33:43.554084+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40521728,"num_objects":9893,"num_object_clones":0,"num_object_copies":19786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3610,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8557,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3174},{"shards":"2","objects":6285}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10851","reported_seq":11006,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:42.819531+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:33:42.819531+0000","last_peered":"2024-09-10T20:33:42.819531+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:33:42.819531+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40873984,"num_objects":9979,"num_object_clones":0,"num_object_copies":19958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3639,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7574,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6341},{"shards":"0,2","objects":3213}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10705","reported_seq":10864,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:42.846664+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:33:42.846664+0000","last_peered":"2024-09-10T20:33:42.846664+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:33:42.846664+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40464384,"num_objects":9879,"num_object_clones":0,"num_object_copies":19758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3525,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9879,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7399,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6357},{"shards":"0,2","objects":3123}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193230+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193230+0000","last_peered":"2024-09-10T20:32:18.193230+0000","last_clean":"2024-09-10T20:32:18.193230+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193230+0000","last_undegraded":"2024-09-10T20:32:18.193230+0000","last_fullsized":"2024-09-10T20:32:18.193230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10907","reported_seq":11053,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:42.738760+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:33:42.738760+0000","last_peered":"2024-09-10T20:33:42.738760+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:33:42.738760+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41283584,"num_objects":10079,"num_object_clones":0,"num_object_copies":20158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3708,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7734,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6373},{"shards":"0,2","objects":3301}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10619","reported_seq":14661,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:33:42.839223+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:33:42.839223+0000","last_peered":"2024-09-10T20:33:42.839223+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:33:42.839223+0000","last_undegraded":"2024-09-10T20:33:42.839223+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40538595,"num_objects":9898,"num_object_clones":0,"num_object_copies":19796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1530,"num_objects_unfound":0,"num_objects_dirty":9898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7226,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1938,"num_bytes_recovered":7938048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8368},{"shards":"0,2","objects":1530}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":650782145,"num_objects":158886,"num_object_clones":0,"num_object_copies":317772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35998,"num_objects_misplaced":1530,"num_objects_unfound":0,"num_objects_dirty":158886,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":136531,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5618,"num_bytes_recovered":22700032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1341341696,"data_stored":1341246840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160635,"ondisk_log_size":160635,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182462,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":887728,"kb_used_data":536800,"kb_used_omap":4,"kb_used_meta":350907,"kb_avail":93484112,"statfs":{"total":96636764160,"available":95727730688,"internally_reserved":0,"allocated":549683200,"data_stored":549440512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":359329450},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,4,1,5,3,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986234,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703988,"kb_used_data":433504,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93667852,"statfs":{"total":96636764160,"available":95915880448,"internally_reserved":0,"allocated":443908096,"data_stored":443665413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276950698},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,0,2,2,0,0,0,0,0,0,0,2],"upper_bound":16384},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659181,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612868,"kb_used_data":342388,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93758972,"statfs":{"total":96636764160,"available":96009187328,"internally_reserved":0,"allocated":350605312,"data_stored":350282679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":276950702},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":548229120,"data_stored":548225507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":350101504,"data_stored":350017946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":443011072,"data_stored":443003387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:33:47.502 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:33:48.057 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:48.059 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:33:48.088 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":208,"stamp":"2024-09-10T20:33:46.917063+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650782164,"num_objects":158888,"num_object_clones":0,"num_object_copies":317776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35998,"num_objects_misplaced":1530,"num_objects_unfound":0,"num_objects_dirty":158888,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":136533,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5620,"num_bytes_recovered":22700051,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160638,"ondisk_log_size":160638,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2204584,"kb_used_data":1312692,"kb_used_omap":12,"kb_used_meta":891827,"kb_avail":280910936,"statfs":{"total":289910292480,"available":287652798464,"internally_reserved":0,"allocated":1344196608,"data_stored":1343388604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":913230850},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,10,1,10,5,7,0,0,0,0,0,0,2],"upper_bound":16384},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18001920,"num_objects":-4395,"num_object_clones":0,"num_object_copies":-8790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-48,"num_objects_misplaced":-331,"num_objects_unfound":0,"num_objects_dirty":-4395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4395,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":379,"num_bytes_recovered":1355776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004314"},"pg_stats":[{"pgid":"2.f","version":"45'10726","reported_seq":10821,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:42.709985+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:33:42.709985+0000","last_peered":"2024-09-10T20:33:42.709985+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:33:42.709985+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40681472,"num_objects":9932,"num_object_clones":0,"num_object_copies":19864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3600,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9932,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10726,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6332},{"shards":"0,2","objects":3210}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10914","reported_seq":11023,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:43.584851+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:33:43.584851+0000","last_peered":"2024-09-10T20:33:43.584851+0000","last_clean":"2024-09-10T20:33:43.584851+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:33:43.584851+0000","last_undegraded":"2024-09-10T20:33:43.584851+0000","last_fullsized":"2024-09-10T20:33:43.584851+0000","mapping_epoch":35,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41074688,"num_objects":10028,"num_object_clones":0,"num_object_copies":20056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9100,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10814","reported_seq":18161,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:44.143866+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:33:44.143866+0000","last_peered":"2024-09-10T20:33:44.143866+0000","last_clean":"2024-09-10T20:33:44.143866+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:33:44.143866+0000","last_undegraded":"2024-09-10T20:33:44.143866+0000","last_fullsized":"2024-09-10T20:33:44.143866+0000","mapping_epoch":44,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40554520,"num_objects":9902,"num_object_clones":0,"num_object_copies":19804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9902,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8942,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10745","reported_seq":10826,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:43.584150+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:33:43.584150+0000","last_peered":"2024-09-10T20:33:43.584150+0000","last_clean":"2024-09-10T20:33:43.584150+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:33:43.584150+0000","last_undegraded":"2024-09-10T20:33:43.584150+0000","last_fullsized":"2024-09-10T20:33:43.584150+0000","mapping_epoch":35,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40382464,"num_objects":9859,"num_object_clones":0,"num_object_copies":19718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10745,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10946","reported_seq":11073,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:42.794807+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:33:42.794807+0000","last_peered":"2024-09-10T20:33:42.794807+0000","last_clean":"2024-09-10T20:33:42.794807+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:33:42.794807+0000","last_undegraded":"2024-09-10T20:33:42.794807+0000","last_fullsized":"2024-09-10T20:33:42.794807+0000","mapping_epoch":34,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41463808,"num_objects":10123,"num_object_clones":0,"num_object_copies":20246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7942,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10620","reported_seq":10765,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:42.816471+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:33:42.816471+0000","last_peered":"2024-09-10T20:33:42.816471+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:33:42.816471+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40349696,"num_objects":9851,"num_object_clones":0,"num_object_copies":19702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3611,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7484,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6242},{"shards":"0,2","objects":3232}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10725","reported_seq":10898,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:42.827835+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:33:42.827835+0000","last_peered":"2024-09-10T20:33:42.827835+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:33:42.827835+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40480768,"num_objects":9883,"num_object_clones":0,"num_object_copies":19766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7320,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6347},{"shards":"0,2","objects":3137}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10759","reported_seq":10884,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:42.812633+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:33:42.812633+0000","last_peered":"2024-09-10T20:33:42.812633+0000","last_clean":"2024-09-10T20:33:42.812633+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:33:42.812633+0000","last_undegraded":"2024-09-10T20:33:42.812633+0000","last_fullsized":"2024-09-10T20:33:42.812633+0000","mapping_epoch":34,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40865792,"num_objects":9977,"num_object_clones":0,"num_object_copies":19954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7830,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10667","reported_seq":10803,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:43.583031+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:33:43.583031+0000","last_peered":"2024-09-10T20:33:43.583031+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:33:43.583031+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40026112,"num_objects":9772,"num_object_clones":0,"num_object_copies":19544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8660,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3081},{"shards":"2","objects":6248}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190273+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190273+0000","last_peered":"2024-09-10T20:32:18.190273+0000","last_clean":"2024-09-10T20:32:18.190273+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190273+0000","last_undegraded":"2024-09-10T20:32:18.190273+0000","last_fullsized":"2024-09-10T20:32:18.190273+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10777","reported_seq":10904,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:42.800810+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:33:42.800810+0000","last_peered":"2024-09-10T20:33:42.800810+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:33:42.800810+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40768454,"num_objects":9955,"num_object_clones":0,"num_object_copies":19910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10775,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6349},{"shards":"0,2","objects":3207}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10689","reported_seq":10821,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:43.619910+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:33:43.619910+0000","last_peered":"2024-09-10T20:33:43.619910+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:33:43.619910+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40452096,"num_objects":9876,"num_object_clones":0,"num_object_copies":19752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3633,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8517,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3229},{"shards":"2","objects":6245}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10771","reported_seq":10904,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:43.554084+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:33:43.554084+0000","last_peered":"2024-09-10T20:33:43.554084+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:33:43.554084+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40521728,"num_objects":9893,"num_object_clones":0,"num_object_copies":19786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3610,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8557,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3174},{"shards":"2","objects":6285}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10851","reported_seq":11006,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:42.819531+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:33:42.819531+0000","last_peered":"2024-09-10T20:33:42.819531+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:33:42.819531+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40873984,"num_objects":9979,"num_object_clones":0,"num_object_copies":19958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3639,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7574,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6341},{"shards":"0,2","objects":3213}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10705","reported_seq":10864,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:42.846664+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:33:42.846664+0000","last_peered":"2024-09-10T20:33:42.846664+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:33:42.846664+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40464384,"num_objects":9879,"num_object_clones":0,"num_object_copies":19758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3525,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9879,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7399,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6357},{"shards":"0,2","objects":3123}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193230+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193230+0000","last_peered":"2024-09-10T20:32:18.193230+0000","last_clean":"2024-09-10T20:32:18.193230+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193230+0000","last_undegraded":"2024-09-10T20:32:18.193230+0000","last_fullsized":"2024-09-10T20:32:18.193230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10907","reported_seq":11053,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:42.738760+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:33:42.738760+0000","last_peered":"2024-09-10T20:33:42.738760+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:33:42.738760+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41283584,"num_objects":10079,"num_object_clones":0,"num_object_copies":20158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3708,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7734,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6373},{"shards":"0,2","objects":3301}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10619","reported_seq":14661,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:33:42.839223+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:33:42.839223+0000","last_peered":"2024-09-10T20:33:42.839223+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:33:42.839223+0000","last_undegraded":"2024-09-10T20:33:42.839223+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40538595,"num_objects":9898,"num_object_clones":0,"num_object_copies":19796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1530,"num_objects_unfound":0,"num_objects_dirty":9898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7226,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1938,"num_bytes_recovered":7938048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8368},{"shards":"0,2","objects":1530}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":650782145,"num_objects":158886,"num_object_clones":0,"num_object_copies":317772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35998,"num_objects_misplaced":1530,"num_objects_unfound":0,"num_objects_dirty":158886,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":136531,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5618,"num_bytes_recovered":22700032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1341341696,"data_stored":1341246840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160635,"ondisk_log_size":160635,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182462,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":887728,"kb_used_data":536800,"kb_used_omap":4,"kb_used_meta":350907,"kb_avail":93484112,"statfs":{"total":96636764160,"available":95727730688,"internally_reserved":0,"allocated":549683200,"data_stored":549440512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":359329450},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,4,1,5,3,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986234,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703988,"kb_used_data":433504,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93667852,"statfs":{"total":96636764160,"available":95915880448,"internally_reserved":0,"allocated":443908096,"data_stored":443665413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276950698},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,0,2,2,0,0,0,0,0,0,0,2],"upper_bound":16384},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659181,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612868,"kb_used_data":342388,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93758972,"statfs":{"total":96636764160,"available":96009187328,"internally_reserved":0,"allocated":350605312,"data_stored":350282679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":276950702},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":548229120,"data_stored":548225507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":350101504,"data_stored":350017946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":443011072,"data_stored":443003387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:33:48.089 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:33:48.089 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'10726", 'reported_seq': 10821, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:42.709985+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:33:42.709985+0000', 'last_peered': '2024-09-10T20:33:42.709985+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:33:42.709985+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40681472, 'num_objects': 9932, 'num_object_clones': 0, 'num_object_copies': 19864, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3600, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9932, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10726, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 8, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6332}, {'shards': '0,2', 'objects': 3210}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:48.089 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:33:48.090 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'10620", 'reported_seq': 10765, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:42.816471+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:33:42.816471+0000', 'last_peered': '2024-09-10T20:33:42.816471+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:33:42.816471+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "36'600", 'ondisk_log_start': "36'600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40349696, 'num_objects': 9851, 'num_object_clones': 0, 'num_object_copies': 19702, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3611, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9851, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7484, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 5, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6242}, {'shards': '0,2', 'objects': 3232}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:48.090 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:33:48.090 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'10725", 'reported_seq': 10898, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:42.827835+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:33:42.827835+0000', 'last_peered': '2024-09-10T20:33:42.827835+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:33:42.827835+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40480768, 'num_objects': 9883, 'num_object_clones': 0, 'num_object_copies': 19766, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9883, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7320, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 19, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6347}, {'shards': '0,2', 'objects': 3137}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:48.090 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:33:48.090 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'10667", 'reported_seq': 10803, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:43.583031+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:33:43.583031+0000', 'last_peered': '2024-09-10T20:33:43.583031+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:33:43.583031+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "36'600", 'ondisk_log_start': "36'600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40026112, 'num_objects': 9772, 'num_object_clones': 0, 'num_object_copies': 19544, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9772, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8660, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3081}, {'shards': '2', 'objects': 6248}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:33:48.090 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:33:48.090 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'10777", 'reported_seq': 10904, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:42.800810+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:33:42.800810+0000', 'last_peered': '2024-09-10T20:33:42.800810+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:33:42.800810+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40768454, 'num_objects': 9955, 'num_object_clones': 0, 'num_object_copies': 19910, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9955, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10775, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 11, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6349}, {'shards': '0,2', 'objects': 3207}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:48.090 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:33:48.092 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'10689", 'reported_seq': 10821, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:43.619910+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:33:43.619910+0000', 'last_peered': '2024-09-10T20:33:43.619910+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:33:43.619910+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "36'600", 'ondisk_log_start': "36'600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40452096, 'num_objects': 9876, 'num_object_clones': 0, 'num_object_copies': 19752, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3633, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9876, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8517, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3229}, {'shards': '2', 'objects': 6245}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:33:48.092 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:33:48.092 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'10771", 'reported_seq': 10904, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:43.554084+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:33:43.554084+0000', 'last_peered': '2024-09-10T20:33:43.554084+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:33:43.554084+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40521728, 'num_objects': 9893, 'num_object_clones': 0, 'num_object_copies': 19786, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3610, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9893, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8557, 'num_write_kb': 32484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3174}, {'shards': '2', 'objects': 6285}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:33:48.093 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:33:48.093 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'10851", 'reported_seq': 11006, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:42.819531+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:33:42.819531+0000', 'last_peered': '2024-09-10T20:33:42.819531+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:33:42.819531+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40873984, 'num_objects': 9979, 'num_object_clones': 0, 'num_object_copies': 19958, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3639, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9979, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7574, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 10, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6341}, {'shards': '0,2', 'objects': 3213}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:48.093 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:33:48.093 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'10705", 'reported_seq': 10864, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:42.846664+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:33:42.846664+0000', 'last_peered': '2024-09-10T20:33:42.846664+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:33:42.846664+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40464384, 'num_objects': 9879, 'num_object_clones': 0, 'num_object_copies': 19758, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3525, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9879, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7399, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 10, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6357}, {'shards': '0,2', 'objects': 3123}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:48.093 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:33:48.093 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'10907", 'reported_seq': 11053, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:42.738760+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:33:42.738760+0000', 'last_peered': '2024-09-10T20:33:42.738760+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:33:42.738760+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "36'900", 'ondisk_log_start': "36'900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41283584, 'num_objects': 10079, 'num_object_clones': 0, 'num_object_copies': 20158, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3708, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10079, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7734, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 7, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6373}, {'shards': '0,2', 'objects': 3301}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:48.093 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:33:48.093 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'10619", 'reported_seq': 14661, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:33:42.839223+0000', 'last_change': '2024-09-10T20:32:16.657652+0000', 'last_active': '2024-09-10T20:33:42.839223+0000', 'last_peered': '2024-09-10T20:33:42.839223+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:33:42.839223+0000', 'last_undegraded': '2024-09-10T20:33:42.839223+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "36'600", 'ondisk_log_start': "36'600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40538595, 'num_objects': 9898, 'num_object_clones': 0, 'num_object_copies': 19796, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1530, 'num_objects_unfound': 0, 'num_objects_dirty': 9898, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7226, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1938, 'num_bytes_recovered': 7938048, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 8368}, {'shards': '0,2', 'objects': 1530}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:48.094 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:33:48.779 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:48.779 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35998/317776 objects degraded (11.328%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":340,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":158888,"data_bytes":650782164,"bytes_used":2257494016,"bytes_avail":287652798464,"bytes_total":289910292480,"degraded_objects":35998,"degraded_total":317776,"degraded_ratio":0.113281053320578,"misplaced_objects":1530,"misplaced_total":317776,"misplaced_ratio":0.0048147122501384625,"recovering_objects_per_sec":31,"recovering_bytes_per_sec":112940,"recovering_keys_per_sec":0,"num_objects_recovered":379,"num_bytes_recovered":1355776,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":366},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:33:48.801 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:33:48.801 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:33:49.388 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:49.389 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:33:49.417 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":209,"stamp":"2024-09-10T20:33:48.917786+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":642004436,"num_objects":156745,"num_object_clones":0,"num_object_copies":313490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35988,"num_objects_misplaced":1373,"num_objects_unfound":0,"num_objects_dirty":156745,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":138676,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5787,"num_bytes_recovered":23343123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160881,"ondisk_log_size":160881,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2184768,"kb_used_data":1301192,"kb_used_omap":12,"kb_used_meta":883507,"kb_avail":280930752,"statfs":{"total":289910292480,"available":287673090048,"internally_reserved":0,"allocated":1332420608,"data_stored":1331612604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":904711170},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,8,6,4,7,5,2,0,3],"upper_bound":1024},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26779648,"num_objects":-6538,"num_object_clones":0,"num_object_copies":-13076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-58,"num_objects_misplaced":-488,"num_objects_unfound":0,"num_objects_dirty":-6538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6538,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":546,"num_bytes_recovered":1998848,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004496"},"pg_stats":[{"pgid":"2.f","version":"45'10875","reported_seq":10972,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:47.839589+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:33:47.839589+0000","last_peered":"2024-09-10T20:33:47.839589+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:33:47.839589+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40071168,"num_objects":9783,"num_object_clones":0,"num_object_copies":19566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3599,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9783,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10875,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6184},{"shards":"0,2","objects":3061}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11067","reported_seq":11178,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:48.631711+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:33:48.631711+0000","last_peered":"2024-09-10T20:33:48.631711+0000","last_clean":"2024-09-10T20:33:48.631711+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:33:48.631711+0000","last_undegraded":"2024-09-10T20:33:48.631711+0000","last_fullsized":"2024-09-10T20:33:48.631711+0000","mapping_epoch":35,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40448000,"num_objects":9875,"num_object_clones":0,"num_object_copies":19750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9253,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10814","reported_seq":18161,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:44.143866+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:33:44.143866+0000","last_peered":"2024-09-10T20:33:44.143866+0000","last_clean":"2024-09-10T20:33:44.143866+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:33:44.143866+0000","last_undegraded":"2024-09-10T20:33:44.143866+0000","last_fullsized":"2024-09-10T20:33:44.143866+0000","mapping_epoch":44,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40554520,"num_objects":9902,"num_object_clones":0,"num_object_copies":19804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9902,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8942,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10886","reported_seq":10967,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:48.577587+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:33:48.577587+0000","last_peered":"2024-09-10T20:33:48.577587+0000","last_clean":"2024-09-10T20:33:48.577587+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:33:48.577587+0000","last_undegraded":"2024-09-10T20:33:48.577587+0000","last_fullsized":"2024-09-10T20:33:48.577587+0000","mapping_epoch":35,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39804928,"num_objects":9718,"num_object_clones":0,"num_object_copies":19436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10886,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11101","reported_seq":11228,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:47.878159+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:33:47.878159+0000","last_peered":"2024-09-10T20:33:47.878159+0000","last_clean":"2024-09-10T20:33:47.878159+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:33:47.878159+0000","last_undegraded":"2024-09-10T20:33:47.878159+0000","last_fullsized":"2024-09-10T20:33:47.878159+0000","mapping_epoch":34,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40828928,"num_objects":9968,"num_object_clones":0,"num_object_copies":19936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8097,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10748","reported_seq":10896,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:47.876504+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:33:47.876504+0000","last_peered":"2024-09-10T20:33:47.876504+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:33:47.876504+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39825408,"num_objects":9723,"num_object_clones":0,"num_object_copies":19446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3610,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7612,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6115},{"shards":"0,2","objects":3104}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10886","reported_seq":11064,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:47.800849+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:33:47.800849+0000","last_peered":"2024-09-10T20:33:47.800849+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:33:47.800849+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39821312,"num_objects":9722,"num_object_clones":0,"num_object_copies":19444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3537,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7481,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6187},{"shards":"0,2","objects":2976}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10899","reported_seq":11024,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:47.770349+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:33:47.770349+0000","last_peered":"2024-09-10T20:33:47.770349+0000","last_clean":"2024-09-10T20:33:47.770349+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:33:47.770349+0000","last_undegraded":"2024-09-10T20:33:47.770349+0000","last_fullsized":"2024-09-10T20:33:47.770349+0000","mapping_epoch":34,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40292352,"num_objects":9837,"num_object_clones":0,"num_object_copies":19674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7970,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10815","reported_seq":10952,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:48.553194+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:33:48.553194+0000","last_peered":"2024-09-10T20:33:48.553194+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:33:48.553194+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39419904,"num_objects":9624,"num_object_clones":0,"num_object_copies":19248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3525,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8808,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2933},{"shards":"2","objects":6101}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190273+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190273+0000","last_peered":"2024-09-10T20:32:18.190273+0000","last_clean":"2024-09-10T20:32:18.190273+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190273+0000","last_undegraded":"2024-09-10T20:32:18.190273+0000","last_fullsized":"2024-09-10T20:32:18.190273+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10895","reported_seq":11026,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:47.874896+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:33:47.874896+0000","last_peered":"2024-09-10T20:33:47.874896+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:33:47.874896+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40285126,"num_objects":9837,"num_object_clones":0,"num_object_copies":19674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3606,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10893,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6233},{"shards":"0,2","objects":3089}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10839","reported_seq":10972,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:48.595625+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:33:48.595625+0000","last_peered":"2024-09-10T20:33:48.595625+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:33:48.595625+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39837696,"num_objects":9726,"num_object_clones":0,"num_object_copies":19452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3633,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8667,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3079},{"shards":"2","objects":6095}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10922","reported_seq":11057,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:48.552444+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:33:48.552444+0000","last_peered":"2024-09-10T20:33:48.552444+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:33:48.552444+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39903232,"num_objects":9742,"num_object_clones":0,"num_object_copies":19484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3609,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8708,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3023},{"shards":"2","objects":6135}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10987","reported_seq":11140,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:47.850849+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:33:47.850849+0000","last_peered":"2024-09-10T20:33:47.850849+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:33:47.850849+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40316928,"num_objects":9843,"num_object_clones":0,"num_object_copies":19686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3639,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7710,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6205},{"shards":"0,2","objects":3077}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10844","reported_seq":11005,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:47.868924+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:33:47.868924+0000","last_peered":"2024-09-10T20:33:47.868924+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:33:47.868924+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39895040,"num_objects":9740,"num_object_clones":0,"num_object_copies":19480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7538,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6219},{"shards":"0,2","objects":2984}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193230+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193230+0000","last_peered":"2024-09-10T20:32:18.193230+0000","last_clean":"2024-09-10T20:32:18.193230+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193230+0000","last_undegraded":"2024-09-10T20:32:18.193230+0000","last_fullsized":"2024-09-10T20:32:18.193230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11045","reported_seq":11195,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:47.907646+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:33:47.907646+0000","last_peered":"2024-09-10T20:33:47.907646+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:33:47.907646+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40718336,"num_objects":9941,"num_object_clones":0,"num_object_copies":19882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3706,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7872,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6237},{"shards":"0,2","objects":3163}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10755","reported_seq":15112,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:33:47.912812+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:33:47.912812+0000","last_peered":"2024-09-10T20:33:47.912812+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:33:47.912812+0000","last_undegraded":"2024-09-10T20:33:47.912812+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39981539,"num_objects":9762,"num_object_clones":0,"num_object_copies":19524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1373,"num_objects_unfound":0,"num_objects_dirty":9762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7362,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2095,"num_bytes_recovered":8581120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8389},{"shards":"0,2","objects":1373}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":642004417,"num_objects":156743,"num_object_clones":0,"num_object_copies":313486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35988,"num_objects_misplaced":1373,"num_objects_unfound":0,"num_objects_dirty":156743,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":138674,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5785,"num_bytes_recovered":23343104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1329573888,"data_stored":1329479032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160878,"ondisk_log_size":160878,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182463,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":868532,"kb_used_data":530076,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93503308,"statfs":{"total":96636764160,"available":95747387392,"internally_reserved":0,"allocated":542797824,"data_stored":542555136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,4,0,6,1,2,0,3],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986235,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703368,"kb_used_data":428728,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93668472,"statfs":{"total":96636764160,"available":95916515328,"internally_reserved":0,"allocated":439017472,"data_stored":438774789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281210538},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659181,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612868,"kb_used_data":342388,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93758972,"statfs":{"total":96636764160,"available":96009187328,"internally_reserved":0,"allocated":350605312,"data_stored":350282679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":276950702},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":541274112,"data_stored":541270499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":350101504,"data_stored":350017946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":438198272,"data_stored":438190587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:33:52.419 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:33:52.993 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:52.995 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:33:53.028 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":211,"stamp":"2024-09-10T20:33:52.918923+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":641451476,"num_objects":156610,"num_object_clones":0,"num_object_copies":313220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35988,"num_objects_misplaced":1373,"num_objects_unfound":0,"num_objects_dirty":156610,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":138811,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5787,"num_bytes_recovered":23343123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160916,"ondisk_log_size":160916,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2192592,"kb_used_data":1300696,"kb_used_omap":12,"kb_used_meta":891827,"kb_avail":280922928,"statfs":{"total":289910292480,"available":287665078272,"internally_reserved":0,"allocated":1331912704,"data_stored":1331104700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":913230850},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,6,3,10,5,2,0,3],"upper_bound":1024},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18145280,"num_objects":-4430,"num_object_clones":0,"num_object_copies":-8860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-43,"num_objects_misplaced":-331,"num_objects_unfound":0,"num_objects_dirty":-4430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4430,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":374,"num_bytes_recovered":1355776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004191"},"pg_stats":[{"pgid":"2.f","version":"45'10875","reported_seq":10972,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:47.839589+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:33:47.839589+0000","last_peered":"2024-09-10T20:33:47.839589+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:33:47.839589+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40071168,"num_objects":9783,"num_object_clones":0,"num_object_copies":19566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3599,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9783,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10875,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6184},{"shards":"0,2","objects":3061}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11067","reported_seq":11178,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:48.631711+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:33:48.631711+0000","last_peered":"2024-09-10T20:33:48.631711+0000","last_clean":"2024-09-10T20:33:48.631711+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:33:48.631711+0000","last_undegraded":"2024-09-10T20:33:48.631711+0000","last_fullsized":"2024-09-10T20:33:48.631711+0000","mapping_epoch":35,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40448000,"num_objects":9875,"num_object_clones":0,"num_object_copies":19750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9253,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10949","reported_seq":18297,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:49.164258+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:33:49.164258+0000","last_peered":"2024-09-10T20:33:49.164258+0000","last_clean":"2024-09-10T20:33:49.164258+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:33:49.164258+0000","last_undegraded":"2024-09-10T20:33:49.164258+0000","last_fullsized":"2024-09-10T20:33:49.164258+0000","mapping_epoch":44,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40001560,"num_objects":9767,"num_object_clones":0,"num_object_copies":19534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9767,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9077,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10886","reported_seq":10967,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:48.577587+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:33:48.577587+0000","last_peered":"2024-09-10T20:33:48.577587+0000","last_clean":"2024-09-10T20:33:48.577587+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:33:48.577587+0000","last_undegraded":"2024-09-10T20:33:48.577587+0000","last_fullsized":"2024-09-10T20:33:48.577587+0000","mapping_epoch":35,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39804928,"num_objects":9718,"num_object_clones":0,"num_object_copies":19436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10886,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11101","reported_seq":11228,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:47.878159+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:33:47.878159+0000","last_peered":"2024-09-10T20:33:47.878159+0000","last_clean":"2024-09-10T20:33:47.878159+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:33:47.878159+0000","last_undegraded":"2024-09-10T20:33:47.878159+0000","last_fullsized":"2024-09-10T20:33:47.878159+0000","mapping_epoch":34,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40828928,"num_objects":9968,"num_object_clones":0,"num_object_copies":19936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8097,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10748","reported_seq":10896,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:47.876504+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:33:47.876504+0000","last_peered":"2024-09-10T20:33:47.876504+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:33:47.876504+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39825408,"num_objects":9723,"num_object_clones":0,"num_object_copies":19446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3610,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7612,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6115},{"shards":"0,2","objects":3104}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10886","reported_seq":11064,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:47.800849+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:33:47.800849+0000","last_peered":"2024-09-10T20:33:47.800849+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:33:47.800849+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39821312,"num_objects":9722,"num_object_clones":0,"num_object_copies":19444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3537,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7481,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6187},{"shards":"0,2","objects":2976}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10899","reported_seq":11024,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:47.770349+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:33:47.770349+0000","last_peered":"2024-09-10T20:33:47.770349+0000","last_clean":"2024-09-10T20:33:47.770349+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:33:47.770349+0000","last_undegraded":"2024-09-10T20:33:47.770349+0000","last_fullsized":"2024-09-10T20:33:47.770349+0000","mapping_epoch":34,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40292352,"num_objects":9837,"num_object_clones":0,"num_object_copies":19674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7970,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10815","reported_seq":10952,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:48.553194+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:33:48.553194+0000","last_peered":"2024-09-10T20:33:48.553194+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:33:48.553194+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39419904,"num_objects":9624,"num_object_clones":0,"num_object_copies":19248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3525,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8808,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2933},{"shards":"2","objects":6101}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190273+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190273+0000","last_peered":"2024-09-10T20:32:18.190273+0000","last_clean":"2024-09-10T20:32:18.190273+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190273+0000","last_undegraded":"2024-09-10T20:32:18.190273+0000","last_fullsized":"2024-09-10T20:32:18.190273+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10895","reported_seq":11026,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:47.874896+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:33:47.874896+0000","last_peered":"2024-09-10T20:33:47.874896+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:33:47.874896+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40285126,"num_objects":9837,"num_object_clones":0,"num_object_copies":19674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3606,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10893,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6233},{"shards":"0,2","objects":3089}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10839","reported_seq":10972,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:48.595625+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:33:48.595625+0000","last_peered":"2024-09-10T20:33:48.595625+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:33:48.595625+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39837696,"num_objects":9726,"num_object_clones":0,"num_object_copies":19452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3633,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8667,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3079},{"shards":"2","objects":6095}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10922","reported_seq":11057,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:48.552444+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:33:48.552444+0000","last_peered":"2024-09-10T20:33:48.552444+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:33:48.552444+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39903232,"num_objects":9742,"num_object_clones":0,"num_object_copies":19484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3609,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8708,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3023},{"shards":"2","objects":6135}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10987","reported_seq":11140,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:47.850849+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:33:47.850849+0000","last_peered":"2024-09-10T20:33:47.850849+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:33:47.850849+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40316928,"num_objects":9843,"num_object_clones":0,"num_object_copies":19686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3639,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7710,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6205},{"shards":"0,2","objects":3077}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10844","reported_seq":11005,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:47.868924+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:33:47.868924+0000","last_peered":"2024-09-10T20:33:47.868924+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:33:47.868924+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39895040,"num_objects":9740,"num_object_clones":0,"num_object_copies":19480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7538,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6219},{"shards":"0,2","objects":2984}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193230+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193230+0000","last_peered":"2024-09-10T20:32:18.193230+0000","last_clean":"2024-09-10T20:32:18.193230+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193230+0000","last_undegraded":"2024-09-10T20:32:18.193230+0000","last_fullsized":"2024-09-10T20:32:18.193230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11045","reported_seq":11195,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:47.907646+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:33:47.907646+0000","last_peered":"2024-09-10T20:33:47.907646+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:33:47.907646+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40718336,"num_objects":9941,"num_object_clones":0,"num_object_copies":19882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3706,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7872,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6237},{"shards":"0,2","objects":3163}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10755","reported_seq":15112,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:33:47.912812+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:33:47.912812+0000","last_peered":"2024-09-10T20:33:47.912812+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:33:47.912812+0000","last_undegraded":"2024-09-10T20:33:47.912812+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39981539,"num_objects":9762,"num_object_clones":0,"num_object_copies":19524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1373,"num_objects_unfound":0,"num_objects_dirty":9762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7362,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2095,"num_bytes_recovered":8581120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8389},{"shards":"0,2","objects":1373}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":641451457,"num_objects":156608,"num_object_clones":0,"num_object_copies":313216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35988,"num_objects_misplaced":1373,"num_objects_unfound":0,"num_objects_dirty":156608,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":138809,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5785,"num_bytes_recovered":23343104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1329065984,"data_stored":1328971128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160913,"ondisk_log_size":160913,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182463,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":868532,"kb_used_data":530076,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93503308,"statfs":{"total":96636764160,"available":95747387392,"internally_reserved":0,"allocated":542797824,"data_stored":542555136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,4,0,6,1,2,0,3],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986235,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703368,"kb_used_data":428728,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93668472,"statfs":{"total":96636764160,"available":95916515328,"internally_reserved":0,"allocated":439017472,"data_stored":438774789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281210538},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659182,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":620692,"kb_used_data":341892,"kb_used_omap":4,"kb_used_meta":278779,"kb_avail":93751148,"statfs":{"total":96636764160,"available":96001175552,"internally_reserved":0,"allocated":350097408,"data_stored":349774775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":285470382},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":541274112,"data_stored":541270499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":349593600,"data_stored":349510042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":438198272,"data_stored":438190587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:33:53.029 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:33:53.595 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:53.597 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:33:53.617 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":211,"stamp":"2024-09-10T20:33:52.918923+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":641451476,"num_objects":156610,"num_object_clones":0,"num_object_copies":313220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35988,"num_objects_misplaced":1373,"num_objects_unfound":0,"num_objects_dirty":156610,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":138811,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5787,"num_bytes_recovered":23343123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160916,"ondisk_log_size":160916,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2192592,"kb_used_data":1300696,"kb_used_omap":12,"kb_used_meta":891827,"kb_avail":280922928,"statfs":{"total":289910292480,"available":287665078272,"internally_reserved":0,"allocated":1331912704,"data_stored":1331104700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":913230850},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,6,3,10,5,2,0,3],"upper_bound":1024},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18145280,"num_objects":-4430,"num_object_clones":0,"num_object_copies":-8860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-43,"num_objects_misplaced":-331,"num_objects_unfound":0,"num_objects_dirty":-4430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4430,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":374,"num_bytes_recovered":1355776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004191"},"pg_stats":[{"pgid":"2.f","version":"45'10875","reported_seq":10972,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:47.839589+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:33:47.839589+0000","last_peered":"2024-09-10T20:33:47.839589+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:33:47.839589+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40071168,"num_objects":9783,"num_object_clones":0,"num_object_copies":19566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3599,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9783,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10875,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6184},{"shards":"0,2","objects":3061}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11067","reported_seq":11178,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:48.631711+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:33:48.631711+0000","last_peered":"2024-09-10T20:33:48.631711+0000","last_clean":"2024-09-10T20:33:48.631711+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:33:48.631711+0000","last_undegraded":"2024-09-10T20:33:48.631711+0000","last_fullsized":"2024-09-10T20:33:48.631711+0000","mapping_epoch":35,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40448000,"num_objects":9875,"num_object_clones":0,"num_object_copies":19750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9253,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10949","reported_seq":18297,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:49.164258+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:33:49.164258+0000","last_peered":"2024-09-10T20:33:49.164258+0000","last_clean":"2024-09-10T20:33:49.164258+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:33:49.164258+0000","last_undegraded":"2024-09-10T20:33:49.164258+0000","last_fullsized":"2024-09-10T20:33:49.164258+0000","mapping_epoch":44,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40001560,"num_objects":9767,"num_object_clones":0,"num_object_copies":19534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9767,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9077,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10886","reported_seq":10967,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:48.577587+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:33:48.577587+0000","last_peered":"2024-09-10T20:33:48.577587+0000","last_clean":"2024-09-10T20:33:48.577587+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:33:48.577587+0000","last_undegraded":"2024-09-10T20:33:48.577587+0000","last_fullsized":"2024-09-10T20:33:48.577587+0000","mapping_epoch":35,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39804928,"num_objects":9718,"num_object_clones":0,"num_object_copies":19436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10886,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11101","reported_seq":11228,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:47.878159+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:33:47.878159+0000","last_peered":"2024-09-10T20:33:47.878159+0000","last_clean":"2024-09-10T20:33:47.878159+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:33:47.878159+0000","last_undegraded":"2024-09-10T20:33:47.878159+0000","last_fullsized":"2024-09-10T20:33:47.878159+0000","mapping_epoch":34,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40828928,"num_objects":9968,"num_object_clones":0,"num_object_copies":19936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8097,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10748","reported_seq":10896,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:47.876504+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:33:47.876504+0000","last_peered":"2024-09-10T20:33:47.876504+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:33:47.876504+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39825408,"num_objects":9723,"num_object_clones":0,"num_object_copies":19446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3610,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7612,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6115},{"shards":"0,2","objects":3104}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10886","reported_seq":11064,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:47.800849+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:33:47.800849+0000","last_peered":"2024-09-10T20:33:47.800849+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:33:47.800849+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39821312,"num_objects":9722,"num_object_clones":0,"num_object_copies":19444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3537,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7481,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6187},{"shards":"0,2","objects":2976}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10899","reported_seq":11024,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:47.770349+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:33:47.770349+0000","last_peered":"2024-09-10T20:33:47.770349+0000","last_clean":"2024-09-10T20:33:47.770349+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:33:47.770349+0000","last_undegraded":"2024-09-10T20:33:47.770349+0000","last_fullsized":"2024-09-10T20:33:47.770349+0000","mapping_epoch":34,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40292352,"num_objects":9837,"num_object_clones":0,"num_object_copies":19674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7970,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10815","reported_seq":10952,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:48.553194+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:33:48.553194+0000","last_peered":"2024-09-10T20:33:48.553194+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:33:48.553194+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39419904,"num_objects":9624,"num_object_clones":0,"num_object_copies":19248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3525,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8808,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2933},{"shards":"2","objects":6101}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190273+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190273+0000","last_peered":"2024-09-10T20:32:18.190273+0000","last_clean":"2024-09-10T20:32:18.190273+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190273+0000","last_undegraded":"2024-09-10T20:32:18.190273+0000","last_fullsized":"2024-09-10T20:32:18.190273+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10895","reported_seq":11026,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:47.874896+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:33:47.874896+0000","last_peered":"2024-09-10T20:33:47.874896+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:33:47.874896+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40285126,"num_objects":9837,"num_object_clones":0,"num_object_copies":19674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3606,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10893,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6233},{"shards":"0,2","objects":3089}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10839","reported_seq":10972,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:48.595625+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:33:48.595625+0000","last_peered":"2024-09-10T20:33:48.595625+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:33:48.595625+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39837696,"num_objects":9726,"num_object_clones":0,"num_object_copies":19452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3633,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8667,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3079},{"shards":"2","objects":6095}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10922","reported_seq":11057,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:48.552444+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:33:48.552444+0000","last_peered":"2024-09-10T20:33:48.552444+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:33:48.552444+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39903232,"num_objects":9742,"num_object_clones":0,"num_object_copies":19484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3609,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8708,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3023},{"shards":"2","objects":6135}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10987","reported_seq":11140,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:47.850849+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:33:47.850849+0000","last_peered":"2024-09-10T20:33:47.850849+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:33:47.850849+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40316928,"num_objects":9843,"num_object_clones":0,"num_object_copies":19686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3639,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7710,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6205},{"shards":"0,2","objects":3077}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10844","reported_seq":11005,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:47.868924+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:33:47.868924+0000","last_peered":"2024-09-10T20:33:47.868924+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:33:47.868924+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39895040,"num_objects":9740,"num_object_clones":0,"num_object_copies":19480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7538,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6219},{"shards":"0,2","objects":2984}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193230+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193230+0000","last_peered":"2024-09-10T20:32:18.193230+0000","last_clean":"2024-09-10T20:32:18.193230+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193230+0000","last_undegraded":"2024-09-10T20:32:18.193230+0000","last_fullsized":"2024-09-10T20:32:18.193230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11045","reported_seq":11195,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:47.907646+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:33:47.907646+0000","last_peered":"2024-09-10T20:33:47.907646+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:33:47.907646+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40718336,"num_objects":9941,"num_object_clones":0,"num_object_copies":19882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3706,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7872,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6237},{"shards":"0,2","objects":3163}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10755","reported_seq":15112,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:33:47.912812+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:33:47.912812+0000","last_peered":"2024-09-10T20:33:47.912812+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:33:47.912812+0000","last_undegraded":"2024-09-10T20:33:47.912812+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39981539,"num_objects":9762,"num_object_clones":0,"num_object_copies":19524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1373,"num_objects_unfound":0,"num_objects_dirty":9762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7362,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2095,"num_bytes_recovered":8581120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8389},{"shards":"0,2","objects":1373}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":641451457,"num_objects":156608,"num_object_clones":0,"num_object_copies":313216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35988,"num_objects_misplaced":1373,"num_objects_unfound":0,"num_objects_dirty":156608,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":138809,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5785,"num_bytes_recovered":23343104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1329065984,"data_stored":1328971128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160913,"ondisk_log_size":160913,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182463,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":868532,"kb_used_data":530076,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93503308,"statfs":{"total":96636764160,"available":95747387392,"internally_reserved":0,"allocated":542797824,"data_stored":542555136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,4,0,6,1,2,0,3],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986235,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703368,"kb_used_data":428728,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93668472,"statfs":{"total":96636764160,"available":95916515328,"internally_reserved":0,"allocated":439017472,"data_stored":438774789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281210538},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659182,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":620692,"kb_used_data":341892,"kb_used_omap":4,"kb_used_meta":278779,"kb_avail":93751148,"statfs":{"total":96636764160,"available":96001175552,"internally_reserved":0,"allocated":350097408,"data_stored":349774775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":285470382},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":541274112,"data_stored":541270499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":349593600,"data_stored":349510042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":438198272,"data_stored":438190587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:33:53.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:33:53.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'10875", 'reported_seq': 10972, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:47.839589+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:33:47.839589+0000', 'last_peered': '2024-09-10T20:33:47.839589+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:33:47.839589+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40071168, 'num_objects': 9783, 'num_object_clones': 0, 'num_object_copies': 19566, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3599, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9783, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10875, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 9, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6184}, {'shards': '0,2', 'objects': 3061}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:53.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:33:53.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'10748", 'reported_seq': 10896, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:47.876504+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:33:47.876504+0000', 'last_peered': '2024-09-10T20:33:47.876504+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:33:47.876504+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39825408, 'num_objects': 9723, 'num_object_clones': 0, 'num_object_copies': 19446, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3610, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9723, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7612, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 6, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6115}, {'shards': '0,2', 'objects': 3104}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:53.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:33:53.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'10886", 'reported_seq': 11064, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:47.800849+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:33:47.800849+0000', 'last_peered': '2024-09-10T20:33:47.800849+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:33:47.800849+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39821312, 'num_objects': 9722, 'num_object_clones': 0, 'num_object_copies': 19444, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3537, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9722, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7481, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 20, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6187}, {'shards': '0,2', 'objects': 2976}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:53.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:33:53.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'10815", 'reported_seq': 10952, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:48.553194+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:33:48.553194+0000', 'last_peered': '2024-09-10T20:33:48.553194+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:33:48.553194+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39419904, 'num_objects': 9624, 'num_object_clones': 0, 'num_object_copies': 19248, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3525, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9624, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8808, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2933}, {'shards': '2', 'objects': 6101}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:33:53.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:33:53.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'10895", 'reported_seq': 11026, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:47.874896+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:33:47.874896+0000', 'last_peered': '2024-09-10T20:33:47.874896+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:33:47.874896+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40285126, 'num_objects': 9837, 'num_object_clones': 0, 'num_object_copies': 19674, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3606, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9837, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10893, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 13, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6233}, {'shards': '0,2', 'objects': 3089}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:53.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:33:53.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'10839", 'reported_seq': 10972, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:48.595625+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:33:48.595625+0000', 'last_peered': '2024-09-10T20:33:48.595625+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:33:48.595625+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39837696, 'num_objects': 9726, 'num_object_clones': 0, 'num_object_copies': 19452, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3633, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9726, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8667, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3079}, {'shards': '2', 'objects': 6095}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:33:53.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:33:53.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'10922", 'reported_seq': 11057, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:48.552444+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:33:48.552444+0000', 'last_peered': '2024-09-10T20:33:48.552444+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:33:48.552444+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "36'900", 'ondisk_log_start': "36'900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39903232, 'num_objects': 9742, 'num_object_clones': 0, 'num_object_copies': 19484, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3609, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9742, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8708, 'num_write_kb': 32484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3023}, {'shards': '2', 'objects': 6135}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:33:53.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:33:53.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'10987", 'reported_seq': 11140, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:47.850849+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:33:47.850849+0000', 'last_peered': '2024-09-10T20:33:47.850849+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:33:47.850849+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "36'900", 'ondisk_log_start': "36'900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40316928, 'num_objects': 9843, 'num_object_clones': 0, 'num_object_copies': 19686, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3639, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9843, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7710, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 10, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6205}, {'shards': '0,2', 'objects': 3077}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:53.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:33:53.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'10844", 'reported_seq': 11005, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:47.868924+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:33:47.868924+0000', 'last_peered': '2024-09-10T20:33:47.868924+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:33:47.868924+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39895040, 'num_objects': 9740, 'num_object_clones': 0, 'num_object_copies': 19480, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9740, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7538, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 11, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6219}, {'shards': '0,2', 'objects': 2984}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:53.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:33:53.621 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'11045", 'reported_seq': 11195, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:47.907646+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:33:47.907646+0000', 'last_peered': '2024-09-10T20:33:47.907646+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:33:47.907646+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "36'1000", 'ondisk_log_start': "36'1000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40718336, 'num_objects': 9941, 'num_object_clones': 0, 'num_object_copies': 19882, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3706, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9941, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7872, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 9, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6237}, {'shards': '0,2', 'objects': 3163}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:53.621 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:33:53.621 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'10755", 'reported_seq': 15112, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:33:47.912812+0000', 'last_change': '2024-09-10T20:32:16.657652+0000', 'last_active': '2024-09-10T20:33:47.912812+0000', 'last_peered': '2024-09-10T20:33:47.912812+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:33:47.912812+0000', 'last_undegraded': '2024-09-10T20:33:47.912812+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39981539, 'num_objects': 9762, 'num_object_clones': 0, 'num_object_copies': 19524, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1373, 'num_objects_unfound': 0, 'num_objects_dirty': 9762, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7362, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2095, 'num_bytes_recovered': 8581120, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 8389}, {'shards': '0,2', 'objects': 1373}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:53.621 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:33:54.306 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:54.306 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35988/313220 objects degraded (11.490%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":345,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":156610,"data_bytes":641451476,"bytes_used":2245214208,"bytes_avail":287665078272,"bytes_total":289910292480,"degraded_objects":35988,"degraded_total":313220,"degraded_ratio":0.11489687759402337,"misplaced_objects":1373,"misplaced_total":313220,"misplaced_ratio":0.0043835004150437392,"recovering_objects_per_sec":31,"recovering_bytes_per_sec":112941,"recovering_keys_per_sec":0,"num_objects_recovered":374,"num_bytes_recovered":1355776,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":369},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:33:54.325 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:33:54.326 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:33:54.868 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:54.871 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:33:54.891 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":211,"stamp":"2024-09-10T20:33:52.918923+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":641451476,"num_objects":156610,"num_object_clones":0,"num_object_copies":313220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35988,"num_objects_misplaced":1373,"num_objects_unfound":0,"num_objects_dirty":156610,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":138811,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5787,"num_bytes_recovered":23343123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160916,"ondisk_log_size":160916,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2192592,"kb_used_data":1300696,"kb_used_omap":12,"kb_used_meta":891827,"kb_avail":280922928,"statfs":{"total":289910292480,"available":287665078272,"internally_reserved":0,"allocated":1331912704,"data_stored":1331104700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":913230850},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,6,3,10,5,2,0,3],"upper_bound":1024},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18145280,"num_objects":-4430,"num_object_clones":0,"num_object_copies":-8860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-43,"num_objects_misplaced":-331,"num_objects_unfound":0,"num_objects_dirty":-4430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4430,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":374,"num_bytes_recovered":1355776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004191"},"pg_stats":[{"pgid":"2.f","version":"45'10875","reported_seq":10972,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:47.839589+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:33:47.839589+0000","last_peered":"2024-09-10T20:33:47.839589+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:33:47.839589+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40071168,"num_objects":9783,"num_object_clones":0,"num_object_copies":19566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3599,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9783,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10875,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6184},{"shards":"0,2","objects":3061}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11067","reported_seq":11178,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:48.631711+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:33:48.631711+0000","last_peered":"2024-09-10T20:33:48.631711+0000","last_clean":"2024-09-10T20:33:48.631711+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:33:48.631711+0000","last_undegraded":"2024-09-10T20:33:48.631711+0000","last_fullsized":"2024-09-10T20:33:48.631711+0000","mapping_epoch":35,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40448000,"num_objects":9875,"num_object_clones":0,"num_object_copies":19750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9253,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10949","reported_seq":18297,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:49.164258+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:33:49.164258+0000","last_peered":"2024-09-10T20:33:49.164258+0000","last_clean":"2024-09-10T20:33:49.164258+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:33:49.164258+0000","last_undegraded":"2024-09-10T20:33:49.164258+0000","last_fullsized":"2024-09-10T20:33:49.164258+0000","mapping_epoch":44,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40001560,"num_objects":9767,"num_object_clones":0,"num_object_copies":19534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9767,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9077,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10886","reported_seq":10967,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:48.577587+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:33:48.577587+0000","last_peered":"2024-09-10T20:33:48.577587+0000","last_clean":"2024-09-10T20:33:48.577587+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:33:48.577587+0000","last_undegraded":"2024-09-10T20:33:48.577587+0000","last_fullsized":"2024-09-10T20:33:48.577587+0000","mapping_epoch":35,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39804928,"num_objects":9718,"num_object_clones":0,"num_object_copies":19436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10886,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11101","reported_seq":11228,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:47.878159+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:33:47.878159+0000","last_peered":"2024-09-10T20:33:47.878159+0000","last_clean":"2024-09-10T20:33:47.878159+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:33:47.878159+0000","last_undegraded":"2024-09-10T20:33:47.878159+0000","last_fullsized":"2024-09-10T20:33:47.878159+0000","mapping_epoch":34,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40828928,"num_objects":9968,"num_object_clones":0,"num_object_copies":19936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8097,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10748","reported_seq":10896,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:47.876504+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:33:47.876504+0000","last_peered":"2024-09-10T20:33:47.876504+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:33:47.876504+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39825408,"num_objects":9723,"num_object_clones":0,"num_object_copies":19446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3610,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7612,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6115},{"shards":"0,2","objects":3104}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10886","reported_seq":11064,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:47.800849+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:33:47.800849+0000","last_peered":"2024-09-10T20:33:47.800849+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:33:47.800849+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39821312,"num_objects":9722,"num_object_clones":0,"num_object_copies":19444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3537,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7481,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6187},{"shards":"0,2","objects":2976}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10899","reported_seq":11024,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:47.770349+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:33:47.770349+0000","last_peered":"2024-09-10T20:33:47.770349+0000","last_clean":"2024-09-10T20:33:47.770349+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:33:47.770349+0000","last_undegraded":"2024-09-10T20:33:47.770349+0000","last_fullsized":"2024-09-10T20:33:47.770349+0000","mapping_epoch":34,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40292352,"num_objects":9837,"num_object_clones":0,"num_object_copies":19674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7970,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10815","reported_seq":10952,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:48.553194+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:33:48.553194+0000","last_peered":"2024-09-10T20:33:48.553194+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:33:48.553194+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39419904,"num_objects":9624,"num_object_clones":0,"num_object_copies":19248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3525,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8808,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2933},{"shards":"2","objects":6101}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190273+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190273+0000","last_peered":"2024-09-10T20:32:18.190273+0000","last_clean":"2024-09-10T20:32:18.190273+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190273+0000","last_undegraded":"2024-09-10T20:32:18.190273+0000","last_fullsized":"2024-09-10T20:32:18.190273+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10895","reported_seq":11026,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:47.874896+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:33:47.874896+0000","last_peered":"2024-09-10T20:33:47.874896+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:33:47.874896+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40285126,"num_objects":9837,"num_object_clones":0,"num_object_copies":19674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3606,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10893,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6233},{"shards":"0,2","objects":3089}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10839","reported_seq":10972,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:48.595625+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:33:48.595625+0000","last_peered":"2024-09-10T20:33:48.595625+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:33:48.595625+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39837696,"num_objects":9726,"num_object_clones":0,"num_object_copies":19452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3633,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8667,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3079},{"shards":"2","objects":6095}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10922","reported_seq":11057,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:48.552444+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:33:48.552444+0000","last_peered":"2024-09-10T20:33:48.552444+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:33:48.552444+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39903232,"num_objects":9742,"num_object_clones":0,"num_object_copies":19484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3609,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8708,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3023},{"shards":"2","objects":6135}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10987","reported_seq":11140,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:47.850849+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:33:47.850849+0000","last_peered":"2024-09-10T20:33:47.850849+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:33:47.850849+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40316928,"num_objects":9843,"num_object_clones":0,"num_object_copies":19686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3639,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7710,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6205},{"shards":"0,2","objects":3077}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10844","reported_seq":11005,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:47.868924+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:33:47.868924+0000","last_peered":"2024-09-10T20:33:47.868924+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:33:47.868924+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39895040,"num_objects":9740,"num_object_clones":0,"num_object_copies":19480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7538,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6219},{"shards":"0,2","objects":2984}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193230+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193230+0000","last_peered":"2024-09-10T20:32:18.193230+0000","last_clean":"2024-09-10T20:32:18.193230+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193230+0000","last_undegraded":"2024-09-10T20:32:18.193230+0000","last_fullsized":"2024-09-10T20:32:18.193230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11045","reported_seq":11195,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:47.907646+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:33:47.907646+0000","last_peered":"2024-09-10T20:33:47.907646+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:33:47.907646+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40718336,"num_objects":9941,"num_object_clones":0,"num_object_copies":19882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3706,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7872,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6237},{"shards":"0,2","objects":3163}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10755","reported_seq":15112,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:33:47.912812+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:33:47.912812+0000","last_peered":"2024-09-10T20:33:47.912812+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:33:47.912812+0000","last_undegraded":"2024-09-10T20:33:47.912812+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39981539,"num_objects":9762,"num_object_clones":0,"num_object_copies":19524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1373,"num_objects_unfound":0,"num_objects_dirty":9762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7362,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2095,"num_bytes_recovered":8581120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8389},{"shards":"0,2","objects":1373}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":641451457,"num_objects":156608,"num_object_clones":0,"num_object_copies":313216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35988,"num_objects_misplaced":1373,"num_objects_unfound":0,"num_objects_dirty":156608,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":138809,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5785,"num_bytes_recovered":23343104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1329065984,"data_stored":1328971128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160913,"ondisk_log_size":160913,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182463,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":868532,"kb_used_data":530076,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93503308,"statfs":{"total":96636764160,"available":95747387392,"internally_reserved":0,"allocated":542797824,"data_stored":542555136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,4,0,6,1,2,0,3],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986235,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703368,"kb_used_data":428728,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93668472,"statfs":{"total":96636764160,"available":95916515328,"internally_reserved":0,"allocated":439017472,"data_stored":438774789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281210538},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659182,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":620692,"kb_used_data":341892,"kb_used_omap":4,"kb_used_meta":278779,"kb_avail":93751148,"statfs":{"total":96636764160,"available":96001175552,"internally_reserved":0,"allocated":350097408,"data_stored":349774775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":285470382},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":541274112,"data_stored":541270499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":349593600,"data_stored":349510042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":438198272,"data_stored":438190587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:33:57.893 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:33:58.455 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:58.456 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:33:58.477 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":213,"stamp":"2024-09-10T20:33:56.920355+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":632051156,"num_objects":154315,"num_object_clones":0,"num_object_copies":308630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35972,"num_objects_misplaced":1245,"num_objects_unfound":0,"num_objects_dirty":154315,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":141106,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5934,"num_bytes_recovered":23867411,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160811,"ondisk_log_size":160811,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2159784,"kb_used_data":1288684,"kb_used_omap":12,"kb_used_meta":871027,"kb_avail":280955736,"statfs":{"total":289910292480,"available":287698673664,"internally_reserved":0,"allocated":1319612416,"data_stored":1318815251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":891931650},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,3,4,8,11,3,0,0,2,0,0,3],"upper_bound":8192},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18731008,"num_objects":-4573,"num_object_clones":0,"num_object_copies":-9146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-26,"num_objects_misplaced":-285,"num_objects_unfound":0,"num_objects_dirty":-4573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4573,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":314,"num_bytes_recovered":1167360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004040"},"pg_stats":[{"pgid":"2.f","version":"45'11025","reported_seq":11126,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:52.926324+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:33:52.926324+0000","last_peered":"2024-09-10T20:33:52.926324+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:33:52.926324+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39456768,"num_objects":9633,"num_object_clones":0,"num_object_copies":19266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3597,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11025,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6036},{"shards":"0,2","objects":2911}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11227","reported_seq":11336,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:53.708435+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:33:53.708435+0000","last_peered":"2024-09-10T20:33:53.708435+0000","last_clean":"2024-09-10T20:33:53.708435+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:33:53.708435+0000","last_undegraded":"2024-09-10T20:33:53.708435+0000","last_fullsized":"2024-09-10T20:33:53.708435+0000","mapping_epoch":35,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39792640,"num_objects":9715,"num_object_clones":0,"num_object_copies":19430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9413,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11083","reported_seq":18431,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:54.120454+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:33:54.120454+0000","last_peered":"2024-09-10T20:33:54.120454+0000","last_clean":"2024-09-10T20:33:54.120454+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:33:54.120454+0000","last_undegraded":"2024-09-10T20:33:54.120454+0000","last_fullsized":"2024-09-10T20:33:54.120454+0000","mapping_epoch":44,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39452696,"num_objects":9633,"num_object_clones":0,"num_object_copies":19266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9633,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9211,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'11032","reported_seq":11113,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:53.643319+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:33:53.643319+0000","last_peered":"2024-09-10T20:33:53.643319+0000","last_clean":"2024-09-10T20:33:53.643319+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:33:53.643319+0000","last_undegraded":"2024-09-10T20:33:53.643319+0000","last_fullsized":"2024-09-10T20:33:53.643319+0000","mapping_epoch":35,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39206912,"num_objects":9572,"num_object_clones":0,"num_object_copies":19144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11032,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11256","reported_seq":11383,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:52.886968+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:33:52.886968+0000","last_peered":"2024-09-10T20:33:52.886968+0000","last_clean":"2024-09-10T20:33:52.886968+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:33:52.886968+0000","last_undegraded":"2024-09-10T20:33:52.886968+0000","last_fullsized":"2024-09-10T20:33:52.886968+0000","mapping_epoch":34,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40194048,"num_objects":9813,"num_object_clones":0,"num_object_copies":19626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8252,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10895","reported_seq":11047,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:52.878430+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:33:52.878430+0000","last_peered":"2024-09-10T20:33:52.878430+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:33:52.878430+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39223296,"num_objects":9576,"num_object_clones":0,"num_object_copies":19152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7759,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5970},{"shards":"0,2","objects":2957}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11013","reported_seq":11199,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:52.902849+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:33:52.902849+0000","last_peered":"2024-09-10T20:33:52.902849+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:33:52.902849+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39301120,"num_objects":9595,"num_object_clones":0,"num_object_copies":19190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3533,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7608,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6064},{"shards":"0,2","objects":2849}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11026","reported_seq":11151,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:52.850472+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:33:52.850472+0000","last_peered":"2024-09-10T20:33:52.850472+0000","last_clean":"2024-09-10T20:33:52.850472+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:33:52.850472+0000","last_undegraded":"2024-09-10T20:33:52.850472+0000","last_fullsized":"2024-09-10T20:33:52.850472+0000","mapping_epoch":34,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39772160,"num_objects":9710,"num_object_clones":0,"num_object_copies":19420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8097,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10959","reported_seq":11101,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:53.632828+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:33:53.632828+0000","last_peered":"2024-09-10T20:33:53.632828+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:33:53.632828+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38830080,"num_objects":9480,"num_object_clones":0,"num_object_copies":18960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3523,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9480,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8952,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2789},{"shards":"2","objects":5959}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190273+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190273+0000","last_peered":"2024-09-10T20:32:18.190273+0000","last_clean":"2024-09-10T20:32:18.190273+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190273+0000","last_undegraded":"2024-09-10T20:32:18.190273+0000","last_fullsized":"2024-09-10T20:32:18.190273+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11045","reported_seq":11176,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:52.869615+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:33:52.869615+0000","last_peered":"2024-09-10T20:33:52.869615+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:33:52.869615+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39670726,"num_objects":9687,"num_object_clones":0,"num_object_copies":19374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3606,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9687,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11043,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6083},{"shards":"0,2","objects":2939}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10989","reported_seq":11122,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:53.618829+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:33:53.618829+0000","last_peered":"2024-09-10T20:33:53.618829+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:33:53.618829+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39223296,"num_objects":9576,"num_object_clones":0,"num_object_copies":19152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3633,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8817,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2929},{"shards":"2","objects":5945}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11064","reported_seq":11199,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:53.640261+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:33:53.640261+0000","last_peered":"2024-09-10T20:33:53.640261+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:33:53.640261+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39321600,"num_objects":9600,"num_object_clones":0,"num_object_copies":19200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3609,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8850,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2881},{"shards":"2","objects":5993}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11123","reported_seq":11278,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:52.854583+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:33:52.854583+0000","last_peered":"2024-09-10T20:33:52.854583+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:33:52.854583+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39759872,"num_objects":9707,"num_object_clones":0,"num_object_copies":19414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3639,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7846,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6069},{"shards":"0,2","objects":2941}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10990","reported_seq":11159,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:52.772686+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:33:52.772686+0000","last_peered":"2024-09-10T20:33:52.772686+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:33:52.772686+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39297024,"num_objects":9594,"num_object_clones":0,"num_object_copies":19188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3520,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9594,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7684,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6077},{"shards":"0,2","objects":2838}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193230+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193230+0000","last_peered":"2024-09-10T20:32:18.193230+0000","last_clean":"2024-09-10T20:32:18.193230+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193230+0000","last_undegraded":"2024-09-10T20:32:18.193230+0000","last_fullsized":"2024-09-10T20:32:18.193230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11169","reported_seq":11323,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:52.923724+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:33:52.923724+0000","last_peered":"2024-09-10T20:33:52.923724+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:33:52.923724+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40210432,"num_objects":9817,"num_object_clones":0,"num_object_copies":19634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3704,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7996,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6115},{"shards":"0,2","objects":3039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10912","reported_seq":15531,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:33:52.913858+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:33:52.913858+0000","last_peered":"2024-09-10T20:33:52.913858+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:33:52.913858+0000","last_undegraded":"2024-09-10T20:33:52.913858+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39338467,"num_objects":9605,"num_object_clones":0,"num_object_copies":19210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1245,"num_objects_unfound":0,"num_objects_dirty":9605,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7519,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2226,"num_bytes_recovered":9105408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8360},{"shards":"0,2","objects":1245}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":632051137,"num_objects":154313,"num_object_clones":0,"num_object_copies":308626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35972,"num_objects_misplaced":1245,"num_objects_unfound":0,"num_objects_dirty":154313,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":141104,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5932,"num_bytes_recovered":23867392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1316638720,"data_stored":1316554703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160808,"ondisk_log_size":160808,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182464,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":865916,"kb_used_data":523296,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93505924,"statfs":{"total":96636764160,"available":95750066176,"internally_reserved":0,"allocated":535855104,"data_stored":535612416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,1,7,3,0,0,0,0,0,3],"upper_bound":8192},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986236,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":686300,"kb_used_data":424140,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93685540,"statfs":{"total":96636764160,"available":95933992960,"internally_reserved":0,"allocated":434319360,"data_stored":434076677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268431018},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,1,3,0,0,0,0,2],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659183,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":607568,"kb_used_data":341248,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93764272,"statfs":{"total":96636764160,"available":96014614528,"internally_reserved":0,"allocated":349437952,"data_stored":349126158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":272690862},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,4,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":534228992,"data_stored":534225379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":348987392,"data_stored":348914673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":433422336,"data_stored":433414651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:33:58.479 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:33:59.057 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:59.057 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:33:59.081 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":214,"stamp":"2024-09-10T20:33:58.921097+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":623355348,"num_objects":152192,"num_object_clones":0,"num_object_copies":304384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35947,"num_objects_misplaced":1094,"num_objects_unfound":0,"num_objects_dirty":152192,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":143229,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6110,"num_bytes_recovered":24485907,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160934,"ondisk_log_size":160934,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2174768,"kb_used_data":1277312,"kb_used_omap":12,"kb_used_meta":897395,"kb_avail":280940752,"statfs":{"total":289910292480,"available":287683330048,"internally_reserved":0,"allocated":1307967488,"data_stored":1307170323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":918932482},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,0,5,8,11,3,0,0,7],"upper_bound":1024},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27426816,"num_objects":-6696,"num_object_clones":0,"num_object_copies":-13392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-51,"num_objects_misplaced":-436,"num_objects_unfound":0,"num_objects_dirty":-6696,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6696,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":490,"num_bytes_recovered":1785856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004033"},"pg_stats":[{"pgid":"2.f","version":"45'11178","reported_seq":11281,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:57.882195+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:33:57.882195+0000","last_peered":"2024-09-10T20:33:57.882195+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:33:57.882195+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38830080,"num_objects":9480,"num_object_clones":0,"num_object_copies":18960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3596,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9480,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11178,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5884},{"shards":"0,2","objects":2758}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11374","reported_seq":11485,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:58.537580+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:33:58.537580+0000","last_peered":"2024-09-10T20:33:58.537580+0000","last_clean":"2024-09-10T20:33:58.537580+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:33:58.537580+0000","last_undegraded":"2024-09-10T20:33:58.537580+0000","last_fullsized":"2024-09-10T20:33:58.537580+0000","mapping_epoch":35,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39190528,"num_objects":9568,"num_object_clones":0,"num_object_copies":19136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9560,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11083","reported_seq":18431,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:54.120454+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:33:54.120454+0000","last_peered":"2024-09-10T20:33:54.120454+0000","last_clean":"2024-09-10T20:33:54.120454+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:33:54.120454+0000","last_undegraded":"2024-09-10T20:33:54.120454+0000","last_fullsized":"2024-09-10T20:33:54.120454+0000","mapping_epoch":44,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39452696,"num_objects":9633,"num_object_clones":0,"num_object_copies":19266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9633,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9211,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'11183","reported_seq":11264,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:58.701377+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:33:58.701377+0000","last_peered":"2024-09-10T20:33:58.701377+0000","last_clean":"2024-09-10T20:33:58.701377+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:33:58.701377+0000","last_undegraded":"2024-09-10T20:33:58.701377+0000","last_fullsized":"2024-09-10T20:33:58.701377+0000","mapping_epoch":35,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38588416,"num_objects":9421,"num_object_clones":0,"num_object_copies":18842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11183,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11390","reported_seq":11517,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:57.887924+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:33:57.887924+0000","last_peered":"2024-09-10T20:33:57.887924+0000","last_clean":"2024-09-10T20:33:57.887924+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:33:57.887924+0000","last_undegraded":"2024-09-10T20:33:57.887924+0000","last_fullsized":"2024-09-10T20:33:57.887924+0000","mapping_epoch":34,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39645184,"num_objects":9679,"num_object_clones":0,"num_object_copies":19358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8386,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11042","reported_seq":11203,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:57.911101+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:33:57.911101+0000","last_peered":"2024-09-10T20:33:57.911101+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:33:57.911101+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38621184,"num_objects":9429,"num_object_clones":0,"num_object_copies":18858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3604,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7906,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5827},{"shards":"0,2","objects":2810}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11145","reported_seq":11337,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:57.812584+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:33:57.812584+0000","last_peered":"2024-09-10T20:33:57.812584+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:33:57.812584+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38760448,"num_objects":9463,"num_object_clones":0,"num_object_copies":18926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3530,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7740,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5935},{"shards":"0,2","objects":2717}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11171","reported_seq":11296,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:57.957987+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:33:57.957987+0000","last_peered":"2024-09-10T20:33:57.957987+0000","last_clean":"2024-09-10T20:33:57.957987+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:33:57.957987+0000","last_undegraded":"2024-09-10T20:33:57.957987+0000","last_fullsized":"2024-09-10T20:33:57.957987+0000","mapping_epoch":34,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39178240,"num_objects":9565,"num_object_clones":0,"num_object_copies":19130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8242,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11090","reported_seq":11233,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:58.629816+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:33:58.629816+0000","last_peered":"2024-09-10T20:33:58.629816+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:33:58.629816+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38293504,"num_objects":9349,"num_object_clones":0,"num_object_copies":18698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3523,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9083,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2658},{"shards":"2","objects":5828}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190273+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190273+0000","last_peered":"2024-09-10T20:32:18.190273+0000","last_clean":"2024-09-10T20:32:18.190273+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190273+0000","last_undegraded":"2024-09-10T20:32:18.190273+0000","last_fullsized":"2024-09-10T20:32:18.190273+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11205","reported_seq":11342,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:57.847679+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:33:57.847679+0000","last_peered":"2024-09-10T20:33:57.847679+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:33:57.847679+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39015366,"num_objects":9527,"num_object_clones":0,"num_object_copies":19054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3603,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11203,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5926},{"shards":"0,2","objects":2779}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11120","reported_seq":11250,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:58.710336+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:33:58.710336+0000","last_peered":"2024-09-10T20:33:58.710336+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:33:58.710336+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38686720,"num_objects":9445,"num_object_clones":0,"num_object_copies":18890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3632,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8948,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2798},{"shards":"2","objects":5815}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11185","reported_seq":11322,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:58.654380+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:33:58.654380+0000","last_peered":"2024-09-10T20:33:58.654380+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:33:58.654380+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38825984,"num_objects":9479,"num_object_clones":0,"num_object_copies":18958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8971,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2760},{"shards":"2","objects":5873}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11253","reported_seq":11412,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:57.860195+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:33:57.860195+0000","last_peered":"2024-09-10T20:33:57.860195+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:33:57.860195+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39227392,"num_objects":9577,"num_object_clones":0,"num_object_copies":19154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3637,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7976,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5941},{"shards":"0,2","objects":2811}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11120","reported_seq":11295,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:57.970015+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:33:57.970015+0000","last_peered":"2024-09-10T20:33:57.970015+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:33:57.970015+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38764544,"num_objects":9464,"num_object_clones":0,"num_object_copies":18928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7814,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5950},{"shards":"0,2","objects":2708}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193230+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193230+0000","last_peered":"2024-09-10T20:32:18.193230+0000","last_clean":"2024-09-10T20:32:18.193230+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193230+0000","last_undegraded":"2024-09-10T20:32:18.193230+0000","last_fullsized":"2024-09-10T20:32:18.193230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11323","reported_seq":11491,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:57.879865+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:33:57.879865+0000","last_peered":"2024-09-10T20:33:57.879865+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:33:57.879865+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39579648,"num_objects":9663,"num_object_clones":0,"num_object_copies":19326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8150,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5968},{"shards":"0,2","objects":2885}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11069","reported_seq":15990,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:33:57.958421+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:33:57.958421+0000","last_peered":"2024-09-10T20:33:57.958421+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:33:57.958421+0000","last_undegraded":"2024-09-10T20:33:57.958421+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38695395,"num_objects":9448,"num_object_clones":0,"num_object_copies":18896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1094,"num_objects_unfound":0,"num_objects_dirty":9448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7676,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2377,"num_bytes_recovered":9723904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8354},{"shards":"0,2","objects":1094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":623355329,"num_objects":152190,"num_object_clones":0,"num_object_copies":304380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35947,"num_objects_misplaced":1094,"num_objects_unfound":0,"num_objects_dirty":152190,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":143227,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6108,"num_bytes_recovered":24485888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1304875008,"data_stored":1304790991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182465,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":863228,"kb_used_data":516460,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93508612,"statfs":{"total":96636764160,"available":95752818688,"internally_reserved":0,"allocated":528855040,"data_stored":528612352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,1,7,3,0,0,5],"upper_bound":1024},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986237,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703972,"kb_used_data":419604,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93667868,"statfs":{"total":96636764160,"available":95915896832,"internally_reserved":0,"allocated":429674496,"data_stored":429431813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":291172010},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,1,3,0,0,0,0,2],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659183,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":607568,"kb_used_data":341248,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93764272,"statfs":{"total":96636764160,"available":96014614528,"internally_reserved":0,"allocated":349437952,"data_stored":349126158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":272690862},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,4,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":527073280,"data_stored":527069667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":348987392,"data_stored":348914673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":428814336,"data_stored":428806651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:33:59.082 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:33:59.082 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'11178", 'reported_seq': 11281, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:57.882195+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:33:57.882195+0000', 'last_peered': '2024-09-10T20:33:57.882195+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:33:57.882195+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38830080, 'num_objects': 9480, 'num_object_clones': 0, 'num_object_copies': 18960, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3596, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9480, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11178, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 12, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5884}, {'shards': '0,2', 'objects': 2758}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:59.083 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:33:59.083 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'11042", 'reported_seq': 11203, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:57.911101+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:33:57.911101+0000', 'last_peered': '2024-09-10T20:33:57.911101+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:33:57.911101+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "36'1000", 'ondisk_log_start': "36'1000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38621184, 'num_objects': 9429, 'num_object_clones': 0, 'num_object_copies': 18858, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3604, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9429, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7906, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 12, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5827}, {'shards': '0,2', 'objects': 2810}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:59.083 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:33:59.083 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'11145", 'reported_seq': 11337, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:57.812584+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:33:57.812584+0000', 'last_peered': '2024-09-10T20:33:57.812584+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:33:57.812584+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38760448, 'num_objects': 9463, 'num_object_clones': 0, 'num_object_copies': 18926, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3530, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9463, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7740, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5935}, {'shards': '0,2', 'objects': 2717}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:59.083 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:33:59.083 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'11090", 'reported_seq': 11233, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:58.629816+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:33:58.629816+0000', 'last_peered': '2024-09-10T20:33:58.629816+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:33:58.629816+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "36'1000", 'ondisk_log_start': "36'1000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38293504, 'num_objects': 9349, 'num_object_clones': 0, 'num_object_copies': 18698, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3523, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9349, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9083, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 5, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2658}, {'shards': '2', 'objects': 5828}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:33:59.083 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:33:59.083 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'11205", 'reported_seq': 11342, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:57.847679+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:33:57.847679+0000', 'last_peered': '2024-09-10T20:33:57.847679+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:33:57.847679+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "36'1200", 'ondisk_log_start': "36'1200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39015366, 'num_objects': 9527, 'num_object_clones': 0, 'num_object_copies': 19054, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3603, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9527, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11203, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 16, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5926}, {'shards': '0,2', 'objects': 2779}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:59.083 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:33:59.084 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'11120", 'reported_seq': 11250, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:58.710336+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:33:58.710336+0000', 'last_peered': '2024-09-10T20:33:58.710336+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:33:58.710336+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38686720, 'num_objects': 9445, 'num_object_clones': 0, 'num_object_copies': 18890, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3632, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9445, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8948, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2798}, {'shards': '2', 'objects': 5815}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:33:59.084 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:33:59.084 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'11185", 'reported_seq': 11322, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:58.654380+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:33:58.654380+0000', 'last_peered': '2024-09-10T20:33:58.654380+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:33:58.654380+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38825984, 'num_objects': 9479, 'num_object_clones': 0, 'num_object_copies': 18958, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9479, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8971, 'num_write_kb': 32484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2760}, {'shards': '2', 'objects': 5873}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:33:59.084 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:33:59.084 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'11253", 'reported_seq': 11412, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:57.860195+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:33:57.860195+0000', 'last_peered': '2024-09-10T20:33:57.860195+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:33:57.860195+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "36'1200", 'ondisk_log_start': "36'1200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39227392, 'num_objects': 9577, 'num_object_clones': 0, 'num_object_copies': 19154, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3637, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9577, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7976, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 12, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5941}, {'shards': '0,2', 'objects': 2811}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:59.084 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:33:59.084 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'11120", 'reported_seq': 11295, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:57.970015+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:33:57.970015+0000', 'last_peered': '2024-09-10T20:33:57.970015+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:33:57.970015+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38764544, 'num_objects': 9464, 'num_object_clones': 0, 'num_object_copies': 18928, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9464, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7814, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 18, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5950}, {'shards': '0,2', 'objects': 2708}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:59.084 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:33:59.084 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'11323", 'reported_seq': 11491, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:57.879865+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:33:57.879865+0000', 'last_peered': '2024-09-10T20:33:57.879865+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:33:57.879865+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39579648, 'num_objects': 9663, 'num_object_clones': 0, 'num_object_copies': 19326, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9663, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8150, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 18, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5968}, {'shards': '0,2', 'objects': 2885}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:59.084 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:33:59.085 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'11069", 'reported_seq': 15990, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:33:57.958421+0000', 'last_change': '2024-09-10T20:32:16.657652+0000', 'last_active': '2024-09-10T20:33:57.958421+0000', 'last_peered': '2024-09-10T20:33:57.958421+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:33:57.958421+0000', 'last_undegraded': '2024-09-10T20:33:57.958421+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "36'1000", 'ondisk_log_start': "36'1000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38695395, 'num_objects': 9448, 'num_object_clones': 0, 'num_object_copies': 18896, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1094, 'num_objects_unfound': 0, 'num_objects_dirty': 9448, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7676, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2377, 'num_bytes_recovered': 9723904, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 8354}, {'shards': '0,2', 'objects': 1094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:33:59.085 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:33:59.785 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:33:59.785 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35947/304384 objects degraded (11.810%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":351,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":152192,"data_bytes":623355348,"bytes_used":2226962432,"bytes_avail":287683330048,"bytes_total":289910292480,"degraded_objects":35947,"degraded_total":304384,"degraded_ratio":0.11809753469301935,"misplaced_objects":1094,"misplaced_total":304384,"misplaced_ratio":0.0035941442388561819,"recovering_objects_per_sec":40,"recovering_bytes_per_sec":148771,"recovering_keys_per_sec":0,"num_objects_recovered":490,"num_bytes_recovered":1785856,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":557},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:33:59.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:33:59.805 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:34:00.325 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:00.326 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:34:00.348 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":214,"stamp":"2024-09-10T20:33:58.921097+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":623355348,"num_objects":152192,"num_object_clones":0,"num_object_copies":304384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35947,"num_objects_misplaced":1094,"num_objects_unfound":0,"num_objects_dirty":152192,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":143229,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6110,"num_bytes_recovered":24485907,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160934,"ondisk_log_size":160934,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2174768,"kb_used_data":1277312,"kb_used_omap":12,"kb_used_meta":897395,"kb_avail":280940752,"statfs":{"total":289910292480,"available":287683330048,"internally_reserved":0,"allocated":1307967488,"data_stored":1307170323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":918932482},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,0,5,8,11,3,0,0,7],"upper_bound":1024},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27426816,"num_objects":-6696,"num_object_clones":0,"num_object_copies":-13392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-51,"num_objects_misplaced":-436,"num_objects_unfound":0,"num_objects_dirty":-6696,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6696,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":490,"num_bytes_recovered":1785856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004033"},"pg_stats":[{"pgid":"2.f","version":"45'11178","reported_seq":11281,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:57.882195+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:33:57.882195+0000","last_peered":"2024-09-10T20:33:57.882195+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:33:57.882195+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38830080,"num_objects":9480,"num_object_clones":0,"num_object_copies":18960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3596,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9480,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11178,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5884},{"shards":"0,2","objects":2758}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11374","reported_seq":11485,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:58.537580+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:33:58.537580+0000","last_peered":"2024-09-10T20:33:58.537580+0000","last_clean":"2024-09-10T20:33:58.537580+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:33:58.537580+0000","last_undegraded":"2024-09-10T20:33:58.537580+0000","last_fullsized":"2024-09-10T20:33:58.537580+0000","mapping_epoch":35,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39190528,"num_objects":9568,"num_object_clones":0,"num_object_copies":19136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9560,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11083","reported_seq":18431,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:54.120454+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:33:54.120454+0000","last_peered":"2024-09-10T20:33:54.120454+0000","last_clean":"2024-09-10T20:33:54.120454+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:33:54.120454+0000","last_undegraded":"2024-09-10T20:33:54.120454+0000","last_fullsized":"2024-09-10T20:33:54.120454+0000","mapping_epoch":44,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39452696,"num_objects":9633,"num_object_clones":0,"num_object_copies":19266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9633,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9211,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'11183","reported_seq":11264,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:58.701377+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:33:58.701377+0000","last_peered":"2024-09-10T20:33:58.701377+0000","last_clean":"2024-09-10T20:33:58.701377+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:33:58.701377+0000","last_undegraded":"2024-09-10T20:33:58.701377+0000","last_fullsized":"2024-09-10T20:33:58.701377+0000","mapping_epoch":35,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38588416,"num_objects":9421,"num_object_clones":0,"num_object_copies":18842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11183,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11390","reported_seq":11517,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:57.887924+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:33:57.887924+0000","last_peered":"2024-09-10T20:33:57.887924+0000","last_clean":"2024-09-10T20:33:57.887924+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:33:57.887924+0000","last_undegraded":"2024-09-10T20:33:57.887924+0000","last_fullsized":"2024-09-10T20:33:57.887924+0000","mapping_epoch":34,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39645184,"num_objects":9679,"num_object_clones":0,"num_object_copies":19358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8386,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11042","reported_seq":11203,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:57.911101+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:33:57.911101+0000","last_peered":"2024-09-10T20:33:57.911101+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:33:57.911101+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38621184,"num_objects":9429,"num_object_clones":0,"num_object_copies":18858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3604,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7906,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5827},{"shards":"0,2","objects":2810}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11145","reported_seq":11337,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:57.812584+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:33:57.812584+0000","last_peered":"2024-09-10T20:33:57.812584+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:33:57.812584+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38760448,"num_objects":9463,"num_object_clones":0,"num_object_copies":18926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3530,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7740,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5935},{"shards":"0,2","objects":2717}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11171","reported_seq":11296,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:57.957987+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:33:57.957987+0000","last_peered":"2024-09-10T20:33:57.957987+0000","last_clean":"2024-09-10T20:33:57.957987+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:33:57.957987+0000","last_undegraded":"2024-09-10T20:33:57.957987+0000","last_fullsized":"2024-09-10T20:33:57.957987+0000","mapping_epoch":34,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39178240,"num_objects":9565,"num_object_clones":0,"num_object_copies":19130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8242,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11090","reported_seq":11233,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:58.629816+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:33:58.629816+0000","last_peered":"2024-09-10T20:33:58.629816+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:33:58.629816+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38293504,"num_objects":9349,"num_object_clones":0,"num_object_copies":18698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3523,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9083,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2658},{"shards":"2","objects":5828}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190273+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190273+0000","last_peered":"2024-09-10T20:32:18.190273+0000","last_clean":"2024-09-10T20:32:18.190273+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190273+0000","last_undegraded":"2024-09-10T20:32:18.190273+0000","last_fullsized":"2024-09-10T20:32:18.190273+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11205","reported_seq":11342,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:57.847679+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:33:57.847679+0000","last_peered":"2024-09-10T20:33:57.847679+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:33:57.847679+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39015366,"num_objects":9527,"num_object_clones":0,"num_object_copies":19054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3603,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11203,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5926},{"shards":"0,2","objects":2779}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11120","reported_seq":11250,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:58.710336+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:33:58.710336+0000","last_peered":"2024-09-10T20:33:58.710336+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:33:58.710336+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38686720,"num_objects":9445,"num_object_clones":0,"num_object_copies":18890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3632,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8948,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2798},{"shards":"2","objects":5815}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11185","reported_seq":11322,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:58.654380+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:33:58.654380+0000","last_peered":"2024-09-10T20:33:58.654380+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:33:58.654380+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38825984,"num_objects":9479,"num_object_clones":0,"num_object_copies":18958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8971,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2760},{"shards":"2","objects":5873}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11253","reported_seq":11412,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:57.860195+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:33:57.860195+0000","last_peered":"2024-09-10T20:33:57.860195+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:33:57.860195+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39227392,"num_objects":9577,"num_object_clones":0,"num_object_copies":19154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3637,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7976,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5941},{"shards":"0,2","objects":2811}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11120","reported_seq":11295,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:57.970015+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:33:57.970015+0000","last_peered":"2024-09-10T20:33:57.970015+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:33:57.970015+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38764544,"num_objects":9464,"num_object_clones":0,"num_object_copies":18928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7814,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5950},{"shards":"0,2","objects":2708}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193230+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193230+0000","last_peered":"2024-09-10T20:32:18.193230+0000","last_clean":"2024-09-10T20:32:18.193230+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193230+0000","last_undegraded":"2024-09-10T20:32:18.193230+0000","last_fullsized":"2024-09-10T20:32:18.193230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11323","reported_seq":11491,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:57.879865+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:33:57.879865+0000","last_peered":"2024-09-10T20:33:57.879865+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:33:57.879865+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39579648,"num_objects":9663,"num_object_clones":0,"num_object_copies":19326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8150,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5968},{"shards":"0,2","objects":2885}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11069","reported_seq":15990,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:33:57.958421+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:33:57.958421+0000","last_peered":"2024-09-10T20:33:57.958421+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:33:57.958421+0000","last_undegraded":"2024-09-10T20:33:57.958421+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38695395,"num_objects":9448,"num_object_clones":0,"num_object_copies":18896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1094,"num_objects_unfound":0,"num_objects_dirty":9448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7676,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2377,"num_bytes_recovered":9723904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8354},{"shards":"0,2","objects":1094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":623355329,"num_objects":152190,"num_object_clones":0,"num_object_copies":304380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35947,"num_objects_misplaced":1094,"num_objects_unfound":0,"num_objects_dirty":152190,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":143227,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6108,"num_bytes_recovered":24485888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1304875008,"data_stored":1304790991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182465,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":863228,"kb_used_data":516460,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93508612,"statfs":{"total":96636764160,"available":95752818688,"internally_reserved":0,"allocated":528855040,"data_stored":528612352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,1,7,3,0,0,5],"upper_bound":1024},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986237,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703972,"kb_used_data":419604,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93667868,"statfs":{"total":96636764160,"available":95915896832,"internally_reserved":0,"allocated":429674496,"data_stored":429431813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":291172010},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,1,3,0,0,0,0,2],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659183,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":607568,"kb_used_data":341248,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93764272,"statfs":{"total":96636764160,"available":96014614528,"internally_reserved":0,"allocated":349437952,"data_stored":349126158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":272690862},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,4,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":527073280,"data_stored":527069667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":348987392,"data_stored":348914673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":428814336,"data_stored":428806651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:34:03.351 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:34:03.920 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:03.930 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:34:03.952 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":216,"stamp":"2024-09-10T20:34:02.922714+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":622683604,"num_objects":152028,"num_object_clones":0,"num_object_copies":304056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35947,"num_objects_misplaced":1094,"num_objects_unfound":0,"num_objects_dirty":152028,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":143393,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6110,"num_bytes_recovered":24485907,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160898,"ondisk_log_size":160898,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2178268,"kb_used_data":1276652,"kb_used_omap":12,"kb_used_meta":901555,"kb_avail":280937252,"statfs":{"total":289910292480,"available":287679746048,"internally_reserved":0,"allocated":1307291648,"data_stored":1306494483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":923192322},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,0,4,9,10,3,0,0,7],"upper_bound":1024},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18767872,"num_objects":-4582,"num_object_clones":0,"num_object_copies":-9164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-41,"num_objects_misplaced":-279,"num_objects_unfound":0,"num_objects_dirty":-4582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4582,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":323,"num_bytes_recovered":1142784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004313"},"pg_stats":[{"pgid":"2.f","version":"45'11178","reported_seq":11281,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:57.882195+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:33:57.882195+0000","last_peered":"2024-09-10T20:33:57.882195+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:33:57.882195+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38830080,"num_objects":9480,"num_object_clones":0,"num_object_copies":18960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3596,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9480,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11178,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5884},{"shards":"0,2","objects":2758}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11374","reported_seq":11485,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:58.537580+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:33:58.537580+0000","last_peered":"2024-09-10T20:33:58.537580+0000","last_clean":"2024-09-10T20:33:58.537580+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:33:58.537580+0000","last_undegraded":"2024-09-10T20:33:58.537580+0000","last_fullsized":"2024-09-10T20:33:58.537580+0000","mapping_epoch":35,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39190528,"num_objects":9568,"num_object_clones":0,"num_object_copies":19136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9560,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11247","reported_seq":18595,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:59.105992+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:33:59.105992+0000","last_peered":"2024-09-10T20:33:59.105992+0000","last_clean":"2024-09-10T20:33:59.105992+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:33:59.105992+0000","last_undegraded":"2024-09-10T20:33:59.105992+0000","last_fullsized":"2024-09-10T20:33:59.105992+0000","mapping_epoch":44,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38780952,"num_objects":9469,"num_object_clones":0,"num_object_copies":18938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9469,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9375,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'11183","reported_seq":11264,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:58.701377+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:33:58.701377+0000","last_peered":"2024-09-10T20:33:58.701377+0000","last_clean":"2024-09-10T20:33:58.701377+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:33:58.701377+0000","last_undegraded":"2024-09-10T20:33:58.701377+0000","last_fullsized":"2024-09-10T20:33:58.701377+0000","mapping_epoch":35,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38588416,"num_objects":9421,"num_object_clones":0,"num_object_copies":18842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11183,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11390","reported_seq":11517,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:57.887924+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:33:57.887924+0000","last_peered":"2024-09-10T20:33:57.887924+0000","last_clean":"2024-09-10T20:33:57.887924+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:33:57.887924+0000","last_undegraded":"2024-09-10T20:33:57.887924+0000","last_fullsized":"2024-09-10T20:33:57.887924+0000","mapping_epoch":34,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39645184,"num_objects":9679,"num_object_clones":0,"num_object_copies":19358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8386,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11042","reported_seq":11203,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:57.911101+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:33:57.911101+0000","last_peered":"2024-09-10T20:33:57.911101+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:33:57.911101+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38621184,"num_objects":9429,"num_object_clones":0,"num_object_copies":18858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3604,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7906,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5827},{"shards":"0,2","objects":2810}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11145","reported_seq":11337,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:57.812584+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:33:57.812584+0000","last_peered":"2024-09-10T20:33:57.812584+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:33:57.812584+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38760448,"num_objects":9463,"num_object_clones":0,"num_object_copies":18926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3530,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7740,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5935},{"shards":"0,2","objects":2717}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11171","reported_seq":11296,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:57.957987+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:33:57.957987+0000","last_peered":"2024-09-10T20:33:57.957987+0000","last_clean":"2024-09-10T20:33:57.957987+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:33:57.957987+0000","last_undegraded":"2024-09-10T20:33:57.957987+0000","last_fullsized":"2024-09-10T20:33:57.957987+0000","mapping_epoch":34,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39178240,"num_objects":9565,"num_object_clones":0,"num_object_copies":19130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8242,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11090","reported_seq":11233,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:58.629816+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:33:58.629816+0000","last_peered":"2024-09-10T20:33:58.629816+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:33:58.629816+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38293504,"num_objects":9349,"num_object_clones":0,"num_object_copies":18698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3523,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9083,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2658},{"shards":"2","objects":5828}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190273+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190273+0000","last_peered":"2024-09-10T20:32:18.190273+0000","last_clean":"2024-09-10T20:32:18.190273+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190273+0000","last_undegraded":"2024-09-10T20:32:18.190273+0000","last_fullsized":"2024-09-10T20:32:18.190273+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11205","reported_seq":11342,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:57.847679+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:33:57.847679+0000","last_peered":"2024-09-10T20:33:57.847679+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:33:57.847679+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39015366,"num_objects":9527,"num_object_clones":0,"num_object_copies":19054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3603,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11203,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5926},{"shards":"0,2","objects":2779}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11120","reported_seq":11250,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:58.710336+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:33:58.710336+0000","last_peered":"2024-09-10T20:33:58.710336+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:33:58.710336+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38686720,"num_objects":9445,"num_object_clones":0,"num_object_copies":18890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3632,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8948,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2798},{"shards":"2","objects":5815}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11185","reported_seq":11322,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:58.654380+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:33:58.654380+0000","last_peered":"2024-09-10T20:33:58.654380+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:33:58.654380+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38825984,"num_objects":9479,"num_object_clones":0,"num_object_copies":18958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8971,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2760},{"shards":"2","objects":5873}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11253","reported_seq":11412,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:57.860195+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:33:57.860195+0000","last_peered":"2024-09-10T20:33:57.860195+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:33:57.860195+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39227392,"num_objects":9577,"num_object_clones":0,"num_object_copies":19154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3637,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7976,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5941},{"shards":"0,2","objects":2811}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11120","reported_seq":11295,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:57.970015+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:33:57.970015+0000","last_peered":"2024-09-10T20:33:57.970015+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:33:57.970015+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38764544,"num_objects":9464,"num_object_clones":0,"num_object_copies":18928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7814,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5950},{"shards":"0,2","objects":2708}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193230+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193230+0000","last_peered":"2024-09-10T20:32:18.193230+0000","last_clean":"2024-09-10T20:32:18.193230+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193230+0000","last_undegraded":"2024-09-10T20:32:18.193230+0000","last_fullsized":"2024-09-10T20:32:18.193230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11323","reported_seq":11491,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:57.879865+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:33:57.879865+0000","last_peered":"2024-09-10T20:33:57.879865+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:33:57.879865+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39579648,"num_objects":9663,"num_object_clones":0,"num_object_copies":19326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8150,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5968},{"shards":"0,2","objects":2885}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11069","reported_seq":15990,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:33:57.958421+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:33:57.958421+0000","last_peered":"2024-09-10T20:33:57.958421+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:33:57.958421+0000","last_undegraded":"2024-09-10T20:33:57.958421+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38695395,"num_objects":9448,"num_object_clones":0,"num_object_copies":18896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1094,"num_objects_unfound":0,"num_objects_dirty":9448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7676,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2377,"num_bytes_recovered":9723904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8354},{"shards":"0,2","objects":1094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":622683585,"num_objects":152026,"num_object_clones":0,"num_object_copies":304052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35947,"num_objects_misplaced":1094,"num_objects_unfound":0,"num_objects_dirty":152026,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":143391,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6108,"num_bytes_recovered":24485888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1304182784,"data_stored":1304098767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160895,"ondisk_log_size":160895,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182465,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":863228,"kb_used_data":516460,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93508612,"statfs":{"total":96636764160,"available":95752818688,"internally_reserved":0,"allocated":528855040,"data_stored":528612352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,1,7,3,0,0,5],"upper_bound":1024},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986237,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703972,"kb_used_data":419604,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93667868,"statfs":{"total":96636764160,"available":95915896832,"internally_reserved":0,"allocated":429674496,"data_stored":429431813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":291172010},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,1,3,0,0,0,0,2],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659184,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":611068,"kb_used_data":340588,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93760772,"statfs":{"total":96636764160,"available":96011030528,"internally_reserved":0,"allocated":348762112,"data_stored":348450318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":276950702},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,5,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":527073280,"data_stored":527069667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":348295168,"data_stored":348222449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":428814336,"data_stored":428806651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:34:03.953 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:34:04.530 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:04.533 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:34:04.557 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":216,"stamp":"2024-09-10T20:34:02.922714+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":622683604,"num_objects":152028,"num_object_clones":0,"num_object_copies":304056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35947,"num_objects_misplaced":1094,"num_objects_unfound":0,"num_objects_dirty":152028,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":143393,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6110,"num_bytes_recovered":24485907,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160898,"ondisk_log_size":160898,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2178268,"kb_used_data":1276652,"kb_used_omap":12,"kb_used_meta":901555,"kb_avail":280937252,"statfs":{"total":289910292480,"available":287679746048,"internally_reserved":0,"allocated":1307291648,"data_stored":1306494483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":923192322},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,0,4,9,10,3,0,0,7],"upper_bound":1024},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18767872,"num_objects":-4582,"num_object_clones":0,"num_object_copies":-9164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-41,"num_objects_misplaced":-279,"num_objects_unfound":0,"num_objects_dirty":-4582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4582,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":323,"num_bytes_recovered":1142784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004313"},"pg_stats":[{"pgid":"2.f","version":"45'11178","reported_seq":11281,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:57.882195+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:33:57.882195+0000","last_peered":"2024-09-10T20:33:57.882195+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:33:57.882195+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38830080,"num_objects":9480,"num_object_clones":0,"num_object_copies":18960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3596,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9480,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11178,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5884},{"shards":"0,2","objects":2758}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11374","reported_seq":11485,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:58.537580+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:33:58.537580+0000","last_peered":"2024-09-10T20:33:58.537580+0000","last_clean":"2024-09-10T20:33:58.537580+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:33:58.537580+0000","last_undegraded":"2024-09-10T20:33:58.537580+0000","last_fullsized":"2024-09-10T20:33:58.537580+0000","mapping_epoch":35,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39190528,"num_objects":9568,"num_object_clones":0,"num_object_copies":19136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9560,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11247","reported_seq":18595,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:59.105992+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:33:59.105992+0000","last_peered":"2024-09-10T20:33:59.105992+0000","last_clean":"2024-09-10T20:33:59.105992+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:33:59.105992+0000","last_undegraded":"2024-09-10T20:33:59.105992+0000","last_fullsized":"2024-09-10T20:33:59.105992+0000","mapping_epoch":44,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38780952,"num_objects":9469,"num_object_clones":0,"num_object_copies":18938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9469,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9375,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'11183","reported_seq":11264,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:58.701377+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:33:58.701377+0000","last_peered":"2024-09-10T20:33:58.701377+0000","last_clean":"2024-09-10T20:33:58.701377+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:33:58.701377+0000","last_undegraded":"2024-09-10T20:33:58.701377+0000","last_fullsized":"2024-09-10T20:33:58.701377+0000","mapping_epoch":35,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38588416,"num_objects":9421,"num_object_clones":0,"num_object_copies":18842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11183,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11390","reported_seq":11517,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:57.887924+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:33:57.887924+0000","last_peered":"2024-09-10T20:33:57.887924+0000","last_clean":"2024-09-10T20:33:57.887924+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:33:57.887924+0000","last_undegraded":"2024-09-10T20:33:57.887924+0000","last_fullsized":"2024-09-10T20:33:57.887924+0000","mapping_epoch":34,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39645184,"num_objects":9679,"num_object_clones":0,"num_object_copies":19358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8386,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11042","reported_seq":11203,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:57.911101+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:33:57.911101+0000","last_peered":"2024-09-10T20:33:57.911101+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:33:57.911101+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38621184,"num_objects":9429,"num_object_clones":0,"num_object_copies":18858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3604,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7906,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5827},{"shards":"0,2","objects":2810}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11145","reported_seq":11337,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:57.812584+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:33:57.812584+0000","last_peered":"2024-09-10T20:33:57.812584+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:33:57.812584+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38760448,"num_objects":9463,"num_object_clones":0,"num_object_copies":18926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3530,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7740,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5935},{"shards":"0,2","objects":2717}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11171","reported_seq":11296,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:33:57.957987+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:33:57.957987+0000","last_peered":"2024-09-10T20:33:57.957987+0000","last_clean":"2024-09-10T20:33:57.957987+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:33:57.957987+0000","last_undegraded":"2024-09-10T20:33:57.957987+0000","last_fullsized":"2024-09-10T20:33:57.957987+0000","mapping_epoch":34,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39178240,"num_objects":9565,"num_object_clones":0,"num_object_copies":19130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8242,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187395+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187395+0000","last_peered":"2024-09-10T20:32:18.187395+0000","last_clean":"2024-09-10T20:32:18.187395+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187395+0000","last_undegraded":"2024-09-10T20:32:18.187395+0000","last_fullsized":"2024-09-10T20:32:18.187395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11090","reported_seq":11233,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:58.629816+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:33:58.629816+0000","last_peered":"2024-09-10T20:33:58.629816+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:33:58.629816+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38293504,"num_objects":9349,"num_object_clones":0,"num_object_copies":18698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3523,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9083,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2658},{"shards":"2","objects":5828}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190273+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190273+0000","last_peered":"2024-09-10T20:32:18.190273+0000","last_clean":"2024-09-10T20:32:18.190273+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190273+0000","last_undegraded":"2024-09-10T20:32:18.190273+0000","last_fullsized":"2024-09-10T20:32:18.190273+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11205","reported_seq":11342,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:57.847679+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:33:57.847679+0000","last_peered":"2024-09-10T20:33:57.847679+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:33:57.847679+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39015366,"num_objects":9527,"num_object_clones":0,"num_object_copies":19054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3603,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11203,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5926},{"shards":"0,2","objects":2779}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11120","reported_seq":11250,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:58.710336+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:33:58.710336+0000","last_peered":"2024-09-10T20:33:58.710336+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:33:58.710336+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38686720,"num_objects":9445,"num_object_clones":0,"num_object_copies":18890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3632,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8948,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2798},{"shards":"2","objects":5815}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11185","reported_seq":11322,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:58.654380+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:33:58.654380+0000","last_peered":"2024-09-10T20:33:58.654380+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:33:58.654380+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38825984,"num_objects":9479,"num_object_clones":0,"num_object_copies":18958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8971,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2760},{"shards":"2","objects":5873}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11253","reported_seq":11412,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:57.860195+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:33:57.860195+0000","last_peered":"2024-09-10T20:33:57.860195+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:33:57.860195+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39227392,"num_objects":9577,"num_object_clones":0,"num_object_copies":19154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3637,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7976,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5941},{"shards":"0,2","objects":2811}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11120","reported_seq":11295,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:57.970015+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:33:57.970015+0000","last_peered":"2024-09-10T20:33:57.970015+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:33:57.970015+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38764544,"num_objects":9464,"num_object_clones":0,"num_object_copies":18928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7814,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5950},{"shards":"0,2","objects":2708}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193230+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193230+0000","last_peered":"2024-09-10T20:32:18.193230+0000","last_clean":"2024-09-10T20:32:18.193230+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193230+0000","last_undegraded":"2024-09-10T20:32:18.193230+0000","last_fullsized":"2024-09-10T20:32:18.193230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11323","reported_seq":11491,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:33:57.879865+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:33:57.879865+0000","last_peered":"2024-09-10T20:33:57.879865+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:33:57.879865+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39579648,"num_objects":9663,"num_object_clones":0,"num_object_copies":19326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8150,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5968},{"shards":"0,2","objects":2885}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11069","reported_seq":15990,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:33:57.958421+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:33:57.958421+0000","last_peered":"2024-09-10T20:33:57.958421+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:33:57.958421+0000","last_undegraded":"2024-09-10T20:33:57.958421+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38695395,"num_objects":9448,"num_object_clones":0,"num_object_copies":18896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1094,"num_objects_unfound":0,"num_objects_dirty":9448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7676,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2377,"num_bytes_recovered":9723904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8354},{"shards":"0,2","objects":1094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":622683585,"num_objects":152026,"num_object_clones":0,"num_object_copies":304052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35947,"num_objects_misplaced":1094,"num_objects_unfound":0,"num_objects_dirty":152026,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":143391,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6108,"num_bytes_recovered":24485888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1304182784,"data_stored":1304098767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160895,"ondisk_log_size":160895,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182465,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":863228,"kb_used_data":516460,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93508612,"statfs":{"total":96636764160,"available":95752818688,"internally_reserved":0,"allocated":528855040,"data_stored":528612352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,1,7,3,0,0,5],"upper_bound":1024},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986237,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703972,"kb_used_data":419604,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93667868,"statfs":{"total":96636764160,"available":95915896832,"internally_reserved":0,"allocated":429674496,"data_stored":429431813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":291172010},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,1,3,0,0,0,0,2],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659184,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":611068,"kb_used_data":340588,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93760772,"statfs":{"total":96636764160,"available":96011030528,"internally_reserved":0,"allocated":348762112,"data_stored":348450318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":276950702},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,5,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":527073280,"data_stored":527069667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":348295168,"data_stored":348222449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":428814336,"data_stored":428806651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:34:04.558 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:34:04.558 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'11178", 'reported_seq': 11281, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:57.882195+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:33:57.882195+0000', 'last_peered': '2024-09-10T20:33:57.882195+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:33:57.882195+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38830080, 'num_objects': 9480, 'num_object_clones': 0, 'num_object_copies': 18960, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3596, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9480, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11178, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 12, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5884}, {'shards': '0,2', 'objects': 2758}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:04.558 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:34:04.558 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'11042", 'reported_seq': 11203, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:57.911101+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:33:57.911101+0000', 'last_peered': '2024-09-10T20:33:57.911101+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:33:57.911101+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "36'1000", 'ondisk_log_start': "36'1000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38621184, 'num_objects': 9429, 'num_object_clones': 0, 'num_object_copies': 18858, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3604, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9429, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7906, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 12, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5827}, {'shards': '0,2', 'objects': 2810}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:04.558 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:34:04.558 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'11145", 'reported_seq': 11337, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:57.812584+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:33:57.812584+0000', 'last_peered': '2024-09-10T20:33:57.812584+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:33:57.812584+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38760448, 'num_objects': 9463, 'num_object_clones': 0, 'num_object_copies': 18926, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3530, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9463, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7740, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5935}, {'shards': '0,2', 'objects': 2717}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:04.559 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:34:04.559 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'11090", 'reported_seq': 11233, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:58.629816+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:33:58.629816+0000', 'last_peered': '2024-09-10T20:33:58.629816+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:33:58.629816+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "36'1000", 'ondisk_log_start': "36'1000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38293504, 'num_objects': 9349, 'num_object_clones': 0, 'num_object_copies': 18698, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3523, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9349, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9083, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 5, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2658}, {'shards': '2', 'objects': 5828}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:34:04.559 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:34:04.559 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'11205", 'reported_seq': 11342, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:57.847679+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:33:57.847679+0000', 'last_peered': '2024-09-10T20:33:57.847679+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:33:57.847679+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "36'1200", 'ondisk_log_start': "36'1200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39015366, 'num_objects': 9527, 'num_object_clones': 0, 'num_object_copies': 19054, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3603, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9527, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11203, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 16, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5926}, {'shards': '0,2', 'objects': 2779}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:04.559 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:34:04.559 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'11120", 'reported_seq': 11250, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:58.710336+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:33:58.710336+0000', 'last_peered': '2024-09-10T20:33:58.710336+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:33:58.710336+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38686720, 'num_objects': 9445, 'num_object_clones': 0, 'num_object_copies': 18890, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3632, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9445, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8948, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2798}, {'shards': '2', 'objects': 5815}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:34:04.559 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:34:04.559 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'11185", 'reported_seq': 11322, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:58.654380+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:33:58.654380+0000', 'last_peered': '2024-09-10T20:33:58.654380+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:33:58.654380+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38825984, 'num_objects': 9479, 'num_object_clones': 0, 'num_object_copies': 18958, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9479, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8971, 'num_write_kb': 32484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2760}, {'shards': '2', 'objects': 5873}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:34:04.559 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:34:04.559 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'11253", 'reported_seq': 11412, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:57.860195+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:33:57.860195+0000', 'last_peered': '2024-09-10T20:33:57.860195+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:33:57.860195+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "36'1200", 'ondisk_log_start': "36'1200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39227392, 'num_objects': 9577, 'num_object_clones': 0, 'num_object_copies': 19154, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3637, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9577, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7976, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 12, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5941}, {'shards': '0,2', 'objects': 2811}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:04.560 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:34:04.560 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'11120", 'reported_seq': 11295, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:57.970015+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:33:57.970015+0000', 'last_peered': '2024-09-10T20:33:57.970015+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:33:57.970015+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38764544, 'num_objects': 9464, 'num_object_clones': 0, 'num_object_copies': 18928, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9464, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7814, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 18, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5950}, {'shards': '0,2', 'objects': 2708}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:04.560 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:34:04.560 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'11323", 'reported_seq': 11491, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:33:57.879865+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:33:57.879865+0000', 'last_peered': '2024-09-10T20:33:57.879865+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:33:57.879865+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39579648, 'num_objects': 9663, 'num_object_clones': 0, 'num_object_copies': 19326, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9663, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8150, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 18, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5968}, {'shards': '0,2', 'objects': 2885}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:04.560 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:34:04.560 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'11069", 'reported_seq': 15990, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:33:57.958421+0000', 'last_change': '2024-09-10T20:32:16.657652+0000', 'last_active': '2024-09-10T20:33:57.958421+0000', 'last_peered': '2024-09-10T20:33:57.958421+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:33:57.958421+0000', 'last_undegraded': '2024-09-10T20:33:57.958421+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "36'1000", 'ondisk_log_start': "36'1000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38695395, 'num_objects': 9448, 'num_object_clones': 0, 'num_object_copies': 18896, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1094, 'num_objects_unfound': 0, 'num_objects_dirty': 9448, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7676, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2377, 'num_bytes_recovered': 9723904, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 8354}, {'shards': '0,2', 'objects': 1094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:04.560 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:34:05.253 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:05.253 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35947/304056 objects degraded (11.822%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":356,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":152028,"data_bytes":622683604,"bytes_used":2230546432,"bytes_avail":287679746048,"bytes_total":289910292480,"degraded_objects":35947,"degraded_total":304056,"degraded_ratio":0.1182249322493225,"misplaced_objects":1094,"misplaced_total":304056,"misplaced_ratio":0.003598021417107375,"recovering_objects_per_sec":26,"recovering_bytes_per_sec":95197,"recovering_keys_per_sec":0,"num_objects_recovered":323,"num_bytes_recovered":1142784,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":381},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:34:05.286 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:34:05.287 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:34:05.870 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:05.871 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:34:05.897 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":217,"stamp":"2024-09-10T20:34:04.925805+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":613361108,"num_objects":149752,"num_object_clones":0,"num_object_copies":299504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35918,"num_objects_misplaced":925,"num_objects_unfound":0,"num_objects_dirty":149752,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":145669,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6308,"num_bytes_recovered":25178131,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160674,"ondisk_log_size":160674,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2154052,"kb_used_data":1264904,"kb_used_omap":12,"kb_used_meta":889075,"kb_avail":280961468,"statfs":{"total":289910292480,"available":287704543232,"internally_reserved":0,"allocated":1295261696,"data_stored":1294464531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":910412802},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,6,6,7,8,4,0,0,3],"upper_bound":1024},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28090368,"num_objects":-6858,"num_object_clones":0,"num_object_copies":-13716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-70,"num_objects_misplaced":-448,"num_objects_unfound":0,"num_objects_dirty":-6858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6858,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":521,"num_bytes_recovered":1835008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006882"},"pg_stats":[{"pgid":"2.f","version":"45'11332","reported_seq":11443,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:02.979399+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:34:02.979399+0000","last_peered":"2024-09-10T20:34:02.979399+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:34:02.979399+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38199296,"num_objects":9326,"num_object_clones":0,"num_object_copies":18652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3592,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11332,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5734},{"shards":"0,2","objects":2604}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11531","reported_seq":11642,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:03.724141+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:34:03.724141+0000","last_peered":"2024-09-10T20:34:03.724141+0000","last_clean":"2024-09-10T20:34:03.724141+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:34:03.724141+0000","last_undegraded":"2024-09-10T20:34:03.724141+0000","last_fullsized":"2024-09-10T20:34:03.724141+0000","mapping_epoch":35,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38547456,"num_objects":9411,"num_object_clones":0,"num_object_copies":18822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9717,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11374","reported_seq":18722,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:04.045583+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:34:04.045583+0000","last_peered":"2024-09-10T20:34:04.045583+0000","last_clean":"2024-09-10T20:34:04.045583+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:34:04.045583+0000","last_undegraded":"2024-09-10T20:34:04.045583+0000","last_fullsized":"2024-09-10T20:34:04.045583+0000","mapping_epoch":44,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38260760,"num_objects":9342,"num_object_clones":0,"num_object_copies":18684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9342,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9502,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'11317","reported_seq":11398,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:03.745562+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:34:03.745562+0000","last_peered":"2024-09-10T20:34:03.745562+0000","last_clean":"2024-09-10T20:34:03.745562+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:34:03.745562+0000","last_undegraded":"2024-09-10T20:34:03.745562+0000","last_fullsized":"2024-09-10T20:34:03.745562+0000","mapping_epoch":35,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38039552,"num_objects":9287,"num_object_clones":0,"num_object_copies":18574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11317,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11539","reported_seq":11665,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:02.951101+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:34:02.951101+0000","last_peered":"2024-09-10T20:34:02.951101+0000","last_clean":"2024-09-10T20:34:02.951101+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:34:02.951101+0000","last_undegraded":"2024-09-10T20:34:02.951101+0000","last_fullsized":"2024-09-10T20:34:02.951101+0000","mapping_epoch":34,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39034880,"num_objects":9530,"num_object_clones":0,"num_object_copies":19060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8535,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11203","reported_seq":11378,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:02.969466+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:34:02.969466+0000","last_peered":"2024-09-10T20:34:02.969466+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:34:02.969466+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37961728,"num_objects":9268,"num_object_clones":0,"num_object_copies":18536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3597,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8067,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5673},{"shards":"0,2","objects":2649}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11289","reported_seq":11487,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:02.975819+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:34:02.975819+0000","last_peered":"2024-09-10T20:34:02.975819+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:34:02.975819+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38170624,"num_objects":9319,"num_object_clones":0,"num_object_copies":18638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7884,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5794},{"shards":"0,2","objects":2573}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11307","reported_seq":11432,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:02.974456+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:34:02.974456+0000","last_peered":"2024-09-10T20:34:02.974456+0000","last_clean":"2024-09-10T20:34:02.974456+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:34:02.974456+0000","last_undegraded":"2024-09-10T20:34:02.974456+0000","last_fullsized":"2024-09-10T20:34:02.974456+0000","mapping_epoch":34,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38621184,"num_objects":9429,"num_object_clones":0,"num_object_copies":18858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8378,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187449+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187449+0000","last_peered":"2024-09-10T20:32:18.187449+0000","last_clean":"2024-09-10T20:32:18.187449+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187449+0000","last_undegraded":"2024-09-10T20:32:18.187449+0000","last_fullsized":"2024-09-10T20:32:18.187449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11229","reported_seq":11372,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:03.751467+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:34:03.751467+0000","last_peered":"2024-09-10T20:34:03.751467+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:34:03.751467+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37724160,"num_objects":9210,"num_object_clones":0,"num_object_copies":18420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3523,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9222,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2519},{"shards":"2","objects":5689}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190273+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190273+0000","last_peered":"2024-09-10T20:32:18.190273+0000","last_clean":"2024-09-10T20:32:18.190273+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190273+0000","last_undegraded":"2024-09-10T20:32:18.190273+0000","last_fullsized":"2024-09-10T20:32:18.190273+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11342","reported_seq":11489,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:02.922452+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:34:02.922452+0000","last_peered":"2024-09-10T20:34:02.922452+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:34:02.922452+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38454214,"num_objects":9390,"num_object_clones":0,"num_object_copies":18780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3598,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11340,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5794},{"shards":"0,2","objects":2642}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11273","reported_seq":11408,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:03.691927+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:34:03.691927+0000","last_peered":"2024-09-10T20:34:03.691927+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:34:03.691927+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38060032,"num_objects":9292,"num_object_clones":0,"num_object_copies":18584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3632,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9101,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2645},{"shards":"2","objects":5662}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11331","reported_seq":11468,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:03.654141+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:34:03.654141+0000","last_peered":"2024-09-10T20:34:03.654141+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:34:03.654141+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38227968,"num_objects":9333,"num_object_clones":0,"num_object_copies":18666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9117,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2614},{"shards":"2","objects":5727}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11398","reported_seq":11568,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:02.968719+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:34:02.968719+0000","last_peered":"2024-09-10T20:34:02.968719+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:34:02.968719+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38633472,"num_objects":9432,"num_object_clones":0,"num_object_copies":18864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3631,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8121,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5802},{"shards":"0,2","objects":2666}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11255","reported_seq":11434,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:02.911997+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:34:02.911997+0000","last_peered":"2024-09-10T20:34:02.911997+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:34:02.911997+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38211584,"num_objects":9329,"num_object_clones":0,"num_object_copies":18658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3515,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7949,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5817},{"shards":"0,2","objects":2573}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193230+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193230+0000","last_peered":"2024-09-10T20:32:18.193230+0000","last_clean":"2024-09-10T20:32:18.193230+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193230+0000","last_undegraded":"2024-09-10T20:32:18.193230+0000","last_fullsized":"2024-09-10T20:32:18.193230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11449","reported_seq":11621,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:02.947839+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:34:02.947839+0000","last_peered":"2024-09-10T20:34:02.947839+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:34:02.947839+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39063552,"num_objects":9537,"num_object_clones":0,"num_object_copies":19074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3695,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8276,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5844},{"shards":"0,2","objects":2759}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11202","reported_seq":16461,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:34:02.990349+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:34:02.990349+0000","last_peered":"2024-09-10T20:34:02.990349+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:34:02.990349+0000","last_undegraded":"2024-09-10T20:34:02.990349+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38150627,"num_objects":9315,"num_object_clones":0,"num_object_copies":18630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":925,"num_objects_unfound":0,"num_objects_dirty":9315,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7809,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2546,"num_bytes_recovered":10416128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8390},{"shards":"0,2","objects":925}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":613361089,"num_objects":149750,"num_object_clones":0,"num_object_copies":299500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35918,"num_objects_misplaced":925,"num_objects_unfound":0,"num_objects_dirty":149750,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":145667,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6306,"num_bytes_recovered":25178112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1292165120,"data_stored":1292081103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160671,"ondisk_log_size":160671,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182466,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":848060,"kb_used_data":509608,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93523780,"statfs":{"total":96636764160,"available":95768350720,"internally_reserved":0,"allocated":521838592,"data_stored":521595904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,2,4,2,3,0,0,3],"upper_bound":1024},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986238,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703620,"kb_used_data":415084,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93668220,"statfs":{"total":96636764160,"available":95916257280,"internally_reserved":0,"allocated":425046016,"data_stored":424803333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":295431850},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659185,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":602372,"kb_used_data":340212,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93769468,"statfs":{"total":96636764160,"available":96019935232,"internally_reserved":0,"allocated":348377088,"data_stored":348065294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":268431022},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,3,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":520065024,"data_stored":520061411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":347951104,"data_stored":347878385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":424148992,"data_stored":424141307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:34:08.899 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:34:09.434 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:09.434 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:34:09.453 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":219,"stamp":"2024-09-10T20:34:08.927765+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":604751316,"num_objects":147650,"num_object_clones":0,"num_object_copies":295300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35894,"num_objects_misplaced":719,"num_objects_unfound":0,"num_objects_dirty":147650,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":147771,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6538,"num_bytes_recovered":26021907,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160976,"ondisk_log_size":160976,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2151024,"kb_used_data":1253556,"kb_used_omap":12,"kb_used_meta":897395,"kb_avail":280964496,"statfs":{"total":289910292480,"available":287707643904,"internally_reserved":0,"allocated":1283641344,"data_stored":1282844179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":918932482},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,3,2,7,6,4,3,0,0,0,6],"upper_bound":4096},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27299840,"num_objects":-6665,"num_object_clones":0,"num_object_copies":-13330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-78,"num_objects_misplaced":-526,"num_objects_unfound":0,"num_objects_dirty":-6665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6665,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":604,"num_bytes_recovered":2154496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007409"},"pg_stats":[{"pgid":"2.f","version":"45'11474","reported_seq":11595,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:07.984176+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:34:07.984176+0000","last_peered":"2024-09-10T20:34:07.984176+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:34:07.984176+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37617664,"num_objects":9184,"num_object_clones":0,"num_object_copies":18368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3587,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11474,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5597},{"shards":"0,2","objects":2462}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11685","reported_seq":11796,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:08.730146+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:34:08.730146+0000","last_peered":"2024-09-10T20:34:08.730146+0000","last_clean":"2024-09-10T20:34:08.730146+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:34:08.730146+0000","last_undegraded":"2024-09-10T20:34:08.730146+0000","last_fullsized":"2024-09-10T20:34:08.730146+0000","mapping_epoch":35,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37916672,"num_objects":9257,"num_object_clones":0,"num_object_copies":18514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9871,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11374","reported_seq":18722,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:04.045583+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:34:04.045583+0000","last_peered":"2024-09-10T20:34:04.045583+0000","last_clean":"2024-09-10T20:34:04.045583+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:34:04.045583+0000","last_undegraded":"2024-09-10T20:34:04.045583+0000","last_fullsized":"2024-09-10T20:34:04.045583+0000","mapping_epoch":44,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38260760,"num_objects":9342,"num_object_clones":0,"num_object_copies":18684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9342,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9502,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'11445","reported_seq":11526,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:08.724268+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:34:08.724268+0000","last_peered":"2024-09-10T20:34:08.724268+0000","last_clean":"2024-09-10T20:34:08.724268+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:34:08.724268+0000","last_undegraded":"2024-09-10T20:34:08.724268+0000","last_fullsized":"2024-09-10T20:34:08.724268+0000","mapping_epoch":35,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37515264,"num_objects":9159,"num_object_clones":0,"num_object_copies":18318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11445,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11697","reported_seq":11824,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:07.998829+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:34:07.998829+0000","last_peered":"2024-09-10T20:34:07.998829+0000","last_clean":"2024-09-10T20:34:07.998829+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:34:07.998829+0000","last_undegraded":"2024-09-10T20:34:07.998829+0000","last_fullsized":"2024-09-10T20:34:07.998829+0000","mapping_epoch":34,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38387712,"num_objects":9372,"num_object_clones":0,"num_object_copies":18744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8693,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11332","reported_seq":11509,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:07.989315+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:34:07.989315+0000","last_peered":"2024-09-10T20:34:07.989315+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:34:07.989315+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37433344,"num_objects":9139,"num_object_clones":0,"num_object_copies":18278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3596,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8196,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5545},{"shards":"0,2","objects":2520}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11421","reported_seq":11627,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:07.923440+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:34:07.923440+0000","last_peered":"2024-09-10T20:34:07.923440+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:34:07.923440+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629952,"num_objects":9187,"num_object_clones":0,"num_object_copies":18374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3523,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8016,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5666},{"shards":"0,2","objects":2441}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11473","reported_seq":11598,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:08.025004+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:34:08.025004+0000","last_peered":"2024-09-10T20:34:08.025004+0000","last_clean":"2024-09-10T20:34:08.025004+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:34:08.025004+0000","last_undegraded":"2024-09-10T20:34:08.025004+0000","last_fullsized":"2024-09-10T20:34:08.025004+0000","mapping_epoch":34,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37941248,"num_objects":9263,"num_object_clones":0,"num_object_copies":18526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8544,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187449+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187449+0000","last_peered":"2024-09-10T20:32:18.187449+0000","last_clean":"2024-09-10T20:32:18.187449+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187449+0000","last_undegraded":"2024-09-10T20:32:18.187449+0000","last_fullsized":"2024-09-10T20:32:18.187449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11338","reported_seq":11483,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:08.783045+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:34:08.783045+0000","last_peered":"2024-09-10T20:34:08.783045+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:34:08.783045+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37277696,"num_objects":9101,"num_object_clones":0,"num_object_copies":18202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3522,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9331,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2410},{"shards":"2","objects":5581}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190273+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190273+0000","last_peered":"2024-09-10T20:32:18.190273+0000","last_clean":"2024-09-10T20:32:18.190273+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190273+0000","last_undegraded":"2024-09-10T20:32:18.190273+0000","last_fullsized":"2024-09-10T20:32:18.190273+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11487","reported_seq":11638,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:07.992533+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:34:07.992533+0000","last_peered":"2024-09-10T20:34:07.992533+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:34:07.992533+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37860294,"num_objects":9245,"num_object_clones":0,"num_object_copies":18490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3596,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11485,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5651},{"shards":"0,2","objects":2497}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11443","reported_seq":11588,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:08.583236+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:34:08.583236+0000","last_peered":"2024-09-10T20:34:08.583236+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:34:08.583236+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37363712,"num_objects":9122,"num_object_clones":0,"num_object_copies":18244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3627,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9271,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2475},{"shards":"2","objects":5497}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11470","reported_seq":11609,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:08.669019+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:34:08.669019+0000","last_peered":"2024-09-10T20:34:08.669019+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:34:08.669019+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37658624,"num_objects":9194,"num_object_clones":0,"num_object_copies":18388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9256,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2475},{"shards":"2","objects":5589}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11527","reported_seq":11702,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:08.006495+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:34:08.006495+0000","last_peered":"2024-09-10T20:34:08.006495+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:34:08.006495+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38105088,"num_objects":9303,"num_object_clones":0,"num_object_copies":18606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8250,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5675},{"shards":"0,2","objects":2537}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11396","reported_seq":11579,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:07.950783+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:34:07.950783+0000","last_peered":"2024-09-10T20:34:07.950783+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:34:07.950783+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37634048,"num_objects":9188,"num_object_clones":0,"num_object_copies":18376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3513,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8090,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5678},{"shards":"0,2","objects":2432}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193230+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193230+0000","last_peered":"2024-09-10T20:32:18.193230+0000","last_clean":"2024-09-10T20:32:18.193230+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193230+0000","last_undegraded":"2024-09-10T20:32:18.193230+0000","last_fullsized":"2024-09-10T20:32:18.193230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11594","reported_seq":11768,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:07.876728+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:34:07.876728+0000","last_peered":"2024-09-10T20:34:07.876728+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:34:07.876728+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38469632,"num_objects":9392,"num_object_clones":0,"num_object_copies":18784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3694,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8421,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5700},{"shards":"0,2","objects":2614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11317","reported_seq":16988,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:34:08.031953+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:34:08.031953+0000","last_peered":"2024-09-10T20:34:08.031953+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:34:08.031953+0000","last_undegraded":"2024-09-10T20:34:08.031953+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37679587,"num_objects":9200,"num_object_clones":0,"num_object_copies":18400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":719,"num_objects_unfound":0,"num_objects_dirty":9200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7924,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2752,"num_bytes_recovered":11259904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8481},{"shards":"0,2","objects":719}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":604751297,"num_objects":147648,"num_object_clones":0,"num_object_copies":295296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35894,"num_objects_misplaced":719,"num_objects_unfound":0,"num_objects_dirty":147648,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":147769,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6536,"num_bytes_recovered":26021888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1280548864,"data_stored":1280464847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160973,"ondisk_log_size":160973,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182467,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":845388,"kb_used_data":502780,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93526452,"statfs":{"total":96636764160,"available":95771086848,"internally_reserved":0,"allocated":514846720,"data_stored":514604032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,0,3,1,0,0,0,0,0,6],"upper_bound":4096},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986239,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703264,"kb_used_data":410564,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93668576,"statfs":{"total":96636764160,"available":95916621824,"internally_reserved":0,"allocated":420417536,"data_stored":420174853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":299691690},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,1,0,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659185,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":602372,"kb_used_data":340212,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93769468,"statfs":{"total":96636764160,"available":96019935232,"internally_reserved":0,"allocated":348377088,"data_stored":348065294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":268431022},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,3,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":513228800,"data_stored":513225187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":347951104,"data_stored":347878385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":419368960,"data_stored":419361275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:34:09.455 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:34:09.995 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:09.996 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:34:10.017 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":219,"stamp":"2024-09-10T20:34:08.927765+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":604751316,"num_objects":147650,"num_object_clones":0,"num_object_copies":295300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35894,"num_objects_misplaced":719,"num_objects_unfound":0,"num_objects_dirty":147650,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":147771,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6538,"num_bytes_recovered":26021907,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160976,"ondisk_log_size":160976,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2151024,"kb_used_data":1253556,"kb_used_omap":12,"kb_used_meta":897395,"kb_avail":280964496,"statfs":{"total":289910292480,"available":287707643904,"internally_reserved":0,"allocated":1283641344,"data_stored":1282844179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":918932482},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,3,2,7,6,4,3,0,0,0,6],"upper_bound":4096},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27299840,"num_objects":-6665,"num_object_clones":0,"num_object_copies":-13330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-78,"num_objects_misplaced":-526,"num_objects_unfound":0,"num_objects_dirty":-6665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6665,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":604,"num_bytes_recovered":2154496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007409"},"pg_stats":[{"pgid":"2.f","version":"45'11474","reported_seq":11595,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:07.984176+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:34:07.984176+0000","last_peered":"2024-09-10T20:34:07.984176+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:34:07.984176+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37617664,"num_objects":9184,"num_object_clones":0,"num_object_copies":18368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3587,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11474,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5597},{"shards":"0,2","objects":2462}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11685","reported_seq":11796,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:08.730146+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:34:08.730146+0000","last_peered":"2024-09-10T20:34:08.730146+0000","last_clean":"2024-09-10T20:34:08.730146+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:34:08.730146+0000","last_undegraded":"2024-09-10T20:34:08.730146+0000","last_fullsized":"2024-09-10T20:34:08.730146+0000","mapping_epoch":35,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37916672,"num_objects":9257,"num_object_clones":0,"num_object_copies":18514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9871,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11374","reported_seq":18722,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:04.045583+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:34:04.045583+0000","last_peered":"2024-09-10T20:34:04.045583+0000","last_clean":"2024-09-10T20:34:04.045583+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:34:04.045583+0000","last_undegraded":"2024-09-10T20:34:04.045583+0000","last_fullsized":"2024-09-10T20:34:04.045583+0000","mapping_epoch":44,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38260760,"num_objects":9342,"num_object_clones":0,"num_object_copies":18684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9342,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9502,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'11445","reported_seq":11526,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:08.724268+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:34:08.724268+0000","last_peered":"2024-09-10T20:34:08.724268+0000","last_clean":"2024-09-10T20:34:08.724268+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:34:08.724268+0000","last_undegraded":"2024-09-10T20:34:08.724268+0000","last_fullsized":"2024-09-10T20:34:08.724268+0000","mapping_epoch":35,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37515264,"num_objects":9159,"num_object_clones":0,"num_object_copies":18318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11445,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11697","reported_seq":11824,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:07.998829+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:34:07.998829+0000","last_peered":"2024-09-10T20:34:07.998829+0000","last_clean":"2024-09-10T20:34:07.998829+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:34:07.998829+0000","last_undegraded":"2024-09-10T20:34:07.998829+0000","last_fullsized":"2024-09-10T20:34:07.998829+0000","mapping_epoch":34,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38387712,"num_objects":9372,"num_object_clones":0,"num_object_copies":18744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8693,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11332","reported_seq":11509,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:07.989315+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:34:07.989315+0000","last_peered":"2024-09-10T20:34:07.989315+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:34:07.989315+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37433344,"num_objects":9139,"num_object_clones":0,"num_object_copies":18278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3596,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8196,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5545},{"shards":"0,2","objects":2520}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11421","reported_seq":11627,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:07.923440+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:34:07.923440+0000","last_peered":"2024-09-10T20:34:07.923440+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:34:07.923440+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629952,"num_objects":9187,"num_object_clones":0,"num_object_copies":18374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3523,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8016,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5666},{"shards":"0,2","objects":2441}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11473","reported_seq":11598,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:08.025004+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:34:08.025004+0000","last_peered":"2024-09-10T20:34:08.025004+0000","last_clean":"2024-09-10T20:34:08.025004+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:34:08.025004+0000","last_undegraded":"2024-09-10T20:34:08.025004+0000","last_fullsized":"2024-09-10T20:34:08.025004+0000","mapping_epoch":34,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37941248,"num_objects":9263,"num_object_clones":0,"num_object_copies":18526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8544,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187449+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187449+0000","last_peered":"2024-09-10T20:32:18.187449+0000","last_clean":"2024-09-10T20:32:18.187449+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187449+0000","last_undegraded":"2024-09-10T20:32:18.187449+0000","last_fullsized":"2024-09-10T20:32:18.187449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11338","reported_seq":11483,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:08.783045+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:34:08.783045+0000","last_peered":"2024-09-10T20:34:08.783045+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:34:08.783045+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37277696,"num_objects":9101,"num_object_clones":0,"num_object_copies":18202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3522,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9331,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2410},{"shards":"2","objects":5581}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190273+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190273+0000","last_peered":"2024-09-10T20:32:18.190273+0000","last_clean":"2024-09-10T20:32:18.190273+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190273+0000","last_undegraded":"2024-09-10T20:32:18.190273+0000","last_fullsized":"2024-09-10T20:32:18.190273+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11487","reported_seq":11638,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:07.992533+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:34:07.992533+0000","last_peered":"2024-09-10T20:34:07.992533+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:34:07.992533+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37860294,"num_objects":9245,"num_object_clones":0,"num_object_copies":18490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3596,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11485,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5651},{"shards":"0,2","objects":2497}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11443","reported_seq":11588,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:08.583236+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:34:08.583236+0000","last_peered":"2024-09-10T20:34:08.583236+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:34:08.583236+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37363712,"num_objects":9122,"num_object_clones":0,"num_object_copies":18244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3627,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9271,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2475},{"shards":"2","objects":5497}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11470","reported_seq":11609,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:08.669019+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:34:08.669019+0000","last_peered":"2024-09-10T20:34:08.669019+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:34:08.669019+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37658624,"num_objects":9194,"num_object_clones":0,"num_object_copies":18388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9256,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2475},{"shards":"2","objects":5589}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11527","reported_seq":11702,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:08.006495+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:34:08.006495+0000","last_peered":"2024-09-10T20:34:08.006495+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:34:08.006495+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38105088,"num_objects":9303,"num_object_clones":0,"num_object_copies":18606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8250,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5675},{"shards":"0,2","objects":2537}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11396","reported_seq":11579,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:07.950783+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:34:07.950783+0000","last_peered":"2024-09-10T20:34:07.950783+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:34:07.950783+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37634048,"num_objects":9188,"num_object_clones":0,"num_object_copies":18376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3513,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8090,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5678},{"shards":"0,2","objects":2432}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193230+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193230+0000","last_peered":"2024-09-10T20:32:18.193230+0000","last_clean":"2024-09-10T20:32:18.193230+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193230+0000","last_undegraded":"2024-09-10T20:32:18.193230+0000","last_fullsized":"2024-09-10T20:32:18.193230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11594","reported_seq":11768,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:07.876728+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:34:07.876728+0000","last_peered":"2024-09-10T20:34:07.876728+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:34:07.876728+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38469632,"num_objects":9392,"num_object_clones":0,"num_object_copies":18784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3694,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8421,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5700},{"shards":"0,2","objects":2614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11317","reported_seq":16988,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:34:08.031953+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:34:08.031953+0000","last_peered":"2024-09-10T20:34:08.031953+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:34:08.031953+0000","last_undegraded":"2024-09-10T20:34:08.031953+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37679587,"num_objects":9200,"num_object_clones":0,"num_object_copies":18400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":719,"num_objects_unfound":0,"num_objects_dirty":9200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7924,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2752,"num_bytes_recovered":11259904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8481},{"shards":"0,2","objects":719}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":604751297,"num_objects":147648,"num_object_clones":0,"num_object_copies":295296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35894,"num_objects_misplaced":719,"num_objects_unfound":0,"num_objects_dirty":147648,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":147769,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6536,"num_bytes_recovered":26021888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1280548864,"data_stored":1280464847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160973,"ondisk_log_size":160973,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182467,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":845388,"kb_used_data":502780,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93526452,"statfs":{"total":96636764160,"available":95771086848,"internally_reserved":0,"allocated":514846720,"data_stored":514604032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,0,3,1,0,0,0,0,0,6],"upper_bound":4096},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986239,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703264,"kb_used_data":410564,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93668576,"statfs":{"total":96636764160,"available":95916621824,"internally_reserved":0,"allocated":420417536,"data_stored":420174853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":299691690},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,1,0,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659185,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":602372,"kb_used_data":340212,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93769468,"statfs":{"total":96636764160,"available":96019935232,"internally_reserved":0,"allocated":348377088,"data_stored":348065294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":268431022},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,3,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":513228800,"data_stored":513225187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":347951104,"data_stored":347878385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":419368960,"data_stored":419361275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:34:10.018 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:34:10.018 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'11474", 'reported_seq': 11595, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:07.984176+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:34:07.984176+0000', 'last_peered': '2024-09-10T20:34:07.984176+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:34:07.984176+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37617664, 'num_objects': 9184, 'num_object_clones': 0, 'num_object_copies': 18368, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3587, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9184, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11474, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 21, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5597}, {'shards': '0,2', 'objects': 2462}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:10.018 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:34:10.018 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'11332", 'reported_seq': 11509, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:07.989315+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:34:07.989315+0000', 'last_peered': '2024-09-10T20:34:07.989315+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:34:07.989315+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37433344, 'num_objects': 9139, 'num_object_clones': 0, 'num_object_copies': 18278, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3596, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9139, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8196, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 20, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5545}, {'shards': '0,2', 'objects': 2520}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:10.019 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:34:10.019 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'11421", 'reported_seq': 11627, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:07.923440+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:34:07.923440+0000', 'last_peered': '2024-09-10T20:34:07.923440+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:34:07.923440+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37629952, 'num_objects': 9187, 'num_object_clones': 0, 'num_object_copies': 18374, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3523, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9187, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8016, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 34, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5666}, {'shards': '0,2', 'objects': 2441}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:10.019 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:34:10.019 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'11338", 'reported_seq': 11483, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:08.783045+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:34:08.783045+0000', 'last_peered': '2024-09-10T20:34:08.783045+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:34:08.783045+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37277696, 'num_objects': 9101, 'num_object_clones': 0, 'num_object_copies': 18202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3522, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9101, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9331, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 6, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2410}, {'shards': '2', 'objects': 5581}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:34:10.019 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:34:10.019 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'11487", 'reported_seq': 11638, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:07.992533+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:34:07.992533+0000', 'last_peered': '2024-09-10T20:34:07.992533+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:34:07.992533+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37860294, 'num_objects': 9245, 'num_object_clones': 0, 'num_object_copies': 18490, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3596, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9245, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11485, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 23, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5651}, {'shards': '0,2', 'objects': 2497}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:10.019 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:34:10.019 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'11443", 'reported_seq': 11588, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:08.583236+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:34:08.583236+0000', 'last_peered': '2024-09-10T20:34:08.583236+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:34:08.583236+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37363712, 'num_objects': 9122, 'num_object_clones': 0, 'num_object_copies': 18244, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3627, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9122, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9271, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 8, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2475}, {'shards': '2', 'objects': 5497}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:34:10.019 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:34:10.019 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'11470", 'reported_seq': 11609, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:08.669019+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:34:08.669019+0000', 'last_peered': '2024-09-10T20:34:08.669019+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:34:08.669019+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37658624, 'num_objects': 9194, 'num_object_clones': 0, 'num_object_copies': 18388, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3607, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9194, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9256, 'num_write_kb': 32484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 5, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2475}, {'shards': '2', 'objects': 5589}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:34:10.019 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:34:10.020 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'11527", 'reported_seq': 11702, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:08.006495+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:34:08.006495+0000', 'last_peered': '2024-09-10T20:34:08.006495+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:34:08.006495+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38105088, 'num_objects': 9303, 'num_object_clones': 0, 'num_object_copies': 18606, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9303, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8250, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 20, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5675}, {'shards': '0,2', 'objects': 2537}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:10.020 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:34:10.020 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'11396", 'reported_seq': 11579, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:07.950783+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:34:07.950783+0000', 'last_peered': '2024-09-10T20:34:07.950783+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:34:07.950783+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37634048, 'num_objects': 9188, 'num_object_clones': 0, 'num_object_copies': 18376, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3513, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9188, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8090, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 22, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5678}, {'shards': '0,2', 'objects': 2432}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:10.020 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:34:10.020 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'11594", 'reported_seq': 11768, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:07.876728+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:34:07.876728+0000', 'last_peered': '2024-09-10T20:34:07.876728+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:34:07.876728+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38469632, 'num_objects': 9392, 'num_object_clones': 0, 'num_object_copies': 18784, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3694, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9392, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8421, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 21, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5700}, {'shards': '0,2', 'objects': 2614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:10.020 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:34:10.020 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'11317", 'reported_seq': 16988, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:34:08.031953+0000', 'last_change': '2024-09-10T20:32:16.657652+0000', 'last_active': '2024-09-10T20:34:08.031953+0000', 'last_peered': '2024-09-10T20:34:08.031953+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:34:08.031953+0000', 'last_undegraded': '2024-09-10T20:34:08.031953+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37679587, 'num_objects': 9200, 'num_object_clones': 0, 'num_object_copies': 18400, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 719, 'num_objects_unfound': 0, 'num_objects_dirty': 9200, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7924, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2752, 'num_bytes_recovered': 11259904, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 8481}, {'shards': '0,2', 'objects': 719}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:10.021 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:34:10.689 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:10.689 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35894/295300 objects degraded (12.155%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":362,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":147650,"data_bytes":604751316,"bytes_used":2202648576,"bytes_avail":287707643904,"bytes_total":289910292480,"degraded_objects":35894,"degraded_total":295300,"degraded_ratio":0.12155096512021672,"misplaced_objects":719,"misplaced_total":295300,"misplaced_ratio":0.0024348120555367425,"recovering_objects_per_sec":50,"recovering_bytes_per_sec":179430,"recovering_keys_per_sec":0,"num_objects_recovered":604,"num_bytes_recovered":2154496,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":555},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:34:10.714 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:34:10.715 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:34:11.269 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:11.269 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:34:11.289 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2024-09-10T20:34:10.928336+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":604177876,"num_objects":147510,"num_object_clones":0,"num_object_copies":295020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35894,"num_objects_misplaced":719,"num_objects_unfound":0,"num_objects_dirty":147510,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":147911,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6538,"num_bytes_recovered":26021907,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160916,"ondisk_log_size":160916,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2155056,"kb_used_data":1253428,"kb_used_omap":12,"kb_used_meta":901555,"kb_avail":280960464,"statfs":{"total":289910292480,"available":287703515136,"internally_reserved":0,"allocated":1283510272,"data_stored":1282713107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":923192322},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,1,0,6,4,5,6,0,0,0,6],"upper_bound":4096},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19177472,"num_objects":-4682,"num_object_clones":0,"num_object_copies":-9364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-53,"num_objects_misplaced":-375,"num_objects_unfound":0,"num_objects_dirty":-4682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4682,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":428,"num_bytes_recovered":1536000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007238"},"pg_stats":[{"pgid":"2.f","version":"45'11474","reported_seq":11595,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:07.984176+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:34:07.984176+0000","last_peered":"2024-09-10T20:34:07.984176+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:34:07.984176+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37617664,"num_objects":9184,"num_object_clones":0,"num_object_copies":18368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3587,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11474,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5597},{"shards":"0,2","objects":2462}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11685","reported_seq":11796,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:08.730146+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:34:08.730146+0000","last_peered":"2024-09-10T20:34:08.730146+0000","last_clean":"2024-09-10T20:34:08.730146+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:34:08.730146+0000","last_undegraded":"2024-09-10T20:34:08.730146+0000","last_fullsized":"2024-09-10T20:34:08.730146+0000","mapping_epoch":35,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37916672,"num_objects":9257,"num_object_clones":0,"num_object_copies":18514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9871,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11514","reported_seq":18862,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:09.236799+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:34:09.236799+0000","last_peered":"2024-09-10T20:34:09.236799+0000","last_clean":"2024-09-10T20:34:09.236799+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:34:09.236799+0000","last_undegraded":"2024-09-10T20:34:09.236799+0000","last_fullsized":"2024-09-10T20:34:09.236799+0000","mapping_epoch":44,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37687320,"num_objects":9202,"num_object_clones":0,"num_object_copies":18404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9202,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9642,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'11445","reported_seq":11526,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:08.724268+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:34:08.724268+0000","last_peered":"2024-09-10T20:34:08.724268+0000","last_clean":"2024-09-10T20:34:08.724268+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:34:08.724268+0000","last_undegraded":"2024-09-10T20:34:08.724268+0000","last_fullsized":"2024-09-10T20:34:08.724268+0000","mapping_epoch":35,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37515264,"num_objects":9159,"num_object_clones":0,"num_object_copies":18318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11445,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11697","reported_seq":11824,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:07.998829+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:34:07.998829+0000","last_peered":"2024-09-10T20:34:07.998829+0000","last_clean":"2024-09-10T20:34:07.998829+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:34:07.998829+0000","last_undegraded":"2024-09-10T20:34:07.998829+0000","last_fullsized":"2024-09-10T20:34:07.998829+0000","mapping_epoch":34,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38387712,"num_objects":9372,"num_object_clones":0,"num_object_copies":18744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8693,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11332","reported_seq":11509,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:07.989315+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:34:07.989315+0000","last_peered":"2024-09-10T20:34:07.989315+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:34:07.989315+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37433344,"num_objects":9139,"num_object_clones":0,"num_object_copies":18278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3596,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8196,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5545},{"shards":"0,2","objects":2520}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11421","reported_seq":11627,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:07.923440+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:34:07.923440+0000","last_peered":"2024-09-10T20:34:07.923440+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:34:07.923440+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629952,"num_objects":9187,"num_object_clones":0,"num_object_copies":18374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3523,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8016,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5666},{"shards":"0,2","objects":2441}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11473","reported_seq":11598,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:08.025004+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:34:08.025004+0000","last_peered":"2024-09-10T20:34:08.025004+0000","last_clean":"2024-09-10T20:34:08.025004+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:34:08.025004+0000","last_undegraded":"2024-09-10T20:34:08.025004+0000","last_fullsized":"2024-09-10T20:34:08.025004+0000","mapping_epoch":34,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37941248,"num_objects":9263,"num_object_clones":0,"num_object_copies":18526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8544,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187449+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187449+0000","last_peered":"2024-09-10T20:32:18.187449+0000","last_clean":"2024-09-10T20:32:18.187449+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187449+0000","last_undegraded":"2024-09-10T20:32:18.187449+0000","last_fullsized":"2024-09-10T20:32:18.187449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11338","reported_seq":11483,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:08.783045+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:34:08.783045+0000","last_peered":"2024-09-10T20:34:08.783045+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:34:08.783045+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37277696,"num_objects":9101,"num_object_clones":0,"num_object_copies":18202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3522,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9331,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2410},{"shards":"2","objects":5581}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190273+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190273+0000","last_peered":"2024-09-10T20:32:18.190273+0000","last_clean":"2024-09-10T20:32:18.190273+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190273+0000","last_undegraded":"2024-09-10T20:32:18.190273+0000","last_fullsized":"2024-09-10T20:32:18.190273+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11487","reported_seq":11638,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:07.992533+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:34:07.992533+0000","last_peered":"2024-09-10T20:34:07.992533+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:34:07.992533+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37860294,"num_objects":9245,"num_object_clones":0,"num_object_copies":18490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3596,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11485,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5651},{"shards":"0,2","objects":2497}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11443","reported_seq":11588,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:08.583236+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:34:08.583236+0000","last_peered":"2024-09-10T20:34:08.583236+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:34:08.583236+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37363712,"num_objects":9122,"num_object_clones":0,"num_object_copies":18244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3627,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9271,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2475},{"shards":"2","objects":5497}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11470","reported_seq":11609,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:08.669019+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:34:08.669019+0000","last_peered":"2024-09-10T20:34:08.669019+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:34:08.669019+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37658624,"num_objects":9194,"num_object_clones":0,"num_object_copies":18388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9256,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2475},{"shards":"2","objects":5589}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11527","reported_seq":11702,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:08.006495+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:34:08.006495+0000","last_peered":"2024-09-10T20:34:08.006495+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:34:08.006495+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38105088,"num_objects":9303,"num_object_clones":0,"num_object_copies":18606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8250,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5675},{"shards":"0,2","objects":2537}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11396","reported_seq":11579,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:07.950783+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:34:07.950783+0000","last_peered":"2024-09-10T20:34:07.950783+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:34:07.950783+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37634048,"num_objects":9188,"num_object_clones":0,"num_object_copies":18376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3513,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8090,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5678},{"shards":"0,2","objects":2432}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193230+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193230+0000","last_peered":"2024-09-10T20:32:18.193230+0000","last_clean":"2024-09-10T20:32:18.193230+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193230+0000","last_undegraded":"2024-09-10T20:32:18.193230+0000","last_fullsized":"2024-09-10T20:32:18.193230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11594","reported_seq":11768,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:07.876728+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:34:07.876728+0000","last_peered":"2024-09-10T20:34:07.876728+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:34:07.876728+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38469632,"num_objects":9392,"num_object_clones":0,"num_object_copies":18784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3694,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8421,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5700},{"shards":"0,2","objects":2614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11317","reported_seq":16988,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:34:08.031953+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:34:08.031953+0000","last_peered":"2024-09-10T20:34:08.031953+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:34:08.031953+0000","last_undegraded":"2024-09-10T20:34:08.031953+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37679587,"num_objects":9200,"num_object_clones":0,"num_object_copies":18400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":719,"num_objects_unfound":0,"num_objects_dirty":9200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7924,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2752,"num_bytes_recovered":11259904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8481},{"shards":"0,2","objects":719}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":604177857,"num_objects":147508,"num_object_clones":0,"num_object_copies":295016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35894,"num_objects_misplaced":719,"num_objects_unfound":0,"num_objects_dirty":147508,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":147909,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6536,"num_bytes_recovered":26021888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1280380928,"data_stored":1280296911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160913,"ondisk_log_size":160913,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182467,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":845388,"kb_used_data":502780,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93526452,"statfs":{"total":96636764160,"available":95771086848,"internally_reserved":0,"allocated":514846720,"data_stored":514604032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,0,3,1,0,0,0,0,0,6],"upper_bound":4096},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986239,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703264,"kb_used_data":410564,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93668576,"statfs":{"total":96636764160,"available":95916621824,"internally_reserved":0,"allocated":420417536,"data_stored":420174853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":299691690},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,1,0,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659186,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":606404,"kb_used_data":340084,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93765436,"statfs":{"total":96636764160,"available":96015806464,"internally_reserved":0,"allocated":348246016,"data_stored":347934222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":272690862},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,3,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":513228800,"data_stored":513225187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":347783168,"data_stored":347710449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":419368960,"data_stored":419361275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:34:14.291 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:34:14.891 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:14.892 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:34:14.915 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2024-09-10T20:34:12.928986+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":604177876,"num_objects":147510,"num_object_clones":0,"num_object_copies":295020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35894,"num_objects_misplaced":719,"num_objects_unfound":0,"num_objects_dirty":147510,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":147911,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6538,"num_bytes_recovered":26021907,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160916,"ondisk_log_size":160916,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2155056,"kb_used_data":1253428,"kb_used_omap":12,"kb_used_meta":901555,"kb_avail":280960464,"statfs":{"total":289910292480,"available":287703515136,"internally_reserved":0,"allocated":1283510272,"data_stored":1282713107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":923192322},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,1,0,6,4,5,6,0,0,0,6],"upper_bound":4096},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18505728,"num_objects":-4518,"num_object_clones":0,"num_object_copies":-9036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-53,"num_objects_misplaced":-375,"num_objects_unfound":0,"num_objects_dirty":-4518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4518,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":428,"num_bytes_recovered":1536000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007099"},"pg_stats":[{"pgid":"2.f","version":"45'11474","reported_seq":11595,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:07.984176+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:34:07.984176+0000","last_peered":"2024-09-10T20:34:07.984176+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:34:07.984176+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37617664,"num_objects":9184,"num_object_clones":0,"num_object_copies":18368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3587,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11474,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5597},{"shards":"0,2","objects":2462}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11685","reported_seq":11796,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:08.730146+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:34:08.730146+0000","last_peered":"2024-09-10T20:34:08.730146+0000","last_clean":"2024-09-10T20:34:08.730146+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:34:08.730146+0000","last_undegraded":"2024-09-10T20:34:08.730146+0000","last_fullsized":"2024-09-10T20:34:08.730146+0000","mapping_epoch":35,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37916672,"num_objects":9257,"num_object_clones":0,"num_object_copies":18514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9871,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11514","reported_seq":18862,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:09.236799+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:34:09.236799+0000","last_peered":"2024-09-10T20:34:09.236799+0000","last_clean":"2024-09-10T20:34:09.236799+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:34:09.236799+0000","last_undegraded":"2024-09-10T20:34:09.236799+0000","last_fullsized":"2024-09-10T20:34:09.236799+0000","mapping_epoch":44,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37687320,"num_objects":9202,"num_object_clones":0,"num_object_copies":18404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9202,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9642,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'11445","reported_seq":11526,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:08.724268+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:34:08.724268+0000","last_peered":"2024-09-10T20:34:08.724268+0000","last_clean":"2024-09-10T20:34:08.724268+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:34:08.724268+0000","last_undegraded":"2024-09-10T20:34:08.724268+0000","last_fullsized":"2024-09-10T20:34:08.724268+0000","mapping_epoch":35,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37515264,"num_objects":9159,"num_object_clones":0,"num_object_copies":18318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11445,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11697","reported_seq":11824,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:07.998829+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:34:07.998829+0000","last_peered":"2024-09-10T20:34:07.998829+0000","last_clean":"2024-09-10T20:34:07.998829+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:34:07.998829+0000","last_undegraded":"2024-09-10T20:34:07.998829+0000","last_fullsized":"2024-09-10T20:34:07.998829+0000","mapping_epoch":34,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38387712,"num_objects":9372,"num_object_clones":0,"num_object_copies":18744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8693,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11332","reported_seq":11509,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:07.989315+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:34:07.989315+0000","last_peered":"2024-09-10T20:34:07.989315+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:34:07.989315+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37433344,"num_objects":9139,"num_object_clones":0,"num_object_copies":18278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3596,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8196,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5545},{"shards":"0,2","objects":2520}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11421","reported_seq":11627,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:07.923440+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:34:07.923440+0000","last_peered":"2024-09-10T20:34:07.923440+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:34:07.923440+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629952,"num_objects":9187,"num_object_clones":0,"num_object_copies":18374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3523,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8016,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5666},{"shards":"0,2","objects":2441}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11473","reported_seq":11598,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:08.025004+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:34:08.025004+0000","last_peered":"2024-09-10T20:34:08.025004+0000","last_clean":"2024-09-10T20:34:08.025004+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:34:08.025004+0000","last_undegraded":"2024-09-10T20:34:08.025004+0000","last_fullsized":"2024-09-10T20:34:08.025004+0000","mapping_epoch":34,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37941248,"num_objects":9263,"num_object_clones":0,"num_object_copies":18526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8544,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187449+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187449+0000","last_peered":"2024-09-10T20:32:18.187449+0000","last_clean":"2024-09-10T20:32:18.187449+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187449+0000","last_undegraded":"2024-09-10T20:32:18.187449+0000","last_fullsized":"2024-09-10T20:32:18.187449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11338","reported_seq":11483,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:08.783045+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:34:08.783045+0000","last_peered":"2024-09-10T20:34:08.783045+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:34:08.783045+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37277696,"num_objects":9101,"num_object_clones":0,"num_object_copies":18202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3522,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9331,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2410},{"shards":"2","objects":5581}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190273+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190273+0000","last_peered":"2024-09-10T20:32:18.190273+0000","last_clean":"2024-09-10T20:32:18.190273+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190273+0000","last_undegraded":"2024-09-10T20:32:18.190273+0000","last_fullsized":"2024-09-10T20:32:18.190273+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11487","reported_seq":11638,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:07.992533+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:34:07.992533+0000","last_peered":"2024-09-10T20:34:07.992533+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:34:07.992533+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37860294,"num_objects":9245,"num_object_clones":0,"num_object_copies":18490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3596,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11485,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5651},{"shards":"0,2","objects":2497}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11443","reported_seq":11588,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:08.583236+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:34:08.583236+0000","last_peered":"2024-09-10T20:34:08.583236+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:34:08.583236+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37363712,"num_objects":9122,"num_object_clones":0,"num_object_copies":18244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3627,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9271,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2475},{"shards":"2","objects":5497}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187437+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187437+0000","last_peered":"2024-09-10T20:32:18.187437+0000","last_clean":"2024-09-10T20:32:18.187437+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187437+0000","last_undegraded":"2024-09-10T20:32:18.187437+0000","last_fullsized":"2024-09-10T20:32:18.187437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11470","reported_seq":11609,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:08.669019+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:34:08.669019+0000","last_peered":"2024-09-10T20:34:08.669019+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:34:08.669019+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37658624,"num_objects":9194,"num_object_clones":0,"num_object_copies":18388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9256,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2475},{"shards":"2","objects":5589}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11527","reported_seq":11702,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:08.006495+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:34:08.006495+0000","last_peered":"2024-09-10T20:34:08.006495+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:34:08.006495+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38105088,"num_objects":9303,"num_object_clones":0,"num_object_copies":18606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8250,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5675},{"shards":"0,2","objects":2537}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187739+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187739+0000","last_peered":"2024-09-10T20:32:18.187739+0000","last_clean":"2024-09-10T20:32:18.187739+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187739+0000","last_undegraded":"2024-09-10T20:32:18.187739+0000","last_fullsized":"2024-09-10T20:32:18.187739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11396","reported_seq":11579,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:07.950783+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:34:07.950783+0000","last_peered":"2024-09-10T20:34:07.950783+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:34:07.950783+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37634048,"num_objects":9188,"num_object_clones":0,"num_object_copies":18376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3513,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8090,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5678},{"shards":"0,2","objects":2432}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193230+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193230+0000","last_peered":"2024-09-10T20:32:18.193230+0000","last_clean":"2024-09-10T20:32:18.193230+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193230+0000","last_undegraded":"2024-09-10T20:32:18.193230+0000","last_fullsized":"2024-09-10T20:32:18.193230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11594","reported_seq":11768,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:07.876728+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:34:07.876728+0000","last_peered":"2024-09-10T20:34:07.876728+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:34:07.876728+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38469632,"num_objects":9392,"num_object_clones":0,"num_object_copies":18784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3694,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8421,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5700},{"shards":"0,2","objects":2614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11317","reported_seq":16988,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:34:08.031953+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:34:08.031953+0000","last_peered":"2024-09-10T20:34:08.031953+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:34:08.031953+0000","last_undegraded":"2024-09-10T20:34:08.031953+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37679587,"num_objects":9200,"num_object_clones":0,"num_object_copies":18400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":719,"num_objects_unfound":0,"num_objects_dirty":9200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7924,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2752,"num_bytes_recovered":11259904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8481},{"shards":"0,2","objects":719}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":604177857,"num_objects":147508,"num_object_clones":0,"num_object_copies":295016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35894,"num_objects_misplaced":719,"num_objects_unfound":0,"num_objects_dirty":147508,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":147909,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6536,"num_bytes_recovered":26021888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1280380928,"data_stored":1280296911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160913,"ondisk_log_size":160913,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182467,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":845388,"kb_used_data":502780,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93526452,"statfs":{"total":96636764160,"available":95771086848,"internally_reserved":0,"allocated":514846720,"data_stored":514604032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,0,3,1,0,0,0,0,0,6],"upper_bound":4096},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986239,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703264,"kb_used_data":410564,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93668576,"statfs":{"total":96636764160,"available":95916621824,"internally_reserved":0,"allocated":420417536,"data_stored":420174853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":299691690},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,1,0,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659186,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":606404,"kb_used_data":340084,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93765436,"statfs":{"total":96636764160,"available":96015806464,"internally_reserved":0,"allocated":348246016,"data_stored":347934222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":272690862},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,3,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":513228800,"data_stored":513225187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":347783168,"data_stored":347710449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":419368960,"data_stored":419361275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:34:14.917 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:34:15.472 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:15.473 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:34:15.505 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":222,"stamp":"2024-09-10T20:34:14.929509+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595006932,"num_objects":145271,"num_object_clones":0,"num_object_copies":290542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35888,"num_objects_misplaced":516,"num_objects_unfound":0,"num_objects_dirty":145271,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":150150,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6747,"num_bytes_recovered":26853395,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2183620,"kb_used_data":1241748,"kb_used_omap":12,"kb_used_meta":941811,"kb_avail":280931900,"statfs":{"total":289910292480,"available":287674265600,"internally_reserved":0,"allocated":1271549952,"data_stored":1270752787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":964414466},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,1,3,5,4,0,0,0,0,0,1,8],"upper_bound":8192},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27676672,"num_objects":-6757,"num_object_clones":0,"num_object_copies":-13514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-59,"num_objects_misplaced":-578,"num_objects_unfound":0,"num_objects_dirty":-6757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6757,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":637,"num_bytes_recovered":2367488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006795"},"pg_stats":[{"pgid":"2.f","version":"45'11601","reported_seq":11722,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:13.023643+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:34:13.023643+0000","last_peered":"2024-09-10T20:34:13.023643+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:34:13.023643+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37097472,"num_objects":9057,"num_object_clones":0,"num_object_copies":18114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3587,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11601,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5470},{"shards":"0,2","objects":2335}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11836","reported_seq":11947,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:13.810761+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:34:13.810761+0000","last_peered":"2024-09-10T20:34:13.810761+0000","last_clean":"2024-09-10T20:34:13.810761+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:34:13.810761+0000","last_undegraded":"2024-09-10T20:34:13.810761+0000","last_fullsized":"2024-09-10T20:34:13.810761+0000","mapping_epoch":35,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37298176,"num_objects":9106,"num_object_clones":0,"num_object_copies":18212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10022,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11663","reported_seq":19010,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:14.278540+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:34:14.278540+0000","last_peered":"2024-09-10T20:34:14.278540+0000","last_clean":"2024-09-10T20:34:14.278540+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:34:14.278540+0000","last_undegraded":"2024-09-10T20:34:14.278540+0000","last_fullsized":"2024-09-10T20:34:14.278540+0000","mapping_epoch":44,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37077016,"num_objects":9053,"num_object_clones":0,"num_object_copies":18106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9053,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9791,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'11589","reported_seq":11670,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:13.796033+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:34:13.796033+0000","last_peered":"2024-09-10T20:34:13.796033+0000","last_clean":"2024-09-10T20:34:13.796033+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:34:13.796033+0000","last_undegraded":"2024-09-10T20:34:13.796033+0000","last_fullsized":"2024-09-10T20:34:13.796033+0000","mapping_epoch":35,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36925440,"num_objects":9015,"num_object_clones":0,"num_object_copies":18030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11589,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11829","reported_seq":11956,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:13.017463+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:34:13.017463+0000","last_peered":"2024-09-10T20:34:13.017463+0000","last_clean":"2024-09-10T20:34:13.017463+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:34:13.017463+0000","last_undegraded":"2024-09-10T20:34:13.017463+0000","last_fullsized":"2024-09-10T20:34:13.017463+0000","mapping_epoch":34,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37847040,"num_objects":9240,"num_object_clones":0,"num_object_copies":18480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8825,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11460","reported_seq":11639,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:13.008592+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:34:13.008592+0000","last_peered":"2024-09-10T20:34:13.008592+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:34:13.008592+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36909056,"num_objects":9011,"num_object_clones":0,"num_object_copies":18022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3595,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8324,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5418},{"shards":"0,2","objects":2392}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11558","reported_seq":11764,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:12.972137+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:34:12.972137+0000","last_peered":"2024-09-10T20:34:12.972137+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:34:12.972137+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37068800,"num_objects":9050,"num_object_clones":0,"num_object_copies":18100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3523,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8153,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5529},{"shards":"0,2","objects":2304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11617","reported_seq":11742,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:13.017279+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:34:13.017279+0000","last_peered":"2024-09-10T20:34:13.017279+0000","last_clean":"2024-09-10T20:34:13.017279+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:34:13.017279+0000","last_undegraded":"2024-09-10T20:34:13.017279+0000","last_fullsized":"2024-09-10T20:34:13.017279+0000","mapping_epoch":34,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37351424,"num_objects":9119,"num_object_clones":0,"num_object_copies":18238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8688,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187449+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187449+0000","last_peered":"2024-09-10T20:32:18.187449+0000","last_clean":"2024-09-10T20:32:18.187449+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187449+0000","last_undegraded":"2024-09-10T20:32:18.187449+0000","last_fullsized":"2024-09-10T20:32:18.187449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11486","reported_seq":11637,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:13.751714+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:34:13.751714+0000","last_peered":"2024-09-10T20:34:13.751714+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:34:13.751714+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36671488,"num_objects":8953,"num_object_clones":0,"num_object_copies":17906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3519,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9479,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2262},{"shards":"2","objects":5436}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190273+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190273+0000","last_peered":"2024-09-10T20:32:18.190273+0000","last_clean":"2024-09-10T20:32:18.190273+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190273+0000","last_undegraded":"2024-09-10T20:32:18.190273+0000","last_fullsized":"2024-09-10T20:32:18.190273+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11636","reported_seq":11787,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:13.007268+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:34:13.007268+0000","last_peered":"2024-09-10T20:34:13.007268+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:34:13.007268+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37249990,"num_objects":9096,"num_object_clones":0,"num_object_copies":18192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3596,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11634,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5502},{"shards":"0,2","objects":2348}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11593","reported_seq":11738,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:13.710826+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:34:13.710826+0000","last_peered":"2024-09-10T20:34:13.710826+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:34:13.710826+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36749312,"num_objects":8972,"num_object_clones":0,"num_object_copies":17944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3627,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9421,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2325},{"shards":"2","objects":5347}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187486+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187486+0000","last_peered":"2024-09-10T20:32:18.187486+0000","last_clean":"2024-09-10T20:32:18.187486+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187486+0000","last_undegraded":"2024-09-10T20:32:18.187486+0000","last_fullsized":"2024-09-10T20:32:18.187486+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11595","reported_seq":11736,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:13.777750+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:34:13.777750+0000","last_peered":"2024-09-10T20:34:13.777750+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:34:13.777750+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37146624,"num_objects":9069,"num_object_clones":0,"num_object_copies":18138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3606,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9381,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2350},{"shards":"2","objects":5465}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11663","reported_seq":11838,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:13.021512+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:34:13.021512+0000","last_peered":"2024-09-10T20:34:13.021512+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:34:13.021512+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37548032,"num_objects":9167,"num_object_clones":0,"num_object_copies":18334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8386,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5539},{"shards":"0,2","objects":2401}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187775+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187775+0000","last_peered":"2024-09-10T20:32:18.187775+0000","last_clean":"2024-09-10T20:32:18.187775+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187775+0000","last_undegraded":"2024-09-10T20:32:18.187775+0000","last_fullsized":"2024-09-10T20:32:18.187775+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11543","reported_seq":11728,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:12.986261+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:34:12.986261+0000","last_peered":"2024-09-10T20:34:12.986261+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:34:12.986261+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37031936,"num_objects":9041,"num_object_clones":0,"num_object_copies":18082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3512,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8237,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5532},{"shards":"0,2","objects":2285}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193230+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193230+0000","last_peered":"2024-09-10T20:32:18.193230+0000","last_clean":"2024-09-10T20:32:18.193230+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193230+0000","last_undegraded":"2024-09-10T20:32:18.193230+0000","last_fullsized":"2024-09-10T20:32:18.193230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11716","reported_seq":11890,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:13.035012+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:34:13.035012+0000","last_peered":"2024-09-10T20:34:13.035012+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:34:13.035012+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37969920,"num_objects":9270,"num_object_clones":0,"num_object_copies":18540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3694,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8543,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5578},{"shards":"0,2","objects":2492}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11467","reported_seq":17542,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:34:13.052787+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:34:13.052787+0000","last_peered":"2024-09-10T20:34:13.052787+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:34:13.052787+0000","last_undegraded":"2024-09-10T20:34:13.052787+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37065187,"num_objects":9050,"num_object_clones":0,"num_object_copies":18100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":516,"num_objects_unfound":0,"num_objects_dirty":9050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8074,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2955,"num_bytes_recovered":12091392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8534},{"shards":"0,2","objects":516}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":595006913,"num_objects":145269,"num_object_clones":0,"num_object_copies":290538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35888,"num_objects_misplaced":516,"num_objects_unfound":0,"num_objects_dirty":145269,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":150148,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6745,"num_bytes_recovered":26853376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1268559872,"data_stored":1268475855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182468,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":865132,"kb_used_data":496156,"kb_used_omap":4,"kb_used_meta":368955,"kb_avail":93506708,"statfs":{"total":96636764160,"available":95750868992,"internally_reserved":0,"allocated":508063744,"data_stored":507821056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":377810602},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,2,3,0,0,0,0,0,0,8],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986240,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":686100,"kb_used_data":405892,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93685740,"statfs":{"total":96636764160,"available":95934197760,"internally_reserved":0,"allocated":415633408,"data_stored":415390725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":286912170},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,1,0,0,0,0,0,1],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659187,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":632388,"kb_used_data":339700,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93739452,"statfs":{"total":96636764160,"available":95989198848,"internally_reserved":0,"allocated":347852800,"data_stored":347541006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":299691694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":506413056,"data_stored":506409443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":347426816,"data_stored":347354097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":414720000,"data_stored":414712315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:34:15.506 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:34:15.506 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'11601", 'reported_seq': 11722, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:13.023643+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:34:13.023643+0000', 'last_peered': '2024-09-10T20:34:13.023643+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:34:13.023643+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37097472, 'num_objects': 9057, 'num_object_clones': 0, 'num_object_copies': 18114, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3587, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9057, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11601, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 21, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5470}, {'shards': '0,2', 'objects': 2335}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:15.507 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:34:15.507 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'11460", 'reported_seq': 11639, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:13.008592+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:34:13.008592+0000', 'last_peered': '2024-09-10T20:34:13.008592+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:34:13.008592+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36909056, 'num_objects': 9011, 'num_object_clones': 0, 'num_object_copies': 18022, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3595, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9011, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8324, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 21, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5418}, {'shards': '0,2', 'objects': 2392}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:15.507 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:34:15.507 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'11558", 'reported_seq': 11764, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:12.972137+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:34:12.972137+0000', 'last_peered': '2024-09-10T20:34:12.972137+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:34:12.972137+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37068800, 'num_objects': 9050, 'num_object_clones': 0, 'num_object_copies': 18100, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3523, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9050, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8153, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 34, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5529}, {'shards': '0,2', 'objects': 2304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:15.507 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:34:15.507 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'11486", 'reported_seq': 11637, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:13.751714+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:34:13.751714+0000', 'last_peered': '2024-09-10T20:34:13.751714+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:34:13.751714+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36671488, 'num_objects': 8953, 'num_object_clones': 0, 'num_object_copies': 17906, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3519, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8953, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9479, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 9, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2262}, {'shards': '2', 'objects': 5436}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:34:15.507 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:34:15.507 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'11636", 'reported_seq': 11787, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:13.007268+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:34:13.007268+0000', 'last_peered': '2024-09-10T20:34:13.007268+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:34:13.007268+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37249990, 'num_objects': 9096, 'num_object_clones': 0, 'num_object_copies': 18192, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3596, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9096, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11634, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 23, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5502}, {'shards': '0,2', 'objects': 2348}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:15.507 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:34:15.507 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'11593", 'reported_seq': 11738, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:13.710826+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:34:13.710826+0000', 'last_peered': '2024-09-10T20:34:13.710826+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:34:13.710826+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36749312, 'num_objects': 8972, 'num_object_clones': 0, 'num_object_copies': 17944, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3627, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8972, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9421, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 8, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2325}, {'shards': '2', 'objects': 5347}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:34:15.508 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:34:15.508 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'11595", 'reported_seq': 11736, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:13.777750+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:34:13.777750+0000', 'last_peered': '2024-09-10T20:34:13.777750+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:34:13.777750+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37146624, 'num_objects': 9069, 'num_object_clones': 0, 'num_object_copies': 18138, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3606, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9069, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9381, 'num_write_kb': 32484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 6, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2350}, {'shards': '2', 'objects': 5465}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:34:15.508 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:34:15.508 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'11663", 'reported_seq': 11838, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:13.021512+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:34:13.021512+0000', 'last_peered': '2024-09-10T20:34:13.021512+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:34:13.021512+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37548032, 'num_objects': 9167, 'num_object_clones': 0, 'num_object_copies': 18334, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9167, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8386, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 20, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5539}, {'shards': '0,2', 'objects': 2401}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:15.508 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:34:15.508 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'11543", 'reported_seq': 11728, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:12.986261+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:34:12.986261+0000', 'last_peered': '2024-09-10T20:34:12.986261+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:34:12.986261+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37031936, 'num_objects': 9041, 'num_object_clones': 0, 'num_object_copies': 18082, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3512, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9041, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8237, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 23, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5532}, {'shards': '0,2', 'objects': 2285}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:15.508 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:34:15.508 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'11716", 'reported_seq': 11890, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:13.035012+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:34:13.035012+0000', 'last_peered': '2024-09-10T20:34:13.035012+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:34:13.035012+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37969920, 'num_objects': 9270, 'num_object_clones': 0, 'num_object_copies': 18540, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3694, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9270, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8543, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 21, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5578}, {'shards': '0,2', 'objects': 2492}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:15.508 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:34:15.508 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'11467", 'reported_seq': 17542, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:34:13.052787+0000', 'last_change': '2024-09-10T20:32:16.657652+0000', 'last_active': '2024-09-10T20:34:13.052787+0000', 'last_peered': '2024-09-10T20:34:13.052787+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:34:13.052787+0000', 'last_undegraded': '2024-09-10T20:34:13.052787+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37065187, 'num_objects': 9050, 'num_object_clones': 0, 'num_object_copies': 18100, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 516, 'num_objects_unfound': 0, 'num_objects_dirty': 9050, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8074, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2955, 'num_bytes_recovered': 12091392, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 8534}, {'shards': '0,2', 'objects': 516}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:15.509 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:34:16.185 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:16.185 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35888/290542 objects degraded (12.352%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":367,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":145271,"data_bytes":595006932,"bytes_used":2236026880,"bytes_avail":287674265600,"bytes_total":289910292480,"degraded_objects":35888,"degraded_total":290542,"degraded_ratio":0.12352086789517522,"misplaced_objects":516,"misplaced_total":290542,"misplaced_ratio":0.0017759910787424882,"recovering_objects_per_sec":53,"recovering_bytes_per_sec":197179,"recovering_keys_per_sec":0,"num_objects_recovered":637,"num_bytes_recovered":2367488,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":562},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:34:16.205 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:34:16.205 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:34:16.755 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:16.756 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:34:16.779 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":222,"stamp":"2024-09-10T20:34:14.929509+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595006932,"num_objects":145271,"num_object_clones":0,"num_object_copies":290542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35888,"num_objects_misplaced":516,"num_objects_unfound":0,"num_objects_dirty":145271,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":150150,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6747,"num_bytes_recovered":26853395,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2183620,"kb_used_data":1241748,"kb_used_omap":12,"kb_used_meta":941811,"kb_avail":280931900,"statfs":{"total":289910292480,"available":287674265600,"internally_reserved":0,"allocated":1271549952,"data_stored":1270752787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":964414466},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,1,3,5,4,0,0,0,0,0,1,8],"upper_bound":8192},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27676672,"num_objects":-6757,"num_object_clones":0,"num_object_copies":-13514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-59,"num_objects_misplaced":-578,"num_objects_unfound":0,"num_objects_dirty":-6757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6757,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":637,"num_bytes_recovered":2367488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006795"},"pg_stats":[{"pgid":"2.f","version":"45'11601","reported_seq":11722,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:13.023643+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:34:13.023643+0000","last_peered":"2024-09-10T20:34:13.023643+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:34:13.023643+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37097472,"num_objects":9057,"num_object_clones":0,"num_object_copies":18114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3587,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11601,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5470},{"shards":"0,2","objects":2335}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11836","reported_seq":11947,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:13.810761+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:34:13.810761+0000","last_peered":"2024-09-10T20:34:13.810761+0000","last_clean":"2024-09-10T20:34:13.810761+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:34:13.810761+0000","last_undegraded":"2024-09-10T20:34:13.810761+0000","last_fullsized":"2024-09-10T20:34:13.810761+0000","mapping_epoch":35,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37298176,"num_objects":9106,"num_object_clones":0,"num_object_copies":18212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10022,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11663","reported_seq":19010,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:14.278540+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:34:14.278540+0000","last_peered":"2024-09-10T20:34:14.278540+0000","last_clean":"2024-09-10T20:34:14.278540+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:34:14.278540+0000","last_undegraded":"2024-09-10T20:34:14.278540+0000","last_fullsized":"2024-09-10T20:34:14.278540+0000","mapping_epoch":44,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37077016,"num_objects":9053,"num_object_clones":0,"num_object_copies":18106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9053,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9791,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'11589","reported_seq":11670,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:13.796033+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:34:13.796033+0000","last_peered":"2024-09-10T20:34:13.796033+0000","last_clean":"2024-09-10T20:34:13.796033+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:34:13.796033+0000","last_undegraded":"2024-09-10T20:34:13.796033+0000","last_fullsized":"2024-09-10T20:34:13.796033+0000","mapping_epoch":35,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36925440,"num_objects":9015,"num_object_clones":0,"num_object_copies":18030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11589,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11829","reported_seq":11956,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:13.017463+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:34:13.017463+0000","last_peered":"2024-09-10T20:34:13.017463+0000","last_clean":"2024-09-10T20:34:13.017463+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:34:13.017463+0000","last_undegraded":"2024-09-10T20:34:13.017463+0000","last_fullsized":"2024-09-10T20:34:13.017463+0000","mapping_epoch":34,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37847040,"num_objects":9240,"num_object_clones":0,"num_object_copies":18480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8825,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11460","reported_seq":11639,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:13.008592+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:34:13.008592+0000","last_peered":"2024-09-10T20:34:13.008592+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:34:13.008592+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36909056,"num_objects":9011,"num_object_clones":0,"num_object_copies":18022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3595,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8324,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5418},{"shards":"0,2","objects":2392}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11558","reported_seq":11764,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:12.972137+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:34:12.972137+0000","last_peered":"2024-09-10T20:34:12.972137+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:34:12.972137+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37068800,"num_objects":9050,"num_object_clones":0,"num_object_copies":18100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3523,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8153,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5529},{"shards":"0,2","objects":2304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11617","reported_seq":11742,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:13.017279+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:34:13.017279+0000","last_peered":"2024-09-10T20:34:13.017279+0000","last_clean":"2024-09-10T20:34:13.017279+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:34:13.017279+0000","last_undegraded":"2024-09-10T20:34:13.017279+0000","last_fullsized":"2024-09-10T20:34:13.017279+0000","mapping_epoch":34,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37351424,"num_objects":9119,"num_object_clones":0,"num_object_copies":18238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8688,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187449+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187449+0000","last_peered":"2024-09-10T20:32:18.187449+0000","last_clean":"2024-09-10T20:32:18.187449+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187449+0000","last_undegraded":"2024-09-10T20:32:18.187449+0000","last_fullsized":"2024-09-10T20:32:18.187449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11486","reported_seq":11637,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:13.751714+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:34:13.751714+0000","last_peered":"2024-09-10T20:34:13.751714+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:34:13.751714+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36671488,"num_objects":8953,"num_object_clones":0,"num_object_copies":17906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3519,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9479,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2262},{"shards":"2","objects":5436}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190273+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190273+0000","last_peered":"2024-09-10T20:32:18.190273+0000","last_clean":"2024-09-10T20:32:18.190273+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190273+0000","last_undegraded":"2024-09-10T20:32:18.190273+0000","last_fullsized":"2024-09-10T20:32:18.190273+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11636","reported_seq":11787,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:13.007268+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:34:13.007268+0000","last_peered":"2024-09-10T20:34:13.007268+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:34:13.007268+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37249990,"num_objects":9096,"num_object_clones":0,"num_object_copies":18192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3596,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11634,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5502},{"shards":"0,2","objects":2348}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11593","reported_seq":11738,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:13.710826+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:34:13.710826+0000","last_peered":"2024-09-10T20:34:13.710826+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:34:13.710826+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36749312,"num_objects":8972,"num_object_clones":0,"num_object_copies":17944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3627,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9421,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2325},{"shards":"2","objects":5347}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187486+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187486+0000","last_peered":"2024-09-10T20:32:18.187486+0000","last_clean":"2024-09-10T20:32:18.187486+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187486+0000","last_undegraded":"2024-09-10T20:32:18.187486+0000","last_fullsized":"2024-09-10T20:32:18.187486+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11595","reported_seq":11736,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:13.777750+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:34:13.777750+0000","last_peered":"2024-09-10T20:34:13.777750+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:34:13.777750+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37146624,"num_objects":9069,"num_object_clones":0,"num_object_copies":18138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3606,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9381,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2350},{"shards":"2","objects":5465}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11663","reported_seq":11838,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:13.021512+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:34:13.021512+0000","last_peered":"2024-09-10T20:34:13.021512+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:34:13.021512+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37548032,"num_objects":9167,"num_object_clones":0,"num_object_copies":18334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8386,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5539},{"shards":"0,2","objects":2401}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187775+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187775+0000","last_peered":"2024-09-10T20:32:18.187775+0000","last_clean":"2024-09-10T20:32:18.187775+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187775+0000","last_undegraded":"2024-09-10T20:32:18.187775+0000","last_fullsized":"2024-09-10T20:32:18.187775+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11543","reported_seq":11728,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:12.986261+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:34:12.986261+0000","last_peered":"2024-09-10T20:34:12.986261+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:34:12.986261+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37031936,"num_objects":9041,"num_object_clones":0,"num_object_copies":18082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3512,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8237,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5532},{"shards":"0,2","objects":2285}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193230+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193230+0000","last_peered":"2024-09-10T20:32:18.193230+0000","last_clean":"2024-09-10T20:32:18.193230+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193230+0000","last_undegraded":"2024-09-10T20:32:18.193230+0000","last_fullsized":"2024-09-10T20:32:18.193230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11716","reported_seq":11890,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:13.035012+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:34:13.035012+0000","last_peered":"2024-09-10T20:34:13.035012+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:34:13.035012+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37969920,"num_objects":9270,"num_object_clones":0,"num_object_copies":18540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3694,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8543,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5578},{"shards":"0,2","objects":2492}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11467","reported_seq":17542,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:34:13.052787+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:34:13.052787+0000","last_peered":"2024-09-10T20:34:13.052787+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:34:13.052787+0000","last_undegraded":"2024-09-10T20:34:13.052787+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37065187,"num_objects":9050,"num_object_clones":0,"num_object_copies":18100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":516,"num_objects_unfound":0,"num_objects_dirty":9050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8074,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2955,"num_bytes_recovered":12091392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8534},{"shards":"0,2","objects":516}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":595006913,"num_objects":145269,"num_object_clones":0,"num_object_copies":290538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35888,"num_objects_misplaced":516,"num_objects_unfound":0,"num_objects_dirty":145269,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":150148,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6745,"num_bytes_recovered":26853376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1268559872,"data_stored":1268475855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182468,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":865132,"kb_used_data":496156,"kb_used_omap":4,"kb_used_meta":368955,"kb_avail":93506708,"statfs":{"total":96636764160,"available":95750868992,"internally_reserved":0,"allocated":508063744,"data_stored":507821056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":377810602},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,2,3,0,0,0,0,0,0,8],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986240,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":686100,"kb_used_data":405892,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93685740,"statfs":{"total":96636764160,"available":95934197760,"internally_reserved":0,"allocated":415633408,"data_stored":415390725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":286912170},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,1,0,0,0,0,0,1],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659187,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":632388,"kb_used_data":339700,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93739452,"statfs":{"total":96636764160,"available":95989198848,"internally_reserved":0,"allocated":347852800,"data_stored":347541006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":299691694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":506413056,"data_stored":506409443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":347426816,"data_stored":347354097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":414720000,"data_stored":414712315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:34:19.781 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:34:20.354 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:20.355 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:34:20.385 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":224,"stamp":"2024-09-10T20:34:18.931786+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":586364372,"num_objects":143161,"num_object_clones":0,"num_object_copies":286322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35874,"num_objects_misplaced":322,"num_objects_unfound":0,"num_objects_dirty":143161,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":152260,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6955,"num_bytes_recovered":27648019,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160765,"ondisk_log_size":160765,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2182172,"kb_used_data":1230564,"kb_used_omap":12,"kb_used_meta":951539,"kb_avail":280933348,"statfs":{"total":289910292480,"available":287675748352,"internally_reserved":0,"allocated":1260097536,"data_stored":1259300371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":974375938},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,2,2,4,7,0,0,2,0,0,0,8],"upper_bound":8192},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26996736,"num_objects":-6591,"num_object_clones":0,"num_object_copies":-13182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-44,"num_objects_misplaced":-603,"num_objects_unfound":0,"num_objects_dirty":-6591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6591,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":647,"num_bytes_recovered":2469888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005061"},"pg_stats":[{"pgid":"2.f","version":"45'11743","reported_seq":11868,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:17.985579+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:34:17.985579+0000","last_peered":"2024-09-10T20:34:17.985579+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:34:17.985579+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36515840,"num_objects":8915,"num_object_clones":0,"num_object_copies":17830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3585,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11743,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5330},{"shards":"0,2","objects":2193}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11966","reported_seq":12077,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:18.644272+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:34:18.644272+0000","last_peered":"2024-09-10T20:34:18.644272+0000","last_clean":"2024-09-10T20:34:18.644272+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:34:18.644272+0000","last_undegraded":"2024-09-10T20:34:18.644272+0000","last_fullsized":"2024-09-10T20:34:18.644272+0000","mapping_epoch":35,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36765696,"num_objects":8976,"num_object_clones":0,"num_object_copies":17952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10152,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11663","reported_seq":19010,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:14.278540+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:34:14.278540+0000","last_peered":"2024-09-10T20:34:14.278540+0000","last_clean":"2024-09-10T20:34:14.278540+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:34:14.278540+0000","last_undegraded":"2024-09-10T20:34:14.278540+0000","last_fullsized":"2024-09-10T20:34:14.278540+0000","mapping_epoch":44,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37077016,"num_objects":9053,"num_object_clones":0,"num_object_copies":18106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9053,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9791,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'11730","reported_seq":11811,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:18.709650+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:34:18.709650+0000","last_peered":"2024-09-10T20:34:18.709650+0000","last_clean":"2024-09-10T20:34:18.709650+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:34:18.709650+0000","last_undegraded":"2024-09-10T20:34:18.709650+0000","last_fullsized":"2024-09-10T20:34:18.709650+0000","mapping_epoch":35,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36347904,"num_objects":8874,"num_object_clones":0,"num_object_copies":17748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11730,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11958","reported_seq":12085,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:18.003922+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:34:18.003922+0000","last_peered":"2024-09-10T20:34:18.003922+0000","last_clean":"2024-09-10T20:34:18.003922+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:34:18.003922+0000","last_undegraded":"2024-09-10T20:34:18.003922+0000","last_fullsized":"2024-09-10T20:34:18.003922+0000","mapping_epoch":34,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37318656,"num_objects":9111,"num_object_clones":0,"num_object_copies":18222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8954,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11590","reported_seq":11769,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:18.025828+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:34:18.025828+0000","last_peered":"2024-09-10T20:34:18.025828+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:34:18.025828+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36376576,"num_objects":8881,"num_object_clones":0,"num_object_copies":17762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3595,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8454,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5288},{"shards":"0,2","objects":2262}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11714","reported_seq":11922,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:17.981313+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:34:17.981313+0000","last_peered":"2024-09-10T20:34:17.981313+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:34:17.981313+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36429824,"num_objects":8894,"num_object_clones":0,"num_object_copies":17788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3522,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8309,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5374},{"shards":"0,2","objects":2148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11770","reported_seq":11895,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:17.963717+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:34:17.963717+0000","last_peered":"2024-09-10T20:34:17.963717+0000","last_clean":"2024-09-10T20:34:17.963717+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:34:17.963717+0000","last_undegraded":"2024-09-10T20:34:17.963717+0000","last_fullsized":"2024-09-10T20:34:17.963717+0000","mapping_epoch":34,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36724736,"num_objects":8966,"num_object_clones":0,"num_object_copies":17932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8841,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187449+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187449+0000","last_peered":"2024-09-10T20:32:18.187449+0000","last_clean":"2024-09-10T20:32:18.187449+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187449+0000","last_undegraded":"2024-09-10T20:32:18.187449+0000","last_fullsized":"2024-09-10T20:32:18.187449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11612","reported_seq":11767,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:18.740045+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:34:18.740045+0000","last_peered":"2024-09-10T20:34:18.740045+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:34:18.740045+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36155392,"num_objects":8827,"num_object_clones":0,"num_object_copies":17654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9605,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2136},{"shards":"2","objects":5312}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190273+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190273+0000","last_peered":"2024-09-10T20:32:18.190273+0000","last_clean":"2024-09-10T20:32:18.190273+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190273+0000","last_undegraded":"2024-09-10T20:32:18.190273+0000","last_fullsized":"2024-09-10T20:32:18.190273+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11783","reported_seq":11934,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:18.027152+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:34:18.027152+0000","last_peered":"2024-09-10T20:34:18.027152+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:34:18.027152+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36647878,"num_objects":8949,"num_object_clones":0,"num_object_copies":17898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3596,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11781,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5355},{"shards":"0,2","objects":2201}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11739","reported_seq":11886,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:18.790291+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:34:18.790291+0000","last_peered":"2024-09-10T20:34:18.790291+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:34:18.790291+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36151296,"num_objects":8826,"num_object_clones":0,"num_object_copies":17652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3626,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9567,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2179},{"shards":"2","objects":5202}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187486+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187486+0000","last_peered":"2024-09-10T20:32:18.187486+0000","last_clean":"2024-09-10T20:32:18.187486+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187486+0000","last_undegraded":"2024-09-10T20:32:18.187486+0000","last_fullsized":"2024-09-10T20:32:18.187486+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11733","reported_seq":11878,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:18.798217+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:34:18.798217+0000","last_peered":"2024-09-10T20:34:18.798217+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:34:18.798217+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36581376,"num_objects":8931,"num_object_clones":0,"num_object_copies":17862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3604,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9519,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2212},{"shards":"2","objects":5329}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11826","reported_seq":12009,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:18.053575+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:34:18.053575+0000","last_peered":"2024-09-10T20:34:18.053575+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:34:18.053575+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36880384,"num_objects":9004,"num_object_clones":0,"num_object_copies":18008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8549,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5381},{"shards":"0,2","objects":2238}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187775+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187775+0000","last_peered":"2024-09-10T20:32:18.187775+0000","last_clean":"2024-09-10T20:32:18.187775+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187775+0000","last_undegraded":"2024-09-10T20:32:18.187775+0000","last_fullsized":"2024-09-10T20:32:18.187775+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11684","reported_seq":11868,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:18.060822+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:34:18.060822+0000","last_peered":"2024-09-10T20:34:18.060822+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:34:18.060822+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36454400,"num_objects":8900,"num_object_clones":0,"num_object_copies":17800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3512,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8378,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5391},{"shards":"0,2","objects":2144}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193230+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193230+0000","last_peered":"2024-09-10T20:32:18.193230+0000","last_clean":"2024-09-10T20:32:18.193230+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193230+0000","last_undegraded":"2024-09-10T20:32:18.193230+0000","last_fullsized":"2024-09-10T20:32:18.193230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11850","reported_seq":12026,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:17.975819+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:34:17.975819+0000","last_peered":"2024-09-10T20:34:17.975819+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:34:17.975819+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37421056,"num_objects":9136,"num_object_clones":0,"num_object_copies":18272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3693,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8677,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5445},{"shards":"0,2","objects":2358}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11601","reported_seq":18066,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:34:18.056364+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:34:18.056364+0000","last_peered":"2024-09-10T20:34:18.056364+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:34:18.056364+0000","last_undegraded":"2024-09-10T20:34:18.056364+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36516323,"num_objects":8916,"num_object_clones":0,"num_object_copies":17832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":322,"num_objects_unfound":0,"num_objects_dirty":8916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8208,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3149,"num_bytes_recovered":12886016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8594},{"shards":"0,2","objects":322}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":586364353,"num_objects":143159,"num_object_clones":0,"num_object_copies":286318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35874,"num_objects_misplaced":322,"num_objects_unfound":0,"num_objects_dirty":143159,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":152258,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6953,"num_bytes_recovered":27648000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1257009152,"data_stored":1256925135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160762,"ondisk_log_size":160762,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182469,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":845840,"kb_used_data":489340,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93526000,"statfs":{"total":96636764160,"available":95770624000,"internally_reserved":0,"allocated":501084160,"data_stored":500841472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,4,0,0,0,0,0,0,8],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986241,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703944,"kb_used_data":401524,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93667896,"statfs":{"total":96636764160,"available":95915925504,"internally_reserved":0,"allocated":411160576,"data_stored":410917893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309653162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,3,0,0,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659187,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":632388,"kb_used_data":339700,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93739452,"statfs":{"total":96636764160,"available":95989198848,"internally_reserved":0,"allocated":347852800,"data_stored":347541006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":299691694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":499384320,"data_stored":499380707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":347426816,"data_stored":347354097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":410198016,"data_stored":410190331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:34:20.387 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:34:20.916 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:20.916 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:34:20.941 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":224,"stamp":"2024-09-10T20:34:18.931786+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":586364372,"num_objects":143161,"num_object_clones":0,"num_object_copies":286322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35874,"num_objects_misplaced":322,"num_objects_unfound":0,"num_objects_dirty":143161,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":152260,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6955,"num_bytes_recovered":27648019,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160765,"ondisk_log_size":160765,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2182172,"kb_used_data":1230564,"kb_used_omap":12,"kb_used_meta":951539,"kb_avail":280933348,"statfs":{"total":289910292480,"available":287675748352,"internally_reserved":0,"allocated":1260097536,"data_stored":1259300371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":974375938},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,2,2,4,7,0,0,2,0,0,0,8],"upper_bound":8192},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26996736,"num_objects":-6591,"num_object_clones":0,"num_object_copies":-13182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-44,"num_objects_misplaced":-603,"num_objects_unfound":0,"num_objects_dirty":-6591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6591,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":647,"num_bytes_recovered":2469888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005061"},"pg_stats":[{"pgid":"2.f","version":"45'11743","reported_seq":11868,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:17.985579+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:34:17.985579+0000","last_peered":"2024-09-10T20:34:17.985579+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:34:17.985579+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36515840,"num_objects":8915,"num_object_clones":0,"num_object_copies":17830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3585,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11743,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5330},{"shards":"0,2","objects":2193}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11966","reported_seq":12077,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:18.644272+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:34:18.644272+0000","last_peered":"2024-09-10T20:34:18.644272+0000","last_clean":"2024-09-10T20:34:18.644272+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:34:18.644272+0000","last_undegraded":"2024-09-10T20:34:18.644272+0000","last_fullsized":"2024-09-10T20:34:18.644272+0000","mapping_epoch":35,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36765696,"num_objects":8976,"num_object_clones":0,"num_object_copies":17952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10152,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11663","reported_seq":19010,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:14.278540+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:34:14.278540+0000","last_peered":"2024-09-10T20:34:14.278540+0000","last_clean":"2024-09-10T20:34:14.278540+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:34:14.278540+0000","last_undegraded":"2024-09-10T20:34:14.278540+0000","last_fullsized":"2024-09-10T20:34:14.278540+0000","mapping_epoch":44,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37077016,"num_objects":9053,"num_object_clones":0,"num_object_copies":18106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9053,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9791,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'11730","reported_seq":11811,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:18.709650+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:34:18.709650+0000","last_peered":"2024-09-10T20:34:18.709650+0000","last_clean":"2024-09-10T20:34:18.709650+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:34:18.709650+0000","last_undegraded":"2024-09-10T20:34:18.709650+0000","last_fullsized":"2024-09-10T20:34:18.709650+0000","mapping_epoch":35,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36347904,"num_objects":8874,"num_object_clones":0,"num_object_copies":17748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11730,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11958","reported_seq":12085,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:18.003922+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:34:18.003922+0000","last_peered":"2024-09-10T20:34:18.003922+0000","last_clean":"2024-09-10T20:34:18.003922+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:34:18.003922+0000","last_undegraded":"2024-09-10T20:34:18.003922+0000","last_fullsized":"2024-09-10T20:34:18.003922+0000","mapping_epoch":34,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37318656,"num_objects":9111,"num_object_clones":0,"num_object_copies":18222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8954,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11590","reported_seq":11769,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:18.025828+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:34:18.025828+0000","last_peered":"2024-09-10T20:34:18.025828+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:34:18.025828+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36376576,"num_objects":8881,"num_object_clones":0,"num_object_copies":17762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3595,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8454,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5288},{"shards":"0,2","objects":2262}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11714","reported_seq":11922,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:17.981313+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:34:17.981313+0000","last_peered":"2024-09-10T20:34:17.981313+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:34:17.981313+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36429824,"num_objects":8894,"num_object_clones":0,"num_object_copies":17788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3522,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8309,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5374},{"shards":"0,2","objects":2148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11770","reported_seq":11895,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:17.963717+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:34:17.963717+0000","last_peered":"2024-09-10T20:34:17.963717+0000","last_clean":"2024-09-10T20:34:17.963717+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:34:17.963717+0000","last_undegraded":"2024-09-10T20:34:17.963717+0000","last_fullsized":"2024-09-10T20:34:17.963717+0000","mapping_epoch":34,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36724736,"num_objects":8966,"num_object_clones":0,"num_object_copies":17932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8841,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187449+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187449+0000","last_peered":"2024-09-10T20:32:18.187449+0000","last_clean":"2024-09-10T20:32:18.187449+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187449+0000","last_undegraded":"2024-09-10T20:32:18.187449+0000","last_fullsized":"2024-09-10T20:32:18.187449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11612","reported_seq":11767,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:18.740045+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:34:18.740045+0000","last_peered":"2024-09-10T20:34:18.740045+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:34:18.740045+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36155392,"num_objects":8827,"num_object_clones":0,"num_object_copies":17654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9605,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2136},{"shards":"2","objects":5312}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190273+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190273+0000","last_peered":"2024-09-10T20:32:18.190273+0000","last_clean":"2024-09-10T20:32:18.190273+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190273+0000","last_undegraded":"2024-09-10T20:32:18.190273+0000","last_fullsized":"2024-09-10T20:32:18.190273+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11783","reported_seq":11934,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:18.027152+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:34:18.027152+0000","last_peered":"2024-09-10T20:34:18.027152+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:34:18.027152+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36647878,"num_objects":8949,"num_object_clones":0,"num_object_copies":17898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3596,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11781,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5355},{"shards":"0,2","objects":2201}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11739","reported_seq":11886,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:18.790291+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:34:18.790291+0000","last_peered":"2024-09-10T20:34:18.790291+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:34:18.790291+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36151296,"num_objects":8826,"num_object_clones":0,"num_object_copies":17652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3626,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9567,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2179},{"shards":"2","objects":5202}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187486+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187486+0000","last_peered":"2024-09-10T20:32:18.187486+0000","last_clean":"2024-09-10T20:32:18.187486+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187486+0000","last_undegraded":"2024-09-10T20:32:18.187486+0000","last_fullsized":"2024-09-10T20:32:18.187486+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11733","reported_seq":11878,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:18.798217+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:34:18.798217+0000","last_peered":"2024-09-10T20:34:18.798217+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:34:18.798217+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36581376,"num_objects":8931,"num_object_clones":0,"num_object_copies":17862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3604,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9519,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2212},{"shards":"2","objects":5329}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11826","reported_seq":12009,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:18.053575+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:34:18.053575+0000","last_peered":"2024-09-10T20:34:18.053575+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:34:18.053575+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36880384,"num_objects":9004,"num_object_clones":0,"num_object_copies":18008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8549,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5381},{"shards":"0,2","objects":2238}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187775+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187775+0000","last_peered":"2024-09-10T20:32:18.187775+0000","last_clean":"2024-09-10T20:32:18.187775+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187775+0000","last_undegraded":"2024-09-10T20:32:18.187775+0000","last_fullsized":"2024-09-10T20:32:18.187775+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11684","reported_seq":11868,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:18.060822+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:34:18.060822+0000","last_peered":"2024-09-10T20:34:18.060822+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:34:18.060822+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36454400,"num_objects":8900,"num_object_clones":0,"num_object_copies":17800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3512,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8378,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5391},{"shards":"0,2","objects":2144}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193230+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193230+0000","last_peered":"2024-09-10T20:32:18.193230+0000","last_clean":"2024-09-10T20:32:18.193230+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193230+0000","last_undegraded":"2024-09-10T20:32:18.193230+0000","last_fullsized":"2024-09-10T20:32:18.193230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11850","reported_seq":12026,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:17.975819+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:34:17.975819+0000","last_peered":"2024-09-10T20:34:17.975819+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:34:17.975819+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37421056,"num_objects":9136,"num_object_clones":0,"num_object_copies":18272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3693,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8677,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5445},{"shards":"0,2","objects":2358}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188283+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188283+0000","last_peered":"2024-09-10T20:32:18.188283+0000","last_clean":"2024-09-10T20:32:18.188283+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188283+0000","last_undegraded":"2024-09-10T20:32:18.188283+0000","last_fullsized":"2024-09-10T20:32:18.188283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11601","reported_seq":18066,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:34:18.056364+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:34:18.056364+0000","last_peered":"2024-09-10T20:34:18.056364+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:34:18.056364+0000","last_undegraded":"2024-09-10T20:34:18.056364+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36516323,"num_objects":8916,"num_object_clones":0,"num_object_copies":17832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":322,"num_objects_unfound":0,"num_objects_dirty":8916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8208,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3149,"num_bytes_recovered":12886016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8594},{"shards":"0,2","objects":322}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":586364353,"num_objects":143159,"num_object_clones":0,"num_object_copies":286318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35874,"num_objects_misplaced":322,"num_objects_unfound":0,"num_objects_dirty":143159,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":152258,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6953,"num_bytes_recovered":27648000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1257009152,"data_stored":1256925135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160762,"ondisk_log_size":160762,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182469,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":845840,"kb_used_data":489340,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93526000,"statfs":{"total":96636764160,"available":95770624000,"internally_reserved":0,"allocated":501084160,"data_stored":500841472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,4,0,0,0,0,0,0,8],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986241,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703944,"kb_used_data":401524,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93667896,"statfs":{"total":96636764160,"available":95915925504,"internally_reserved":0,"allocated":411160576,"data_stored":410917893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309653162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,3,0,0,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659187,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":632388,"kb_used_data":339700,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93739452,"statfs":{"total":96636764160,"available":95989198848,"internally_reserved":0,"allocated":347852800,"data_stored":347541006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":299691694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":499384320,"data_stored":499380707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":347426816,"data_stored":347354097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":410198016,"data_stored":410190331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:34:20.943 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:34:20.943 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'11743", 'reported_seq': 11868, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:17.985579+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:34:17.985579+0000', 'last_peered': '2024-09-10T20:34:17.985579+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:34:17.985579+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36515840, 'num_objects': 8915, 'num_object_clones': 0, 'num_object_copies': 17830, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3585, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8915, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11743, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 23, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5330}, {'shards': '0,2', 'objects': 2193}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:20.943 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:34:20.943 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'11590", 'reported_seq': 11769, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:18.025828+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:34:18.025828+0000', 'last_peered': '2024-09-10T20:34:18.025828+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:34:18.025828+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36376576, 'num_objects': 8881, 'num_object_clones': 0, 'num_object_copies': 17762, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3595, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8881, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8454, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 21, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5288}, {'shards': '0,2', 'objects': 2262}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:20.943 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:34:20.943 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'11714", 'reported_seq': 11922, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:17.981313+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:34:17.981313+0000', 'last_peered': '2024-09-10T20:34:17.981313+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:34:17.981313+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36429824, 'num_objects': 8894, 'num_object_clones': 0, 'num_object_copies': 17788, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3522, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8894, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8309, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 35, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5374}, {'shards': '0,2', 'objects': 2148}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:20.943 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:34:20.943 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'11612", 'reported_seq': 11767, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:18.740045+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:34:18.740045+0000', 'last_peered': '2024-09-10T20:34:18.740045+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:34:18.740045+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36155392, 'num_objects': 8827, 'num_object_clones': 0, 'num_object_copies': 17654, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8827, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9605, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 11, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2136}, {'shards': '2', 'objects': 5312}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:34:20.943 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:34:20.944 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'11783", 'reported_seq': 11934, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:18.027152+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:34:18.027152+0000', 'last_peered': '2024-09-10T20:34:18.027152+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:34:18.027152+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36647878, 'num_objects': 8949, 'num_object_clones': 0, 'num_object_copies': 17898, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3596, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8949, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11781, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 23, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5355}, {'shards': '0,2', 'objects': 2201}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:20.944 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:34:20.944 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'11739", 'reported_seq': 11886, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:18.790291+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:34:18.790291+0000', 'last_peered': '2024-09-10T20:34:18.790291+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:34:18.790291+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36151296, 'num_objects': 8826, 'num_object_clones': 0, 'num_object_copies': 17652, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3626, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8826, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9567, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 9, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2179}, {'shards': '2', 'objects': 5202}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:34:20.944 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:34:20.944 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'11733", 'reported_seq': 11878, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:18.798217+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:34:18.798217+0000', 'last_peered': '2024-09-10T20:34:18.798217+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:34:18.798217+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36581376, 'num_objects': 8931, 'num_object_clones': 0, 'num_object_copies': 17862, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3604, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8931, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9519, 'num_write_kb': 32484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 8, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2212}, {'shards': '2', 'objects': 5329}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:34:20.944 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:34:20.944 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'11826", 'reported_seq': 12009, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:18.053575+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:34:18.053575+0000', 'last_peered': '2024-09-10T20:34:18.053575+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:34:18.053575+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36880384, 'num_objects': 9004, 'num_object_clones': 0, 'num_object_copies': 18008, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9004, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8549, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 25, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5381}, {'shards': '0,2', 'objects': 2238}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:20.944 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:34:20.944 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'11684", 'reported_seq': 11868, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:18.060822+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:34:18.060822+0000', 'last_peered': '2024-09-10T20:34:18.060822+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:34:18.060822+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36454400, 'num_objects': 8900, 'num_object_clones': 0, 'num_object_copies': 17800, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3512, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8900, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8378, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 23, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5391}, {'shards': '0,2', 'objects': 2144}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:20.944 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:34:20.946 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'11850", 'reported_seq': 12026, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:17.975819+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:34:17.975819+0000', 'last_peered': '2024-09-10T20:34:17.975819+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:34:17.975819+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37421056, 'num_objects': 9136, 'num_object_clones': 0, 'num_object_copies': 18272, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3693, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9136, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8677, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 22, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5445}, {'shards': '0,2', 'objects': 2358}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:20.946 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:34:20.946 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'11601", 'reported_seq': 18066, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:34:18.056364+0000', 'last_change': '2024-09-10T20:32:16.657652+0000', 'last_active': '2024-09-10T20:34:18.056364+0000', 'last_peered': '2024-09-10T20:34:18.056364+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:34:18.056364+0000', 'last_undegraded': '2024-09-10T20:34:18.056364+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36516323, 'num_objects': 8916, 'num_object_clones': 0, 'num_object_copies': 17832, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 322, 'num_objects_unfound': 0, 'num_objects_dirty': 8916, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8208, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3149, 'num_bytes_recovered': 12886016, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 8594}, {'shards': '0,2', 'objects': 322}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:20.947 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:34:21.716 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:21.716 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35874/286030 objects degraded (12.542%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":373,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":143015,"data_bytes":585766356,"bytes_used":2221424640,"bytes_avail":287688867840,"bytes_total":289910292480,"degraded_objects":35874,"degraded_total":286030,"degraded_ratio":0.12542041044645666,"misplaced_objects":322,"misplaced_total":286030,"misplaced_ratio":0.0011257560395762682,"recovering_objects_per_sec":34,"recovering_bytes_per_sec":135454,"recovering_keys_per_sec":0,"num_objects_recovered":417,"num_bytes_recovered":1626112,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":386},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:34:21.740 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:34:21.740 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:34:22.300 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:22.300 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:34:22.327 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":225,"stamp":"2024-09-10T20:34:20.932635+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":585766356,"num_objects":143015,"num_object_clones":0,"num_object_copies":286030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35874,"num_objects_misplaced":322,"num_objects_unfound":0,"num_objects_dirty":143015,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":152406,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6955,"num_bytes_recovered":27648019,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160711,"ondisk_log_size":160711,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2169360,"kb_used_data":1230232,"kb_used_omap":12,"kb_used_meta":939059,"kb_avail":280946160,"statfs":{"total":289910292480,"available":287688867840,"internally_reserved":0,"allocated":1259757568,"data_stored":1258960403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":961596418},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,4,4,8,3,0,2,0,0,0,8],"upper_bound":8192},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18984960,"num_objects":-4635,"num_object_clones":0,"num_object_copies":-9270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-20,"num_objects_misplaced":-397,"num_objects_unfound":0,"num_objects_dirty":-4635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4635,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":417,"num_bytes_recovered":1626112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004870"},"pg_stats":[{"pgid":"2.f","version":"45'11743","reported_seq":11868,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:17.985579+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:34:17.985579+0000","last_peered":"2024-09-10T20:34:17.985579+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:34:17.985579+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36515840,"num_objects":8915,"num_object_clones":0,"num_object_copies":17830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3585,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11743,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5330},{"shards":"0,2","objects":2193}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11966","reported_seq":12077,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:18.644272+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:34:18.644272+0000","last_peered":"2024-09-10T20:34:18.644272+0000","last_clean":"2024-09-10T20:34:18.644272+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:34:18.644272+0000","last_undegraded":"2024-09-10T20:34:18.644272+0000","last_fullsized":"2024-09-10T20:34:18.644272+0000","mapping_epoch":35,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36765696,"num_objects":8976,"num_object_clones":0,"num_object_copies":17952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10152,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11809","reported_seq":19157,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:19.327867+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:34:19.327867+0000","last_peered":"2024-09-10T20:34:19.327867+0000","last_clean":"2024-09-10T20:34:19.327867+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:34:19.327867+0000","last_undegraded":"2024-09-10T20:34:19.327867+0000","last_fullsized":"2024-09-10T20:34:19.327867+0000","mapping_epoch":44,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36479000,"num_objects":8907,"num_object_clones":0,"num_object_copies":17814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8907,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9937,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'11730","reported_seq":11811,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:18.709650+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:34:18.709650+0000","last_peered":"2024-09-10T20:34:18.709650+0000","last_clean":"2024-09-10T20:34:18.709650+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:34:18.709650+0000","last_undegraded":"2024-09-10T20:34:18.709650+0000","last_fullsized":"2024-09-10T20:34:18.709650+0000","mapping_epoch":35,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36347904,"num_objects":8874,"num_object_clones":0,"num_object_copies":17748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11730,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11958","reported_seq":12085,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:18.003922+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:34:18.003922+0000","last_peered":"2024-09-10T20:34:18.003922+0000","last_clean":"2024-09-10T20:34:18.003922+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:34:18.003922+0000","last_undegraded":"2024-09-10T20:34:18.003922+0000","last_fullsized":"2024-09-10T20:34:18.003922+0000","mapping_epoch":34,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37318656,"num_objects":9111,"num_object_clones":0,"num_object_copies":18222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8954,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11590","reported_seq":11769,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:18.025828+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:34:18.025828+0000","last_peered":"2024-09-10T20:34:18.025828+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:34:18.025828+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36376576,"num_objects":8881,"num_object_clones":0,"num_object_copies":17762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3595,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8454,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5288},{"shards":"0,2","objects":2262}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11714","reported_seq":11922,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:17.981313+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:34:17.981313+0000","last_peered":"2024-09-10T20:34:17.981313+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:34:17.981313+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36429824,"num_objects":8894,"num_object_clones":0,"num_object_copies":17788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3522,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8309,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5374},{"shards":"0,2","objects":2148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11770","reported_seq":11895,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:17.963717+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:34:17.963717+0000","last_peered":"2024-09-10T20:34:17.963717+0000","last_clean":"2024-09-10T20:34:17.963717+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:34:17.963717+0000","last_undegraded":"2024-09-10T20:34:17.963717+0000","last_fullsized":"2024-09-10T20:34:17.963717+0000","mapping_epoch":34,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36724736,"num_objects":8966,"num_object_clones":0,"num_object_copies":17932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8841,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187449+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187449+0000","last_peered":"2024-09-10T20:32:18.187449+0000","last_clean":"2024-09-10T20:32:18.187449+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187449+0000","last_undegraded":"2024-09-10T20:32:18.187449+0000","last_fullsized":"2024-09-10T20:32:18.187449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11612","reported_seq":11767,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:18.740045+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:34:18.740045+0000","last_peered":"2024-09-10T20:34:18.740045+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:34:18.740045+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36155392,"num_objects":8827,"num_object_clones":0,"num_object_copies":17654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9605,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2136},{"shards":"2","objects":5312}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190273+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190273+0000","last_peered":"2024-09-10T20:32:18.190273+0000","last_clean":"2024-09-10T20:32:18.190273+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190273+0000","last_undegraded":"2024-09-10T20:32:18.190273+0000","last_fullsized":"2024-09-10T20:32:18.190273+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11783","reported_seq":11934,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:18.027152+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:34:18.027152+0000","last_peered":"2024-09-10T20:34:18.027152+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:34:18.027152+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36647878,"num_objects":8949,"num_object_clones":0,"num_object_copies":17898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3596,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11781,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5355},{"shards":"0,2","objects":2201}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11739","reported_seq":11886,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:18.790291+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:34:18.790291+0000","last_peered":"2024-09-10T20:34:18.790291+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:34:18.790291+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36151296,"num_objects":8826,"num_object_clones":0,"num_object_copies":17652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3626,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9567,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2179},{"shards":"2","objects":5202}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187486+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187486+0000","last_peered":"2024-09-10T20:32:18.187486+0000","last_clean":"2024-09-10T20:32:18.187486+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187486+0000","last_undegraded":"2024-09-10T20:32:18.187486+0000","last_fullsized":"2024-09-10T20:32:18.187486+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11733","reported_seq":11878,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:18.798217+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:34:18.798217+0000","last_peered":"2024-09-10T20:34:18.798217+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:34:18.798217+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36581376,"num_objects":8931,"num_object_clones":0,"num_object_copies":17862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3604,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9519,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2212},{"shards":"2","objects":5329}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11826","reported_seq":12009,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:18.053575+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:34:18.053575+0000","last_peered":"2024-09-10T20:34:18.053575+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:34:18.053575+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36880384,"num_objects":9004,"num_object_clones":0,"num_object_copies":18008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8549,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5381},{"shards":"0,2","objects":2238}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187775+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187775+0000","last_peered":"2024-09-10T20:32:18.187775+0000","last_clean":"2024-09-10T20:32:18.187775+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187775+0000","last_undegraded":"2024-09-10T20:32:18.187775+0000","last_fullsized":"2024-09-10T20:32:18.187775+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11684","reported_seq":11868,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:18.060822+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:34:18.060822+0000","last_peered":"2024-09-10T20:34:18.060822+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:34:18.060822+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36454400,"num_objects":8900,"num_object_clones":0,"num_object_copies":17800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3512,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8378,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5391},{"shards":"0,2","objects":2144}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193230+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193230+0000","last_peered":"2024-09-10T20:32:18.193230+0000","last_clean":"2024-09-10T20:32:18.193230+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193230+0000","last_undegraded":"2024-09-10T20:32:18.193230+0000","last_fullsized":"2024-09-10T20:32:18.193230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11850","reported_seq":12026,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:17.975819+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:34:17.975819+0000","last_peered":"2024-09-10T20:34:17.975819+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:34:17.975819+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37421056,"num_objects":9136,"num_object_clones":0,"num_object_copies":18272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3693,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8677,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5445},{"shards":"0,2","objects":2358}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188324+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188324+0000","last_peered":"2024-09-10T20:32:18.188324+0000","last_clean":"2024-09-10T20:32:18.188324+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188324+0000","last_undegraded":"2024-09-10T20:32:18.188324+0000","last_fullsized":"2024-09-10T20:32:18.188324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188284+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188284+0000","last_peered":"2024-09-10T20:32:18.188284+0000","last_clean":"2024-09-10T20:32:18.188284+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188284+0000","last_undegraded":"2024-09-10T20:32:18.188284+0000","last_fullsized":"2024-09-10T20:32:18.188284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11601","reported_seq":18066,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:34:18.056364+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:34:18.056364+0000","last_peered":"2024-09-10T20:34:18.056364+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:34:18.056364+0000","last_undegraded":"2024-09-10T20:34:18.056364+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36516323,"num_objects":8916,"num_object_clones":0,"num_object_copies":17832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":322,"num_objects_unfound":0,"num_objects_dirty":8916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8208,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3149,"num_bytes_recovered":12886016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8594},{"shards":"0,2","objects":322}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":585766337,"num_objects":143013,"num_object_clones":0,"num_object_copies":286026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35874,"num_objects_misplaced":322,"num_objects_unfound":0,"num_objects_dirty":143013,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":152404,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6953,"num_bytes_recovered":27648000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1256607744,"data_stored":1256523727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160708,"ondisk_log_size":160708,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182469,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":845840,"kb_used_data":489340,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93526000,"statfs":{"total":96636764160,"available":95770624000,"internally_reserved":0,"allocated":501084160,"data_stored":500841472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,4,0,0,0,0,0,0,8],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986241,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703944,"kb_used_data":401524,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93667896,"statfs":{"total":96636764160,"available":95915925504,"internally_reserved":0,"allocated":411160576,"data_stored":410917893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309653162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,3,0,0,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659188,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":619576,"kb_used_data":339368,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93752264,"statfs":{"total":96636764160,"available":96002318336,"internally_reserved":0,"allocated":347512832,"data_stored":347201038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":286912174},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,4,1,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":499384320,"data_stored":499380707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":347025408,"data_stored":346952689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":410198016,"data_stored":410190331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:34:25.331 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:34:25.849 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:25.850 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:34:25.876 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":227,"stamp":"2024-09-10T20:34:24.934322+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":576865748,"num_objects":140842,"num_object_clones":0,"num_object_copies":281684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35862,"num_objects_misplaced":131,"num_objects_unfound":0,"num_objects_dirty":140842,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":154579,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7158,"num_bytes_recovered":28430355,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160884,"ondisk_log_size":160884,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2174996,"kb_used_data":1219232,"kb_used_omap":12,"kb_used_meta":955699,"kb_avail":280940524,"statfs":{"total":289910292480,"available":287683096576,"internally_reserved":0,"allocated":1248493568,"data_stored":1247696403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":978635778},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,2,6,8,3,0,0,10],"upper_bound":1024},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27312128,"num_objects":-6668,"num_object_clones":0,"num_object_copies":-13336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-32,"num_objects_misplaced":-588,"num_objects_unfound":0,"num_objects_dirty":-6668,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6668,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":620,"num_bytes_recovered":2408448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005336"},"pg_stats":[{"pgid":"2.f","version":"45'11873","reported_seq":12010,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:23.048586+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:34:23.048586+0000","last_peered":"2024-09-10T20:34:23.048586+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:34:23.048586+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35983360,"num_objects":8785,"num_object_clones":0,"num_object_copies":17570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3579,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11873,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5206},{"shards":"0,2","objects":2063}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12091","reported_seq":12202,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:23.796067+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:34:23.796067+0000","last_peered":"2024-09-10T20:34:23.796067+0000","last_clean":"2024-09-10T20:34:23.796067+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:34:23.796067+0000","last_undegraded":"2024-09-10T20:34:23.796067+0000","last_fullsized":"2024-09-10T20:34:23.796067+0000","mapping_epoch":35,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36253696,"num_objects":8851,"num_object_clones":0,"num_object_copies":17702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10277,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11944","reported_seq":19292,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:24.360399+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:34:24.360399+0000","last_peered":"2024-09-10T20:34:24.360399+0000","last_clean":"2024-09-10T20:34:24.360399+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:34:24.360399+0000","last_undegraded":"2024-09-10T20:34:24.360399+0000","last_fullsized":"2024-09-10T20:34:24.360399+0000","mapping_epoch":44,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35926040,"num_objects":8772,"num_object_clones":0,"num_object_copies":17544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8772,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10072,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'11862","reported_seq":11943,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:23.703980+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:34:23.703980+0000","last_peered":"2024-09-10T20:34:23.703980+0000","last_clean":"2024-09-10T20:34:23.703980+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:34:23.703980+0000","last_undegraded":"2024-09-10T20:34:23.703980+0000","last_fullsized":"2024-09-10T20:34:23.703980+0000","mapping_epoch":35,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35807232,"num_objects":8742,"num_object_clones":0,"num_object_copies":17484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11862,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12089","reported_seq":12216,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:23.076415+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:34:23.076415+0000","last_peered":"2024-09-10T20:34:23.076415+0000","last_clean":"2024-09-10T20:34:23.076415+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:34:23.076415+0000","last_undegraded":"2024-09-10T20:34:23.076415+0000","last_fullsized":"2024-09-10T20:34:23.076415+0000","mapping_epoch":34,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36782080,"num_objects":8980,"num_object_clones":0,"num_object_copies":17960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8980,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9085,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11732","reported_seq":11913,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:22.995921+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:34:22.995921+0000","last_peered":"2024-09-10T20:34:22.995921+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:34:22.995921+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35794944,"num_objects":8739,"num_object_clones":0,"num_object_copies":17478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3594,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8596,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5147},{"shards":"0,2","objects":2120}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11852","reported_seq":12061,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:23.091195+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:34:23.091195+0000","last_peered":"2024-09-10T20:34:23.091195+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:34:23.091195+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35864576,"num_objects":8756,"num_object_clones":0,"num_object_copies":17512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3522,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8447,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5236},{"shards":"0,2","objects":2010}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11889","reported_seq":12014,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:22.797257+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:34:22.797257+0000","last_peered":"2024-09-10T20:34:22.797257+0000","last_clean":"2024-09-10T20:34:22.797257+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:34:22.797257+0000","last_undegraded":"2024-09-10T20:34:22.797257+0000","last_fullsized":"2024-09-10T20:34:22.797257+0000","mapping_epoch":34,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36237312,"num_objects":8847,"num_object_clones":0,"num_object_copies":17694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8847,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8960,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187449+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187449+0000","last_peered":"2024-09-10T20:32:18.187449+0000","last_clean":"2024-09-10T20:32:18.187449+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187449+0000","last_undegraded":"2024-09-10T20:32:18.187449+0000","last_fullsized":"2024-09-10T20:32:18.187449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11745","reported_seq":11900,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:23.689469+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:34:23.689469+0000","last_peered":"2024-09-10T20:34:23.689469+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:34:23.689469+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35610624,"num_objects":8694,"num_object_clones":0,"num_object_copies":17388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9738,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2003},{"shards":"2","objects":5179}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190273+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190273+0000","last_peered":"2024-09-10T20:32:18.190273+0000","last_clean":"2024-09-10T20:32:18.190273+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190273+0000","last_undegraded":"2024-09-10T20:32:18.190273+0000","last_fullsized":"2024-09-10T20:32:18.190273+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11912","reported_seq":12065,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:22.996717+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:34:22.996717+0000","last_peered":"2024-09-10T20:34:22.996717+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:34:22.996717+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36119494,"num_objects":8820,"num_object_clones":0,"num_object_copies":17640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3595,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8820,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11910,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5227},{"shards":"0,2","objects":2072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11856","reported_seq":12003,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:23.807747+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:34:23.807747+0000","last_peered":"2024-09-10T20:34:23.807747+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:34:23.807747+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35672064,"num_objects":8709,"num_object_clones":0,"num_object_copies":17418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3626,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9684,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2062},{"shards":"2","objects":5085}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187486+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187486+0000","last_peered":"2024-09-10T20:32:18.187486+0000","last_clean":"2024-09-10T20:32:18.187486+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187486+0000","last_undegraded":"2024-09-10T20:32:18.187486+0000","last_fullsized":"2024-09-10T20:32:18.187486+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11896","reported_seq":12041,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:23.616611+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:34:23.616611+0000","last_peered":"2024-09-10T20:34:23.616611+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:34:23.616611+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35913728,"num_objects":8768,"num_object_clones":0,"num_object_copies":17536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3604,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9682,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2049},{"shards":"2","objects":5166}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11980","reported_seq":12168,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:23.021047+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:34:23.021047+0000","last_peered":"2024-09-10T20:34:23.021047+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:34:23.021047+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36249600,"num_objects":8850,"num_object_clones":0,"num_object_copies":17700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3623,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8703,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5228},{"shards":"0,2","objects":2084}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187775+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187775+0000","last_peered":"2024-09-10T20:32:18.187775+0000","last_clean":"2024-09-10T20:32:18.187775+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187775+0000","last_undegraded":"2024-09-10T20:32:18.187775+0000","last_fullsized":"2024-09-10T20:32:18.187775+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11813","reported_seq":11998,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:23.053925+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:34:23.053925+0000","last_peered":"2024-09-10T20:34:23.053925+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:34:23.053925+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35926016,"num_objects":8771,"num_object_clones":0,"num_object_copies":17542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3512,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8507,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5262},{"shards":"0,2","objects":2015}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193230+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193230+0000","last_peered":"2024-09-10T20:32:18.193230+0000","last_clean":"2024-09-10T20:32:18.193230+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193230+0000","last_undegraded":"2024-09-10T20:32:18.193230+0000","last_fullsized":"2024-09-10T20:32:18.193230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12007","reported_seq":12189,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:23.028359+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:34:23.028359+0000","last_peered":"2024-09-10T20:34:23.028359+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:34:23.028359+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36777984,"num_objects":8979,"num_object_clones":0,"num_object_copies":17958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3690,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8834,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5291},{"shards":"0,2","objects":2201}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188324+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188324+0000","last_peered":"2024-09-10T20:32:18.188324+0000","last_clean":"2024-09-10T20:32:18.188324+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188324+0000","last_undegraded":"2024-09-10T20:32:18.188324+0000","last_fullsized":"2024-09-10T20:32:18.188324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188372+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188372+0000","last_peered":"2024-09-10T20:32:18.188372+0000","last_clean":"2024-09-10T20:32:18.188372+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188372+0000","last_undegraded":"2024-09-10T20:32:18.188372+0000","last_fullsized":"2024-09-10T20:32:18.188372+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11740","reported_seq":18587,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:34:23.100958+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:34:23.100958+0000","last_peered":"2024-09-10T20:34:23.100958+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:34:23.100958+0000","last_undegraded":"2024-09-10T20:34:23.100958+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35946979,"num_objects":8777,"num_object_clones":0,"num_object_copies":17554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":131,"num_objects_unfound":0,"num_objects_dirty":8777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8347,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13668352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8646},{"shards":"0,2","objects":131}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":576865729,"num_objects":140840,"num_object_clones":0,"num_object_copies":281680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35862,"num_objects_misplaced":131,"num_objects_unfound":0,"num_objects_dirty":140840,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":154577,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7156,"num_bytes_recovered":28430336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1245405184,"data_stored":1245321167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160881,"ondisk_log_size":160881,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182470,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":843572,"kb_used_data":482912,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93528268,"statfs":{"total":96636764160,"available":95772946432,"internally_reserved":0,"allocated":494501888,"data_stored":494259200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,3,2,0,0,6],"upper_bound":1024},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986242,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703812,"kb_used_data":397236,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93668028,"statfs":{"total":96636764160,"available":95916060672,"internally_reserved":0,"allocated":406769664,"data_stored":406526981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313913002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,0,1,3,0,0,0,4],"upper_bound":1024},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659189,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627612,"kb_used_data":339084,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93744228,"statfs":{"total":96636764160,"available":95994089472,"internally_reserved":0,"allocated":347222016,"data_stored":346910222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":295431854},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":492748800,"data_stored":492745187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":346791936,"data_stored":346719217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":405864448,"data_stored":405856763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:34:25.878 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:34:26.395 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:26.396 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:34:26.410 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":228,"stamp":"2024-09-10T20:34:26.076119+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":576865748,"num_objects":140842,"num_object_clones":0,"num_object_copies":281684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35862,"num_objects_misplaced":131,"num_objects_unfound":0,"num_objects_dirty":140842,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":154579,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7158,"num_bytes_recovered":28430355,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160884,"ondisk_log_size":160884,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2174996,"kb_used_data":1219232,"kb_used_omap":12,"kb_used_meta":955699,"kb_avail":280940524,"statfs":{"total":289910292480,"available":287683096576,"internally_reserved":0,"allocated":1248493568,"data_stored":1247696403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":978635778},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,2,6,8,3,0,0,10],"upper_bound":1024},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18141184,"num_objects":-4429,"num_object_clones":0,"num_object_copies":-8858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-26,"num_objects_misplaced":-385,"num_objects_unfound":0,"num_objects_dirty":-4429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4429,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":411,"num_bytes_recovered":1576960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.146610"},"pg_stats":[{"pgid":"2.f","version":"45'11873","reported_seq":12010,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:23.048586+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:34:23.048586+0000","last_peered":"2024-09-10T20:34:23.048586+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:34:23.048586+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35983360,"num_objects":8785,"num_object_clones":0,"num_object_copies":17570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3579,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11873,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5206},{"shards":"0,2","objects":2063}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12091","reported_seq":12202,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:23.796067+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:34:23.796067+0000","last_peered":"2024-09-10T20:34:23.796067+0000","last_clean":"2024-09-10T20:34:23.796067+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:34:23.796067+0000","last_undegraded":"2024-09-10T20:34:23.796067+0000","last_fullsized":"2024-09-10T20:34:23.796067+0000","mapping_epoch":35,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36253696,"num_objects":8851,"num_object_clones":0,"num_object_copies":17702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10277,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11944","reported_seq":19292,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:24.360399+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:34:24.360399+0000","last_peered":"2024-09-10T20:34:24.360399+0000","last_clean":"2024-09-10T20:34:24.360399+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:34:24.360399+0000","last_undegraded":"2024-09-10T20:34:24.360399+0000","last_fullsized":"2024-09-10T20:34:24.360399+0000","mapping_epoch":44,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35926040,"num_objects":8772,"num_object_clones":0,"num_object_copies":17544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8772,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10072,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'11862","reported_seq":11943,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:23.703980+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:34:23.703980+0000","last_peered":"2024-09-10T20:34:23.703980+0000","last_clean":"2024-09-10T20:34:23.703980+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:34:23.703980+0000","last_undegraded":"2024-09-10T20:34:23.703980+0000","last_fullsized":"2024-09-10T20:34:23.703980+0000","mapping_epoch":35,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35807232,"num_objects":8742,"num_object_clones":0,"num_object_copies":17484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11862,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12089","reported_seq":12216,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:23.076415+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:34:23.076415+0000","last_peered":"2024-09-10T20:34:23.076415+0000","last_clean":"2024-09-10T20:34:23.076415+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:34:23.076415+0000","last_undegraded":"2024-09-10T20:34:23.076415+0000","last_fullsized":"2024-09-10T20:34:23.076415+0000","mapping_epoch":34,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36782080,"num_objects":8980,"num_object_clones":0,"num_object_copies":17960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8980,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9085,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11732","reported_seq":11913,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:22.995921+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:34:22.995921+0000","last_peered":"2024-09-10T20:34:22.995921+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:34:22.995921+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35794944,"num_objects":8739,"num_object_clones":0,"num_object_copies":17478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3594,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8596,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5147},{"shards":"0,2","objects":2120}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11852","reported_seq":12061,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:23.091195+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:34:23.091195+0000","last_peered":"2024-09-10T20:34:23.091195+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:34:23.091195+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35864576,"num_objects":8756,"num_object_clones":0,"num_object_copies":17512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3522,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8447,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5236},{"shards":"0,2","objects":2010}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11889","reported_seq":12014,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:22.797257+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:34:22.797257+0000","last_peered":"2024-09-10T20:34:22.797257+0000","last_clean":"2024-09-10T20:34:22.797257+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:34:22.797257+0000","last_undegraded":"2024-09-10T20:34:22.797257+0000","last_fullsized":"2024-09-10T20:34:22.797257+0000","mapping_epoch":34,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36237312,"num_objects":8847,"num_object_clones":0,"num_object_copies":17694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8847,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8960,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187449+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187449+0000","last_peered":"2024-09-10T20:32:18.187449+0000","last_clean":"2024-09-10T20:32:18.187449+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187449+0000","last_undegraded":"2024-09-10T20:32:18.187449+0000","last_fullsized":"2024-09-10T20:32:18.187449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11745","reported_seq":11900,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:23.689469+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:34:23.689469+0000","last_peered":"2024-09-10T20:34:23.689469+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:34:23.689469+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35610624,"num_objects":8694,"num_object_clones":0,"num_object_copies":17388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9738,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2003},{"shards":"2","objects":5179}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190273+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190273+0000","last_peered":"2024-09-10T20:32:18.190273+0000","last_clean":"2024-09-10T20:32:18.190273+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190273+0000","last_undegraded":"2024-09-10T20:32:18.190273+0000","last_fullsized":"2024-09-10T20:32:18.190273+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11912","reported_seq":12065,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:22.996717+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:34:22.996717+0000","last_peered":"2024-09-10T20:34:22.996717+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:34:22.996717+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36119494,"num_objects":8820,"num_object_clones":0,"num_object_copies":17640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3595,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8820,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11910,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5227},{"shards":"0,2","objects":2072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11856","reported_seq":12003,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:23.807747+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:34:23.807747+0000","last_peered":"2024-09-10T20:34:23.807747+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:34:23.807747+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35672064,"num_objects":8709,"num_object_clones":0,"num_object_copies":17418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3626,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9684,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2062},{"shards":"2","objects":5085}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187486+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187486+0000","last_peered":"2024-09-10T20:32:18.187486+0000","last_clean":"2024-09-10T20:32:18.187486+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187486+0000","last_undegraded":"2024-09-10T20:32:18.187486+0000","last_fullsized":"2024-09-10T20:32:18.187486+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11896","reported_seq":12041,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:23.616611+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:34:23.616611+0000","last_peered":"2024-09-10T20:34:23.616611+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:34:23.616611+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35913728,"num_objects":8768,"num_object_clones":0,"num_object_copies":17536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3604,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9682,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2049},{"shards":"2","objects":5166}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11980","reported_seq":12168,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:23.021047+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:34:23.021047+0000","last_peered":"2024-09-10T20:34:23.021047+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:34:23.021047+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36249600,"num_objects":8850,"num_object_clones":0,"num_object_copies":17700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3623,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8703,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5228},{"shards":"0,2","objects":2084}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187775+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187775+0000","last_peered":"2024-09-10T20:32:18.187775+0000","last_clean":"2024-09-10T20:32:18.187775+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187775+0000","last_undegraded":"2024-09-10T20:32:18.187775+0000","last_fullsized":"2024-09-10T20:32:18.187775+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11813","reported_seq":11998,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:23.053925+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:34:23.053925+0000","last_peered":"2024-09-10T20:34:23.053925+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:34:23.053925+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35926016,"num_objects":8771,"num_object_clones":0,"num_object_copies":17542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3512,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8507,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5262},{"shards":"0,2","objects":2015}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193230+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193230+0000","last_peered":"2024-09-10T20:32:18.193230+0000","last_clean":"2024-09-10T20:32:18.193230+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193230+0000","last_undegraded":"2024-09-10T20:32:18.193230+0000","last_fullsized":"2024-09-10T20:32:18.193230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12007","reported_seq":12189,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:23.028359+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:34:23.028359+0000","last_peered":"2024-09-10T20:34:23.028359+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:34:23.028359+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36777984,"num_objects":8979,"num_object_clones":0,"num_object_copies":17958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3690,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8834,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5291},{"shards":"0,2","objects":2201}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188324+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188324+0000","last_peered":"2024-09-10T20:32:18.188324+0000","last_clean":"2024-09-10T20:32:18.188324+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188324+0000","last_undegraded":"2024-09-10T20:32:18.188324+0000","last_fullsized":"2024-09-10T20:32:18.188324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188372+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188372+0000","last_peered":"2024-09-10T20:32:18.188372+0000","last_clean":"2024-09-10T20:32:18.188372+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188372+0000","last_undegraded":"2024-09-10T20:32:18.188372+0000","last_fullsized":"2024-09-10T20:32:18.188372+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11740","reported_seq":18587,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:34:23.100958+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:34:23.100958+0000","last_peered":"2024-09-10T20:34:23.100958+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:34:23.100958+0000","last_undegraded":"2024-09-10T20:34:23.100958+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35946979,"num_objects":8777,"num_object_clones":0,"num_object_copies":17554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":131,"num_objects_unfound":0,"num_objects_dirty":8777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8347,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13668352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8646},{"shards":"0,2","objects":131}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":576865729,"num_objects":140840,"num_object_clones":0,"num_object_copies":281680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35862,"num_objects_misplaced":131,"num_objects_unfound":0,"num_objects_dirty":140840,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":154577,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7156,"num_bytes_recovered":28430336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1245405184,"data_stored":1245321167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160881,"ondisk_log_size":160881,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182470,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":843572,"kb_used_data":482912,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93528268,"statfs":{"total":96636764160,"available":95772946432,"internally_reserved":0,"allocated":494501888,"data_stored":494259200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,3,2,0,0,6],"upper_bound":1024},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986242,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703812,"kb_used_data":397236,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93668028,"statfs":{"total":96636764160,"available":95916060672,"internally_reserved":0,"allocated":406769664,"data_stored":406526981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313913002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,0,1,3,0,0,0,4],"upper_bound":1024},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659189,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627612,"kb_used_data":339084,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93744228,"statfs":{"total":96636764160,"available":95994089472,"internally_reserved":0,"allocated":347222016,"data_stored":346910222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":295431854},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":492748800,"data_stored":492745187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":346791936,"data_stored":346719217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":405864448,"data_stored":405856763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:34:26.412 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:34:26.413 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'11873", 'reported_seq': 12010, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:23.048586+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:34:23.048586+0000', 'last_peered': '2024-09-10T20:34:23.048586+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:34:23.048586+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35983360, 'num_objects': 8785, 'num_object_clones': 0, 'num_object_copies': 17570, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3579, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8785, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11873, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5206}, {'shards': '0,2', 'objects': 2063}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:26.413 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:34:26.413 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'11732", 'reported_seq': 11913, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:22.995921+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:34:22.995921+0000', 'last_peered': '2024-09-10T20:34:22.995921+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:34:22.995921+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35794944, 'num_objects': 8739, 'num_object_clones': 0, 'num_object_copies': 17478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3594, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8739, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8596, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 22, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5147}, {'shards': '0,2', 'objects': 2120}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:26.413 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:34:26.413 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'11852", 'reported_seq': 12061, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:23.091195+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:34:23.091195+0000', 'last_peered': '2024-09-10T20:34:23.091195+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:34:23.091195+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35864576, 'num_objects': 8756, 'num_object_clones': 0, 'num_object_copies': 17512, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3522, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8756, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8447, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 35, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5236}, {'shards': '0,2', 'objects': 2010}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:26.414 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:34:26.414 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'11745", 'reported_seq': 11900, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:23.689469+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:34:23.689469+0000', 'last_peered': '2024-09-10T20:34:23.689469+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:34:23.689469+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35610624, 'num_objects': 8694, 'num_object_clones': 0, 'num_object_copies': 17388, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8694, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9738, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 11, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2003}, {'shards': '2', 'objects': 5179}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:34:26.414 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:34:26.414 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'11912", 'reported_seq': 12065, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:22.996717+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:34:22.996717+0000', 'last_peered': '2024-09-10T20:34:22.996717+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:34:22.996717+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36119494, 'num_objects': 8820, 'num_object_clones': 0, 'num_object_copies': 17640, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3595, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8820, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11910, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 24, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5227}, {'shards': '0,2', 'objects': 2072}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:26.414 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:34:26.414 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'11856", 'reported_seq': 12003, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:23.807747+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:34:23.807747+0000', 'last_peered': '2024-09-10T20:34:23.807747+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:34:23.807747+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35672064, 'num_objects': 8709, 'num_object_clones': 0, 'num_object_copies': 17418, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3626, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8709, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9684, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 9, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2062}, {'shards': '2', 'objects': 5085}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:34:26.414 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:34:26.414 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'11896", 'reported_seq': 12041, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:23.616611+0000', 'last_change': '2024-09-10T20:30:05.647133+0000', 'last_active': '2024-09-10T20:34:23.616611+0000', 'last_peered': '2024-09-10T20:34:23.616611+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:34:23.616611+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35913728, 'num_objects': 8768, 'num_object_clones': 0, 'num_object_copies': 17536, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3604, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8768, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9682, 'num_write_kb': 32484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 8, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2049}, {'shards': '2', 'objects': 5166}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:34:26.414 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:34:26.415 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'11980", 'reported_seq': 12168, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:23.021047+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:34:23.021047+0000', 'last_peered': '2024-09-10T20:34:23.021047+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:34:23.021047+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36249600, 'num_objects': 8850, 'num_object_clones': 0, 'num_object_copies': 17700, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3623, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8850, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8703, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 26, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5228}, {'shards': '0,2', 'objects': 2084}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:26.415 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:34:26.415 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'11813", 'reported_seq': 11998, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:23.053925+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:34:23.053925+0000', 'last_peered': '2024-09-10T20:34:23.053925+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:34:23.053925+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35926016, 'num_objects': 8771, 'num_object_clones': 0, 'num_object_copies': 17542, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3512, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8771, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8507, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 23, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5262}, {'shards': '0,2', 'objects': 2015}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:26.415 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:34:26.415 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'12007", 'reported_seq': 12189, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:23.028359+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:34:23.028359+0000', 'last_peered': '2024-09-10T20:34:23.028359+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:34:23.028359+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36777984, 'num_objects': 8979, 'num_object_clones': 0, 'num_object_copies': 17958, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3690, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8979, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8834, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 25, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5291}, {'shards': '0,2', 'objects': 2201}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:26.415 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T20:34:26.415 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'11740", 'reported_seq': 18587, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T20:34:23.100958+0000', 'last_change': '2024-09-10T20:32:16.657652+0000', 'last_active': '2024-09-10T20:34:23.100958+0000', 'last_peered': '2024-09-10T20:34:23.100958+0000', 'last_clean': '2024-09-10T20:30:03.302909+0000', 'last_became_active': '2024-09-10T20:30:05.684434+0000', 'last_became_peered': '2024-09-10T20:30:05.684434+0000', 'last_unstale': '2024-09-10T20:34:23.100958+0000', 'last_undegraded': '2024-09-10T20:34:23.100958+0000', 'last_fullsized': '2024-09-10T20:30:05.305646+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3446", 'last_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:02.819432+0000', 'objects_scrubbed': 3382, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1287, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35946979, 'num_objects': 8777, 'num_object_clones': 0, 'num_object_copies': 17554, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 131, 'num_objects_unfound': 0, 'num_objects_dirty': 8777, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8347, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3340, 'num_bytes_recovered': 13668352, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 8646}, {'shards': '0,2', 'objects': 131}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:26.416 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:34:26.887 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:26.887 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35862/281684 objects degraded (12.731%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":378,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":140842,"data_bytes":576865748,"bytes_used":2227195904,"bytes_avail":287683096576,"bytes_total":289910292480,"degraded_objects":35862,"degraded_total":281684,"degraded_ratio":0.12731287542068417,"misplaced_objects":131,"misplaced_total":281684,"misplaced_ratio":0.0004650601383110152,"recovering_objects_per_sec":51,"recovering_bytes_per_sec":200614,"recovering_keys_per_sec":0,"num_objects_recovered":620,"num_bytes_recovered":2408448,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":555},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:34:26.902 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:34:26.902 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:34:27.298 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:27.300 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:34:27.327 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":230,"stamp":"2024-09-10T20:34:27.073628+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":576865748,"num_objects":140842,"num_object_clones":0,"num_object_copies":281684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35862,"num_objects_misplaced":131,"num_objects_unfound":0,"num_objects_dirty":140842,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":154579,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7158,"num_bytes_recovered":28430355,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160884,"ondisk_log_size":160884,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2174996,"kb_used_data":1219232,"kb_used_omap":12,"kb_used_meta":955699,"kb_avail":280940524,"statfs":{"total":289910292480,"available":287683096576,"internally_reserved":0,"allocated":1248493568,"data_stored":1247696403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":978635778},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,2,6,8,3,0,0,10],"upper_bound":1024},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-9498624,"num_objects":-2319,"num_object_clones":0,"num_object_copies":-4638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-12,"num_objects_misplaced":-191,"num_objects_unfound":0,"num_objects_dirty":-2319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2319,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":782336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.141841"},"pg_stats":[{"pgid":"2.f","version":"45'11873","reported_seq":12010,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:23.048586+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:34:23.048586+0000","last_peered":"2024-09-10T20:34:23.048586+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:34:23.048586+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35983360,"num_objects":8785,"num_object_clones":0,"num_object_copies":17570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3579,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11873,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5206},{"shards":"0,2","objects":2063}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12091","reported_seq":12202,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:23.796067+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:34:23.796067+0000","last_peered":"2024-09-10T20:34:23.796067+0000","last_clean":"2024-09-10T20:34:23.796067+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:34:23.796067+0000","last_undegraded":"2024-09-10T20:34:23.796067+0000","last_fullsized":"2024-09-10T20:34:23.796067+0000","mapping_epoch":35,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36253696,"num_objects":8851,"num_object_clones":0,"num_object_copies":17702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10277,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11944","reported_seq":19292,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:24.360399+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:34:24.360399+0000","last_peered":"2024-09-10T20:34:24.360399+0000","last_clean":"2024-09-10T20:34:24.360399+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:34:24.360399+0000","last_undegraded":"2024-09-10T20:34:24.360399+0000","last_fullsized":"2024-09-10T20:34:24.360399+0000","mapping_epoch":44,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35926040,"num_objects":8772,"num_object_clones":0,"num_object_copies":17544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8772,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10072,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'11862","reported_seq":11943,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:23.703980+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:34:23.703980+0000","last_peered":"2024-09-10T20:34:23.703980+0000","last_clean":"2024-09-10T20:34:23.703980+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:34:23.703980+0000","last_undegraded":"2024-09-10T20:34:23.703980+0000","last_fullsized":"2024-09-10T20:34:23.703980+0000","mapping_epoch":35,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35807232,"num_objects":8742,"num_object_clones":0,"num_object_copies":17484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11862,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12089","reported_seq":12216,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:23.076415+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:34:23.076415+0000","last_peered":"2024-09-10T20:34:23.076415+0000","last_clean":"2024-09-10T20:34:23.076415+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:34:23.076415+0000","last_undegraded":"2024-09-10T20:34:23.076415+0000","last_fullsized":"2024-09-10T20:34:23.076415+0000","mapping_epoch":34,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36782080,"num_objects":8980,"num_object_clones":0,"num_object_copies":17960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8980,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9085,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11732","reported_seq":11913,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:22.995921+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:34:22.995921+0000","last_peered":"2024-09-10T20:34:22.995921+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:34:22.995921+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35794944,"num_objects":8739,"num_object_clones":0,"num_object_copies":17478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3594,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8596,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5147},{"shards":"0,2","objects":2120}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11852","reported_seq":12061,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:23.091195+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:34:23.091195+0000","last_peered":"2024-09-10T20:34:23.091195+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:34:23.091195+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35864576,"num_objects":8756,"num_object_clones":0,"num_object_copies":17512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3522,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8447,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5236},{"shards":"0,2","objects":2010}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11889","reported_seq":12014,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:22.797257+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:34:22.797257+0000","last_peered":"2024-09-10T20:34:22.797257+0000","last_clean":"2024-09-10T20:34:22.797257+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:34:22.797257+0000","last_undegraded":"2024-09-10T20:34:22.797257+0000","last_fullsized":"2024-09-10T20:34:22.797257+0000","mapping_epoch":34,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36237312,"num_objects":8847,"num_object_clones":0,"num_object_copies":17694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8847,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8960,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187449+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187449+0000","last_peered":"2024-09-10T20:32:18.187449+0000","last_clean":"2024-09-10T20:32:18.187449+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187449+0000","last_undegraded":"2024-09-10T20:32:18.187449+0000","last_fullsized":"2024-09-10T20:32:18.187449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11745","reported_seq":11900,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:23.689469+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:34:23.689469+0000","last_peered":"2024-09-10T20:34:23.689469+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:34:23.689469+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35610624,"num_objects":8694,"num_object_clones":0,"num_object_copies":17388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9738,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2003},{"shards":"2","objects":5179}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.190273+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:32:18.190273+0000","last_peered":"2024-09-10T20:32:18.190273+0000","last_clean":"2024-09-10T20:32:18.190273+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:32:18.190273+0000","last_undegraded":"2024-09-10T20:32:18.190273+0000","last_fullsized":"2024-09-10T20:32:18.190273+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11912","reported_seq":12065,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:22.996717+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:34:22.996717+0000","last_peered":"2024-09-10T20:34:22.996717+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:34:22.996717+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36119494,"num_objects":8820,"num_object_clones":0,"num_object_copies":17640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3595,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8820,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11910,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5227},{"shards":"0,2","objects":2072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.182790+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:32:18.182790+0000","last_peered":"2024-09-10T20:32:18.182790+0000","last_clean":"2024-09-10T20:32:18.182790+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:32:18.182790+0000","last_undegraded":"2024-09-10T20:32:18.182790+0000","last_fullsized":"2024-09-10T20:32:18.182790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11856","reported_seq":12003,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:23.807747+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:34:23.807747+0000","last_peered":"2024-09-10T20:34:23.807747+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:34:23.807747+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35672064,"num_objects":8709,"num_object_clones":0,"num_object_copies":17418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3626,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9684,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2062},{"shards":"2","objects":5085}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187486+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187486+0000","last_peered":"2024-09-10T20:32:18.187486+0000","last_clean":"2024-09-10T20:32:18.187486+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187486+0000","last_undegraded":"2024-09-10T20:32:18.187486+0000","last_fullsized":"2024-09-10T20:32:18.187486+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11896","reported_seq":12041,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:23.616611+0000","last_change":"2024-09-10T20:30:05.647133+0000","last_active":"2024-09-10T20:34:23.616611+0000","last_peered":"2024-09-10T20:34:23.616611+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:34:23.616611+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35913728,"num_objects":8768,"num_object_clones":0,"num_object_copies":17536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3604,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9682,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2049},{"shards":"2","objects":5166}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11980","reported_seq":12168,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:23.021047+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:34:23.021047+0000","last_peered":"2024-09-10T20:34:23.021047+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:34:23.021047+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36249600,"num_objects":8850,"num_object_clones":0,"num_object_copies":17700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3623,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8703,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5228},{"shards":"0,2","objects":2084}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187775+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187775+0000","last_peered":"2024-09-10T20:32:18.187775+0000","last_clean":"2024-09-10T20:32:18.187775+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187775+0000","last_undegraded":"2024-09-10T20:32:18.187775+0000","last_fullsized":"2024-09-10T20:32:18.187775+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11813","reported_seq":11998,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:23.053925+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:34:23.053925+0000","last_peered":"2024-09-10T20:34:23.053925+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:34:23.053925+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35926016,"num_objects":8771,"num_object_clones":0,"num_object_copies":17542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3512,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8507,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5262},{"shards":"0,2","objects":2015}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.193230+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:32:18.193230+0000","last_peered":"2024-09-10T20:32:18.193230+0000","last_clean":"2024-09-10T20:32:18.193230+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:32:18.193230+0000","last_undegraded":"2024-09-10T20:32:18.193230+0000","last_fullsized":"2024-09-10T20:32:18.193230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12007","reported_seq":12189,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:23.028359+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:34:23.028359+0000","last_peered":"2024-09-10T20:34:23.028359+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:34:23.028359+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36777984,"num_objects":8979,"num_object_clones":0,"num_object_copies":17958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3690,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8834,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5291},{"shards":"0,2","objects":2201}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188324+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188324+0000","last_peered":"2024-09-10T20:32:18.188324+0000","last_clean":"2024-09-10T20:32:18.188324+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188324+0000","last_undegraded":"2024-09-10T20:32:18.188324+0000","last_fullsized":"2024-09-10T20:32:18.188324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188372+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188372+0000","last_peered":"2024-09-10T20:32:18.188372+0000","last_clean":"2024-09-10T20:32:18.188372+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188372+0000","last_undegraded":"2024-09-10T20:32:18.188372+0000","last_fullsized":"2024-09-10T20:32:18.188372+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11740","reported_seq":18587,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:34:23.100958+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:34:23.100958+0000","last_peered":"2024-09-10T20:34:23.100958+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:34:23.100958+0000","last_undegraded":"2024-09-10T20:34:23.100958+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35946979,"num_objects":8777,"num_object_clones":0,"num_object_copies":17554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":131,"num_objects_unfound":0,"num_objects_dirty":8777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8347,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13668352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8646},{"shards":"0,2","objects":131}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":576865729,"num_objects":140840,"num_object_clones":0,"num_object_copies":281680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35862,"num_objects_misplaced":131,"num_objects_unfound":0,"num_objects_dirty":140840,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":154577,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7156,"num_bytes_recovered":28430336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1245405184,"data_stored":1245321167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160881,"ondisk_log_size":160881,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182470,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":843572,"kb_used_data":482912,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93528268,"statfs":{"total":96636764160,"available":95772946432,"internally_reserved":0,"allocated":494501888,"data_stored":494259200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,3,2,0,0,6],"upper_bound":1024},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986242,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703812,"kb_used_data":397236,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93668028,"statfs":{"total":96636764160,"available":95916060672,"internally_reserved":0,"allocated":406769664,"data_stored":406526981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313913002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,0,1,3,0,0,0,4],"upper_bound":1024},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659189,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627612,"kb_used_data":339084,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93744228,"statfs":{"total":96636764160,"available":95994089472,"internally_reserved":0,"allocated":347222016,"data_stored":346910222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":295431854},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":492748800,"data_stored":492745187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":346791936,"data_stored":346719217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":405864448,"data_stored":405856763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:34:30.330 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:34:30.889 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:30.889 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:34:30.913 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2024-09-10T20:34:28.936907+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":570062292,"num_objects":139181,"num_object_clones":0,"num_object_copies":278362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33852,"num_objects_misplaced":1992,"num_objects_unfound":0,"num_objects_dirty":139181,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":156240,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7305,"num_bytes_recovered":29007891,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2173824,"kb_used_data":1209728,"kb_used_omap":12,"kb_used_meta":964019,"kb_avail":280941696,"statfs":{"total":289910292480,"available":287684296704,"internally_reserved":0,"allocated":1238761472,"data_stored":1237945097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":987155458},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,3,10,5,3,0,0,6,0,0,4],"upper_bound":8192},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-15704064,"num_objects":-3834,"num_object_clones":0,"num_object_copies":-7668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2022,"num_objects_misplaced":1670,"num_objects_unfound":0,"num_objects_dirty":-3834,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3834,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":350,"num_bytes_recovered":1359872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.004271"},"pg_stats":[{"pgid":"2.f","version":"47'11980","reported_seq":12123,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:28.058573+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:34:28.058573+0000","last_peered":"2024-09-10T20:34:28.058573+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:34:28.058573+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35545088,"num_objects":8678,"num_object_clones":0,"num_object_copies":17356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8678,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11980,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5100},{"shards":"0,2","objects":1956}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'12206","reported_seq":12321,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:28.838317+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:34:28.838317+0000","last_peered":"2024-09-10T20:34:28.838317+0000","last_clean":"2024-09-10T20:34:28.838317+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:34:28.838317+0000","last_undegraded":"2024-09-10T20:34:28.838317+0000","last_fullsized":"2024-09-10T20:34:28.838317+0000","mapping_epoch":35,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35782656,"num_objects":8736,"num_object_clones":0,"num_object_copies":17472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10392,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11944","reported_seq":19292,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:34:24.360399+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:34:24.360399+0000","last_peered":"2024-09-10T20:34:24.360399+0000","last_clean":"2024-09-10T20:34:24.360399+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:34:24.360399+0000","last_undegraded":"2024-09-10T20:34:24.360399+0000","last_fullsized":"2024-09-10T20:34:24.360399+0000","mapping_epoch":44,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35926040,"num_objects":8772,"num_object_clones":0,"num_object_copies":17544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8772,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10072,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'11971","reported_seq":12056,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:28.828378+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:34:28.828378+0000","last_peered":"2024-09-10T20:34:28.828378+0000","last_clean":"2024-09-10T20:34:28.828378+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:34:28.828378+0000","last_undegraded":"2024-09-10T20:34:28.828378+0000","last_fullsized":"2024-09-10T20:34:28.828378+0000","mapping_epoch":35,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35360768,"num_objects":8633,"num_object_clones":0,"num_object_copies":17266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11971,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'12198","reported_seq":12329,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.975225+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:34:27.975225+0000","last_peered":"2024-09-10T20:34:27.975225+0000","last_clean":"2024-09-10T20:34:27.975225+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:34:27.975225+0000","last_undegraded":"2024-09-10T20:34:27.975225+0000","last_fullsized":"2024-09-10T20:34:27.975225+0000","mapping_epoch":34,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36335616,"num_objects":8871,"num_object_clones":0,"num_object_copies":17742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8871,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9194,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'11863","reported_seq":12048,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:28.072642+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:34:28.072642+0000","last_peered":"2024-09-10T20:34:28.072642+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:34:28.072642+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35258368,"num_objects":8608,"num_object_clones":0,"num_object_copies":17216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3594,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8727,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5016},{"shards":"0,2","objects":1989}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'11988","reported_seq":12202,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:28.092257+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:34:28.092257+0000","last_peered":"2024-09-10T20:34:28.092257+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:34:28.092257+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35307520,"num_objects":8620,"num_object_clones":0,"num_object_copies":17240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8583,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5101},{"shards":"0,2","objects":1874}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'12014","reported_seq":12143,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:28.059585+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:34:28.059585+0000","last_peered":"2024-09-10T20:34:28.059585+0000","last_clean":"2024-09-10T20:34:28.059585+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:34:28.059585+0000","last_undegraded":"2024-09-10T20:34:28.059585+0000","last_fullsized":"2024-09-10T20:34:28.059585+0000","mapping_epoch":34,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35725312,"num_objects":8722,"num_object_clones":0,"num_object_copies":17444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9085,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187449+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:32:18.187449+0000","last_peered":"2024-09-10T20:32:18.187449+0000","last_clean":"2024-09-10T20:32:18.187449+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:32:18.187449+0000","last_undegraded":"2024-09-10T20:32:18.187449+0000","last_fullsized":"2024-09-10T20:32:18.187449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'11855","reported_seq":12014,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:28.744954+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:34:28.744954+0000","last_peered":"2024-09-10T20:34:28.744954+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:34:28.744954+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35160064,"num_objects":8584,"num_object_clones":0,"num_object_copies":17168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9848,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1893},{"shards":"2","objects":5069}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086228+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086228+0000","last_peered":"2024-09-10T20:34:27.086228+0000","last_clean":"2024-09-10T20:34:27.086228+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086228+0000","last_undegraded":"2024-09-10T20:34:27.086228+0000","last_fullsized":"2024-09-10T20:34:27.086228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'12037","reported_seq":12194,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:27.986762+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:34:27.986762+0000","last_peered":"2024-09-10T20:34:27.986762+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:34:27.986762+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35607494,"num_objects":8695,"num_object_clones":0,"num_object_copies":17390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3595,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12035,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5102},{"shards":"0,2","objects":1947}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.075947+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.075947+0000","last_peered":"2024-09-10T20:34:27.075947+0000","last_clean":"2024-09-10T20:34:27.075947+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.075947+0000","last_undegraded":"2024-09-10T20:34:27.075947+0000","last_fullsized":"2024-09-10T20:34:27.075947+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'11979","reported_seq":12130,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:28.820407+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:34:28.820407+0000","last_peered":"2024-09-10T20:34:28.820407+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:34:28.820407+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35168256,"num_objects":8586,"num_object_clones":0,"num_object_copies":17172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3626,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9807,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1939},{"shards":"2","objects":4962}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187486+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:32:18.187486+0000","last_peered":"2024-09-10T20:32:18.187486+0000","last_clean":"2024-09-10T20:32:18.187486+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:32:18.187486+0000","last_undegraded":"2024-09-10T20:32:18.187486+0000","last_fullsized":"2024-09-10T20:32:18.187486+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'12017","reported_seq":12449,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:28.847641+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:34:28.847641+0000","last_peered":"2024-09-10T20:34:28.847641+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:34:28.847641+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35418112,"num_objects":8647,"num_object_clones":0,"num_object_copies":17294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1600,"num_objects_misplaced":1861,"num_objects_unfound":0,"num_objects_dirty":8647,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9803,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":577536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1861},{"shards":"2","objects":5186}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'12083","reported_seq":12279,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:28.075041+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:34:28.075041+0000","last_peered":"2024-09-10T20:34:28.075041+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:34:28.075041+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35827712,"num_objects":8747,"num_object_clones":0,"num_object_copies":17494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3621,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8806,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5127},{"shards":"0,2","objects":1981}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.187775+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:32:18.187775+0000","last_peered":"2024-09-10T20:32:18.187775+0000","last_clean":"2024-09-10T20:32:18.187775+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:32:18.187775+0000","last_undegraded":"2024-09-10T20:32:18.187775+0000","last_fullsized":"2024-09-10T20:32:18.187775+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'11924","reported_seq":12115,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:28.099809+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:34:28.099809+0000","last_peered":"2024-09-10T20:34:28.099809+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:34:28.099809+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35471360,"num_objects":8660,"num_object_clones":0,"num_object_copies":17320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3511,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8660,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8618,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5152},{"shards":"0,2","objects":1904}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086748+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086748+0000","last_peered":"2024-09-10T20:34:27.086748+0000","last_clean":"2024-09-10T20:34:27.086748+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086748+0000","last_undegraded":"2024-09-10T20:34:27.086748+0000","last_fullsized":"2024-09-10T20:34:27.086748+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'12143","reported_seq":12331,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:28.050961+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:34:28.050961+0000","last_peered":"2024-09-10T20:34:28.050961+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:34:28.050961+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36220928,"num_objects":8843,"num_object_clones":0,"num_object_copies":17686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3689,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8970,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5156},{"shards":"0,2","objects":2065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188324+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:32:18.188324+0000","last_peered":"2024-09-10T20:32:18.188324+0000","last_clean":"2024-09-10T20:32:18.188324+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:32:18.188324+0000","last_undegraded":"2024-09-10T20:32:18.188324+0000","last_fullsized":"2024-09-10T20:32:18.188324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T20:32:18.188372+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:32:18.188372+0000","last_peered":"2024-09-10T20:32:18.188372+0000","last_clean":"2024-09-10T20:32:18.188372+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:32:18.188372+0000","last_undegraded":"2024-09-10T20:32:18.188372+0000","last_fullsized":"2024-09-10T20:32:18.188372+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11740","reported_seq":18587,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T20:34:23.100958+0000","last_change":"2024-09-10T20:32:16.657652+0000","last_active":"2024-09-10T20:34:23.100958+0000","last_peered":"2024-09-10T20:34:23.100958+0000","last_clean":"2024-09-10T20:30:03.302909+0000","last_became_active":"2024-09-10T20:30:05.684434+0000","last_became_peered":"2024-09-10T20:30:05.684434+0000","last_unstale":"2024-09-10T20:34:23.100958+0000","last_undegraded":"2024-09-10T20:34:23.100958+0000","last_fullsized":"2024-09-10T20:30:05.305646+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35946979,"num_objects":8777,"num_object_clones":0,"num_object_copies":17554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":131,"num_objects_unfound":0,"num_objects_dirty":8777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8347,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13668352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8646},{"shards":"0,2","objects":131}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":570062273,"num_objects":139179,"num_object_clones":0,"num_object_copies":278358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33852,"num_objects_misplaced":1992,"num_objects_unfound":0,"num_objects_dirty":139179,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":156238,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7303,"num_bytes_recovered":29007872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1235849216,"data_stored":1235765199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160842,"ondisk_log_size":160842,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182471,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":841832,"kb_used_data":477000,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93530008,"statfs":{"total":96636764160,"available":95774728192,"internally_reserved":0,"allocated":488448000,"data_stored":488195707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,3,2,0,0,6],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986243,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":704380,"kb_used_data":393644,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93667460,"statfs":{"total":96636764160,"available":95915479040,"internally_reserved":0,"allocated":403091456,"data_stored":402839168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318172842},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,5,0,0,0,0,0,0,0,4],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659189,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627612,"kb_used_data":339084,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93744228,"statfs":{"total":96636764160,"available":95994089472,"internally_reserved":0,"allocated":347222016,"data_stored":346910222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":295431854},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":486940672,"data_stored":486937059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":346791936,"data_stored":346719217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":402116608,"data_stored":402108923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:34:30.914 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:34:31.465 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:31.466 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:34:31.490 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":232,"stamp":"2024-09-10T20:34:30.937814+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":568939988,"num_objects":138907,"num_object_clones":0,"num_object_copies":277814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33852,"num_objects_misplaced":1861,"num_objects_unfound":0,"num_objects_dirty":138907,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":156514,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7436,"num_bytes_recovered":29544467,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160919,"ondisk_log_size":160919,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2177664,"kb_used_data":1209400,"kb_used_omap":12,"kb_used_meta":968179,"kb_avail":280937856,"statfs":{"total":289910292480,"available":287680364544,"internally_reserved":0,"allocated":1238425600,"data_stored":1237599620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":991415298},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,8,3,2,2,1,12,1,0,4],"upper_bound":8192},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16826368,"num_objects":-4108,"num_object_clones":0,"num_object_copies":-8216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2022,"num_objects_misplaced":1539,"num_objects_unfound":0,"num_objects_dirty":-4108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4108,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":481,"num_bytes_recovered":1896448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.004308"},"pg_stats":[{"pgid":"2.f","version":"47'11980","reported_seq":12123,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:28.058573+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:34:28.058573+0000","last_peered":"2024-09-10T20:34:28.058573+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:34:28.058573+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35545088,"num_objects":8678,"num_object_clones":0,"num_object_copies":17356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8678,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11980,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5100},{"shards":"0,2","objects":1956}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'12206","reported_seq":12321,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:28.838317+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:34:28.838317+0000","last_peered":"2024-09-10T20:34:28.838317+0000","last_clean":"2024-09-10T20:34:28.838317+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:34:28.838317+0000","last_undegraded":"2024-09-10T20:34:28.838317+0000","last_fullsized":"2024-09-10T20:34:28.838317+0000","mapping_epoch":35,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35782656,"num_objects":8736,"num_object_clones":0,"num_object_copies":17472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10392,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'12060","reported_seq":19412,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:29.371546+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:34:29.371546+0000","last_peered":"2024-09-10T20:34:29.371546+0000","last_clean":"2024-09-10T20:34:29.371546+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:34:29.371546+0000","last_undegraded":"2024-09-10T20:34:29.371546+0000","last_fullsized":"2024-09-10T20:34:29.371546+0000","mapping_epoch":44,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35450904,"num_objects":8656,"num_object_clones":0,"num_object_copies":17312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8656,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10188,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'11971","reported_seq":12056,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:28.828378+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:34:28.828378+0000","last_peered":"2024-09-10T20:34:28.828378+0000","last_clean":"2024-09-10T20:34:28.828378+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:34:28.828378+0000","last_undegraded":"2024-09-10T20:34:28.828378+0000","last_fullsized":"2024-09-10T20:34:28.828378+0000","mapping_epoch":35,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35360768,"num_objects":8633,"num_object_clones":0,"num_object_copies":17266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11971,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'12198","reported_seq":12329,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.975225+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:34:27.975225+0000","last_peered":"2024-09-10T20:34:27.975225+0000","last_clean":"2024-09-10T20:34:27.975225+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:34:27.975225+0000","last_undegraded":"2024-09-10T20:34:27.975225+0000","last_fullsized":"2024-09-10T20:34:27.975225+0000","mapping_epoch":34,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36335616,"num_objects":8871,"num_object_clones":0,"num_object_copies":17742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8871,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9194,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'11863","reported_seq":12048,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:28.072642+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:34:28.072642+0000","last_peered":"2024-09-10T20:34:28.072642+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:34:28.072642+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35258368,"num_objects":8608,"num_object_clones":0,"num_object_copies":17216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3594,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8727,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5016},{"shards":"0,2","objects":1989}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'11988","reported_seq":12202,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:28.092257+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:34:28.092257+0000","last_peered":"2024-09-10T20:34:28.092257+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:34:28.092257+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35307520,"num_objects":8620,"num_object_clones":0,"num_object_copies":17240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8583,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5101},{"shards":"0,2","objects":1874}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'12014","reported_seq":12143,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:28.059585+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:34:28.059585+0000","last_peered":"2024-09-10T20:34:28.059585+0000","last_clean":"2024-09-10T20:34:28.059585+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:34:28.059585+0000","last_undegraded":"2024-09-10T20:34:28.059585+0000","last_fullsized":"2024-09-10T20:34:28.059585+0000","mapping_epoch":34,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35725312,"num_objects":8722,"num_object_clones":0,"num_object_copies":17444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9085,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'11855","reported_seq":12014,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:28.744954+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:34:28.744954+0000","last_peered":"2024-09-10T20:34:28.744954+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:34:28.744954+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35160064,"num_objects":8584,"num_object_clones":0,"num_object_copies":17168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9848,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1893},{"shards":"2","objects":5069}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086228+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086228+0000","last_peered":"2024-09-10T20:34:27.086228+0000","last_clean":"2024-09-10T20:34:27.086228+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086228+0000","last_undegraded":"2024-09-10T20:34:27.086228+0000","last_fullsized":"2024-09-10T20:34:27.086228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'12037","reported_seq":12194,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:27.986762+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:34:27.986762+0000","last_peered":"2024-09-10T20:34:27.986762+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:34:27.986762+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35607494,"num_objects":8695,"num_object_clones":0,"num_object_copies":17390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3595,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12035,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5102},{"shards":"0,2","objects":1947}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.075947+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.075947+0000","last_peered":"2024-09-10T20:34:27.075947+0000","last_clean":"2024-09-10T20:34:27.075947+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.075947+0000","last_undegraded":"2024-09-10T20:34:27.075947+0000","last_fullsized":"2024-09-10T20:34:27.075947+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'11979","reported_seq":12130,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:28.820407+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:34:28.820407+0000","last_peered":"2024-09-10T20:34:28.820407+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:34:28.820407+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35168256,"num_objects":8586,"num_object_clones":0,"num_object_copies":17172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3626,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9807,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1939},{"shards":"2","objects":4962}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'12017","reported_seq":12449,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:28.847641+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:34:28.847641+0000","last_peered":"2024-09-10T20:34:28.847641+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:34:28.847641+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35418112,"num_objects":8647,"num_object_clones":0,"num_object_copies":17294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1600,"num_objects_misplaced":1861,"num_objects_unfound":0,"num_objects_dirty":8647,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9803,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":577536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1861},{"shards":"2","objects":5186}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'12083","reported_seq":12279,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:28.075041+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:34:28.075041+0000","last_peered":"2024-09-10T20:34:28.075041+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:34:28.075041+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35827712,"num_objects":8747,"num_object_clones":0,"num_object_copies":17494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3621,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8806,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5127},{"shards":"0,2","objects":1981}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'11924","reported_seq":12115,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:28.099809+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:34:28.099809+0000","last_peered":"2024-09-10T20:34:28.099809+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:34:28.099809+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35471360,"num_objects":8660,"num_object_clones":0,"num_object_copies":17320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3511,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8660,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8618,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5152},{"shards":"0,2","objects":1904}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086748+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086748+0000","last_peered":"2024-09-10T20:34:27.086748+0000","last_clean":"2024-09-10T20:34:27.086748+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086748+0000","last_undegraded":"2024-09-10T20:34:27.086748+0000","last_fullsized":"2024-09-10T20:34:27.086748+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'12143","reported_seq":12331,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:28.050961+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:34:28.050961+0000","last_peered":"2024-09-10T20:34:28.050961+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:34:28.050961+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36220928,"num_objects":8843,"num_object_clones":0,"num_object_copies":17686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3689,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8970,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5156},{"shards":"0,2","objects":2065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'11898","reported_seq":19020,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:29.389123+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:34:29.389123+0000","last_peered":"2024-09-10T20:34:29.389123+0000","last_clean":"2024-09-10T20:34:29.389123+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:34:29.389123+0000","last_undegraded":"2024-09-10T20:34:29.389123+0000","last_fullsized":"2024-09-10T20:34:29.389123+0000","mapping_epoch":46,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35299811,"num_objects":8619,"num_object_clones":0,"num_object_copies":17238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8505,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":568939969,"num_objects":138905,"num_object_clones":0,"num_object_copies":277810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33852,"num_objects_misplaced":1861,"num_objects_unfound":0,"num_objects_dirty":138905,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":156512,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7434,"num_bytes_recovered":29544448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1235394560,"data_stored":1235310543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160916,"ondisk_log_size":160916,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182471,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":841832,"kb_used_data":477000,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93530008,"statfs":{"total":96636764160,"available":95774728192,"internally_reserved":0,"allocated":488448000,"data_stored":488195707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,3,2,0,0,6],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986243,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":704380,"kb_used_data":393644,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93667460,"statfs":{"total":96636764160,"available":95915479040,"internally_reserved":0,"allocated":403091456,"data_stored":402839168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318172842},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,5,0,0,0,0,0,0,0,4],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659190,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":631452,"kb_used_data":338756,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93740388,"statfs":{"total":96636764160,"available":95990157312,"internally_reserved":0,"allocated":346886144,"data_stored":346564745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":299691694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,2,1,6,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":486940672,"data_stored":486937059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":346337280,"data_stored":346264561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":402116608,"data_stored":402108923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:34:31.492 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:34:31.492 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'11980", 'reported_seq': 12123, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:28.058573+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:34:28.058573+0000', 'last_peered': '2024-09-10T20:34:28.058573+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:34:28.058573+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35545088, 'num_objects': 8678, 'num_object_clones': 0, 'num_object_copies': 17356, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8678, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11980, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5100}, {'shards': '0,2', 'objects': 1956}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:31.492 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:34:31.492 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'11863", 'reported_seq': 12048, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:28.072642+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:34:28.072642+0000', 'last_peered': '2024-09-10T20:34:28.072642+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:34:28.072642+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35258368, 'num_objects': 8608, 'num_object_clones': 0, 'num_object_copies': 17216, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3594, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8608, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8727, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 22, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5016}, {'shards': '0,2', 'objects': 1989}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:31.492 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:34:31.492 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'11988", 'reported_seq': 12202, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:28.092257+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:34:28.092257+0000', 'last_peered': '2024-09-10T20:34:28.092257+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:34:28.092257+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35307520, 'num_objects': 8620, 'num_object_clones': 0, 'num_object_copies': 17240, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8620, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8583, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 36, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5101}, {'shards': '0,2', 'objects': 1874}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:31.492 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:34:31.492 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'11855", 'reported_seq': 12014, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:28.744954+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:34:28.744954+0000', 'last_peered': '2024-09-10T20:34:28.744954+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:34:28.744954+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35160064, 'num_objects': 8584, 'num_object_clones': 0, 'num_object_copies': 17168, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8584, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9848, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 11, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1893}, {'shards': '2', 'objects': 5069}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:34:31.493 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:34:31.493 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'12037", 'reported_seq': 12194, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:27.986762+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:34:27.986762+0000', 'last_peered': '2024-09-10T20:34:27.986762+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:34:27.986762+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35607494, 'num_objects': 8695, 'num_object_clones': 0, 'num_object_copies': 17390, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3595, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8695, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12035, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 24, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5102}, {'shards': '0,2', 'objects': 1947}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:31.493 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:34:31.493 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'11979", 'reported_seq': 12130, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:28.820407+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:34:28.820407+0000', 'last_peered': '2024-09-10T20:34:28.820407+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:34:28.820407+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35168256, 'num_objects': 8586, 'num_object_clones': 0, 'num_object_copies': 17172, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3626, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8586, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9807, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 9, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1939}, {'shards': '2', 'objects': 4962}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:34:31.493 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:34:31.493 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'12017", 'reported_seq': 12449, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:28.847641+0000', 'last_change': '2024-09-10T20:34:25.241937+0000', 'last_active': '2024-09-10T20:34:28.847641+0000', 'last_peered': '2024-09-10T20:34:28.847641+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:34:28.847641+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35418112, 'num_objects': 8647, 'num_object_clones': 0, 'num_object_copies': 17294, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1600, 'num_objects_misplaced': 1861, 'num_objects_unfound': 0, 'num_objects_dirty': 8647, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9803, 'num_write_kb': 32484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 149, 'num_bytes_recovered': 577536, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1861}, {'shards': '2', 'objects': 5186}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:34:31.493 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:34:31.493 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'12083", 'reported_seq': 12279, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:28.075041+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:34:28.075041+0000', 'last_peered': '2024-09-10T20:34:28.075041+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:34:28.075041+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35827712, 'num_objects': 8747, 'num_object_clones': 0, 'num_object_copies': 17494, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3621, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8747, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8806, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 28, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5127}, {'shards': '0,2', 'objects': 1981}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:31.493 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:34:31.493 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'11924", 'reported_seq': 12115, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:28.099809+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:34:28.099809+0000', 'last_peered': '2024-09-10T20:34:28.099809+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:34:28.099809+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35471360, 'num_objects': 8660, 'num_object_clones': 0, 'num_object_copies': 17320, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3511, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8660, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8618, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 24, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5152}, {'shards': '0,2', 'objects': 1904}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:31.494 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:34:31.494 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'12143", 'reported_seq': 12331, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:28.050961+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:34:28.050961+0000', 'last_peered': '2024-09-10T20:34:28.050961+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:34:28.050961+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "36'2100", 'ondisk_log_start': "36'2100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36220928, 'num_objects': 8843, 'num_object_clones': 0, 'num_object_copies': 17686, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3689, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8843, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8970, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 26, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5156}, {'shards': '0,2', 'objects': 2065}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:31.494 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:34:32.186 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:32.186 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33852/277814 objects degraded (12.185%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":383,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":138907,"data_bytes":568939988,"bytes_used":2229927936,"bytes_avail":287680364544,"bytes_total":289910292480,"degraded_objects":33852,"degraded_total":277814,"degraded_ratio":0.12185131058909918,"misplaced_objects":1861,"misplaced_total":277814,"misplaced_ratio":0.006698726486066218,"recovering_objects_per_sec":60,"recovering_bytes_per_sec":236928,"recovering_keys_per_sec":0,"num_objects_recovered":481,"num_bytes_recovered":1896448,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":513},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:34:32.205 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:34:32.205 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:34:32.756 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:32.757 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:34:32.781 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":232,"stamp":"2024-09-10T20:34:30.937814+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":568939988,"num_objects":138907,"num_object_clones":0,"num_object_copies":277814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33852,"num_objects_misplaced":1861,"num_objects_unfound":0,"num_objects_dirty":138907,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":156514,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7436,"num_bytes_recovered":29544467,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160919,"ondisk_log_size":160919,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2177664,"kb_used_data":1209400,"kb_used_omap":12,"kb_used_meta":968179,"kb_avail":280937856,"statfs":{"total":289910292480,"available":287680364544,"internally_reserved":0,"allocated":1238425600,"data_stored":1237599620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":991415298},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,8,3,2,2,1,12,1,0,4],"upper_bound":8192},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16826368,"num_objects":-4108,"num_object_clones":0,"num_object_copies":-8216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2022,"num_objects_misplaced":1539,"num_objects_unfound":0,"num_objects_dirty":-4108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4108,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":481,"num_bytes_recovered":1896448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.004308"},"pg_stats":[{"pgid":"2.f","version":"47'11980","reported_seq":12123,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:28.058573+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:34:28.058573+0000","last_peered":"2024-09-10T20:34:28.058573+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:34:28.058573+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35545088,"num_objects":8678,"num_object_clones":0,"num_object_copies":17356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8678,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11980,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5100},{"shards":"0,2","objects":1956}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'12206","reported_seq":12321,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:28.838317+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:34:28.838317+0000","last_peered":"2024-09-10T20:34:28.838317+0000","last_clean":"2024-09-10T20:34:28.838317+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:34:28.838317+0000","last_undegraded":"2024-09-10T20:34:28.838317+0000","last_fullsized":"2024-09-10T20:34:28.838317+0000","mapping_epoch":35,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35782656,"num_objects":8736,"num_object_clones":0,"num_object_copies":17472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10392,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'12060","reported_seq":19412,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:29.371546+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:34:29.371546+0000","last_peered":"2024-09-10T20:34:29.371546+0000","last_clean":"2024-09-10T20:34:29.371546+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:34:29.371546+0000","last_undegraded":"2024-09-10T20:34:29.371546+0000","last_fullsized":"2024-09-10T20:34:29.371546+0000","mapping_epoch":44,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35450904,"num_objects":8656,"num_object_clones":0,"num_object_copies":17312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8656,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10188,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'11971","reported_seq":12056,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:28.828378+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:34:28.828378+0000","last_peered":"2024-09-10T20:34:28.828378+0000","last_clean":"2024-09-10T20:34:28.828378+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:34:28.828378+0000","last_undegraded":"2024-09-10T20:34:28.828378+0000","last_fullsized":"2024-09-10T20:34:28.828378+0000","mapping_epoch":35,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35360768,"num_objects":8633,"num_object_clones":0,"num_object_copies":17266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11971,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'12198","reported_seq":12329,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.975225+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:34:27.975225+0000","last_peered":"2024-09-10T20:34:27.975225+0000","last_clean":"2024-09-10T20:34:27.975225+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:34:27.975225+0000","last_undegraded":"2024-09-10T20:34:27.975225+0000","last_fullsized":"2024-09-10T20:34:27.975225+0000","mapping_epoch":34,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36335616,"num_objects":8871,"num_object_clones":0,"num_object_copies":17742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8871,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9194,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'11863","reported_seq":12048,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:28.072642+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:34:28.072642+0000","last_peered":"2024-09-10T20:34:28.072642+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:34:28.072642+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35258368,"num_objects":8608,"num_object_clones":0,"num_object_copies":17216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3594,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8727,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5016},{"shards":"0,2","objects":1989}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'11988","reported_seq":12202,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:28.092257+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:34:28.092257+0000","last_peered":"2024-09-10T20:34:28.092257+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:34:28.092257+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35307520,"num_objects":8620,"num_object_clones":0,"num_object_copies":17240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8583,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5101},{"shards":"0,2","objects":1874}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'12014","reported_seq":12143,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:28.059585+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:34:28.059585+0000","last_peered":"2024-09-10T20:34:28.059585+0000","last_clean":"2024-09-10T20:34:28.059585+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:34:28.059585+0000","last_undegraded":"2024-09-10T20:34:28.059585+0000","last_fullsized":"2024-09-10T20:34:28.059585+0000","mapping_epoch":34,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35725312,"num_objects":8722,"num_object_clones":0,"num_object_copies":17444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9085,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'11855","reported_seq":12014,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:28.744954+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:34:28.744954+0000","last_peered":"2024-09-10T20:34:28.744954+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:34:28.744954+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35160064,"num_objects":8584,"num_object_clones":0,"num_object_copies":17168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9848,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1893},{"shards":"2","objects":5069}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086228+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086228+0000","last_peered":"2024-09-10T20:34:27.086228+0000","last_clean":"2024-09-10T20:34:27.086228+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086228+0000","last_undegraded":"2024-09-10T20:34:27.086228+0000","last_fullsized":"2024-09-10T20:34:27.086228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'12037","reported_seq":12194,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:27.986762+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:34:27.986762+0000","last_peered":"2024-09-10T20:34:27.986762+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:34:27.986762+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35607494,"num_objects":8695,"num_object_clones":0,"num_object_copies":17390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3595,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12035,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5102},{"shards":"0,2","objects":1947}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.075947+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.075947+0000","last_peered":"2024-09-10T20:34:27.075947+0000","last_clean":"2024-09-10T20:34:27.075947+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.075947+0000","last_undegraded":"2024-09-10T20:34:27.075947+0000","last_fullsized":"2024-09-10T20:34:27.075947+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'11979","reported_seq":12130,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:28.820407+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:34:28.820407+0000","last_peered":"2024-09-10T20:34:28.820407+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:34:28.820407+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35168256,"num_objects":8586,"num_object_clones":0,"num_object_copies":17172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3626,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9807,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1939},{"shards":"2","objects":4962}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'12017","reported_seq":12449,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:28.847641+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:34:28.847641+0000","last_peered":"2024-09-10T20:34:28.847641+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:34:28.847641+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35418112,"num_objects":8647,"num_object_clones":0,"num_object_copies":17294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1600,"num_objects_misplaced":1861,"num_objects_unfound":0,"num_objects_dirty":8647,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9803,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":577536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1861},{"shards":"2","objects":5186}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'12083","reported_seq":12279,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:28.075041+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:34:28.075041+0000","last_peered":"2024-09-10T20:34:28.075041+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:34:28.075041+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35827712,"num_objects":8747,"num_object_clones":0,"num_object_copies":17494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3621,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8806,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5127},{"shards":"0,2","objects":1981}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'11924","reported_seq":12115,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:28.099809+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:34:28.099809+0000","last_peered":"2024-09-10T20:34:28.099809+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:34:28.099809+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35471360,"num_objects":8660,"num_object_clones":0,"num_object_copies":17320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3511,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8660,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8618,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5152},{"shards":"0,2","objects":1904}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086748+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086748+0000","last_peered":"2024-09-10T20:34:27.086748+0000","last_clean":"2024-09-10T20:34:27.086748+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086748+0000","last_undegraded":"2024-09-10T20:34:27.086748+0000","last_fullsized":"2024-09-10T20:34:27.086748+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'12143","reported_seq":12331,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:28.050961+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:34:28.050961+0000","last_peered":"2024-09-10T20:34:28.050961+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:34:28.050961+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36220928,"num_objects":8843,"num_object_clones":0,"num_object_copies":17686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3689,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8970,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5156},{"shards":"0,2","objects":2065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'11898","reported_seq":19020,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:29.389123+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:34:29.389123+0000","last_peered":"2024-09-10T20:34:29.389123+0000","last_clean":"2024-09-10T20:34:29.389123+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:34:29.389123+0000","last_undegraded":"2024-09-10T20:34:29.389123+0000","last_fullsized":"2024-09-10T20:34:29.389123+0000","mapping_epoch":46,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35299811,"num_objects":8619,"num_object_clones":0,"num_object_copies":17238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8505,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":568939969,"num_objects":138905,"num_object_clones":0,"num_object_copies":277810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33852,"num_objects_misplaced":1861,"num_objects_unfound":0,"num_objects_dirty":138905,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":156512,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7434,"num_bytes_recovered":29544448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1235394560,"data_stored":1235310543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160916,"ondisk_log_size":160916,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182471,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":841832,"kb_used_data":477000,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93530008,"statfs":{"total":96636764160,"available":95774728192,"internally_reserved":0,"allocated":488448000,"data_stored":488195707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,3,2,0,0,6],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986243,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":704380,"kb_used_data":393644,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93667460,"statfs":{"total":96636764160,"available":95915479040,"internally_reserved":0,"allocated":403091456,"data_stored":402839168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318172842},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,5,0,0,0,0,0,0,0,4],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659190,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":631452,"kb_used_data":338756,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93740388,"statfs":{"total":96636764160,"available":95990157312,"internally_reserved":0,"allocated":346886144,"data_stored":346564745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":299691694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,2,1,6,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":486940672,"data_stored":486937059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":346337280,"data_stored":346264561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":402116608,"data_stored":402108923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:34:35.783 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:34:36.346 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:36.348 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:34:36.373 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":234,"stamp":"2024-09-10T20:34:34.939459+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":559830484,"num_objects":136683,"num_object_clones":0,"num_object_copies":273366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33840,"num_objects_misplaced":1725,"num_objects_unfound":0,"num_objects_dirty":136683,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":158738,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7584,"num_bytes_recovered":30101523,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160643,"ondisk_log_size":160643,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2136400,"kb_used_data":1197276,"kb_used_omap":12,"kb_used_meta":939059,"kb_avail":280979120,"statfs":{"total":289910292480,"available":287722618880,"internally_reserved":0,"allocated":1226010624,"data_stored":1225184644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":961596418},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,5,4,2,0,0,0,0,0,7,4],"upper_bound":16384},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17035264,"num_objects":-4159,"num_object_clones":0,"num_object_copies":-8318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2022,"num_objects_misplaced":1594,"num_objects_unfound":0,"num_objects_dirty":-4159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4159,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":426,"num_bytes_recovered":1671168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.863339"},"pg_stats":[{"pgid":"2.f","version":"47'12124","reported_seq":12267,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:33.088956+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:34:33.088956+0000","last_peered":"2024-09-10T20:34:33.088956+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:34:33.088956+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34955264,"num_objects":8534,"num_object_clones":0,"num_object_copies":17068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12124,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4956},{"shards":"0,2","objects":1812}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'12364","reported_seq":12479,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:33.762876+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:34:33.762876+0000","last_peered":"2024-09-10T20:34:33.762876+0000","last_clean":"2024-09-10T20:34:33.762876+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:34:33.762876+0000","last_undegraded":"2024-09-10T20:34:33.762876+0000","last_fullsized":"2024-09-10T20:34:33.762876+0000","mapping_epoch":35,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35135488,"num_objects":8578,"num_object_clones":0,"num_object_copies":17156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10550,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'12206","reported_seq":19559,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:34.404954+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:34:34.404954+0000","last_peered":"2024-09-10T20:34:34.404954+0000","last_clean":"2024-09-10T20:34:34.404954+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:34:34.404954+0000","last_undegraded":"2024-09-10T20:34:34.404954+0000","last_fullsized":"2024-09-10T20:34:34.404954+0000","mapping_epoch":44,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34852888,"num_objects":8510,"num_object_clones":0,"num_object_copies":17020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8510,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10334,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'12109","reported_seq":12194,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:33.828733+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:34:33.828733+0000","last_peered":"2024-09-10T20:34:33.828733+0000","last_clean":"2024-09-10T20:34:33.828733+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:34:33.828733+0000","last_undegraded":"2024-09-10T20:34:33.828733+0000","last_fullsized":"2024-09-10T20:34:33.828733+0000","mapping_epoch":35,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34795520,"num_objects":8495,"num_object_clones":0,"num_object_copies":16990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12109,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'12321","reported_seq":12452,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:33.117390+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:34:33.117390+0000","last_peered":"2024-09-10T20:34:33.117390+0000","last_clean":"2024-09-10T20:34:33.117390+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:34:33.117390+0000","last_undegraded":"2024-09-10T20:34:33.117390+0000","last_fullsized":"2024-09-10T20:34:33.117390+0000","mapping_epoch":34,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35831808,"num_objects":8748,"num_object_clones":0,"num_object_copies":17496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9317,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'12011","reported_seq":12198,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:33.076331+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:34:33.076331+0000","last_peered":"2024-09-10T20:34:33.076331+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:34:33.076331+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34652160,"num_objects":8460,"num_object_clones":0,"num_object_copies":16920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8875,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4869},{"shards":"0,2","objects":1841}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'12092","reported_seq":12311,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:33.111045+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:34:33.111045+0000","last_peered":"2024-09-10T20:34:33.111045+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:34:33.111045+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34881536,"num_objects":8516,"num_object_clones":0,"num_object_copies":17032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3519,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8687,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4999},{"shards":"0,2","objects":1770}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'12152","reported_seq":12281,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:33.052989+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:34:33.052989+0000","last_peered":"2024-09-10T20:34:33.052989+0000","last_clean":"2024-09-10T20:34:33.052989+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:34:33.052989+0000","last_undegraded":"2024-09-10T20:34:33.052989+0000","last_fullsized":"2024-09-10T20:34:33.052989+0000","mapping_epoch":34,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35160064,"num_objects":8584,"num_object_clones":0,"num_object_copies":17168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9223,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'11987","reported_seq":12148,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:33.808018+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:34:33.808018+0000","last_peered":"2024-09-10T20:34:33.808018+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:34:33.808018+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34619392,"num_objects":8452,"num_object_clones":0,"num_object_copies":16904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3516,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9980,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1761},{"shards":"2","objects":4938}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086228+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086228+0000","last_peered":"2024-09-10T20:34:27.086228+0000","last_clean":"2024-09-10T20:34:27.086228+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086228+0000","last_undegraded":"2024-09-10T20:34:27.086228+0000","last_fullsized":"2024-09-10T20:34:27.086228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'12179","reported_seq":12336,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:33.076257+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:34:33.076257+0000","last_peered":"2024-09-10T20:34:33.076257+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:34:33.076257+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35025862,"num_objects":8553,"num_object_clones":0,"num_object_copies":17106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3595,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8553,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12177,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4960},{"shards":"0,2","objects":1805}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.075947+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.075947+0000","last_peered":"2024-09-10T20:34:27.075947+0000","last_clean":"2024-09-10T20:34:27.075947+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.075947+0000","last_undegraded":"2024-09-10T20:34:27.075947+0000","last_fullsized":"2024-09-10T20:34:27.075947+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'12101","reported_seq":12266,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:33.754205+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:34:33.754205+0000","last_peered":"2024-09-10T20:34:33.754205+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:34:33.754205+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34668544,"num_objects":8464,"num_object_clones":0,"num_object_copies":16928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3619,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9929,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1817},{"shards":"2","objects":4847}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'12161","reported_seq":12865,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:33.835128+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:34:33.835128+0000","last_peered":"2024-09-10T20:34:33.835128+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:34:33.835128+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34828288,"num_objects":8503,"num_object_clones":0,"num_object_copies":17006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1600,"num_objects_misplaced":1725,"num_objects_unfound":0,"num_objects_dirty":8503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9947,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":285,"num_bytes_recovered":1134592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1725},{"shards":"2","objects":5178}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'12225","reported_seq":12423,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:32.963752+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:34:32.963752+0000","last_peered":"2024-09-10T20:34:32.963752+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:34:32.963752+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35246080,"num_objects":8605,"num_object_clones":0,"num_object_copies":17210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3620,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8605,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8948,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4986},{"shards":"0,2","objects":1839}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'12050","reported_seq":12241,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:33.069950+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:34:33.069950+0000","last_peered":"2024-09-10T20:34:33.069950+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:34:33.069950+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34955264,"num_objects":8534,"num_object_clones":0,"num_object_copies":17068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3511,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8744,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5026},{"shards":"0,2","objects":1778}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086748+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086748+0000","last_peered":"2024-09-10T20:34:27.086748+0000","last_clean":"2024-09-10T20:34:27.086748+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086748+0000","last_undegraded":"2024-09-10T20:34:27.086748+0000","last_fullsized":"2024-09-10T20:34:27.086748+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'12303","reported_seq":12491,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:33.088718+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:34:33.088718+0000","last_peered":"2024-09-10T20:34:33.088718+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:34:33.088718+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35565568,"num_objects":8683,"num_object_clones":0,"num_object_copies":17366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3689,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9130,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4996},{"shards":"0,2","objects":1905}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'12055","reported_seq":19177,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:34.383950+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:34:34.383950+0000","last_peered":"2024-09-10T20:34:34.383950+0000","last_clean":"2024-09-10T20:34:34.383950+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:34:34.383950+0000","last_undegraded":"2024-09-10T20:34:34.383950+0000","last_fullsized":"2024-09-10T20:34:34.383950+0000","mapping_epoch":46,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34656739,"num_objects":8462,"num_object_clones":0,"num_object_copies":16924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8662,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":559830465,"num_objects":136681,"num_object_clones":0,"num_object_copies":273362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33840,"num_objects_misplaced":1725,"num_objects_unfound":0,"num_objects_dirty":136681,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":158736,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7582,"num_bytes_recovered":30101504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1222643712,"data_stored":1222559695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160640,"ondisk_log_size":160640,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182472,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":826800,"kb_used_data":470304,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93545040,"statfs":{"total":96636764160,"available":95790120960,"internally_reserved":0,"allocated":481591296,"data_stored":481339003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3,2,0,0,0,0,0,7],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986244,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687664,"kb_used_data":389408,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93684176,"statfs":{"total":96636764160,"available":95932596224,"internally_reserved":0,"allocated":398753792,"data_stored":398501504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305393322},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,0,0,0,0,0,0,0,0,4],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659191,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621936,"kb_used_data":337564,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93749904,"statfs":{"total":96636764160,"available":95999901696,"internally_reserved":0,"allocated":345665536,"data_stored":345344137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":291172014},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":480059392,"data_stored":480055779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":345047040,"data_stored":344974321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":397537280,"data_stored":397529595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:34:36.375 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:34:36.926 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:36.929 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:34:36.956 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":234,"stamp":"2024-09-10T20:34:34.939459+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":559830484,"num_objects":136683,"num_object_clones":0,"num_object_copies":273366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33840,"num_objects_misplaced":1725,"num_objects_unfound":0,"num_objects_dirty":136683,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":158738,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7584,"num_bytes_recovered":30101523,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160643,"ondisk_log_size":160643,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2136400,"kb_used_data":1197276,"kb_used_omap":12,"kb_used_meta":939059,"kb_avail":280979120,"statfs":{"total":289910292480,"available":287722618880,"internally_reserved":0,"allocated":1226010624,"data_stored":1225184644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":961596418},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,5,4,2,0,0,0,0,0,7,4],"upper_bound":16384},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17035264,"num_objects":-4159,"num_object_clones":0,"num_object_copies":-8318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2022,"num_objects_misplaced":1594,"num_objects_unfound":0,"num_objects_dirty":-4159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4159,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":426,"num_bytes_recovered":1671168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.863339"},"pg_stats":[{"pgid":"2.f","version":"47'12124","reported_seq":12267,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:33.088956+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:34:33.088956+0000","last_peered":"2024-09-10T20:34:33.088956+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:34:33.088956+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34955264,"num_objects":8534,"num_object_clones":0,"num_object_copies":17068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12124,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4956},{"shards":"0,2","objects":1812}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'12364","reported_seq":12479,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:33.762876+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:34:33.762876+0000","last_peered":"2024-09-10T20:34:33.762876+0000","last_clean":"2024-09-10T20:34:33.762876+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:34:33.762876+0000","last_undegraded":"2024-09-10T20:34:33.762876+0000","last_fullsized":"2024-09-10T20:34:33.762876+0000","mapping_epoch":35,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35135488,"num_objects":8578,"num_object_clones":0,"num_object_copies":17156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10550,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'12206","reported_seq":19559,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:34.404954+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:34:34.404954+0000","last_peered":"2024-09-10T20:34:34.404954+0000","last_clean":"2024-09-10T20:34:34.404954+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:34:34.404954+0000","last_undegraded":"2024-09-10T20:34:34.404954+0000","last_fullsized":"2024-09-10T20:34:34.404954+0000","mapping_epoch":44,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34852888,"num_objects":8510,"num_object_clones":0,"num_object_copies":17020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8510,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10334,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'12109","reported_seq":12194,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:33.828733+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:34:33.828733+0000","last_peered":"2024-09-10T20:34:33.828733+0000","last_clean":"2024-09-10T20:34:33.828733+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:34:33.828733+0000","last_undegraded":"2024-09-10T20:34:33.828733+0000","last_fullsized":"2024-09-10T20:34:33.828733+0000","mapping_epoch":35,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34795520,"num_objects":8495,"num_object_clones":0,"num_object_copies":16990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12109,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'12321","reported_seq":12452,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:33.117390+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:34:33.117390+0000","last_peered":"2024-09-10T20:34:33.117390+0000","last_clean":"2024-09-10T20:34:33.117390+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:34:33.117390+0000","last_undegraded":"2024-09-10T20:34:33.117390+0000","last_fullsized":"2024-09-10T20:34:33.117390+0000","mapping_epoch":34,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35831808,"num_objects":8748,"num_object_clones":0,"num_object_copies":17496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9317,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'12011","reported_seq":12198,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:33.076331+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:34:33.076331+0000","last_peered":"2024-09-10T20:34:33.076331+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:34:33.076331+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34652160,"num_objects":8460,"num_object_clones":0,"num_object_copies":16920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8875,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4869},{"shards":"0,2","objects":1841}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'12092","reported_seq":12311,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:33.111045+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:34:33.111045+0000","last_peered":"2024-09-10T20:34:33.111045+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:34:33.111045+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34881536,"num_objects":8516,"num_object_clones":0,"num_object_copies":17032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3519,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8687,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4999},{"shards":"0,2","objects":1770}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'12152","reported_seq":12281,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:33.052989+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:34:33.052989+0000","last_peered":"2024-09-10T20:34:33.052989+0000","last_clean":"2024-09-10T20:34:33.052989+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:34:33.052989+0000","last_undegraded":"2024-09-10T20:34:33.052989+0000","last_fullsized":"2024-09-10T20:34:33.052989+0000","mapping_epoch":34,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35160064,"num_objects":8584,"num_object_clones":0,"num_object_copies":17168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9223,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'11987","reported_seq":12148,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:33.808018+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:34:33.808018+0000","last_peered":"2024-09-10T20:34:33.808018+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:34:33.808018+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34619392,"num_objects":8452,"num_object_clones":0,"num_object_copies":16904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3516,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9980,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1761},{"shards":"2","objects":4938}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086228+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086228+0000","last_peered":"2024-09-10T20:34:27.086228+0000","last_clean":"2024-09-10T20:34:27.086228+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086228+0000","last_undegraded":"2024-09-10T20:34:27.086228+0000","last_fullsized":"2024-09-10T20:34:27.086228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'12179","reported_seq":12336,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:33.076257+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:34:33.076257+0000","last_peered":"2024-09-10T20:34:33.076257+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:34:33.076257+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35025862,"num_objects":8553,"num_object_clones":0,"num_object_copies":17106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3595,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8553,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12177,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4960},{"shards":"0,2","objects":1805}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.075947+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.075947+0000","last_peered":"2024-09-10T20:34:27.075947+0000","last_clean":"2024-09-10T20:34:27.075947+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.075947+0000","last_undegraded":"2024-09-10T20:34:27.075947+0000","last_fullsized":"2024-09-10T20:34:27.075947+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'12101","reported_seq":12266,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:33.754205+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:34:33.754205+0000","last_peered":"2024-09-10T20:34:33.754205+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:34:33.754205+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34668544,"num_objects":8464,"num_object_clones":0,"num_object_copies":16928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3619,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9929,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1817},{"shards":"2","objects":4847}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'12161","reported_seq":12865,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:33.835128+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:34:33.835128+0000","last_peered":"2024-09-10T20:34:33.835128+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:34:33.835128+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34828288,"num_objects":8503,"num_object_clones":0,"num_object_copies":17006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1600,"num_objects_misplaced":1725,"num_objects_unfound":0,"num_objects_dirty":8503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9947,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":285,"num_bytes_recovered":1134592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1725},{"shards":"2","objects":5178}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'12225","reported_seq":12423,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:32.963752+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:34:32.963752+0000","last_peered":"2024-09-10T20:34:32.963752+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:34:32.963752+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35246080,"num_objects":8605,"num_object_clones":0,"num_object_copies":17210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3620,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8605,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8948,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4986},{"shards":"0,2","objects":1839}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'12050","reported_seq":12241,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:33.069950+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:34:33.069950+0000","last_peered":"2024-09-10T20:34:33.069950+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:34:33.069950+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34955264,"num_objects":8534,"num_object_clones":0,"num_object_copies":17068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3511,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8744,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5026},{"shards":"0,2","objects":1778}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086748+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086748+0000","last_peered":"2024-09-10T20:34:27.086748+0000","last_clean":"2024-09-10T20:34:27.086748+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086748+0000","last_undegraded":"2024-09-10T20:34:27.086748+0000","last_fullsized":"2024-09-10T20:34:27.086748+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'12303","reported_seq":12491,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:33.088718+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:34:33.088718+0000","last_peered":"2024-09-10T20:34:33.088718+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:34:33.088718+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35565568,"num_objects":8683,"num_object_clones":0,"num_object_copies":17366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3689,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9130,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4996},{"shards":"0,2","objects":1905}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'12055","reported_seq":19177,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:34.383950+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:34:34.383950+0000","last_peered":"2024-09-10T20:34:34.383950+0000","last_clean":"2024-09-10T20:34:34.383950+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:34:34.383950+0000","last_undegraded":"2024-09-10T20:34:34.383950+0000","last_fullsized":"2024-09-10T20:34:34.383950+0000","mapping_epoch":46,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34656739,"num_objects":8462,"num_object_clones":0,"num_object_copies":16924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8662,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":559830465,"num_objects":136681,"num_object_clones":0,"num_object_copies":273362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33840,"num_objects_misplaced":1725,"num_objects_unfound":0,"num_objects_dirty":136681,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":158736,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7582,"num_bytes_recovered":30101504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1222643712,"data_stored":1222559695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160640,"ondisk_log_size":160640,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182472,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":826800,"kb_used_data":470304,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93545040,"statfs":{"total":96636764160,"available":95790120960,"internally_reserved":0,"allocated":481591296,"data_stored":481339003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3,2,0,0,0,0,0,7],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986244,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687664,"kb_used_data":389408,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93684176,"statfs":{"total":96636764160,"available":95932596224,"internally_reserved":0,"allocated":398753792,"data_stored":398501504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305393322},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,0,0,0,0,0,0,0,0,4],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659191,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621936,"kb_used_data":337564,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93749904,"statfs":{"total":96636764160,"available":95999901696,"internally_reserved":0,"allocated":345665536,"data_stored":345344137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":291172014},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":480059392,"data_stored":480055779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":345047040,"data_stored":344974321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":397537280,"data_stored":397529595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:34:36.958 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:34:36.958 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'12124", 'reported_seq': 12267, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:33.088956+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:34:33.088956+0000', 'last_peered': '2024-09-10T20:34:33.088956+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:34:33.088956+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "36'2100", 'ondisk_log_start': "36'2100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34955264, 'num_objects': 8534, 'num_object_clones': 0, 'num_object_copies': 17068, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8534, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12124, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4956}, {'shards': '0,2', 'objects': 1812}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:36.958 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:34:36.958 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'12011", 'reported_seq': 12198, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:33.076331+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:34:33.076331+0000', 'last_peered': '2024-09-10T20:34:33.076331+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:34:33.076331+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34652160, 'num_objects': 8460, 'num_object_clones': 0, 'num_object_copies': 16920, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8460, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8875, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 23, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4869}, {'shards': '0,2', 'objects': 1841}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:36.958 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:34:36.958 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'12092", 'reported_seq': 12311, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:33.111045+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:34:33.111045+0000', 'last_peered': '2024-09-10T20:34:33.111045+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:34:33.111045+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34881536, 'num_objects': 8516, 'num_object_clones': 0, 'num_object_copies': 17032, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3519, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8516, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8687, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 38, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4999}, {'shards': '0,2', 'objects': 1770}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:36.959 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:34:36.959 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'11987", 'reported_seq': 12148, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:33.808018+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:34:33.808018+0000', 'last_peered': '2024-09-10T20:34:33.808018+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:34:33.808018+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34619392, 'num_objects': 8452, 'num_object_clones': 0, 'num_object_copies': 16904, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3516, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8452, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9980, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 12, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1761}, {'shards': '2', 'objects': 4938}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:34:36.959 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:34:36.959 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'12179", 'reported_seq': 12336, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:33.076257+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:34:33.076257+0000', 'last_peered': '2024-09-10T20:34:33.076257+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:34:33.076257+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "36'2100", 'ondisk_log_start': "36'2100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35025862, 'num_objects': 8553, 'num_object_clones': 0, 'num_object_copies': 17106, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3595, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8553, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12177, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 24, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4960}, {'shards': '0,2', 'objects': 1805}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:36.959 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:34:36.959 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'12101", 'reported_seq': 12266, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:33.754205+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:34:33.754205+0000', 'last_peered': '2024-09-10T20:34:33.754205+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:34:33.754205+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "36'2100", 'ondisk_log_start': "36'2100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34668544, 'num_objects': 8464, 'num_object_clones': 0, 'num_object_copies': 16928, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3619, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8464, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9929, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 16, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1817}, {'shards': '2', 'objects': 4847}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:34:36.959 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:34:36.959 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'12161", 'reported_seq': 12865, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:33.835128+0000', 'last_change': '2024-09-10T20:34:25.241937+0000', 'last_active': '2024-09-10T20:34:33.835128+0000', 'last_peered': '2024-09-10T20:34:33.835128+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:34:33.835128+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "36'2100", 'ondisk_log_start': "36'2100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34828288, 'num_objects': 8503, 'num_object_clones': 0, 'num_object_copies': 17006, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1600, 'num_objects_misplaced': 1725, 'num_objects_unfound': 0, 'num_objects_dirty': 8503, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9947, 'num_write_kb': 32484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 285, 'num_bytes_recovered': 1134592, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1725}, {'shards': '2', 'objects': 5178}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:34:36.959 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:34:36.959 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'12225", 'reported_seq': 12423, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:32.963752+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:34:32.963752+0000', 'last_peered': '2024-09-10T20:34:32.963752+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:34:32.963752+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35246080, 'num_objects': 8605, 'num_object_clones': 0, 'num_object_copies': 17210, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3620, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8605, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8948, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4986}, {'shards': '0,2', 'objects': 1839}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:36.959 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:34:36.960 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'12050", 'reported_seq': 12241, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:33.069950+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:34:33.069950+0000', 'last_peered': '2024-09-10T20:34:33.069950+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:34:33.069950+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34955264, 'num_objects': 8534, 'num_object_clones': 0, 'num_object_copies': 17068, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3511, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8534, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8744, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 24, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5026}, {'shards': '0,2', 'objects': 1778}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:36.960 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:34:36.960 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'12303", 'reported_seq': 12491, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:33.088718+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:34:33.088718+0000', 'last_peered': '2024-09-10T20:34:33.088718+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:34:33.088718+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35565568, 'num_objects': 8683, 'num_object_clones': 0, 'num_object_copies': 17366, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3689, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8683, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9130, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 26, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4996}, {'shards': '0,2', 'objects': 1905}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:36.960 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:34:37.680 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:37.680 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33840/273366 objects degraded (12.379%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":389,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":136683,"data_bytes":559830484,"bytes_used":2187673600,"bytes_avail":287722618880,"bytes_total":289910292480,"degraded_objects":33840,"degraded_total":273366,"degraded_ratio":0.12379008362415224,"misplaced_objects":1725,"misplaced_total":273366,"misplaced_ratio":0.0063102214613375476,"recovering_objects_per_sec":42,"recovering_bytes_per_sec":167028,"recovering_keys_per_sec":0,"num_objects_recovered":426,"num_bytes_recovered":1671168,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":415},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:34:37.711 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:34:37.711 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:34:38.246 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:38.249 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:34:38.273 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":235,"stamp":"2024-09-10T20:34:36.940299+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":559830484,"num_objects":136683,"num_object_clones":0,"num_object_copies":273366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33840,"num_objects_misplaced":1725,"num_objects_unfound":0,"num_objects_dirty":136683,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":158738,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7584,"num_bytes_recovered":30101523,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160643,"ondisk_log_size":160643,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2136400,"kb_used_data":1197276,"kb_used_omap":12,"kb_used_meta":939059,"kb_avail":280979120,"statfs":{"total":289910292480,"available":287722618880,"internally_reserved":0,"allocated":1226010624,"data_stored":1225184644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":961596418},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,5,4,2,0,0,0,0,0,7,4],"upper_bound":16384},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17035264,"num_objects":-4159,"num_object_clones":0,"num_object_copies":-8318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2022,"num_objects_misplaced":1594,"num_objects_unfound":0,"num_objects_dirty":-4159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4159,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":426,"num_bytes_recovered":1671168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.005304"},"pg_stats":[{"pgid":"2.f","version":"47'12124","reported_seq":12267,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:33.088956+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:34:33.088956+0000","last_peered":"2024-09-10T20:34:33.088956+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:34:33.088956+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34955264,"num_objects":8534,"num_object_clones":0,"num_object_copies":17068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12124,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4956},{"shards":"0,2","objects":1812}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'12364","reported_seq":12479,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:33.762876+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:34:33.762876+0000","last_peered":"2024-09-10T20:34:33.762876+0000","last_clean":"2024-09-10T20:34:33.762876+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:34:33.762876+0000","last_undegraded":"2024-09-10T20:34:33.762876+0000","last_fullsized":"2024-09-10T20:34:33.762876+0000","mapping_epoch":35,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35135488,"num_objects":8578,"num_object_clones":0,"num_object_copies":17156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10550,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'12206","reported_seq":19559,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:34.404954+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:34:34.404954+0000","last_peered":"2024-09-10T20:34:34.404954+0000","last_clean":"2024-09-10T20:34:34.404954+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:34:34.404954+0000","last_undegraded":"2024-09-10T20:34:34.404954+0000","last_fullsized":"2024-09-10T20:34:34.404954+0000","mapping_epoch":44,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34852888,"num_objects":8510,"num_object_clones":0,"num_object_copies":17020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8510,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10334,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'12109","reported_seq":12194,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:33.828733+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:34:33.828733+0000","last_peered":"2024-09-10T20:34:33.828733+0000","last_clean":"2024-09-10T20:34:33.828733+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:34:33.828733+0000","last_undegraded":"2024-09-10T20:34:33.828733+0000","last_fullsized":"2024-09-10T20:34:33.828733+0000","mapping_epoch":35,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34795520,"num_objects":8495,"num_object_clones":0,"num_object_copies":16990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12109,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'12321","reported_seq":12452,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:33.117390+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:34:33.117390+0000","last_peered":"2024-09-10T20:34:33.117390+0000","last_clean":"2024-09-10T20:34:33.117390+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:34:33.117390+0000","last_undegraded":"2024-09-10T20:34:33.117390+0000","last_fullsized":"2024-09-10T20:34:33.117390+0000","mapping_epoch":34,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35831808,"num_objects":8748,"num_object_clones":0,"num_object_copies":17496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9317,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'12011","reported_seq":12198,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:33.076331+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:34:33.076331+0000","last_peered":"2024-09-10T20:34:33.076331+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:34:33.076331+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34652160,"num_objects":8460,"num_object_clones":0,"num_object_copies":16920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8875,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4869},{"shards":"0,2","objects":1841}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'12092","reported_seq":12311,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:33.111045+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:34:33.111045+0000","last_peered":"2024-09-10T20:34:33.111045+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:34:33.111045+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34881536,"num_objects":8516,"num_object_clones":0,"num_object_copies":17032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3519,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8687,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4999},{"shards":"0,2","objects":1770}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'12152","reported_seq":12281,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:33.052989+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:34:33.052989+0000","last_peered":"2024-09-10T20:34:33.052989+0000","last_clean":"2024-09-10T20:34:33.052989+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:34:33.052989+0000","last_undegraded":"2024-09-10T20:34:33.052989+0000","last_fullsized":"2024-09-10T20:34:33.052989+0000","mapping_epoch":34,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35160064,"num_objects":8584,"num_object_clones":0,"num_object_copies":17168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9223,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'11987","reported_seq":12148,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:33.808018+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:34:33.808018+0000","last_peered":"2024-09-10T20:34:33.808018+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:34:33.808018+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34619392,"num_objects":8452,"num_object_clones":0,"num_object_copies":16904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3516,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9980,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1761},{"shards":"2","objects":4938}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086228+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086228+0000","last_peered":"2024-09-10T20:34:27.086228+0000","last_clean":"2024-09-10T20:34:27.086228+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086228+0000","last_undegraded":"2024-09-10T20:34:27.086228+0000","last_fullsized":"2024-09-10T20:34:27.086228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'12179","reported_seq":12336,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:33.076257+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:34:33.076257+0000","last_peered":"2024-09-10T20:34:33.076257+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:34:33.076257+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35025862,"num_objects":8553,"num_object_clones":0,"num_object_copies":17106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3595,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8553,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12177,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4960},{"shards":"0,2","objects":1805}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.075947+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.075947+0000","last_peered":"2024-09-10T20:34:27.075947+0000","last_clean":"2024-09-10T20:34:27.075947+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.075947+0000","last_undegraded":"2024-09-10T20:34:27.075947+0000","last_fullsized":"2024-09-10T20:34:27.075947+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'12101","reported_seq":12266,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:33.754205+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:34:33.754205+0000","last_peered":"2024-09-10T20:34:33.754205+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:34:33.754205+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34668544,"num_objects":8464,"num_object_clones":0,"num_object_copies":16928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3619,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9929,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1817},{"shards":"2","objects":4847}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'12161","reported_seq":12865,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:33.835128+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:34:33.835128+0000","last_peered":"2024-09-10T20:34:33.835128+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:34:33.835128+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34828288,"num_objects":8503,"num_object_clones":0,"num_object_copies":17006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1600,"num_objects_misplaced":1725,"num_objects_unfound":0,"num_objects_dirty":8503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9947,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":285,"num_bytes_recovered":1134592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1725},{"shards":"2","objects":5178}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'12225","reported_seq":12423,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:32.963752+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:34:32.963752+0000","last_peered":"2024-09-10T20:34:32.963752+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:34:32.963752+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35246080,"num_objects":8605,"num_object_clones":0,"num_object_copies":17210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3620,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8605,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8948,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4986},{"shards":"0,2","objects":1839}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'12050","reported_seq":12241,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:33.069950+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:34:33.069950+0000","last_peered":"2024-09-10T20:34:33.069950+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:34:33.069950+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34955264,"num_objects":8534,"num_object_clones":0,"num_object_copies":17068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3511,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8744,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5026},{"shards":"0,2","objects":1778}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086748+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086748+0000","last_peered":"2024-09-10T20:34:27.086748+0000","last_clean":"2024-09-10T20:34:27.086748+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086748+0000","last_undegraded":"2024-09-10T20:34:27.086748+0000","last_fullsized":"2024-09-10T20:34:27.086748+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'12303","reported_seq":12491,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:33.088718+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:34:33.088718+0000","last_peered":"2024-09-10T20:34:33.088718+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:34:33.088718+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35565568,"num_objects":8683,"num_object_clones":0,"num_object_copies":17366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3689,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9130,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4996},{"shards":"0,2","objects":1905}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'12055","reported_seq":19177,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:34.383950+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:34:34.383950+0000","last_peered":"2024-09-10T20:34:34.383950+0000","last_clean":"2024-09-10T20:34:34.383950+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:34:34.383950+0000","last_undegraded":"2024-09-10T20:34:34.383950+0000","last_fullsized":"2024-09-10T20:34:34.383950+0000","mapping_epoch":46,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34656739,"num_objects":8462,"num_object_clones":0,"num_object_copies":16924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8662,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":559830465,"num_objects":136681,"num_object_clones":0,"num_object_copies":273362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33840,"num_objects_misplaced":1725,"num_objects_unfound":0,"num_objects_dirty":136681,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":158736,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7582,"num_bytes_recovered":30101504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1222643712,"data_stored":1222559695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160640,"ondisk_log_size":160640,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182472,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":826800,"kb_used_data":470304,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93545040,"statfs":{"total":96636764160,"available":95790120960,"internally_reserved":0,"allocated":481591296,"data_stored":481339003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3,2,0,0,0,0,0,7],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986244,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687664,"kb_used_data":389408,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93684176,"statfs":{"total":96636764160,"available":95932596224,"internally_reserved":0,"allocated":398753792,"data_stored":398501504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305393322},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,0,0,0,0,0,0,0,0,4],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659191,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621936,"kb_used_data":337564,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93749904,"statfs":{"total":96636764160,"available":95999901696,"internally_reserved":0,"allocated":345665536,"data_stored":345344137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":291172014},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":480059392,"data_stored":480055779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":345047040,"data_stored":344974321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":397537280,"data_stored":397529595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:34:41.277 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:34:41.830 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:41.830 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:34:41.858 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":237,"stamp":"2024-09-10T20:34:40.942019+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":550770132,"num_objects":134471,"num_object_clones":0,"num_object_copies":268942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33802,"num_objects_misplaced":1603,"num_objects_unfound":0,"num_objects_dirty":134471,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":160950,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7744,"num_bytes_recovered":30601235,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160755,"ondisk_log_size":160755,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2136356,"kb_used_data":1184744,"kb_used_omap":12,"kb_used_meta":951539,"kb_avail":280979164,"statfs":{"total":289910292480,"available":287722663936,"internally_reserved":0,"allocated":1213177856,"data_stored":1212351876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":974375938},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,3,7,11,1,0,0,0,3,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19292160,"num_objects":-4710,"num_object_clones":0,"num_object_copies":-9420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-50,"num_objects_misplaced":-389,"num_objects_unfound":0,"num_objects_dirty":-4710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4710,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":439,"num_bytes_recovered":1593344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005111"},"pg_stats":[{"pgid":"2.f","version":"47'12255","reported_seq":12415,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:38.105038+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:34:38.105038+0000","last_peered":"2024-09-10T20:34:38.105038+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:34:38.105038+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34418688,"num_objects":8403,"num_object_clones":0,"num_object_copies":16806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3569,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12255,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4834},{"shards":"0,2","objects":1681}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'12503","reported_seq":12618,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:38.847177+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:34:38.847177+0000","last_peered":"2024-09-10T20:34:38.847177+0000","last_clean":"2024-09-10T20:34:38.847177+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:34:38.847177+0000","last_undegraded":"2024-09-10T20:34:38.847177+0000","last_fullsized":"2024-09-10T20:34:38.847177+0000","mapping_epoch":35,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34566144,"num_objects":8439,"num_object_clones":0,"num_object_copies":16878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10689,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'12347","reported_seq":19700,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:39.376532+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:34:39.376532+0000","last_peered":"2024-09-10T20:34:39.376532+0000","last_clean":"2024-09-10T20:34:39.376532+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:34:39.376532+0000","last_undegraded":"2024-09-10T20:34:39.376532+0000","last_fullsized":"2024-09-10T20:34:39.376532+0000","mapping_epoch":44,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34275352,"num_objects":8369,"num_object_clones":0,"num_object_copies":16738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8369,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10475,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'12247","reported_seq":12332,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:38.850557+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:34:38.850557+0000","last_peered":"2024-09-10T20:34:38.850557+0000","last_clean":"2024-09-10T20:34:38.850557+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:34:38.850557+0000","last_undegraded":"2024-09-10T20:34:38.850557+0000","last_fullsized":"2024-09-10T20:34:38.850557+0000","mapping_epoch":35,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34230272,"num_objects":8357,"num_object_clones":0,"num_object_copies":16714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12247,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'12465","reported_seq":12596,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:38.137260+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:34:38.137260+0000","last_peered":"2024-09-10T20:34:38.137260+0000","last_clean":"2024-09-10T20:34:38.137260+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:34:38.137260+0000","last_undegraded":"2024-09-10T20:34:38.137260+0000","last_fullsized":"2024-09-10T20:34:38.137260+0000","mapping_epoch":34,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35241984,"num_objects":8604,"num_object_clones":0,"num_object_copies":17208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9461,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'12142","reported_seq":12337,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:38.079597+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:34:38.079597+0000","last_peered":"2024-09-10T20:34:38.079597+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:34:38.079597+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34115584,"num_objects":8329,"num_object_clones":0,"num_object_copies":16658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3589,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9006,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4742},{"shards":"0,2","objects":1710}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'12208","reported_seq":12435,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:38.012208+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:34:38.012208+0000","last_peered":"2024-09-10T20:34:38.012208+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:34:38.012208+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34406400,"num_objects":8400,"num_object_clones":0,"num_object_copies":16800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3515,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8803,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4887},{"shards":"0,2","objects":1654}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'12299","reported_seq":12428,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:38.087976+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:34:38.087976+0000","last_peered":"2024-09-10T20:34:38.087976+0000","last_clean":"2024-09-10T20:34:38.087976+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:34:38.087976+0000","last_undegraded":"2024-09-10T20:34:38.087976+0000","last_fullsized":"2024-09-10T20:34:38.087976+0000","mapping_epoch":34,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34557952,"num_objects":8437,"num_object_clones":0,"num_object_copies":16874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9370,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'12123","reported_seq":12292,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:38.807818+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:34:38.807818+0000","last_peered":"2024-09-10T20:34:38.807818+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:34:38.807818+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34062336,"num_objects":8316,"num_object_clones":0,"num_object_copies":16632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3511,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10116,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1625},{"shards":"2","objects":4807}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086228+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086228+0000","last_peered":"2024-09-10T20:34:27.086228+0000","last_clean":"2024-09-10T20:34:27.086228+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086228+0000","last_undegraded":"2024-09-10T20:34:27.086228+0000","last_fullsized":"2024-09-10T20:34:27.086228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'12321","reported_seq":12488,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:38.141000+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:34:38.141000+0000","last_peered":"2024-09-10T20:34:38.141000+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:34:38.141000+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34444230,"num_objects":8411,"num_object_clones":0,"num_object_copies":16822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3590,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12319,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4823},{"shards":"0,2","objects":1663}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.075947+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.075947+0000","last_peered":"2024-09-10T20:34:27.075947+0000","last_clean":"2024-09-10T20:34:27.075947+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.075947+0000","last_undegraded":"2024-09-10T20:34:27.075947+0000","last_fullsized":"2024-09-10T20:34:27.075947+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'12257","reported_seq":12422,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:38.784893+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:34:38.784893+0000","last_peered":"2024-09-10T20:34:38.784893+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:34:38.784893+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34029568,"num_objects":8308,"num_object_clones":0,"num_object_copies":16616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3619,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10085,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1661},{"shards":"2","objects":4691}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'12301","reported_seq":13246,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:38.851332+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:34:38.851332+0000","last_peered":"2024-09-10T20:34:38.851332+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:34:38.851332+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34254848,"num_objects":8363,"num_object_clones":0,"num_object_copies":16726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1600,"num_objects_misplaced":1603,"num_objects_unfound":0,"num_objects_dirty":8363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10087,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":407,"num_bytes_recovered":1634304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1603},{"shards":"2","objects":5160}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'12378","reported_seq":12589,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:38.106185+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:34:38.106185+0000","last_peered":"2024-09-10T20:34:38.106185+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:34:38.106185+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34619392,"num_objects":8452,"num_object_clones":0,"num_object_copies":16904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3613,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9101,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4840},{"shards":"0,2","objects":1686}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'12174","reported_seq":12371,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:38.054993+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:34:38.054993+0000","last_peered":"2024-09-10T20:34:38.054993+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:34:38.054993+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34447360,"num_objects":8410,"num_object_clones":0,"num_object_copies":16820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3508,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8868,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4905},{"shards":"0,2","objects":1654}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086748+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086748+0000","last_peered":"2024-09-10T20:34:27.086748+0000","last_clean":"2024-09-10T20:34:27.086748+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086748+0000","last_undegraded":"2024-09-10T20:34:27.086748+0000","last_fullsized":"2024-09-10T20:34:27.086748+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'12453","reported_seq":12643,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:38.058498+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:34:38.058498+0000","last_peered":"2024-09-10T20:34:38.058498+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:34:38.058498+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34951168,"num_objects":8533,"num_object_clones":0,"num_object_copies":17066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9280,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4847},{"shards":"0,2","objects":1755}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'12179","reported_seq":19301,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:39.318996+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:34:39.318996+0000","last_peered":"2024-09-10T20:34:39.318996+0000","last_clean":"2024-09-10T20:34:39.318996+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:34:39.318996+0000","last_undegraded":"2024-09-10T20:34:39.318996+0000","last_fullsized":"2024-09-10T20:34:39.318996+0000","mapping_epoch":46,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34148835,"num_objects":8338,"num_object_clones":0,"num_object_copies":16676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8786,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":550770113,"num_objects":134469,"num_object_clones":0,"num_object_copies":268938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33802,"num_objects_misplaced":1603,"num_objects_unfound":0,"num_objects_dirty":134469,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":160948,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7742,"num_bytes_recovered":30601216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1210044416,"data_stored":1209960399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160752,"ondisk_log_size":160752,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182473,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":824244,"kb_used_data":463584,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93547596,"statfs":{"total":96636764160,"available":95792738304,"internally_reserved":0,"allocated":474710016,"data_stored":474457723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1,5,1,0,0,0,3],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986245,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687184,"kb_used_data":384768,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93684656,"statfs":{"total":96636764160,"available":95933087744,"internally_reserved":0,"allocated":394002432,"data_stored":393750144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309653162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,4,2,0,0,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659192,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":624928,"kb_used_data":336392,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93746912,"statfs":{"total":96636764160,"available":95996837888,"internally_reserved":0,"allocated":344465408,"data_stored":344144009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":295431854},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,2,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":473309184,"data_stored":473305571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":343842816,"data_stored":343770097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":392892416,"data_stored":392884731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:34:41.861 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:34:42.414 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:42.416 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:34:42.437 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":237,"stamp":"2024-09-10T20:34:40.942019+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":550770132,"num_objects":134471,"num_object_clones":0,"num_object_copies":268942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33802,"num_objects_misplaced":1603,"num_objects_unfound":0,"num_objects_dirty":134471,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":160950,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7744,"num_bytes_recovered":30601235,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160755,"ondisk_log_size":160755,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2136356,"kb_used_data":1184744,"kb_used_omap":12,"kb_used_meta":951539,"kb_avail":280979164,"statfs":{"total":289910292480,"available":287722663936,"internally_reserved":0,"allocated":1213177856,"data_stored":1212351876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":974375938},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,3,7,11,1,0,0,0,3,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19292160,"num_objects":-4710,"num_object_clones":0,"num_object_copies":-9420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-50,"num_objects_misplaced":-389,"num_objects_unfound":0,"num_objects_dirty":-4710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4710,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":439,"num_bytes_recovered":1593344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005111"},"pg_stats":[{"pgid":"2.f","version":"47'12255","reported_seq":12415,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:38.105038+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:34:38.105038+0000","last_peered":"2024-09-10T20:34:38.105038+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:34:38.105038+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34418688,"num_objects":8403,"num_object_clones":0,"num_object_copies":16806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3569,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12255,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4834},{"shards":"0,2","objects":1681}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'12503","reported_seq":12618,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:38.847177+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:34:38.847177+0000","last_peered":"2024-09-10T20:34:38.847177+0000","last_clean":"2024-09-10T20:34:38.847177+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:34:38.847177+0000","last_undegraded":"2024-09-10T20:34:38.847177+0000","last_fullsized":"2024-09-10T20:34:38.847177+0000","mapping_epoch":35,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34566144,"num_objects":8439,"num_object_clones":0,"num_object_copies":16878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10689,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'12347","reported_seq":19700,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:39.376532+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:34:39.376532+0000","last_peered":"2024-09-10T20:34:39.376532+0000","last_clean":"2024-09-10T20:34:39.376532+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:34:39.376532+0000","last_undegraded":"2024-09-10T20:34:39.376532+0000","last_fullsized":"2024-09-10T20:34:39.376532+0000","mapping_epoch":44,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34275352,"num_objects":8369,"num_object_clones":0,"num_object_copies":16738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8369,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10475,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'12247","reported_seq":12332,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:38.850557+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:34:38.850557+0000","last_peered":"2024-09-10T20:34:38.850557+0000","last_clean":"2024-09-10T20:34:38.850557+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:34:38.850557+0000","last_undegraded":"2024-09-10T20:34:38.850557+0000","last_fullsized":"2024-09-10T20:34:38.850557+0000","mapping_epoch":35,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34230272,"num_objects":8357,"num_object_clones":0,"num_object_copies":16714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12247,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'12465","reported_seq":12596,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:38.137260+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:34:38.137260+0000","last_peered":"2024-09-10T20:34:38.137260+0000","last_clean":"2024-09-10T20:34:38.137260+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:34:38.137260+0000","last_undegraded":"2024-09-10T20:34:38.137260+0000","last_fullsized":"2024-09-10T20:34:38.137260+0000","mapping_epoch":34,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35241984,"num_objects":8604,"num_object_clones":0,"num_object_copies":17208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9461,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'12142","reported_seq":12337,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:38.079597+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:34:38.079597+0000","last_peered":"2024-09-10T20:34:38.079597+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:34:38.079597+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34115584,"num_objects":8329,"num_object_clones":0,"num_object_copies":16658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3589,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9006,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4742},{"shards":"0,2","objects":1710}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'12208","reported_seq":12435,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:38.012208+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:34:38.012208+0000","last_peered":"2024-09-10T20:34:38.012208+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:34:38.012208+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34406400,"num_objects":8400,"num_object_clones":0,"num_object_copies":16800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3515,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8803,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4887},{"shards":"0,2","objects":1654}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'12299","reported_seq":12428,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:38.087976+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:34:38.087976+0000","last_peered":"2024-09-10T20:34:38.087976+0000","last_clean":"2024-09-10T20:34:38.087976+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:34:38.087976+0000","last_undegraded":"2024-09-10T20:34:38.087976+0000","last_fullsized":"2024-09-10T20:34:38.087976+0000","mapping_epoch":34,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34557952,"num_objects":8437,"num_object_clones":0,"num_object_copies":16874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9370,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'12123","reported_seq":12292,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:38.807818+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:34:38.807818+0000","last_peered":"2024-09-10T20:34:38.807818+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:34:38.807818+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34062336,"num_objects":8316,"num_object_clones":0,"num_object_copies":16632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3511,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10116,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1625},{"shards":"2","objects":4807}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086228+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086228+0000","last_peered":"2024-09-10T20:34:27.086228+0000","last_clean":"2024-09-10T20:34:27.086228+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086228+0000","last_undegraded":"2024-09-10T20:34:27.086228+0000","last_fullsized":"2024-09-10T20:34:27.086228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'12321","reported_seq":12488,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:38.141000+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:34:38.141000+0000","last_peered":"2024-09-10T20:34:38.141000+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:34:38.141000+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34444230,"num_objects":8411,"num_object_clones":0,"num_object_copies":16822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3590,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12319,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4823},{"shards":"0,2","objects":1663}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.075947+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.075947+0000","last_peered":"2024-09-10T20:34:27.075947+0000","last_clean":"2024-09-10T20:34:27.075947+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.075947+0000","last_undegraded":"2024-09-10T20:34:27.075947+0000","last_fullsized":"2024-09-10T20:34:27.075947+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'12257","reported_seq":12422,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:38.784893+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:34:38.784893+0000","last_peered":"2024-09-10T20:34:38.784893+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:34:38.784893+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34029568,"num_objects":8308,"num_object_clones":0,"num_object_copies":16616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3619,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10085,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1661},{"shards":"2","objects":4691}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'12301","reported_seq":13246,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:38.851332+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:34:38.851332+0000","last_peered":"2024-09-10T20:34:38.851332+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:34:38.851332+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34254848,"num_objects":8363,"num_object_clones":0,"num_object_copies":16726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1600,"num_objects_misplaced":1603,"num_objects_unfound":0,"num_objects_dirty":8363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10087,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":407,"num_bytes_recovered":1634304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1603},{"shards":"2","objects":5160}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'12378","reported_seq":12589,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:38.106185+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:34:38.106185+0000","last_peered":"2024-09-10T20:34:38.106185+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:34:38.106185+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34619392,"num_objects":8452,"num_object_clones":0,"num_object_copies":16904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3613,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9101,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4840},{"shards":"0,2","objects":1686}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'12174","reported_seq":12371,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:38.054993+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:34:38.054993+0000","last_peered":"2024-09-10T20:34:38.054993+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:34:38.054993+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34447360,"num_objects":8410,"num_object_clones":0,"num_object_copies":16820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3508,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8868,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4905},{"shards":"0,2","objects":1654}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086748+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086748+0000","last_peered":"2024-09-10T20:34:27.086748+0000","last_clean":"2024-09-10T20:34:27.086748+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086748+0000","last_undegraded":"2024-09-10T20:34:27.086748+0000","last_fullsized":"2024-09-10T20:34:27.086748+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'12453","reported_seq":12643,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:38.058498+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:34:38.058498+0000","last_peered":"2024-09-10T20:34:38.058498+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:34:38.058498+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34951168,"num_objects":8533,"num_object_clones":0,"num_object_copies":17066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9280,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4847},{"shards":"0,2","objects":1755}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'12179","reported_seq":19301,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:39.318996+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:34:39.318996+0000","last_peered":"2024-09-10T20:34:39.318996+0000","last_clean":"2024-09-10T20:34:39.318996+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:34:39.318996+0000","last_undegraded":"2024-09-10T20:34:39.318996+0000","last_fullsized":"2024-09-10T20:34:39.318996+0000","mapping_epoch":46,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34148835,"num_objects":8338,"num_object_clones":0,"num_object_copies":16676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8786,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":550770113,"num_objects":134469,"num_object_clones":0,"num_object_copies":268938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33802,"num_objects_misplaced":1603,"num_objects_unfound":0,"num_objects_dirty":134469,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":160948,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7742,"num_bytes_recovered":30601216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1210044416,"data_stored":1209960399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160752,"ondisk_log_size":160752,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182473,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":824244,"kb_used_data":463584,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93547596,"statfs":{"total":96636764160,"available":95792738304,"internally_reserved":0,"allocated":474710016,"data_stored":474457723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1,5,1,0,0,0,3],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986245,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687184,"kb_used_data":384768,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93684656,"statfs":{"total":96636764160,"available":95933087744,"internally_reserved":0,"allocated":394002432,"data_stored":393750144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309653162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,4,2,0,0,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659192,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":624928,"kb_used_data":336392,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93746912,"statfs":{"total":96636764160,"available":95996837888,"internally_reserved":0,"allocated":344465408,"data_stored":344144009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":295431854},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,2,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":473309184,"data_stored":473305571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":343842816,"data_stored":343770097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":392892416,"data_stored":392884731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:34:42.438 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:34:42.438 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'12255", 'reported_seq': 12415, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:38.105038+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:34:38.105038+0000', 'last_peered': '2024-09-10T20:34:38.105038+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:34:38.105038+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34418688, 'num_objects': 8403, 'num_object_clones': 0, 'num_object_copies': 16806, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3569, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8403, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12255, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 39, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4834}, {'shards': '0,2', 'objects': 1681}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:42.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:34:42.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'12142", 'reported_seq': 12337, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:38.079597+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:34:38.079597+0000', 'last_peered': '2024-09-10T20:34:38.079597+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:34:38.079597+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "36'2100", 'ondisk_log_start': "36'2100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34115584, 'num_objects': 8329, 'num_object_clones': 0, 'num_object_copies': 16658, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3589, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8329, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9006, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4742}, {'shards': '0,2', 'objects': 1710}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:42.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:34:42.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'12208", 'reported_seq': 12435, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:38.012208+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:34:38.012208+0000', 'last_peered': '2024-09-10T20:34:38.012208+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:34:38.012208+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34406400, 'num_objects': 8400, 'num_object_clones': 0, 'num_object_copies': 16800, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3515, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8400, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8803, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 42, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4887}, {'shards': '0,2', 'objects': 1654}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:42.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:34:42.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'12123", 'reported_seq': 12292, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:38.807818+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:34:38.807818+0000', 'last_peered': '2024-09-10T20:34:38.807818+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:34:38.807818+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "36'2100", 'ondisk_log_start': "36'2100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34062336, 'num_objects': 8316, 'num_object_clones': 0, 'num_object_copies': 16632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3511, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8316, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10116, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 17, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1625}, {'shards': '2', 'objects': 4807}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:34:42.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:34:42.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'12321", 'reported_seq': 12488, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:38.141000+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:34:38.141000+0000', 'last_peered': '2024-09-10T20:34:38.141000+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:34:38.141000+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34444230, 'num_objects': 8411, 'num_object_clones': 0, 'num_object_copies': 16822, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3590, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8411, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12319, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4823}, {'shards': '0,2', 'objects': 1663}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:42.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:34:42.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'12257", 'reported_seq': 12422, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:38.784893+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:34:38.784893+0000', 'last_peered': '2024-09-10T20:34:38.784893+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:34:38.784893+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34029568, 'num_objects': 8308, 'num_object_clones': 0, 'num_object_copies': 16616, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3619, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8308, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10085, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 16, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1661}, {'shards': '2', 'objects': 4691}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:34:42.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:34:42.440 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'12301", 'reported_seq': 13246, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:38.851332+0000', 'last_change': '2024-09-10T20:34:25.241937+0000', 'last_active': '2024-09-10T20:34:38.851332+0000', 'last_peered': '2024-09-10T20:34:38.851332+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:34:38.851332+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34254848, 'num_objects': 8363, 'num_object_clones': 0, 'num_object_copies': 16726, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1600, 'num_objects_misplaced': 1603, 'num_objects_unfound': 0, 'num_objects_dirty': 8363, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10087, 'num_write_kb': 32484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 407, 'num_bytes_recovered': 1634304, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1603}, {'shards': '2', 'objects': 5160}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:34:42.440 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:34:42.440 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'12378", 'reported_seq': 12589, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:38.106185+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:34:38.106185+0000', 'last_peered': '2024-09-10T20:34:38.106185+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:34:38.106185+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34619392, 'num_objects': 8452, 'num_object_clones': 0, 'num_object_copies': 16904, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3613, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8452, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9101, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 36, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4840}, {'shards': '0,2', 'objects': 1686}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:42.440 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:34:42.440 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'12174", 'reported_seq': 12371, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:38.054993+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:34:38.054993+0000', 'last_peered': '2024-09-10T20:34:38.054993+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:34:38.054993+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "36'2100", 'ondisk_log_start': "36'2100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34447360, 'num_objects': 8410, 'num_object_clones': 0, 'num_object_copies': 16820, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3508, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8410, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8868, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4905}, {'shards': '0,2', 'objects': 1654}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:42.440 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:34:42.440 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'12453", 'reported_seq': 12643, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:38.058498+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:34:38.058498+0000', 'last_peered': '2024-09-10T20:34:38.058498+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:34:38.058498+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34951168, 'num_objects': 8533, 'num_object_clones': 0, 'num_object_copies': 17066, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8533, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9280, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4847}, {'shards': '0,2', 'objects': 1755}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:42.440 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:34:43.118 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:43.118 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33802/268942 objects degraded (12.569%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":394,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":134471,"data_bytes":550770132,"bytes_used":2187628544,"bytes_avail":287722663936,"bytes_total":289910292480,"degraded_objects":33802,"degraded_total":268942,"degraded_ratio":0.12568509195291178,"misplaced_objects":1603,"misplaced_total":268942,"misplaced_ratio":0.0059603929471782023,"recovering_objects_per_sec":36,"recovering_bytes_per_sec":132722,"recovering_keys_per_sec":0,"num_objects_recovered":439,"num_bytes_recovered":1593344,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":392},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:34:43.143 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:34:43.144 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:34:44.337 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:44.339 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:34:44.361 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":238,"stamp":"2024-09-10T20:34:42.942892+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":550770132,"num_objects":134471,"num_object_clones":0,"num_object_copies":268942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33802,"num_objects_misplaced":1603,"num_objects_unfound":0,"num_objects_dirty":134471,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":160950,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7744,"num_bytes_recovered":30601235,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160755,"ondisk_log_size":160755,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2136356,"kb_used_data":1184744,"kb_used_omap":12,"kb_used_meta":951539,"kb_avail":280979164,"statfs":{"total":289910292480,"available":287722663936,"internally_reserved":0,"allocated":1213177856,"data_stored":1212351876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":974375938},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,3,7,11,1,0,0,0,3,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18169856,"num_objects":-4436,"num_object_clones":0,"num_object_copies":-8872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-50,"num_objects_misplaced":-258,"num_objects_unfound":0,"num_objects_dirty":-4436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4436,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":1056768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005077"},"pg_stats":[{"pgid":"2.f","version":"47'12255","reported_seq":12415,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:38.105038+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:34:38.105038+0000","last_peered":"2024-09-10T20:34:38.105038+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:34:38.105038+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34418688,"num_objects":8403,"num_object_clones":0,"num_object_copies":16806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3569,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12255,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4834},{"shards":"0,2","objects":1681}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'12503","reported_seq":12618,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:38.847177+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:34:38.847177+0000","last_peered":"2024-09-10T20:34:38.847177+0000","last_clean":"2024-09-10T20:34:38.847177+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:34:38.847177+0000","last_undegraded":"2024-09-10T20:34:38.847177+0000","last_fullsized":"2024-09-10T20:34:38.847177+0000","mapping_epoch":35,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34566144,"num_objects":8439,"num_object_clones":0,"num_object_copies":16878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10689,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'12347","reported_seq":19700,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:39.376532+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:34:39.376532+0000","last_peered":"2024-09-10T20:34:39.376532+0000","last_clean":"2024-09-10T20:34:39.376532+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:34:39.376532+0000","last_undegraded":"2024-09-10T20:34:39.376532+0000","last_fullsized":"2024-09-10T20:34:39.376532+0000","mapping_epoch":44,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34275352,"num_objects":8369,"num_object_clones":0,"num_object_copies":16738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8369,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10475,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'12247","reported_seq":12332,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:38.850557+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:34:38.850557+0000","last_peered":"2024-09-10T20:34:38.850557+0000","last_clean":"2024-09-10T20:34:38.850557+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:34:38.850557+0000","last_undegraded":"2024-09-10T20:34:38.850557+0000","last_fullsized":"2024-09-10T20:34:38.850557+0000","mapping_epoch":35,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34230272,"num_objects":8357,"num_object_clones":0,"num_object_copies":16714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12247,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'12465","reported_seq":12596,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:38.137260+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:34:38.137260+0000","last_peered":"2024-09-10T20:34:38.137260+0000","last_clean":"2024-09-10T20:34:38.137260+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:34:38.137260+0000","last_undegraded":"2024-09-10T20:34:38.137260+0000","last_fullsized":"2024-09-10T20:34:38.137260+0000","mapping_epoch":34,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35241984,"num_objects":8604,"num_object_clones":0,"num_object_copies":17208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9461,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'12142","reported_seq":12337,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:38.079597+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:34:38.079597+0000","last_peered":"2024-09-10T20:34:38.079597+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:34:38.079597+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34115584,"num_objects":8329,"num_object_clones":0,"num_object_copies":16658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3589,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9006,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4742},{"shards":"0,2","objects":1710}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'12208","reported_seq":12435,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:38.012208+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:34:38.012208+0000","last_peered":"2024-09-10T20:34:38.012208+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:34:38.012208+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34406400,"num_objects":8400,"num_object_clones":0,"num_object_copies":16800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3515,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8803,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4887},{"shards":"0,2","objects":1654}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'12299","reported_seq":12428,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:38.087976+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:34:38.087976+0000","last_peered":"2024-09-10T20:34:38.087976+0000","last_clean":"2024-09-10T20:34:38.087976+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:34:38.087976+0000","last_undegraded":"2024-09-10T20:34:38.087976+0000","last_fullsized":"2024-09-10T20:34:38.087976+0000","mapping_epoch":34,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34557952,"num_objects":8437,"num_object_clones":0,"num_object_copies":16874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9370,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'12123","reported_seq":12292,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:38.807818+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:34:38.807818+0000","last_peered":"2024-09-10T20:34:38.807818+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:34:38.807818+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34062336,"num_objects":8316,"num_object_clones":0,"num_object_copies":16632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3511,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10116,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1625},{"shards":"2","objects":4807}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086228+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086228+0000","last_peered":"2024-09-10T20:34:27.086228+0000","last_clean":"2024-09-10T20:34:27.086228+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086228+0000","last_undegraded":"2024-09-10T20:34:27.086228+0000","last_fullsized":"2024-09-10T20:34:27.086228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'12321","reported_seq":12488,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:38.141000+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:34:38.141000+0000","last_peered":"2024-09-10T20:34:38.141000+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:34:38.141000+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34444230,"num_objects":8411,"num_object_clones":0,"num_object_copies":16822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3590,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12319,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4823},{"shards":"0,2","objects":1663}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.075947+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.075947+0000","last_peered":"2024-09-10T20:34:27.075947+0000","last_clean":"2024-09-10T20:34:27.075947+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.075947+0000","last_undegraded":"2024-09-10T20:34:27.075947+0000","last_fullsized":"2024-09-10T20:34:27.075947+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'12257","reported_seq":12422,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:38.784893+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:34:38.784893+0000","last_peered":"2024-09-10T20:34:38.784893+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:34:38.784893+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34029568,"num_objects":8308,"num_object_clones":0,"num_object_copies":16616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3619,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10085,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1661},{"shards":"2","objects":4691}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'12301","reported_seq":13246,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:38.851332+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:34:38.851332+0000","last_peered":"2024-09-10T20:34:38.851332+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:34:38.851332+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34254848,"num_objects":8363,"num_object_clones":0,"num_object_copies":16726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1600,"num_objects_misplaced":1603,"num_objects_unfound":0,"num_objects_dirty":8363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10087,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":407,"num_bytes_recovered":1634304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1603},{"shards":"2","objects":5160}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'12378","reported_seq":12589,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:38.106185+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:34:38.106185+0000","last_peered":"2024-09-10T20:34:38.106185+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:34:38.106185+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34619392,"num_objects":8452,"num_object_clones":0,"num_object_copies":16904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3613,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9101,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4840},{"shards":"0,2","objects":1686}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'12174","reported_seq":12371,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:38.054993+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:34:38.054993+0000","last_peered":"2024-09-10T20:34:38.054993+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:34:38.054993+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34447360,"num_objects":8410,"num_object_clones":0,"num_object_copies":16820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3508,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8868,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4905},{"shards":"0,2","objects":1654}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086748+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086748+0000","last_peered":"2024-09-10T20:34:27.086748+0000","last_clean":"2024-09-10T20:34:27.086748+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086748+0000","last_undegraded":"2024-09-10T20:34:27.086748+0000","last_fullsized":"2024-09-10T20:34:27.086748+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'12453","reported_seq":12643,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:38.058498+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:34:38.058498+0000","last_peered":"2024-09-10T20:34:38.058498+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:34:38.058498+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34951168,"num_objects":8533,"num_object_clones":0,"num_object_copies":17066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9280,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4847},{"shards":"0,2","objects":1755}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'12179","reported_seq":19301,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:39.318996+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:34:39.318996+0000","last_peered":"2024-09-10T20:34:39.318996+0000","last_clean":"2024-09-10T20:34:39.318996+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:34:39.318996+0000","last_undegraded":"2024-09-10T20:34:39.318996+0000","last_fullsized":"2024-09-10T20:34:39.318996+0000","mapping_epoch":46,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34148835,"num_objects":8338,"num_object_clones":0,"num_object_copies":16676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8786,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":550770113,"num_objects":134469,"num_object_clones":0,"num_object_copies":268938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33802,"num_objects_misplaced":1603,"num_objects_unfound":0,"num_objects_dirty":134469,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":160948,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7742,"num_bytes_recovered":30601216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1210044416,"data_stored":1209960399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160752,"ondisk_log_size":160752,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182473,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":824244,"kb_used_data":463584,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93547596,"statfs":{"total":96636764160,"available":95792738304,"internally_reserved":0,"allocated":474710016,"data_stored":474457723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1,5,1,0,0,0,3],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986245,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687184,"kb_used_data":384768,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93684656,"statfs":{"total":96636764160,"available":95933087744,"internally_reserved":0,"allocated":394002432,"data_stored":393750144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309653162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,4,2,0,0,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659192,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":624928,"kb_used_data":336392,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93746912,"statfs":{"total":96636764160,"available":95996837888,"internally_reserved":0,"allocated":344465408,"data_stored":344144009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":295431854},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,2,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":473309184,"data_stored":473305571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":343842816,"data_stored":343770097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":392892416,"data_stored":392884731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:34:47.363 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:34:47.893 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:47.895 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:34:47.927 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2024-09-10T20:34:46.944605+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":541488596,"num_objects":132205,"num_object_clones":0,"num_object_copies":264410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33788,"num_objects_misplaced":1485,"num_objects_unfound":0,"num_objects_dirty":132205,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":163216,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7878,"num_bytes_recovered":31084563,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2140500,"kb_used_data":1172260,"kb_used_omap":12,"kb_used_meta":968179,"kb_avail":280975020,"statfs":{"total":289910292480,"available":287718420480,"internally_reserved":0,"allocated":1200394240,"data_stored":1199575486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":991415298},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,3,12,9,3,0,0,0,3,1],"upper_bound":4096},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18341888,"num_objects":-4478,"num_object_clones":0,"num_object_copies":-8956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-52,"num_objects_misplaced":-240,"num_objects_unfound":0,"num_objects_dirty":-4478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4478,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":294,"num_bytes_recovered":983040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005145"},"pg_stats":[{"pgid":"2.f","version":"47'12382","reported_seq":12547,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:43.003561+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:34:43.003561+0000","last_peered":"2024-09-10T20:34:43.003561+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:34:43.003561+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33898496,"num_objects":8276,"num_object_clones":0,"num_object_copies":16552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3567,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12382,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4709},{"shards":"0,2","objects":1554}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'12662","reported_seq":12777,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:43.864808+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:34:43.864808+0000","last_peered":"2024-09-10T20:34:43.864808+0000","last_clean":"2024-09-10T20:34:43.864808+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:34:43.864808+0000","last_undegraded":"2024-09-10T20:34:43.864808+0000","last_fullsized":"2024-09-10T20:34:43.864808+0000","mapping_epoch":35,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33914880,"num_objects":8280,"num_object_clones":0,"num_object_copies":16560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10848,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'12490","reported_seq":19843,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:44.432372+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:34:44.432372+0000","last_peered":"2024-09-10T20:34:44.432372+0000","last_clean":"2024-09-10T20:34:44.432372+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:34:44.432372+0000","last_undegraded":"2024-09-10T20:34:44.432372+0000","last_fullsized":"2024-09-10T20:34:44.432372+0000","mapping_epoch":44,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33689624,"num_objects":8226,"num_object_clones":0,"num_object_copies":16452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8226,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10618,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'12390","reported_seq":12475,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:43.834623+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:34:43.834623+0000","last_peered":"2024-09-10T20:34:43.834623+0000","last_clean":"2024-09-10T20:34:43.834623+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:34:43.834623+0000","last_undegraded":"2024-09-10T20:34:43.834623+0000","last_fullsized":"2024-09-10T20:34:43.834623+0000","mapping_epoch":35,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33644544,"num_objects":8214,"num_object_clones":0,"num_object_copies":16428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12390,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'12612","reported_seq":12743,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:43.172994+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:34:43.172994+0000","last_peered":"2024-09-10T20:34:43.172994+0000","last_clean":"2024-09-10T20:34:43.172994+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:34:43.172994+0000","last_undegraded":"2024-09-10T20:34:43.172994+0000","last_fullsized":"2024-09-10T20:34:43.172994+0000","mapping_epoch":34,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34639872,"num_objects":8457,"num_object_clones":0,"num_object_copies":16914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9608,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'12305","reported_seq":12506,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:43.119670+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:34:43.119670+0000","last_peered":"2024-09-10T20:34:43.119670+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:34:43.119670+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33447936,"num_objects":8166,"num_object_clones":0,"num_object_copies":16332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3586,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9169,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4582},{"shards":"0,2","objects":1547}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'12332","reported_seq":12565,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:43.067931+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:34:43.067931+0000","last_peered":"2024-09-10T20:34:43.067931+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:34:43.067931+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33898496,"num_objects":8276,"num_object_clones":0,"num_object_copies":16552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3512,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8927,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4766},{"shards":"0,2","objects":1530}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'12433","reported_seq":12562,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:43.154885+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:34:43.154885+0000","last_peered":"2024-09-10T20:34:43.154885+0000","last_clean":"2024-09-10T20:34:43.154885+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:34:43.154885+0000","last_undegraded":"2024-09-10T20:34:43.154885+0000","last_fullsized":"2024-09-10T20:34:43.154885+0000","mapping_epoch":34,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34009088,"num_objects":8303,"num_object_clones":0,"num_object_copies":16606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9504,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'12261","reported_seq":12432,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:43.758861+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:34:43.758861+0000","last_peered":"2024-09-10T20:34:43.758861+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:34:43.758861+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33497088,"num_objects":8178,"num_object_clones":0,"num_object_copies":16356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3511,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10254,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1487},{"shards":"2","objects":4669}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086228+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086228+0000","last_peered":"2024-09-10T20:34:27.086228+0000","last_clean":"2024-09-10T20:34:27.086228+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086228+0000","last_undegraded":"2024-09-10T20:34:27.086228+0000","last_fullsized":"2024-09-10T20:34:27.086228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'12473","reported_seq":12644,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:43.143027+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:34:43.143027+0000","last_peered":"2024-09-10T20:34:43.143027+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:34:43.143027+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33821638,"num_objects":8259,"num_object_clones":0,"num_object_copies":16518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3587,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12471,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4674},{"shards":"0,2","objects":1511}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.075947+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.075947+0000","last_peered":"2024-09-10T20:34:27.075947+0000","last_clean":"2024-09-10T20:34:27.075947+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.075947+0000","last_undegraded":"2024-09-10T20:34:27.075947+0000","last_fullsized":"2024-09-10T20:34:27.075947+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'12397","reported_seq":12564,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:43.802140+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:34:43.802140+0000","last_peered":"2024-09-10T20:34:43.802140+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:34:43.802140+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33456128,"num_objects":8168,"num_object_clones":0,"num_object_copies":16336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3618,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10225,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1521},{"shards":"2","objects":4552}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'12435","reported_seq":13623,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:43.886590+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:34:43.886590+0000","last_peered":"2024-09-10T20:34:43.886590+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:34:43.886590+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33705984,"num_objects":8229,"num_object_clones":0,"num_object_copies":16458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1600,"num_objects_misplaced":1485,"num_objects_unfound":0,"num_objects_dirty":8229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10221,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":527,"num_bytes_recovered":2117632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1485},{"shards":"2","objects":5144}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'12515","reported_seq":12729,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:43.151624+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:34:43.151624+0000","last_peered":"2024-09-10T20:34:43.151624+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:34:43.151624+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34058240,"num_objects":8315,"num_object_clones":0,"num_object_copies":16630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8315,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9238,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4704},{"shards":"0,2","objects":1549}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'12305","reported_seq":12502,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:43.141963+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:34:43.141963+0000","last_peered":"2024-09-10T20:34:43.141963+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:34:43.141963+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33910784,"num_objects":8279,"num_object_clones":0,"num_object_copies":16558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3508,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8999,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4774},{"shards":"0,2","objects":1523}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086748+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086748+0000","last_peered":"2024-09-10T20:34:27.086748+0000","last_clean":"2024-09-10T20:34:27.086748+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086748+0000","last_undegraded":"2024-09-10T20:34:27.086748+0000","last_fullsized":"2024-09-10T20:34:27.086748+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'12610","reported_seq":12802,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:43.112356+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:34:43.112356+0000","last_peered":"2024-09-10T20:34:43.112356+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:34:43.112356+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34308096,"num_objects":8376,"num_object_clones":0,"num_object_copies":16752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3687,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9437,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4691},{"shards":"0,2","objects":1598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'12316","reported_seq":19438,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:44.308953+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:34:44.308953+0000","last_peered":"2024-09-10T20:34:44.308953+0000","last_clean":"2024-09-10T20:34:44.308953+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:34:44.308953+0000","last_undegraded":"2024-09-10T20:34:44.308953+0000","last_fullsized":"2024-09-10T20:34:44.308953+0000","mapping_epoch":46,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33587683,"num_objects":8201,"num_object_clones":0,"num_object_copies":16402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8923,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":541488577,"num_objects":132203,"num_object_clones":0,"num_object_copies":264406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33788,"num_objects_misplaced":1485,"num_objects_unfound":0,"num_objects_dirty":132203,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":163214,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7876,"num_bytes_recovered":31084544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1197121536,"data_stored":1197044745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182474,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":825780,"kb_used_data":456804,"kb_used_omap":4,"kb_used_meta":368955,"kb_avail":93546060,"statfs":{"total":96636764160,"available":95791165440,"internally_reserved":0,"allocated":467767296,"data_stored":467515003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":377810602},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,6,3,1,0,0,0,3],"upper_bound":2048},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986246,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":686828,"kb_used_data":380252,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93685012,"statfs":{"total":96636764160,"available":95933452288,"internally_reserved":0,"allocated":389378048,"data_stored":389125760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313913002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,1,3,0,0,0,0,0,1],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659193,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627892,"kb_used_data":335204,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93743948,"statfs":{"total":96636764160,"available":95993802752,"internally_reserved":0,"allocated":343248896,"data_stored":342934723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":299691694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,5,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":466223104,"data_stored":466219491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":342654976,"data_stored":342589483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":388243456,"data_stored":388235771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:34:47.928 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:34:48.489 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:48.491 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:34:48.517 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2024-09-10T20:34:46.944605+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":541488596,"num_objects":132205,"num_object_clones":0,"num_object_copies":264410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33788,"num_objects_misplaced":1485,"num_objects_unfound":0,"num_objects_dirty":132205,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":163216,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7878,"num_bytes_recovered":31084563,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2140500,"kb_used_data":1172260,"kb_used_omap":12,"kb_used_meta":968179,"kb_avail":280975020,"statfs":{"total":289910292480,"available":287718420480,"internally_reserved":0,"allocated":1200394240,"data_stored":1199575486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":991415298},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,3,12,9,3,0,0,0,3,1],"upper_bound":4096},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18341888,"num_objects":-4478,"num_object_clones":0,"num_object_copies":-8956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-52,"num_objects_misplaced":-240,"num_objects_unfound":0,"num_objects_dirty":-4478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4478,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":294,"num_bytes_recovered":983040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005145"},"pg_stats":[{"pgid":"2.f","version":"47'12382","reported_seq":12547,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:43.003561+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:34:43.003561+0000","last_peered":"2024-09-10T20:34:43.003561+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:34:43.003561+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33898496,"num_objects":8276,"num_object_clones":0,"num_object_copies":16552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3567,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12382,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4709},{"shards":"0,2","objects":1554}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'12662","reported_seq":12777,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:43.864808+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:34:43.864808+0000","last_peered":"2024-09-10T20:34:43.864808+0000","last_clean":"2024-09-10T20:34:43.864808+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:34:43.864808+0000","last_undegraded":"2024-09-10T20:34:43.864808+0000","last_fullsized":"2024-09-10T20:34:43.864808+0000","mapping_epoch":35,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33914880,"num_objects":8280,"num_object_clones":0,"num_object_copies":16560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10848,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'12490","reported_seq":19843,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:44.432372+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:34:44.432372+0000","last_peered":"2024-09-10T20:34:44.432372+0000","last_clean":"2024-09-10T20:34:44.432372+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:34:44.432372+0000","last_undegraded":"2024-09-10T20:34:44.432372+0000","last_fullsized":"2024-09-10T20:34:44.432372+0000","mapping_epoch":44,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33689624,"num_objects":8226,"num_object_clones":0,"num_object_copies":16452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8226,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10618,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'12390","reported_seq":12475,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:43.834623+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:34:43.834623+0000","last_peered":"2024-09-10T20:34:43.834623+0000","last_clean":"2024-09-10T20:34:43.834623+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:34:43.834623+0000","last_undegraded":"2024-09-10T20:34:43.834623+0000","last_fullsized":"2024-09-10T20:34:43.834623+0000","mapping_epoch":35,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33644544,"num_objects":8214,"num_object_clones":0,"num_object_copies":16428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12390,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'12612","reported_seq":12743,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:43.172994+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:34:43.172994+0000","last_peered":"2024-09-10T20:34:43.172994+0000","last_clean":"2024-09-10T20:34:43.172994+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:34:43.172994+0000","last_undegraded":"2024-09-10T20:34:43.172994+0000","last_fullsized":"2024-09-10T20:34:43.172994+0000","mapping_epoch":34,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34639872,"num_objects":8457,"num_object_clones":0,"num_object_copies":16914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9608,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'12305","reported_seq":12506,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:43.119670+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:34:43.119670+0000","last_peered":"2024-09-10T20:34:43.119670+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:34:43.119670+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33447936,"num_objects":8166,"num_object_clones":0,"num_object_copies":16332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3586,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9169,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4582},{"shards":"0,2","objects":1547}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'12332","reported_seq":12565,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:43.067931+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:34:43.067931+0000","last_peered":"2024-09-10T20:34:43.067931+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:34:43.067931+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33898496,"num_objects":8276,"num_object_clones":0,"num_object_copies":16552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3512,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8927,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4766},{"shards":"0,2","objects":1530}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'12433","reported_seq":12562,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:43.154885+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:34:43.154885+0000","last_peered":"2024-09-10T20:34:43.154885+0000","last_clean":"2024-09-10T20:34:43.154885+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:34:43.154885+0000","last_undegraded":"2024-09-10T20:34:43.154885+0000","last_fullsized":"2024-09-10T20:34:43.154885+0000","mapping_epoch":34,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34009088,"num_objects":8303,"num_object_clones":0,"num_object_copies":16606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9504,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'12261","reported_seq":12432,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:43.758861+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:34:43.758861+0000","last_peered":"2024-09-10T20:34:43.758861+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:34:43.758861+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33497088,"num_objects":8178,"num_object_clones":0,"num_object_copies":16356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3511,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10254,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1487},{"shards":"2","objects":4669}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086228+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086228+0000","last_peered":"2024-09-10T20:34:27.086228+0000","last_clean":"2024-09-10T20:34:27.086228+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086228+0000","last_undegraded":"2024-09-10T20:34:27.086228+0000","last_fullsized":"2024-09-10T20:34:27.086228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'12473","reported_seq":12644,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:43.143027+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:34:43.143027+0000","last_peered":"2024-09-10T20:34:43.143027+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:34:43.143027+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33821638,"num_objects":8259,"num_object_clones":0,"num_object_copies":16518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3587,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12471,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4674},{"shards":"0,2","objects":1511}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.075947+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.075947+0000","last_peered":"2024-09-10T20:34:27.075947+0000","last_clean":"2024-09-10T20:34:27.075947+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.075947+0000","last_undegraded":"2024-09-10T20:34:27.075947+0000","last_fullsized":"2024-09-10T20:34:27.075947+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'12397","reported_seq":12564,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:43.802140+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:34:43.802140+0000","last_peered":"2024-09-10T20:34:43.802140+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:34:43.802140+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33456128,"num_objects":8168,"num_object_clones":0,"num_object_copies":16336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3618,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10225,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1521},{"shards":"2","objects":4552}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'12435","reported_seq":13623,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:43.886590+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:34:43.886590+0000","last_peered":"2024-09-10T20:34:43.886590+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:34:43.886590+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33705984,"num_objects":8229,"num_object_clones":0,"num_object_copies":16458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1600,"num_objects_misplaced":1485,"num_objects_unfound":0,"num_objects_dirty":8229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10221,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":527,"num_bytes_recovered":2117632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1485},{"shards":"2","objects":5144}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'12515","reported_seq":12729,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:43.151624+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:34:43.151624+0000","last_peered":"2024-09-10T20:34:43.151624+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:34:43.151624+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34058240,"num_objects":8315,"num_object_clones":0,"num_object_copies":16630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8315,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9238,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4704},{"shards":"0,2","objects":1549}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'12305","reported_seq":12502,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:43.141963+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:34:43.141963+0000","last_peered":"2024-09-10T20:34:43.141963+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:34:43.141963+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33910784,"num_objects":8279,"num_object_clones":0,"num_object_copies":16558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3508,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8999,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4774},{"shards":"0,2","objects":1523}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086748+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086748+0000","last_peered":"2024-09-10T20:34:27.086748+0000","last_clean":"2024-09-10T20:34:27.086748+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086748+0000","last_undegraded":"2024-09-10T20:34:27.086748+0000","last_fullsized":"2024-09-10T20:34:27.086748+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'12610","reported_seq":12802,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:43.112356+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:34:43.112356+0000","last_peered":"2024-09-10T20:34:43.112356+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:34:43.112356+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34308096,"num_objects":8376,"num_object_clones":0,"num_object_copies":16752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3687,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9437,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4691},{"shards":"0,2","objects":1598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'12316","reported_seq":19438,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:44.308953+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:34:44.308953+0000","last_peered":"2024-09-10T20:34:44.308953+0000","last_clean":"2024-09-10T20:34:44.308953+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:34:44.308953+0000","last_undegraded":"2024-09-10T20:34:44.308953+0000","last_fullsized":"2024-09-10T20:34:44.308953+0000","mapping_epoch":46,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33587683,"num_objects":8201,"num_object_clones":0,"num_object_copies":16402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8923,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":541488577,"num_objects":132203,"num_object_clones":0,"num_object_copies":264406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33788,"num_objects_misplaced":1485,"num_objects_unfound":0,"num_objects_dirty":132203,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":163214,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7876,"num_bytes_recovered":31084544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1197121536,"data_stored":1197044745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182474,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":825780,"kb_used_data":456804,"kb_used_omap":4,"kb_used_meta":368955,"kb_avail":93546060,"statfs":{"total":96636764160,"available":95791165440,"internally_reserved":0,"allocated":467767296,"data_stored":467515003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":377810602},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,6,3,1,0,0,0,3],"upper_bound":2048},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986246,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":686828,"kb_used_data":380252,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93685012,"statfs":{"total":96636764160,"available":95933452288,"internally_reserved":0,"allocated":389378048,"data_stored":389125760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313913002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,1,3,0,0,0,0,0,1],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659193,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627892,"kb_used_data":335204,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93743948,"statfs":{"total":96636764160,"available":95993802752,"internally_reserved":0,"allocated":343248896,"data_stored":342934723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":299691694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,5,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":466223104,"data_stored":466219491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":342654976,"data_stored":342589483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":388243456,"data_stored":388235771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:34:48.518 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:34:48.518 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'12382", 'reported_seq': 12547, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:43.003561+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:34:43.003561+0000', 'last_peered': '2024-09-10T20:34:43.003561+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:34:43.003561+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33898496, 'num_objects': 8276, 'num_object_clones': 0, 'num_object_copies': 16552, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3567, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8276, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12382, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 41, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4709}, {'shards': '0,2', 'objects': 1554}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:48.518 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:34:48.518 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'12305", 'reported_seq': 12506, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:43.119670+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:34:43.119670+0000', 'last_peered': '2024-09-10T20:34:43.119670+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:34:43.119670+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33447936, 'num_objects': 8166, 'num_object_clones': 0, 'num_object_copies': 16332, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3586, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8166, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9169, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4582}, {'shards': '0,2', 'objects': 1547}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:48.519 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:34:48.519 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'12332", 'reported_seq': 12565, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:43.067931+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:34:43.067931+0000', 'last_peered': '2024-09-10T20:34:43.067931+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:34:43.067931+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33898496, 'num_objects': 8276, 'num_object_clones': 0, 'num_object_copies': 16552, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3512, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8276, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8927, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 45, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4766}, {'shards': '0,2', 'objects': 1530}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:48.519 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:34:48.519 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'12261", 'reported_seq': 12432, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:43.758861+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:34:43.758861+0000', 'last_peered': '2024-09-10T20:34:43.758861+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:34:43.758861+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33497088, 'num_objects': 8178, 'num_object_clones': 0, 'num_object_copies': 16356, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3511, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8178, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10254, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 17, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1487}, {'shards': '2', 'objects': 4669}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:34:48.519 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:34:48.519 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'12473", 'reported_seq': 12644, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:43.143027+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:34:43.143027+0000', 'last_peered': '2024-09-10T20:34:43.143027+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:34:43.143027+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33821638, 'num_objects': 8259, 'num_object_clones': 0, 'num_object_copies': 16518, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3587, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8259, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12471, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 32, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4674}, {'shards': '0,2', 'objects': 1511}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:48.519 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:34:48.519 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'12397", 'reported_seq': 12564, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:43.802140+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:34:43.802140+0000', 'last_peered': '2024-09-10T20:34:43.802140+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:34:43.802140+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33456128, 'num_objects': 8168, 'num_object_clones': 0, 'num_object_copies': 16336, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3618, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8168, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10225, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 17, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1521}, {'shards': '2', 'objects': 4552}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:34:48.519 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:34:48.519 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'12435", 'reported_seq': 13623, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:43.886590+0000', 'last_change': '2024-09-10T20:34:25.241937+0000', 'last_active': '2024-09-10T20:34:43.886590+0000', 'last_peered': '2024-09-10T20:34:43.886590+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:34:43.886590+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33705984, 'num_objects': 8229, 'num_object_clones': 0, 'num_object_copies': 16458, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1600, 'num_objects_misplaced': 1485, 'num_objects_unfound': 0, 'num_objects_dirty': 8229, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10221, 'num_write_kb': 32484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 527, 'num_bytes_recovered': 2117632, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1485}, {'shards': '2', 'objects': 5144}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:34:48.519 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:34:48.520 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'12515", 'reported_seq': 12729, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:43.151624+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:34:43.151624+0000', 'last_peered': '2024-09-10T20:34:43.151624+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:34:43.151624+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "36'2500", 'ondisk_log_start': "36'2500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34058240, 'num_objects': 8315, 'num_object_clones': 0, 'num_object_copies': 16630, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8315, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9238, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 37, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4704}, {'shards': '0,2', 'objects': 1549}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:48.520 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:34:48.520 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'12305", 'reported_seq': 12502, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:43.141963+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:34:43.141963+0000', 'last_peered': '2024-09-10T20:34:43.141963+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:34:43.141963+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33910784, 'num_objects': 8279, 'num_object_clones': 0, 'num_object_copies': 16558, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3508, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8279, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8999, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4774}, {'shards': '0,2', 'objects': 1523}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:48.520 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:34:48.520 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'12610", 'reported_seq': 12802, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:43.112356+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:34:43.112356+0000', 'last_peered': '2024-09-10T20:34:43.112356+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:34:43.112356+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34308096, 'num_objects': 8376, 'num_object_clones': 0, 'num_object_copies': 16752, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3687, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8376, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9437, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 28, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4691}, {'shards': '0,2', 'objects': 1598}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:48.520 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:34:49.215 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:49.215 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33788/264410 objects degraded (12.779%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":400,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":132205,"data_bytes":541488596,"bytes_used":2191872000,"bytes_avail":287718420480,"bytes_total":289910292480,"degraded_objects":33788,"degraded_total":264410,"degraded_ratio":0.12778639234522143,"misplaced_objects":1485,"misplaced_total":264410,"misplaced_ratio":0.0056162777504632951,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":81884,"recovering_keys_per_sec":0,"num_objects_recovered":294,"num_bytes_recovered":983040,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":373},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:34:49.246 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:34:49.246 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:34:49.775 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:49.776 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:34:49.800 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":241,"stamp":"2024-09-10T20:34:48.945327+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":533177812,"num_objects":130176,"num_object_clones":0,"num_object_copies":260352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33770,"num_objects_misplaced":1357,"num_objects_unfound":0,"num_objects_dirty":130176,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":165245,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8024,"num_bytes_recovered":31608851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160850,"ondisk_log_size":160850,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2120508,"kb_used_data":1160588,"kb_used_omap":12,"kb_used_meta":959859,"kb_avail":280995012,"statfs":{"total":289910292480,"available":287738892288,"internally_reserved":0,"allocated":1188442112,"data_stored":1187623358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":982895618},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,4,10,5,2,0,0,0,0,0,0,5],"upper_bound":16384},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26652672,"num_objects":-6507,"num_object_clones":0,"num_object_copies":-13014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-70,"num_objects_misplaced":-368,"num_objects_unfound":0,"num_objects_dirty":-6507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6507,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":440,"num_bytes_recovered":1507328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005028"},"pg_stats":[{"pgid":"2.f","version":"47'12531","reported_seq":12696,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:48.207326+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:34:48.207326+0000","last_peered":"2024-09-10T20:34:48.207326+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:34:48.207326+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33288192,"num_objects":8127,"num_object_clones":0,"num_object_copies":16254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3567,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12531,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4560},{"shards":"0,2","objects":1405}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'12807","reported_seq":12922,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:48.867964+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:34:48.867964+0000","last_peered":"2024-09-10T20:34:48.867964+0000","last_clean":"2024-09-10T20:34:48.867964+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:34:48.867964+0000","last_undegraded":"2024-09-10T20:34:48.867964+0000","last_fullsized":"2024-09-10T20:34:48.867964+0000","mapping_epoch":35,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33320960,"num_objects":8135,"num_object_clones":0,"num_object_copies":16270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10993,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'12490","reported_seq":19843,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:44.432372+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:34:44.432372+0000","last_peered":"2024-09-10T20:34:44.432372+0000","last_clean":"2024-09-10T20:34:44.432372+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:34:44.432372+0000","last_undegraded":"2024-09-10T20:34:44.432372+0000","last_fullsized":"2024-09-10T20:34:44.432372+0000","mapping_epoch":44,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33689624,"num_objects":8226,"num_object_clones":0,"num_object_copies":16452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8226,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10618,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'12538","reported_seq":12623,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:48.807784+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:34:48.807784+0000","last_peered":"2024-09-10T20:34:48.807784+0000","last_clean":"2024-09-10T20:34:48.807784+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:34:48.807784+0000","last_undegraded":"2024-09-10T20:34:48.807784+0000","last_fullsized":"2024-09-10T20:34:48.807784+0000","mapping_epoch":35,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33038336,"num_objects":8066,"num_object_clones":0,"num_object_copies":16132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12538,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'12767","reported_seq":12898,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:48.140160+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:34:48.140160+0000","last_peered":"2024-09-10T20:34:48.140160+0000","last_clean":"2024-09-10T20:34:48.140160+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:34:48.140160+0000","last_undegraded":"2024-09-10T20:34:48.140160+0000","last_fullsized":"2024-09-10T20:34:48.140160+0000","mapping_epoch":34,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34004992,"num_objects":8302,"num_object_clones":0,"num_object_copies":16604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9763,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'12449","reported_seq":12652,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:48.147903+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:34:48.147903+0000","last_peered":"2024-09-10T20:34:48.147903+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:34:48.147903+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32858112,"num_objects":8022,"num_object_clones":0,"num_object_copies":16044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3585,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9313,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4439},{"shards":"0,2","objects":1403}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'12474","reported_seq":12712,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:48.162623+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:34:48.162623+0000","last_peered":"2024-09-10T20:34:48.162623+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:34:48.162623+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33316864,"num_objects":8134,"num_object_clones":0,"num_object_copies":16268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3509,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9069,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4627},{"shards":"0,2","objects":1388}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'12592","reported_seq":12721,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:48.127136+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:34:48.127136+0000","last_peered":"2024-09-10T20:34:48.127136+0000","last_clean":"2024-09-10T20:34:48.127136+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:34:48.127136+0000","last_undegraded":"2024-09-10T20:34:48.127136+0000","last_fullsized":"2024-09-10T20:34:48.127136+0000","mapping_epoch":34,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33357824,"num_objects":8144,"num_object_clones":0,"num_object_copies":16288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9663,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'12385","reported_seq":12558,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:48.815583+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:34:48.815583+0000","last_peered":"2024-09-10T20:34:48.815583+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:34:48.815583+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32989184,"num_objects":8054,"num_object_clones":0,"num_object_copies":16108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3510,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10378,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1363},{"shards":"2","objects":4546}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086228+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086228+0000","last_peered":"2024-09-10T20:34:27.086228+0000","last_clean":"2024-09-10T20:34:27.086228+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086228+0000","last_undegraded":"2024-09-10T20:34:27.086228+0000","last_fullsized":"2024-09-10T20:34:27.086228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'12613","reported_seq":12788,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:48.150487+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:34:48.150487+0000","last_peered":"2024-09-10T20:34:48.150487+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:34:48.150487+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33248198,"num_objects":8119,"num_object_clones":0,"num_object_copies":16238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3586,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12611,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4535},{"shards":"0,2","objects":1371}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.075947+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.075947+0000","last_peered":"2024-09-10T20:34:27.075947+0000","last_clean":"2024-09-10T20:34:27.075947+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.075947+0000","last_undegraded":"2024-09-10T20:34:27.075947+0000","last_fullsized":"2024-09-10T20:34:27.075947+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'12533","reported_seq":12706,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:48.799141+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:34:48.799141+0000","last_peered":"2024-09-10T20:34:48.799141+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:34:48.799141+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32899072,"num_objects":8032,"num_object_clones":0,"num_object_copies":16064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3615,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10361,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1385},{"shards":"2","objects":4419}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'12582","reported_seq":14026,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:48.893252+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:34:48.893252+0000","last_peered":"2024-09-10T20:34:48.893252+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:34:48.893252+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33103872,"num_objects":8082,"num_object_clones":0,"num_object_copies":16164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1600,"num_objects_misplaced":1357,"num_objects_unfound":0,"num_objects_dirty":8082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10368,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":655,"num_bytes_recovered":2641920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1357},{"shards":"2","objects":5125}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'12670","reported_seq":12886,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:48.208554+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:34:48.208554+0000","last_peered":"2024-09-10T20:34:48.208554+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:34:48.208554+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33423360,"num_objects":8160,"num_object_clones":0,"num_object_copies":16320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3611,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9393,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4550},{"shards":"0,2","objects":1394}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'12444","reported_seq":12651,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:48.116463+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:34:48.116463+0000","last_peered":"2024-09-10T20:34:48.116463+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:34:48.116463+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33341440,"num_objects":8140,"num_object_clones":0,"num_object_copies":16280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3503,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9138,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4640},{"shards":"0,2","objects":1384}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086748+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086748+0000","last_peered":"2024-09-10T20:34:27.086748+0000","last_clean":"2024-09-10T20:34:27.086748+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086748+0000","last_undegraded":"2024-09-10T20:34:27.086748+0000","last_fullsized":"2024-09-10T20:34:27.086748+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'12756","reported_seq":12954,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:48.083625+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:34:48.083625+0000","last_peered":"2024-09-10T20:34:48.083625+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:34:48.083625+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33710080,"num_objects":8230,"num_object_clones":0,"num_object_copies":16460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3684,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9583,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4548},{"shards":"0,2","objects":1452}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'12316","reported_seq":19438,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:44.308953+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:34:44.308953+0000","last_peered":"2024-09-10T20:34:44.308953+0000","last_clean":"2024-09-10T20:34:44.308953+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:34:44.308953+0000","last_undegraded":"2024-09-10T20:34:44.308953+0000","last_fullsized":"2024-09-10T20:34:44.308953+0000","mapping_epoch":46,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33587683,"num_objects":8201,"num_object_clones":0,"num_object_copies":16402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8923,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":533177793,"num_objects":130174,"num_object_clones":0,"num_object_copies":260348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33770,"num_objects_misplaced":1357,"num_objects_unfound":0,"num_objects_dirty":130174,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":165243,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8022,"num_bytes_recovered":31608832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1185230848,"data_stored":1185154057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160847,"ondisk_log_size":160847,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182475,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":806172,"kb_used_data":449676,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93565668,"statfs":{"total":96636764160,"available":95811244032,"internally_reserved":0,"allocated":460468224,"data_stored":460215931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,4,0,0,0,0,0,0,0,0,5],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986247,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":686444,"kb_used_data":375708,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93685396,"statfs":{"total":96636764160,"available":95933845504,"internally_reserved":0,"allocated":384724992,"data_stored":384472704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318172842},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659193,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627892,"kb_used_data":335204,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93743948,"statfs":{"total":96636764160,"available":95993802752,"internally_reserved":0,"allocated":343248896,"data_stored":342934723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":299691694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,5,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":458981376,"data_stored":458977763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":342654976,"data_stored":342589483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":383594496,"data_stored":383586811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:34:52.803 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:34:53.365 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:53.366 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:34:53.387 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2024-09-10T20:34:52.946798+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":532116948,"num_objects":129917,"num_object_clones":0,"num_object_copies":259834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33770,"num_objects_misplaced":1357,"num_objects_unfound":0,"num_objects_dirty":129917,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":165504,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8024,"num_bytes_recovered":31608851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160809,"ondisk_log_size":160809,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2129068,"kb_used_data":1159420,"kb_used_omap":12,"kb_used_meta":969587,"kb_avail":280986452,"statfs":{"total":289910292480,"available":287730126848,"internally_reserved":0,"allocated":1187246080,"data_stored":1186427326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":992857090},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,1,4,7,3,0,0,0,0,0,0,0,5],"upper_bound":16384},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18653184,"num_objects":-4554,"num_object_clones":0,"num_object_copies":-9108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-32,"num_objects_misplaced":-246,"num_objects_unfound":0,"num_objects_dirty":-4554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4554,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":1007616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004779"},"pg_stats":[{"pgid":"2.f","version":"47'12531","reported_seq":12696,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:48.207326+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:34:48.207326+0000","last_peered":"2024-09-10T20:34:48.207326+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:34:48.207326+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33288192,"num_objects":8127,"num_object_clones":0,"num_object_copies":16254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3567,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12531,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4560},{"shards":"0,2","objects":1405}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'12807","reported_seq":12922,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:48.867964+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:34:48.867964+0000","last_peered":"2024-09-10T20:34:48.867964+0000","last_clean":"2024-09-10T20:34:48.867964+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:34:48.867964+0000","last_undegraded":"2024-09-10T20:34:48.867964+0000","last_fullsized":"2024-09-10T20:34:48.867964+0000","mapping_epoch":35,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33320960,"num_objects":8135,"num_object_clones":0,"num_object_copies":16270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10993,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'12606","reported_seq":19959,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:49.460390+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:34:49.460390+0000","last_peered":"2024-09-10T20:34:49.460390+0000","last_clean":"2024-09-10T20:34:49.460390+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:34:49.460390+0000","last_undegraded":"2024-09-10T20:34:49.460390+0000","last_fullsized":"2024-09-10T20:34:49.460390+0000","mapping_epoch":44,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33214488,"num_objects":8110,"num_object_clones":0,"num_object_copies":16220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8110,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10734,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'12538","reported_seq":12623,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:48.807784+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:34:48.807784+0000","last_peered":"2024-09-10T20:34:48.807784+0000","last_clean":"2024-09-10T20:34:48.807784+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:34:48.807784+0000","last_undegraded":"2024-09-10T20:34:48.807784+0000","last_fullsized":"2024-09-10T20:34:48.807784+0000","mapping_epoch":35,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33038336,"num_objects":8066,"num_object_clones":0,"num_object_copies":16132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12538,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'12767","reported_seq":12898,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:48.140160+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:34:48.140160+0000","last_peered":"2024-09-10T20:34:48.140160+0000","last_clean":"2024-09-10T20:34:48.140160+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:34:48.140160+0000","last_undegraded":"2024-09-10T20:34:48.140160+0000","last_fullsized":"2024-09-10T20:34:48.140160+0000","mapping_epoch":34,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34004992,"num_objects":8302,"num_object_clones":0,"num_object_copies":16604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9763,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'12449","reported_seq":12652,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:48.147903+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:34:48.147903+0000","last_peered":"2024-09-10T20:34:48.147903+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:34:48.147903+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32858112,"num_objects":8022,"num_object_clones":0,"num_object_copies":16044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3585,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9313,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4439},{"shards":"0,2","objects":1403}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'12474","reported_seq":12712,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:48.162623+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:34:48.162623+0000","last_peered":"2024-09-10T20:34:48.162623+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:34:48.162623+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33316864,"num_objects":8134,"num_object_clones":0,"num_object_copies":16268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3509,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9069,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4627},{"shards":"0,2","objects":1388}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'12592","reported_seq":12721,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:48.127136+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:34:48.127136+0000","last_peered":"2024-09-10T20:34:48.127136+0000","last_clean":"2024-09-10T20:34:48.127136+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:34:48.127136+0000","last_undegraded":"2024-09-10T20:34:48.127136+0000","last_fullsized":"2024-09-10T20:34:48.127136+0000","mapping_epoch":34,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33357824,"num_objects":8144,"num_object_clones":0,"num_object_copies":16288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9663,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'12385","reported_seq":12558,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:48.815583+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:34:48.815583+0000","last_peered":"2024-09-10T20:34:48.815583+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:34:48.815583+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32989184,"num_objects":8054,"num_object_clones":0,"num_object_copies":16108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3510,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10378,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1363},{"shards":"2","objects":4546}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086228+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086228+0000","last_peered":"2024-09-10T20:34:27.086228+0000","last_clean":"2024-09-10T20:34:27.086228+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086228+0000","last_undegraded":"2024-09-10T20:34:27.086228+0000","last_fullsized":"2024-09-10T20:34:27.086228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'12613","reported_seq":12788,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:48.150487+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:34:48.150487+0000","last_peered":"2024-09-10T20:34:48.150487+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:34:48.150487+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33248198,"num_objects":8119,"num_object_clones":0,"num_object_copies":16238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3586,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12611,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4535},{"shards":"0,2","objects":1371}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.075947+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.075947+0000","last_peered":"2024-09-10T20:34:27.075947+0000","last_clean":"2024-09-10T20:34:27.075947+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.075947+0000","last_undegraded":"2024-09-10T20:34:27.075947+0000","last_fullsized":"2024-09-10T20:34:27.075947+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'12533","reported_seq":12706,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:48.799141+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:34:48.799141+0000","last_peered":"2024-09-10T20:34:48.799141+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:34:48.799141+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32899072,"num_objects":8032,"num_object_clones":0,"num_object_copies":16064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3615,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10361,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1385},{"shards":"2","objects":4419}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'12582","reported_seq":14026,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:48.893252+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:34:48.893252+0000","last_peered":"2024-09-10T20:34:48.893252+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:34:48.893252+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33103872,"num_objects":8082,"num_object_clones":0,"num_object_copies":16164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1600,"num_objects_misplaced":1357,"num_objects_unfound":0,"num_objects_dirty":8082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10368,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":655,"num_bytes_recovered":2641920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1357},{"shards":"2","objects":5125}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'12670","reported_seq":12886,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:48.208554+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:34:48.208554+0000","last_peered":"2024-09-10T20:34:48.208554+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:34:48.208554+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33423360,"num_objects":8160,"num_object_clones":0,"num_object_copies":16320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3611,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9393,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4550},{"shards":"0,2","objects":1394}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'12444","reported_seq":12651,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:48.116463+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:34:48.116463+0000","last_peered":"2024-09-10T20:34:48.116463+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:34:48.116463+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33341440,"num_objects":8140,"num_object_clones":0,"num_object_copies":16280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3503,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9138,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4640},{"shards":"0,2","objects":1384}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086748+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086748+0000","last_peered":"2024-09-10T20:34:27.086748+0000","last_clean":"2024-09-10T20:34:27.086748+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086748+0000","last_undegraded":"2024-09-10T20:34:27.086748+0000","last_fullsized":"2024-09-10T20:34:27.086748+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'12756","reported_seq":12954,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:48.083625+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:34:48.083625+0000","last_peered":"2024-09-10T20:34:48.083625+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:34:48.083625+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33710080,"num_objects":8230,"num_object_clones":0,"num_object_copies":16460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3684,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9583,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4548},{"shards":"0,2","objects":1452}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'12459","reported_seq":19581,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:49.381327+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:34:49.381327+0000","last_peered":"2024-09-10T20:34:49.381327+0000","last_clean":"2024-09-10T20:34:49.381327+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:34:49.381327+0000","last_undegraded":"2024-09-10T20:34:49.381327+0000","last_fullsized":"2024-09-10T20:34:49.381327+0000","mapping_epoch":46,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33001955,"num_objects":8058,"num_object_clones":0,"num_object_copies":16116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9066,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":532116929,"num_objects":129915,"num_object_clones":0,"num_object_copies":259830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33770,"num_objects_misplaced":1357,"num_objects_unfound":0,"num_objects_dirty":129915,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":165502,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8022,"num_bytes_recovered":31608832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1184112640,"data_stored":1184035849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160806,"ondisk_log_size":160806,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182475,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":806172,"kb_used_data":449676,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93565668,"statfs":{"total":96636764160,"available":95811244032,"internally_reserved":0,"allocated":460468224,"data_stored":460215931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,4,0,0,0,0,0,0,0,0,5],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986247,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":686444,"kb_used_data":375708,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93685396,"statfs":{"total":96636764160,"available":95933845504,"internally_reserved":0,"allocated":384724992,"data_stored":384472704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318172842},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659194,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":636452,"kb_used_data":334036,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93735388,"statfs":{"total":96636764160,"available":95985037312,"internally_reserved":0,"allocated":342052864,"data_stored":341738691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":309653166},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":458981376,"data_stored":458977763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":341536768,"data_stored":341471275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":383594496,"data_stored":383586811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:34:53.388 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:34:53.969 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:53.972 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:34:53.992 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2024-09-10T20:34:52.946798+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":532116948,"num_objects":129917,"num_object_clones":0,"num_object_copies":259834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33770,"num_objects_misplaced":1357,"num_objects_unfound":0,"num_objects_dirty":129917,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":165504,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8024,"num_bytes_recovered":31608851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160809,"ondisk_log_size":160809,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2129068,"kb_used_data":1159420,"kb_used_omap":12,"kb_used_meta":969587,"kb_avail":280986452,"statfs":{"total":289910292480,"available":287730126848,"internally_reserved":0,"allocated":1187246080,"data_stored":1186427326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":992857090},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,1,4,7,3,0,0,0,0,0,0,0,5],"upper_bound":16384},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18653184,"num_objects":-4554,"num_object_clones":0,"num_object_copies":-9108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-32,"num_objects_misplaced":-246,"num_objects_unfound":0,"num_objects_dirty":-4554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4554,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":1007616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004779"},"pg_stats":[{"pgid":"2.f","version":"47'12531","reported_seq":12696,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:48.207326+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:34:48.207326+0000","last_peered":"2024-09-10T20:34:48.207326+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:34:48.207326+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33288192,"num_objects":8127,"num_object_clones":0,"num_object_copies":16254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3567,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12531,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4560},{"shards":"0,2","objects":1405}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'12807","reported_seq":12922,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:48.867964+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:34:48.867964+0000","last_peered":"2024-09-10T20:34:48.867964+0000","last_clean":"2024-09-10T20:34:48.867964+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:34:48.867964+0000","last_undegraded":"2024-09-10T20:34:48.867964+0000","last_fullsized":"2024-09-10T20:34:48.867964+0000","mapping_epoch":35,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33320960,"num_objects":8135,"num_object_clones":0,"num_object_copies":16270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10993,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'12606","reported_seq":19959,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:49.460390+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:34:49.460390+0000","last_peered":"2024-09-10T20:34:49.460390+0000","last_clean":"2024-09-10T20:34:49.460390+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:34:49.460390+0000","last_undegraded":"2024-09-10T20:34:49.460390+0000","last_fullsized":"2024-09-10T20:34:49.460390+0000","mapping_epoch":44,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33214488,"num_objects":8110,"num_object_clones":0,"num_object_copies":16220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8110,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10734,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'12538","reported_seq":12623,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:48.807784+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:34:48.807784+0000","last_peered":"2024-09-10T20:34:48.807784+0000","last_clean":"2024-09-10T20:34:48.807784+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:34:48.807784+0000","last_undegraded":"2024-09-10T20:34:48.807784+0000","last_fullsized":"2024-09-10T20:34:48.807784+0000","mapping_epoch":35,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33038336,"num_objects":8066,"num_object_clones":0,"num_object_copies":16132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12538,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'12767","reported_seq":12898,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:48.140160+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:34:48.140160+0000","last_peered":"2024-09-10T20:34:48.140160+0000","last_clean":"2024-09-10T20:34:48.140160+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:34:48.140160+0000","last_undegraded":"2024-09-10T20:34:48.140160+0000","last_fullsized":"2024-09-10T20:34:48.140160+0000","mapping_epoch":34,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34004992,"num_objects":8302,"num_object_clones":0,"num_object_copies":16604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9763,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'12449","reported_seq":12652,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:48.147903+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:34:48.147903+0000","last_peered":"2024-09-10T20:34:48.147903+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:34:48.147903+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32858112,"num_objects":8022,"num_object_clones":0,"num_object_copies":16044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3585,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9313,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4439},{"shards":"0,2","objects":1403}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'12474","reported_seq":12712,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:48.162623+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:34:48.162623+0000","last_peered":"2024-09-10T20:34:48.162623+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:34:48.162623+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33316864,"num_objects":8134,"num_object_clones":0,"num_object_copies":16268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3509,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9069,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4627},{"shards":"0,2","objects":1388}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'12592","reported_seq":12721,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:48.127136+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:34:48.127136+0000","last_peered":"2024-09-10T20:34:48.127136+0000","last_clean":"2024-09-10T20:34:48.127136+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:34:48.127136+0000","last_undegraded":"2024-09-10T20:34:48.127136+0000","last_fullsized":"2024-09-10T20:34:48.127136+0000","mapping_epoch":34,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33357824,"num_objects":8144,"num_object_clones":0,"num_object_copies":16288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9663,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'12385","reported_seq":12558,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:48.815583+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:34:48.815583+0000","last_peered":"2024-09-10T20:34:48.815583+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:34:48.815583+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32989184,"num_objects":8054,"num_object_clones":0,"num_object_copies":16108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3510,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10378,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1363},{"shards":"2","objects":4546}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086228+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086228+0000","last_peered":"2024-09-10T20:34:27.086228+0000","last_clean":"2024-09-10T20:34:27.086228+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086228+0000","last_undegraded":"2024-09-10T20:34:27.086228+0000","last_fullsized":"2024-09-10T20:34:27.086228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'12613","reported_seq":12788,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:48.150487+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:34:48.150487+0000","last_peered":"2024-09-10T20:34:48.150487+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:34:48.150487+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33248198,"num_objects":8119,"num_object_clones":0,"num_object_copies":16238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3586,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12611,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4535},{"shards":"0,2","objects":1371}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.075947+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.075947+0000","last_peered":"2024-09-10T20:34:27.075947+0000","last_clean":"2024-09-10T20:34:27.075947+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.075947+0000","last_undegraded":"2024-09-10T20:34:27.075947+0000","last_fullsized":"2024-09-10T20:34:27.075947+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'12533","reported_seq":12706,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:48.799141+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:34:48.799141+0000","last_peered":"2024-09-10T20:34:48.799141+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:34:48.799141+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32899072,"num_objects":8032,"num_object_clones":0,"num_object_copies":16064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3615,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10361,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1385},{"shards":"2","objects":4419}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'12582","reported_seq":14026,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:48.893252+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:34:48.893252+0000","last_peered":"2024-09-10T20:34:48.893252+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:34:48.893252+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33103872,"num_objects":8082,"num_object_clones":0,"num_object_copies":16164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1600,"num_objects_misplaced":1357,"num_objects_unfound":0,"num_objects_dirty":8082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10368,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":655,"num_bytes_recovered":2641920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1357},{"shards":"2","objects":5125}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'12670","reported_seq":12886,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:48.208554+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:34:48.208554+0000","last_peered":"2024-09-10T20:34:48.208554+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:34:48.208554+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33423360,"num_objects":8160,"num_object_clones":0,"num_object_copies":16320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3611,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9393,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4550},{"shards":"0,2","objects":1394}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'12444","reported_seq":12651,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:48.116463+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:34:48.116463+0000","last_peered":"2024-09-10T20:34:48.116463+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:34:48.116463+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33341440,"num_objects":8140,"num_object_clones":0,"num_object_copies":16280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3503,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9138,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4640},{"shards":"0,2","objects":1384}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086748+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086748+0000","last_peered":"2024-09-10T20:34:27.086748+0000","last_clean":"2024-09-10T20:34:27.086748+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086748+0000","last_undegraded":"2024-09-10T20:34:27.086748+0000","last_fullsized":"2024-09-10T20:34:27.086748+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'12756","reported_seq":12954,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:48.083625+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:34:48.083625+0000","last_peered":"2024-09-10T20:34:48.083625+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:34:48.083625+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33710080,"num_objects":8230,"num_object_clones":0,"num_object_copies":16460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3684,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9583,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4548},{"shards":"0,2","objects":1452}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'12459","reported_seq":19581,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:49.381327+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:34:49.381327+0000","last_peered":"2024-09-10T20:34:49.381327+0000","last_clean":"2024-09-10T20:34:49.381327+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:34:49.381327+0000","last_undegraded":"2024-09-10T20:34:49.381327+0000","last_fullsized":"2024-09-10T20:34:49.381327+0000","mapping_epoch":46,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33001955,"num_objects":8058,"num_object_clones":0,"num_object_copies":16116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9066,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":532116929,"num_objects":129915,"num_object_clones":0,"num_object_copies":259830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33770,"num_objects_misplaced":1357,"num_objects_unfound":0,"num_objects_dirty":129915,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":165502,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8022,"num_bytes_recovered":31608832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1184112640,"data_stored":1184035849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160806,"ondisk_log_size":160806,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182475,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":806172,"kb_used_data":449676,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93565668,"statfs":{"total":96636764160,"available":95811244032,"internally_reserved":0,"allocated":460468224,"data_stored":460215931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,4,0,0,0,0,0,0,0,0,5],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986247,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":686444,"kb_used_data":375708,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93685396,"statfs":{"total":96636764160,"available":95933845504,"internally_reserved":0,"allocated":384724992,"data_stored":384472704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318172842},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659194,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":636452,"kb_used_data":334036,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93735388,"statfs":{"total":96636764160,"available":95985037312,"internally_reserved":0,"allocated":342052864,"data_stored":341738691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":309653166},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":458981376,"data_stored":458977763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":341536768,"data_stored":341471275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":383594496,"data_stored":383586811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:34:53.994 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:34:53.994 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'12531", 'reported_seq': 12696, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:48.207326+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:34:48.207326+0000', 'last_peered': '2024-09-10T20:34:48.207326+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:34:48.207326+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "36'2500", 'ondisk_log_start': "36'2500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33288192, 'num_objects': 8127, 'num_object_clones': 0, 'num_object_copies': 16254, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3567, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8127, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12531, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 41, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4560}, {'shards': '0,2', 'objects': 1405}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:53.994 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:34:53.994 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'12449", 'reported_seq': 12652, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:48.147903+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:34:48.147903+0000', 'last_peered': '2024-09-10T20:34:48.147903+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:34:48.147903+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32858112, 'num_objects': 8022, 'num_object_clones': 0, 'num_object_copies': 16044, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3585, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8022, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9313, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 31, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4439}, {'shards': '0,2', 'objects': 1403}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:53.994 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:34:53.994 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'12474", 'reported_seq': 12712, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:48.162623+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:34:48.162623+0000', 'last_peered': '2024-09-10T20:34:48.162623+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:34:48.162623+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33316864, 'num_objects': 8134, 'num_object_clones': 0, 'num_object_copies': 16268, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3509, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8134, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9069, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 48, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4627}, {'shards': '0,2', 'objects': 1388}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:53.994 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:34:53.994 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'12385", 'reported_seq': 12558, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:48.815583+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:34:48.815583+0000', 'last_peered': '2024-09-10T20:34:48.815583+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:34:48.815583+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32989184, 'num_objects': 8054, 'num_object_clones': 0, 'num_object_copies': 16108, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3510, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8054, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10378, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 18, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1363}, {'shards': '2', 'objects': 4546}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:34:53.995 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:34:53.995 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'12613", 'reported_seq': 12788, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:48.150487+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:34:48.150487+0000', 'last_peered': '2024-09-10T20:34:48.150487+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:34:48.150487+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33248198, 'num_objects': 8119, 'num_object_clones': 0, 'num_object_copies': 16238, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3586, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8119, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12611, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 33, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4535}, {'shards': '0,2', 'objects': 1371}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:53.995 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:34:53.995 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'12533", 'reported_seq': 12706, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:48.799141+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:34:48.799141+0000', 'last_peered': '2024-09-10T20:34:48.799141+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:34:48.799141+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "36'2500", 'ondisk_log_start': "36'2500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32899072, 'num_objects': 8032, 'num_object_clones': 0, 'num_object_copies': 16064, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3615, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8032, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10361, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 20, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1385}, {'shards': '2', 'objects': 4419}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:34:53.995 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:34:53.995 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'12582", 'reported_seq': 14026, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:48.893252+0000', 'last_change': '2024-09-10T20:34:25.241937+0000', 'last_active': '2024-09-10T20:34:48.893252+0000', 'last_peered': '2024-09-10T20:34:48.893252+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:34:48.893252+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "36'2500", 'ondisk_log_start': "36'2500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33103872, 'num_objects': 8082, 'num_object_clones': 0, 'num_object_copies': 16164, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1600, 'num_objects_misplaced': 1357, 'num_objects_unfound': 0, 'num_objects_dirty': 8082, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10368, 'num_write_kb': 32484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 655, 'num_bytes_recovered': 2641920, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1357}, {'shards': '2', 'objects': 5125}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:34:53.995 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:34:53.995 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'12670", 'reported_seq': 12886, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:48.208554+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:34:48.208554+0000', 'last_peered': '2024-09-10T20:34:48.208554+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:34:48.208554+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33423360, 'num_objects': 8160, 'num_object_clones': 0, 'num_object_copies': 16320, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3611, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8160, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9393, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 38, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4550}, {'shards': '0,2', 'objects': 1394}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:53.995 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:34:53.995 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'12444", 'reported_seq': 12651, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:48.116463+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:34:48.116463+0000', 'last_peered': '2024-09-10T20:34:48.116463+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:34:48.116463+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33341440, 'num_objects': 8140, 'num_object_clones': 0, 'num_object_copies': 16280, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3503, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8140, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9138, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 32, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4640}, {'shards': '0,2', 'objects': 1384}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:53.996 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:34:53.996 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'12756", 'reported_seq': 12954, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:48.083625+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:34:48.083625+0000', 'last_peered': '2024-09-10T20:34:48.083625+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:34:48.083625+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33710080, 'num_objects': 8230, 'num_object_clones': 0, 'num_object_copies': 16460, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3684, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8230, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9583, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 31, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4548}, {'shards': '0,2', 'objects': 1452}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:53.996 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:34:54.671 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:54.671 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33770/259834 objects degraded (12.997%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":406,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":129917,"data_bytes":532116948,"bytes_used":2180165632,"bytes_avail":287730126848,"bytes_total":289910292480,"degraded_objects":33770,"degraded_total":259834,"degraded_ratio":0.12996759469507455,"misplaced_objects":1357,"misplaced_total":259834,"misplaced_ratio":0.0052225651762279005,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":83934,"recovering_keys_per_sec":0,"num_objects_recovered":280,"num_bytes_recovered":1007616,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":379},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:34:54.697 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:34:54.697 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:34:55.247 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:55.247 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:34:55.272 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":244,"stamp":"2024-09-10T20:34:54.947496+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":522671572,"num_objects":127611,"num_object_clones":0,"num_object_copies":255222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33766,"num_objects_misplaced":1220,"num_objects_unfound":0,"num_objects_dirty":127611,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":167810,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8165,"num_bytes_recovered":32075795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160815,"ondisk_log_size":160815,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2112424,"kb_used_data":1146912,"kb_used_omap":12,"kb_used_meta":965427,"kb_avail":281003096,"statfs":{"total":289910292480,"available":287747170304,"internally_reserved":0,"allocated":1174437888,"data_stored":1173619134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":988597250},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,6,5,4,3,5,4,0,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28098560,"num_objects":-6860,"num_object_clones":0,"num_object_copies":-13720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-36,"num_objects_misplaced":-383,"num_objects_unfound":0,"num_objects_dirty":-6860,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6860,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":421,"num_bytes_recovered":1474560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004604"},"pg_stats":[{"pgid":"2.f","version":"47'12660","reported_seq":12831,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:53.184781+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:34:53.184781+0000","last_peered":"2024-09-10T20:34:53.184781+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:34:53.184781+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32759808,"num_objects":7998,"num_object_clones":0,"num_object_copies":15996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3564,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12660,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4434},{"shards":"0,2","objects":1276}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'12958","reported_seq":13073,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:53.871571+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:34:53.871571+0000","last_peered":"2024-09-10T20:34:53.871571+0000","last_clean":"2024-09-10T20:34:53.871571+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:34:53.871571+0000","last_undegraded":"2024-09-10T20:34:53.871571+0000","last_fullsized":"2024-09-10T20:34:53.871571+0000","mapping_epoch":35,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32702464,"num_objects":7984,"num_object_clones":0,"num_object_copies":15968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11144,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'12726","reported_seq":20078,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:54.423137+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:34:54.423137+0000","last_peered":"2024-09-10T20:34:54.423137+0000","last_clean":"2024-09-10T20:34:54.423137+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:34:54.423137+0000","last_undegraded":"2024-09-10T20:34:54.423137+0000","last_fullsized":"2024-09-10T20:34:54.423137+0000","mapping_epoch":44,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32722968,"num_objects":7990,"num_object_clones":0,"num_object_copies":15980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7990,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10854,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'12685","reported_seq":12770,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:53.862098+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:34:53.862098+0000","last_peered":"2024-09-10T20:34:53.862098+0000","last_clean":"2024-09-10T20:34:53.862098+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:34:53.862098+0000","last_undegraded":"2024-09-10T20:34:53.862098+0000","last_fullsized":"2024-09-10T20:34:53.862098+0000","mapping_epoch":35,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32436224,"num_objects":7919,"num_object_clones":0,"num_object_copies":15838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12685,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'12900","reported_seq":13031,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:53.142650+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:34:53.142650+0000","last_peered":"2024-09-10T20:34:53.142650+0000","last_clean":"2024-09-10T20:34:53.142650+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:34:53.142650+0000","last_undegraded":"2024-09-10T20:34:53.142650+0000","last_fullsized":"2024-09-10T20:34:53.142650+0000","mapping_epoch":34,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33460224,"num_objects":8169,"num_object_clones":0,"num_object_copies":16338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9896,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'12600","reported_seq":12819,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:53.222206+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:34:53.222206+0000","last_peered":"2024-09-10T20:34:53.222206+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:34:53.222206+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32239616,"num_objects":7871,"num_object_clones":0,"num_object_copies":15742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3577,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7871,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9464,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4296},{"shards":"0,2","objects":1252}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'12619","reported_seq":12862,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:53.199284+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:34:53.199284+0000","last_peered":"2024-09-10T20:34:53.199284+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:34:53.199284+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32722944,"num_objects":7989,"num_object_clones":0,"num_object_copies":15978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3507,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9214,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4484},{"shards":"0,2","objects":1243}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'12748","reported_seq":12876,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:53.112509+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:34:53.112509+0000","last_peered":"2024-09-10T20:34:53.112509+0000","last_clean":"2024-09-10T20:34:53.112509+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:34:53.112509+0000","last_undegraded":"2024-09-10T20:34:53.112509+0000","last_fullsized":"2024-09-10T20:34:53.112509+0000","mapping_epoch":34,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32718848,"num_objects":7988,"num_object_clones":0,"num_object_copies":15976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9819,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'12523","reported_seq":12700,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:53.668120+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:34:53.668120+0000","last_peered":"2024-09-10T20:34:53.668120+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:34:53.668120+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32423936,"num_objects":7916,"num_object_clones":0,"num_object_copies":15832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3508,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10516,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1225},{"shards":"2","objects":4410}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086228+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086228+0000","last_peered":"2024-09-10T20:34:27.086228+0000","last_clean":"2024-09-10T20:34:27.086228+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086228+0000","last_undegraded":"2024-09-10T20:34:27.086228+0000","last_fullsized":"2024-09-10T20:34:27.086228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'12770","reported_seq":12945,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:53.216112+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:34:53.216112+0000","last_peered":"2024-09-10T20:34:53.216112+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:34:53.216112+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32605126,"num_objects":7962,"num_object_clones":0,"num_object_copies":15924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3586,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12768,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4378},{"shards":"0,2","objects":1214}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.075947+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.075947+0000","last_peered":"2024-09-10T20:34:27.075947+0000","last_clean":"2024-09-10T20:34:27.075947+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.075947+0000","last_undegraded":"2024-09-10T20:34:27.075947+0000","last_fullsized":"2024-09-10T20:34:27.075947+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'12668","reported_seq":12841,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:53.878650+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:34:53.878650+0000","last_peered":"2024-09-10T20:34:53.878650+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:34:53.878650+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32346112,"num_objects":7897,"num_object_clones":0,"num_object_copies":15794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3615,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10496,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1250},{"shards":"2","objects":4284}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'12724","reported_seq":14396,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:53.900328+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:34:53.900328+0000","last_peered":"2024-09-10T20:34:53.900328+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:34:53.900328+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32522240,"num_objects":7940,"num_object_clones":0,"num_object_copies":15880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1623,"num_objects_misplaced":1220,"num_objects_unfound":0,"num_objects_dirty":7940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10510,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":769,"num_bytes_recovered":3108864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1220},{"shards":"2","objects":5097}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'12818","reported_seq":13038,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:53.173826+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:34:53.173826+0000","last_peered":"2024-09-10T20:34:53.173826+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:34:53.173826+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32817152,"num_objects":8012,"num_object_clones":0,"num_object_copies":16024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3609,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9541,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4404},{"shards":"0,2","objects":1246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'12593","reported_seq":12804,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:53.217043+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:34:53.217043+0000","last_peered":"2024-09-10T20:34:53.217043+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:34:53.217043+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32731136,"num_objects":7991,"num_object_clones":0,"num_object_copies":15982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3500,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9287,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4494},{"shards":"0,2","objects":1235}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086748+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086748+0000","last_peered":"2024-09-10T20:34:27.086748+0000","last_clean":"2024-09-10T20:34:27.086748+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086748+0000","last_undegraded":"2024-09-10T20:34:27.086748+0000","last_fullsized":"2024-09-10T20:34:27.086748+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'12916","reported_seq":13127,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:53.176242+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:34:53.176242+0000","last_peered":"2024-09-10T20:34:53.176242+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:34:53.176242+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33054720,"num_objects":8070,"num_object_clones":0,"num_object_copies":16140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3677,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8070,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9743,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4395},{"shards":"0,2","objects":1292}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'12604","reported_seq":19726,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:54.438323+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:34:54.438323+0000","last_peered":"2024-09-10T20:34:54.438323+0000","last_clean":"2024-09-10T20:34:54.438323+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:34:54.438323+0000","last_undegraded":"2024-09-10T20:34:54.438323+0000","last_fullsized":"2024-09-10T20:34:54.438323+0000","mapping_epoch":46,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32408035,"num_objects":7913,"num_object_clones":0,"num_object_copies":15826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9211,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":522671553,"num_objects":127609,"num_object_clones":0,"num_object_copies":255218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33766,"num_objects_misplaced":1220,"num_objects_unfound":0,"num_objects_dirty":127609,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":167808,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8163,"num_bytes_recovered":32075776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1171300352,"data_stored":1171223561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160812,"ondisk_log_size":160812,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182476,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":803400,"kb_used_data":442720,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93568440,"statfs":{"total":96636764160,"available":95814082560,"internally_reserved":0,"allocated":453345280,"data_stored":453092987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,5,1,1,2,4,0,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986248,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":669468,"kb_used_data":371212,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93702372,"statfs":{"total":96636764160,"available":95951228928,"internally_reserved":0,"allocated":380121088,"data_stored":379868800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305393322},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659195,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":639556,"kb_used_data":332980,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93732284,"statfs":{"total":96636764160,"available":95981858816,"internally_reserved":0,"allocated":340971520,"data_stored":340657347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":313913006},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":451813376,"data_stored":451809763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":340447232,"data_stored":340381739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":379039744,"data_stored":379032059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:34:58.276 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:34:58.829 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:58.832 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:34:58.854 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":245,"stamp":"2024-09-10T20:34:56.948377+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":522671572,"num_objects":127611,"num_object_clones":0,"num_object_copies":255222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33766,"num_objects_misplaced":1220,"num_objects_unfound":0,"num_objects_dirty":127611,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":167810,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8165,"num_bytes_recovered":32075795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160815,"ondisk_log_size":160815,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2112424,"kb_used_data":1146912,"kb_used_omap":12,"kb_used_meta":965427,"kb_avail":281003096,"statfs":{"total":289910292480,"available":287747170304,"internally_reserved":0,"allocated":1174437888,"data_stored":1173619134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":988597250},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,6,5,4,3,5,4,0,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18817024,"num_objects":-4594,"num_object_clones":0,"num_object_copies":-9188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-22,"num_objects_misplaced":-265,"num_objects_unfound":0,"num_objects_dirty":-4594,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4594,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":991232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004702"},"pg_stats":[{"pgid":"2.f","version":"47'12660","reported_seq":12831,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:53.184781+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:34:53.184781+0000","last_peered":"2024-09-10T20:34:53.184781+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:34:53.184781+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32759808,"num_objects":7998,"num_object_clones":0,"num_object_copies":15996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3564,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12660,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4434},{"shards":"0,2","objects":1276}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'12958","reported_seq":13073,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:53.871571+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:34:53.871571+0000","last_peered":"2024-09-10T20:34:53.871571+0000","last_clean":"2024-09-10T20:34:53.871571+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:34:53.871571+0000","last_undegraded":"2024-09-10T20:34:53.871571+0000","last_fullsized":"2024-09-10T20:34:53.871571+0000","mapping_epoch":35,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32702464,"num_objects":7984,"num_object_clones":0,"num_object_copies":15968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11144,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'12726","reported_seq":20078,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:54.423137+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:34:54.423137+0000","last_peered":"2024-09-10T20:34:54.423137+0000","last_clean":"2024-09-10T20:34:54.423137+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:34:54.423137+0000","last_undegraded":"2024-09-10T20:34:54.423137+0000","last_fullsized":"2024-09-10T20:34:54.423137+0000","mapping_epoch":44,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32722968,"num_objects":7990,"num_object_clones":0,"num_object_copies":15980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7990,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10854,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'12685","reported_seq":12770,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:53.862098+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:34:53.862098+0000","last_peered":"2024-09-10T20:34:53.862098+0000","last_clean":"2024-09-10T20:34:53.862098+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:34:53.862098+0000","last_undegraded":"2024-09-10T20:34:53.862098+0000","last_fullsized":"2024-09-10T20:34:53.862098+0000","mapping_epoch":35,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32436224,"num_objects":7919,"num_object_clones":0,"num_object_copies":15838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12685,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'12900","reported_seq":13031,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:53.142650+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:34:53.142650+0000","last_peered":"2024-09-10T20:34:53.142650+0000","last_clean":"2024-09-10T20:34:53.142650+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:34:53.142650+0000","last_undegraded":"2024-09-10T20:34:53.142650+0000","last_fullsized":"2024-09-10T20:34:53.142650+0000","mapping_epoch":34,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33460224,"num_objects":8169,"num_object_clones":0,"num_object_copies":16338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9896,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'12600","reported_seq":12819,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:53.222206+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:34:53.222206+0000","last_peered":"2024-09-10T20:34:53.222206+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:34:53.222206+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32239616,"num_objects":7871,"num_object_clones":0,"num_object_copies":15742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3577,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7871,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9464,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4296},{"shards":"0,2","objects":1252}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'12619","reported_seq":12862,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:53.199284+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:34:53.199284+0000","last_peered":"2024-09-10T20:34:53.199284+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:34:53.199284+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32722944,"num_objects":7989,"num_object_clones":0,"num_object_copies":15978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3507,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9214,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4484},{"shards":"0,2","objects":1243}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'12748","reported_seq":12876,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:53.112509+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:34:53.112509+0000","last_peered":"2024-09-10T20:34:53.112509+0000","last_clean":"2024-09-10T20:34:53.112509+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:34:53.112509+0000","last_undegraded":"2024-09-10T20:34:53.112509+0000","last_fullsized":"2024-09-10T20:34:53.112509+0000","mapping_epoch":34,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32718848,"num_objects":7988,"num_object_clones":0,"num_object_copies":15976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9819,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'12523","reported_seq":12700,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:53.668120+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:34:53.668120+0000","last_peered":"2024-09-10T20:34:53.668120+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:34:53.668120+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32423936,"num_objects":7916,"num_object_clones":0,"num_object_copies":15832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3508,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10516,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1225},{"shards":"2","objects":4410}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086228+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086228+0000","last_peered":"2024-09-10T20:34:27.086228+0000","last_clean":"2024-09-10T20:34:27.086228+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086228+0000","last_undegraded":"2024-09-10T20:34:27.086228+0000","last_fullsized":"2024-09-10T20:34:27.086228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'12770","reported_seq":12945,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:53.216112+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:34:53.216112+0000","last_peered":"2024-09-10T20:34:53.216112+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:34:53.216112+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32605126,"num_objects":7962,"num_object_clones":0,"num_object_copies":15924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3586,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12768,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4378},{"shards":"0,2","objects":1214}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.075947+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.075947+0000","last_peered":"2024-09-10T20:34:27.075947+0000","last_clean":"2024-09-10T20:34:27.075947+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.075947+0000","last_undegraded":"2024-09-10T20:34:27.075947+0000","last_fullsized":"2024-09-10T20:34:27.075947+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'12668","reported_seq":12841,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:53.878650+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:34:53.878650+0000","last_peered":"2024-09-10T20:34:53.878650+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:34:53.878650+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32346112,"num_objects":7897,"num_object_clones":0,"num_object_copies":15794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3615,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10496,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1250},{"shards":"2","objects":4284}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'12724","reported_seq":14396,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:53.900328+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:34:53.900328+0000","last_peered":"2024-09-10T20:34:53.900328+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:34:53.900328+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32522240,"num_objects":7940,"num_object_clones":0,"num_object_copies":15880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1623,"num_objects_misplaced":1220,"num_objects_unfound":0,"num_objects_dirty":7940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10510,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":769,"num_bytes_recovered":3108864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1220},{"shards":"2","objects":5097}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'12818","reported_seq":13038,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:53.173826+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:34:53.173826+0000","last_peered":"2024-09-10T20:34:53.173826+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:34:53.173826+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32817152,"num_objects":8012,"num_object_clones":0,"num_object_copies":16024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3609,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9541,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4404},{"shards":"0,2","objects":1246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'12593","reported_seq":12804,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:53.217043+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:34:53.217043+0000","last_peered":"2024-09-10T20:34:53.217043+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:34:53.217043+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32731136,"num_objects":7991,"num_object_clones":0,"num_object_copies":15982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3500,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9287,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4494},{"shards":"0,2","objects":1235}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086748+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086748+0000","last_peered":"2024-09-10T20:34:27.086748+0000","last_clean":"2024-09-10T20:34:27.086748+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086748+0000","last_undegraded":"2024-09-10T20:34:27.086748+0000","last_fullsized":"2024-09-10T20:34:27.086748+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'12916","reported_seq":13127,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:53.176242+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:34:53.176242+0000","last_peered":"2024-09-10T20:34:53.176242+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:34:53.176242+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33054720,"num_objects":8070,"num_object_clones":0,"num_object_copies":16140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3677,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8070,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9743,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4395},{"shards":"0,2","objects":1292}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'12604","reported_seq":19726,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:54.438323+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:34:54.438323+0000","last_peered":"2024-09-10T20:34:54.438323+0000","last_clean":"2024-09-10T20:34:54.438323+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:34:54.438323+0000","last_undegraded":"2024-09-10T20:34:54.438323+0000","last_fullsized":"2024-09-10T20:34:54.438323+0000","mapping_epoch":46,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32408035,"num_objects":7913,"num_object_clones":0,"num_object_copies":15826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9211,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":522671553,"num_objects":127609,"num_object_clones":0,"num_object_copies":255218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33766,"num_objects_misplaced":1220,"num_objects_unfound":0,"num_objects_dirty":127609,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":167808,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8163,"num_bytes_recovered":32075776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1171300352,"data_stored":1171223561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160812,"ondisk_log_size":160812,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182476,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":803400,"kb_used_data":442720,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93568440,"statfs":{"total":96636764160,"available":95814082560,"internally_reserved":0,"allocated":453345280,"data_stored":453092987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,5,1,1,2,4,0,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986248,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":669468,"kb_used_data":371212,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93702372,"statfs":{"total":96636764160,"available":95951228928,"internally_reserved":0,"allocated":380121088,"data_stored":379868800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305393322},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659195,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":639556,"kb_used_data":332980,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93732284,"statfs":{"total":96636764160,"available":95981858816,"internally_reserved":0,"allocated":340971520,"data_stored":340657347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":313913006},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":451813376,"data_stored":451809763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":340447232,"data_stored":340381739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":379039744,"data_stored":379032059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:34:58.855 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:34:59.396 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:34:59.396 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:34:59.417 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":246,"stamp":"2024-09-10T20:34:58.949110+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":514614740,"num_objects":125644,"num_object_clones":0,"num_object_copies":251288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33740,"num_objects_misplaced":1074,"num_objects_unfound":0,"num_objects_dirty":125644,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":169777,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8337,"num_bytes_recovered":32673811,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160682,"ondisk_log_size":160682,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2097344,"kb_used_data":1136016,"kb_used_omap":12,"kb_used_meta":961267,"kb_avail":281018176,"statfs":{"total":289910292480,"available":287762612224,"internally_reserved":0,"allocated":1163280384,"data_stored":1162461630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":984337410},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,6,5,2,5,4,5,0,5,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26873856,"num_objects":-6561,"num_object_clones":0,"num_object_copies":-13122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-48,"num_objects_misplaced":-411,"num_objects_unfound":0,"num_objects_dirty":-6561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6561,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":459,"num_bytes_recovered":1589248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004505"},"pg_stats":[{"pgid":"2.f","version":"47'12813","reported_seq":12996,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:58.182008+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:34:58.182008+0000","last_peered":"2024-09-10T20:34:58.182008+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:34:58.182008+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32133120,"num_objects":7845,"num_object_clones":0,"num_object_copies":15690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3558,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12813,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4287},{"shards":"0,2","objects":1123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13090","reported_seq":13205,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:58.885877+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:34:58.885877+0000","last_peered":"2024-09-10T20:34:58.885877+0000","last_clean":"2024-09-10T20:34:58.885877+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:34:58.885877+0000","last_undegraded":"2024-09-10T20:34:58.885877+0000","last_fullsized":"2024-09-10T20:34:58.885877+0000","mapping_epoch":35,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32161792,"num_objects":7852,"num_object_clones":0,"num_object_copies":15704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11276,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'12726","reported_seq":20078,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:54.423137+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:34:54.423137+0000","last_peered":"2024-09-10T20:34:54.423137+0000","last_clean":"2024-09-10T20:34:54.423137+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:34:54.423137+0000","last_undegraded":"2024-09-10T20:34:54.423137+0000","last_fullsized":"2024-09-10T20:34:54.423137+0000","mapping_epoch":44,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32722968,"num_objects":7990,"num_object_clones":0,"num_object_copies":15980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7990,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10854,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'12810","reported_seq":12895,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:58.855967+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:34:58.855967+0000","last_peered":"2024-09-10T20:34:58.855967+0000","last_clean":"2024-09-10T20:34:58.855967+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:34:58.855967+0000","last_undegraded":"2024-09-10T20:34:58.855967+0000","last_fullsized":"2024-09-10T20:34:58.855967+0000","mapping_epoch":35,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31924224,"num_objects":7794,"num_object_clones":0,"num_object_copies":15588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7794,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12810,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13025","reported_seq":13155,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:58.191921+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:34:58.191921+0000","last_peered":"2024-09-10T20:34:58.191921+0000","last_clean":"2024-09-10T20:34:58.191921+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:34:58.191921+0000","last_undegraded":"2024-09-10T20:34:58.191921+0000","last_fullsized":"2024-09-10T20:34:58.191921+0000","mapping_epoch":34,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32948224,"num_objects":8044,"num_object_clones":0,"num_object_copies":16088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10021,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'12735","reported_seq":12960,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:58.067019+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:34:58.067019+0000","last_peered":"2024-09-10T20:34:58.067019+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:34:58.067019+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31686656,"num_objects":7736,"num_object_clones":0,"num_object_copies":15472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3574,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9599,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4164},{"shards":"0,2","objects":1117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'12755","reported_seq":13008,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:58.208675+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:34:58.208675+0000","last_peered":"2024-09-10T20:34:58.208675+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:34:58.208675+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32165888,"num_objects":7853,"num_object_clones":0,"num_object_copies":15706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3502,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9350,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4353},{"shards":"0,2","objects":1107}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'12900","reported_seq":13029,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:58.221788+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:34:58.221788+0000","last_peered":"2024-09-10T20:34:58.221788+0000","last_clean":"2024-09-10T20:34:58.221788+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:34:58.221788+0000","last_undegraded":"2024-09-10T20:34:58.221788+0000","last_fullsized":"2024-09-10T20:34:58.221788+0000","mapping_epoch":34,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32096256,"num_objects":7836,"num_object_clones":0,"num_object_copies":15672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7836,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9971,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'12655","reported_seq":12834,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:58.801854+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:34:58.801854+0000","last_peered":"2024-09-10T20:34:58.801854+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:34:58.801854+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31883264,"num_objects":7784,"num_object_clones":0,"num_object_copies":15568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3507,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10648,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1093},{"shards":"2","objects":4279}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086228+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086228+0000","last_peered":"2024-09-10T20:34:27.086228+0000","last_clean":"2024-09-10T20:34:27.086228+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086228+0000","last_undegraded":"2024-09-10T20:34:27.086228+0000","last_fullsized":"2024-09-10T20:34:27.086228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'12920","reported_seq":13099,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:58.242179+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:34:58.242179+0000","last_peered":"2024-09-10T20:34:58.242179+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:34:58.242179+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31990726,"num_objects":7812,"num_object_clones":0,"num_object_copies":15624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3583,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7812,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12918,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4231},{"shards":"0,2","objects":1064}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.075947+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.075947+0000","last_peered":"2024-09-10T20:34:27.075947+0000","last_clean":"2024-09-10T20:34:27.075947+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.075947+0000","last_undegraded":"2024-09-10T20:34:27.075947+0000","last_fullsized":"2024-09-10T20:34:27.075947+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'12812","reported_seq":12987,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:58.837920+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:34:58.837920+0000","last_peered":"2024-09-10T20:34:58.837920+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:34:58.837920+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31756288,"num_objects":7753,"num_object_clones":0,"num_object_copies":15506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3614,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10640,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1106},{"shards":"2","objects":4141}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'12851","reported_seq":14813,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:58.878388+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:34:58.878388+0000","last_peered":"2024-09-10T20:34:58.878388+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:34:58.878388+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32002048,"num_objects":7813,"num_object_clones":0,"num_object_copies":15626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1623,"num_objects_misplaced":1074,"num_objects_unfound":0,"num_objects_dirty":7813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10637,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":915,"num_bytes_recovered":3706880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1074},{"shards":"2","objects":5116}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'12965","reported_seq":13191,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:58.198525+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:34:58.198525+0000","last_peered":"2024-09-10T20:34:58.198525+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:34:58.198525+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32215040,"num_objects":7865,"num_object_clones":0,"num_object_copies":15730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3606,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9688,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4260},{"shards":"0,2","objects":1099}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'12740","reported_seq":12955,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:58.228182+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:34:58.228182+0000","last_peered":"2024-09-10T20:34:58.228182+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:34:58.228182+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32129024,"num_objects":7844,"num_object_clones":0,"num_object_copies":15688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9434,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4348},{"shards":"0,2","objects":1088}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086748+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086748+0000","last_peered":"2024-09-10T20:34:27.086748+0000","last_clean":"2024-09-10T20:34:27.086748+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086748+0000","last_undegraded":"2024-09-10T20:34:27.086748+0000","last_fullsized":"2024-09-10T20:34:27.086748+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13078","reported_seq":13296,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:58.217268+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:34:58.217268+0000","last_peered":"2024-09-10T20:34:58.217268+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:34:58.217268+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32391168,"num_objects":7908,"num_object_clones":0,"num_object_copies":15816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3674,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9905,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4236},{"shards":"0,2","objects":1130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'12604","reported_seq":19726,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:54.438323+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:34:54.438323+0000","last_peered":"2024-09-10T20:34:54.438323+0000","last_clean":"2024-09-10T20:34:54.438323+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:34:54.438323+0000","last_undegraded":"2024-09-10T20:34:54.438323+0000","last_fullsized":"2024-09-10T20:34:54.438323+0000","mapping_epoch":46,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32408035,"num_objects":7913,"num_object_clones":0,"num_object_copies":15826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9211,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":514614721,"num_objects":125642,"num_object_clones":0,"num_object_copies":251284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33740,"num_objects_misplaced":1074,"num_objects_unfound":0,"num_objects_dirty":125642,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":169775,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8335,"num_bytes_recovered":32673792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1159806976,"data_stored":1159730185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160679,"ondisk_log_size":160679,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182477,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":788332,"kb_used_data":435996,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93583508,"statfs":{"total":96636764160,"available":95829512192,"internally_reserved":0,"allocated":446459904,"data_stored":446207611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":360771242},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,5,1,1,2,4,0,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986249,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":669456,"kb_used_data":367040,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93702384,"statfs":{"total":96636764160,"available":95951241216,"internally_reserved":0,"allocated":375848960,"data_stored":375596672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309653162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,3,2,1,0,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659195,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":639556,"kb_used_data":332980,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93732284,"statfs":{"total":96636764160,"available":95981858816,"internally_reserved":0,"allocated":340971520,"data_stored":340657347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":313913006},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":444817408,"data_stored":444813795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":340447232,"data_stored":340381739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":374542336,"data_stored":374534651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:34:59.418 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:34:59.418 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'12813", 'reported_seq': 12996, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:58.182008+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:34:58.182008+0000', 'last_peered': '2024-09-10T20:34:58.182008+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:34:58.182008+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "36'2800", 'ondisk_log_start': "36'2800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32133120, 'num_objects': 7845, 'num_object_clones': 0, 'num_object_copies': 15690, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3558, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7845, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12813, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 50, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4287}, {'shards': '0,2', 'objects': 1123}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:59.418 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:34:59.418 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'12735", 'reported_seq': 12960, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:58.067019+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:34:58.067019+0000', 'last_peered': '2024-09-10T20:34:58.067019+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:34:58.067019+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31686656, 'num_objects': 7736, 'num_object_clones': 0, 'num_object_copies': 15472, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3574, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7736, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9599, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 42, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4164}, {'shards': '0,2', 'objects': 1117}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:59.419 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:34:59.419 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'12755", 'reported_seq': 13008, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:58.208675+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:34:58.208675+0000', 'last_peered': '2024-09-10T20:34:58.208675+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:34:58.208675+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32165888, 'num_objects': 7853, 'num_object_clones': 0, 'num_object_copies': 15706, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3502, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7853, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9350, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 55, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4353}, {'shards': '0,2', 'objects': 1107}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:59.419 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:34:59.419 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'12655", 'reported_seq': 12834, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:58.801854+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:34:58.801854+0000', 'last_peered': '2024-09-10T20:34:58.801854+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:34:58.801854+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31883264, 'num_objects': 7784, 'num_object_clones': 0, 'num_object_copies': 15568, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3507, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7784, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10648, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 21, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1093}, {'shards': '2', 'objects': 4279}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:34:59.419 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:34:59.419 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'12920", 'reported_seq': 13099, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:58.242179+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:34:58.242179+0000', 'last_peered': '2024-09-10T20:34:58.242179+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:34:58.242179+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31990726, 'num_objects': 7812, 'num_object_clones': 0, 'num_object_copies': 15624, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3583, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7812, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12918, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 36, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4231}, {'shards': '0,2', 'objects': 1064}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:59.419 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:34:59.419 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'12812", 'reported_seq': 12987, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:58.837920+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:34:58.837920+0000', 'last_peered': '2024-09-10T20:34:58.837920+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:34:58.837920+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "36'2800", 'ondisk_log_start': "36'2800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31756288, 'num_objects': 7753, 'num_object_clones': 0, 'num_object_copies': 15506, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3614, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7753, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10640, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 21, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1106}, {'shards': '2', 'objects': 4141}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:34:59.419 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:34:59.419 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'12851", 'reported_seq': 14813, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:58.878388+0000', 'last_change': '2024-09-10T20:34:25.241937+0000', 'last_active': '2024-09-10T20:34:58.878388+0000', 'last_peered': '2024-09-10T20:34:58.878388+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:34:58.878388+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "36'2800", 'ondisk_log_start': "36'2800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32002048, 'num_objects': 7813, 'num_object_clones': 0, 'num_object_copies': 15626, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1623, 'num_objects_misplaced': 1074, 'num_objects_unfound': 0, 'num_objects_dirty': 7813, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10637, 'num_write_kb': 32484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 915, 'num_bytes_recovered': 3706880, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1074}, {'shards': '2', 'objects': 5116}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:34:59.420 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:34:59.420 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'12965", 'reported_seq': 13191, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:58.198525+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:34:58.198525+0000', 'last_peered': '2024-09-10T20:34:58.198525+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:34:58.198525+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32215040, 'num_objects': 7865, 'num_object_clones': 0, 'num_object_copies': 15730, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3606, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7865, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9688, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4260}, {'shards': '0,2', 'objects': 1099}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:59.420 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:34:59.420 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'12740", 'reported_seq': 12955, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:58.228182+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:34:58.228182+0000', 'last_peered': '2024-09-10T20:34:58.228182+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:34:58.228182+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32129024, 'num_objects': 7844, 'num_object_clones': 0, 'num_object_copies': 15688, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7844, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9434, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 36, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4348}, {'shards': '0,2', 'objects': 1088}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:59.420 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:34:59.420 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'13078", 'reported_seq': 13296, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:58.217268+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:34:58.217268+0000', 'last_peered': '2024-09-10T20:34:58.217268+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:34:58.217268+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32391168, 'num_objects': 7908, 'num_object_clones': 0, 'num_object_copies': 15816, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3674, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7908, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9905, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 41, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4236}, {'shards': '0,2', 'objects': 1130}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:34:59.420 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:35:00.132 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:00.132 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33740/251288 objects degraded (13.427%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":411,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":125644,"data_bytes":514614740,"bytes_used":2147680256,"bytes_avail":287762612224,"bytes_total":289910292480,"degraded_objects":33740,"degraded_total":251288,"degraded_ratio":0.13426824997612302,"misplaced_objects":1074,"misplaced_total":251288,"misplaced_ratio":0.0042739804527076502,"recovering_objects_per_sec":38,"recovering_bytes_per_sec":132387,"recovering_keys_per_sec":0,"num_objects_recovered":459,"num_bytes_recovered":1589248,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":546},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:35:00.164 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:35:00.164 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:35:00.725 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:00.726 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:35:00.766 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":246,"stamp":"2024-09-10T20:34:58.949110+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":514614740,"num_objects":125644,"num_object_clones":0,"num_object_copies":251288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33740,"num_objects_misplaced":1074,"num_objects_unfound":0,"num_objects_dirty":125644,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":169777,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8337,"num_bytes_recovered":32673811,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160682,"ondisk_log_size":160682,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2097344,"kb_used_data":1136016,"kb_used_omap":12,"kb_used_meta":961267,"kb_avail":281018176,"statfs":{"total":289910292480,"available":287762612224,"internally_reserved":0,"allocated":1163280384,"data_stored":1162461630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":984337410},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,6,5,2,5,4,5,0,5,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26873856,"num_objects":-6561,"num_object_clones":0,"num_object_copies":-13122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-48,"num_objects_misplaced":-411,"num_objects_unfound":0,"num_objects_dirty":-6561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6561,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":459,"num_bytes_recovered":1589248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004505"},"pg_stats":[{"pgid":"2.f","version":"47'12813","reported_seq":12996,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:58.182008+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:34:58.182008+0000","last_peered":"2024-09-10T20:34:58.182008+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:34:58.182008+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32133120,"num_objects":7845,"num_object_clones":0,"num_object_copies":15690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3558,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12813,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4287},{"shards":"0,2","objects":1123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13090","reported_seq":13205,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:58.885877+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:34:58.885877+0000","last_peered":"2024-09-10T20:34:58.885877+0000","last_clean":"2024-09-10T20:34:58.885877+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:34:58.885877+0000","last_undegraded":"2024-09-10T20:34:58.885877+0000","last_fullsized":"2024-09-10T20:34:58.885877+0000","mapping_epoch":35,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32161792,"num_objects":7852,"num_object_clones":0,"num_object_copies":15704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11276,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'12726","reported_seq":20078,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:54.423137+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:34:54.423137+0000","last_peered":"2024-09-10T20:34:54.423137+0000","last_clean":"2024-09-10T20:34:54.423137+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:34:54.423137+0000","last_undegraded":"2024-09-10T20:34:54.423137+0000","last_fullsized":"2024-09-10T20:34:54.423137+0000","mapping_epoch":44,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32722968,"num_objects":7990,"num_object_clones":0,"num_object_copies":15980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7990,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10854,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'12810","reported_seq":12895,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:58.855967+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:34:58.855967+0000","last_peered":"2024-09-10T20:34:58.855967+0000","last_clean":"2024-09-10T20:34:58.855967+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:34:58.855967+0000","last_undegraded":"2024-09-10T20:34:58.855967+0000","last_fullsized":"2024-09-10T20:34:58.855967+0000","mapping_epoch":35,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31924224,"num_objects":7794,"num_object_clones":0,"num_object_copies":15588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7794,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12810,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13025","reported_seq":13155,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:58.191921+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:34:58.191921+0000","last_peered":"2024-09-10T20:34:58.191921+0000","last_clean":"2024-09-10T20:34:58.191921+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:34:58.191921+0000","last_undegraded":"2024-09-10T20:34:58.191921+0000","last_fullsized":"2024-09-10T20:34:58.191921+0000","mapping_epoch":34,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32948224,"num_objects":8044,"num_object_clones":0,"num_object_copies":16088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10021,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'12735","reported_seq":12960,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:58.067019+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:34:58.067019+0000","last_peered":"2024-09-10T20:34:58.067019+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:34:58.067019+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31686656,"num_objects":7736,"num_object_clones":0,"num_object_copies":15472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3574,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9599,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4164},{"shards":"0,2","objects":1117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'12755","reported_seq":13008,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:58.208675+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:34:58.208675+0000","last_peered":"2024-09-10T20:34:58.208675+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:34:58.208675+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32165888,"num_objects":7853,"num_object_clones":0,"num_object_copies":15706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3502,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9350,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4353},{"shards":"0,2","objects":1107}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'12900","reported_seq":13029,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:58.221788+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:34:58.221788+0000","last_peered":"2024-09-10T20:34:58.221788+0000","last_clean":"2024-09-10T20:34:58.221788+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:34:58.221788+0000","last_undegraded":"2024-09-10T20:34:58.221788+0000","last_fullsized":"2024-09-10T20:34:58.221788+0000","mapping_epoch":34,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32096256,"num_objects":7836,"num_object_clones":0,"num_object_copies":15672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7836,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9971,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'12655","reported_seq":12834,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:58.801854+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:34:58.801854+0000","last_peered":"2024-09-10T20:34:58.801854+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:34:58.801854+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31883264,"num_objects":7784,"num_object_clones":0,"num_object_copies":15568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3507,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10648,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1093},{"shards":"2","objects":4279}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086228+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086228+0000","last_peered":"2024-09-10T20:34:27.086228+0000","last_clean":"2024-09-10T20:34:27.086228+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086228+0000","last_undegraded":"2024-09-10T20:34:27.086228+0000","last_fullsized":"2024-09-10T20:34:27.086228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'12920","reported_seq":13099,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:58.242179+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:34:58.242179+0000","last_peered":"2024-09-10T20:34:58.242179+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:34:58.242179+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31990726,"num_objects":7812,"num_object_clones":0,"num_object_copies":15624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3583,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7812,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12918,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4231},{"shards":"0,2","objects":1064}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.075947+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.075947+0000","last_peered":"2024-09-10T20:34:27.075947+0000","last_clean":"2024-09-10T20:34:27.075947+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.075947+0000","last_undegraded":"2024-09-10T20:34:27.075947+0000","last_fullsized":"2024-09-10T20:34:27.075947+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'12812","reported_seq":12987,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:58.837920+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:34:58.837920+0000","last_peered":"2024-09-10T20:34:58.837920+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:34:58.837920+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31756288,"num_objects":7753,"num_object_clones":0,"num_object_copies":15506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3614,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10640,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1106},{"shards":"2","objects":4141}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'12851","reported_seq":14813,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:58.878388+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:34:58.878388+0000","last_peered":"2024-09-10T20:34:58.878388+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:34:58.878388+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32002048,"num_objects":7813,"num_object_clones":0,"num_object_copies":15626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1623,"num_objects_misplaced":1074,"num_objects_unfound":0,"num_objects_dirty":7813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10637,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":915,"num_bytes_recovered":3706880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1074},{"shards":"2","objects":5116}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'12965","reported_seq":13191,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:58.198525+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:34:58.198525+0000","last_peered":"2024-09-10T20:34:58.198525+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:34:58.198525+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32215040,"num_objects":7865,"num_object_clones":0,"num_object_copies":15730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3606,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9688,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4260},{"shards":"0,2","objects":1099}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'12740","reported_seq":12955,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:58.228182+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:34:58.228182+0000","last_peered":"2024-09-10T20:34:58.228182+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:34:58.228182+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32129024,"num_objects":7844,"num_object_clones":0,"num_object_copies":15688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9434,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4348},{"shards":"0,2","objects":1088}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086748+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086748+0000","last_peered":"2024-09-10T20:34:27.086748+0000","last_clean":"2024-09-10T20:34:27.086748+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086748+0000","last_undegraded":"2024-09-10T20:34:27.086748+0000","last_fullsized":"2024-09-10T20:34:27.086748+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13078","reported_seq":13296,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:58.217268+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:34:58.217268+0000","last_peered":"2024-09-10T20:34:58.217268+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:34:58.217268+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32391168,"num_objects":7908,"num_object_clones":0,"num_object_copies":15816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3674,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9905,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4236},{"shards":"0,2","objects":1130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'12604","reported_seq":19726,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:54.438323+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:34:54.438323+0000","last_peered":"2024-09-10T20:34:54.438323+0000","last_clean":"2024-09-10T20:34:54.438323+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:34:54.438323+0000","last_undegraded":"2024-09-10T20:34:54.438323+0000","last_fullsized":"2024-09-10T20:34:54.438323+0000","mapping_epoch":46,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32408035,"num_objects":7913,"num_object_clones":0,"num_object_copies":15826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9211,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":514614721,"num_objects":125642,"num_object_clones":0,"num_object_copies":251284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33740,"num_objects_misplaced":1074,"num_objects_unfound":0,"num_objects_dirty":125642,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":169775,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8335,"num_bytes_recovered":32673792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1159806976,"data_stored":1159730185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160679,"ondisk_log_size":160679,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182477,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":788332,"kb_used_data":435996,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93583508,"statfs":{"total":96636764160,"available":95829512192,"internally_reserved":0,"allocated":446459904,"data_stored":446207611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":360771242},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,5,1,1,2,4,0,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986249,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":669456,"kb_used_data":367040,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93702384,"statfs":{"total":96636764160,"available":95951241216,"internally_reserved":0,"allocated":375848960,"data_stored":375596672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309653162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,3,2,1,0,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659195,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":639556,"kb_used_data":332980,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93732284,"statfs":{"total":96636764160,"available":95981858816,"internally_reserved":0,"allocated":340971520,"data_stored":340657347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":313913006},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":444817408,"data_stored":444813795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":340447232,"data_stored":340381739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":374542336,"data_stored":374534651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:35:03.770 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:35:04.298 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:04.299 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:35:04.319 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2024-09-10T20:35:02.950715+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":513435092,"num_objects":125356,"num_object_clones":0,"num_object_copies":250712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33740,"num_objects_misplaced":1074,"num_objects_unfound":0,"num_objects_dirty":125356,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":170065,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8337,"num_bytes_recovered":32673811,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160770,"ondisk_log_size":160770,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2104676,"kb_used_data":1135028,"kb_used_omap":12,"kb_used_meta":969587,"kb_avail":281010844,"statfs":{"total":289910292480,"available":287755104256,"internally_reserved":0,"allocated":1162268672,"data_stored":1161449918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":992857090},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3,5,7,7,6,7,0,5,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18681856,"num_objects":-4561,"num_object_clones":0,"num_object_copies":-9122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-30,"num_objects_misplaced":-283,"num_objects_unfound":0,"num_objects_dirty":-4561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4561,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":1064960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004677"},"pg_stats":[{"pgid":"2.f","version":"47'12813","reported_seq":12996,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:58.182008+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:34:58.182008+0000","last_peered":"2024-09-10T20:34:58.182008+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:34:58.182008+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32133120,"num_objects":7845,"num_object_clones":0,"num_object_copies":15690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3558,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12813,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4287},{"shards":"0,2","objects":1123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13090","reported_seq":13205,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:58.885877+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:34:58.885877+0000","last_peered":"2024-09-10T20:34:58.885877+0000","last_clean":"2024-09-10T20:34:58.885877+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:34:58.885877+0000","last_undegraded":"2024-09-10T20:34:58.885877+0000","last_fullsized":"2024-09-10T20:34:58.885877+0000","mapping_epoch":35,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32161792,"num_objects":7852,"num_object_clones":0,"num_object_copies":15704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11276,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'12884","reported_seq":20237,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:59.437442+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:34:59.437442+0000","last_peered":"2024-09-10T20:34:59.437442+0000","last_clean":"2024-09-10T20:34:59.437442+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:34:59.437442+0000","last_undegraded":"2024-09-10T20:34:59.437442+0000","last_fullsized":"2024-09-10T20:34:59.437442+0000","mapping_epoch":44,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32075800,"num_objects":7832,"num_object_clones":0,"num_object_copies":15664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7832,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11012,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'12810","reported_seq":12895,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:58.855967+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:34:58.855967+0000","last_peered":"2024-09-10T20:34:58.855967+0000","last_clean":"2024-09-10T20:34:58.855967+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:34:58.855967+0000","last_undegraded":"2024-09-10T20:34:58.855967+0000","last_fullsized":"2024-09-10T20:34:58.855967+0000","mapping_epoch":35,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31924224,"num_objects":7794,"num_object_clones":0,"num_object_copies":15588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7794,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12810,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13025","reported_seq":13155,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:58.191921+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:34:58.191921+0000","last_peered":"2024-09-10T20:34:58.191921+0000","last_clean":"2024-09-10T20:34:58.191921+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:34:58.191921+0000","last_undegraded":"2024-09-10T20:34:58.191921+0000","last_fullsized":"2024-09-10T20:34:58.191921+0000","mapping_epoch":34,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32948224,"num_objects":8044,"num_object_clones":0,"num_object_copies":16088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10021,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'12735","reported_seq":12960,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:58.067019+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:34:58.067019+0000","last_peered":"2024-09-10T20:34:58.067019+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:34:58.067019+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31686656,"num_objects":7736,"num_object_clones":0,"num_object_copies":15472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3574,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9599,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4164},{"shards":"0,2","objects":1117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'12755","reported_seq":13008,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:58.208675+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:34:58.208675+0000","last_peered":"2024-09-10T20:34:58.208675+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:34:58.208675+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32165888,"num_objects":7853,"num_object_clones":0,"num_object_copies":15706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3502,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9350,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4353},{"shards":"0,2","objects":1107}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'12900","reported_seq":13029,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:58.221788+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:34:58.221788+0000","last_peered":"2024-09-10T20:34:58.221788+0000","last_clean":"2024-09-10T20:34:58.221788+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:34:58.221788+0000","last_undegraded":"2024-09-10T20:34:58.221788+0000","last_fullsized":"2024-09-10T20:34:58.221788+0000","mapping_epoch":34,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32096256,"num_objects":7836,"num_object_clones":0,"num_object_copies":15672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7836,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9971,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'12655","reported_seq":12834,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:58.801854+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:34:58.801854+0000","last_peered":"2024-09-10T20:34:58.801854+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:34:58.801854+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31883264,"num_objects":7784,"num_object_clones":0,"num_object_copies":15568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3507,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10648,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1093},{"shards":"2","objects":4279}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086228+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086228+0000","last_peered":"2024-09-10T20:34:27.086228+0000","last_clean":"2024-09-10T20:34:27.086228+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086228+0000","last_undegraded":"2024-09-10T20:34:27.086228+0000","last_fullsized":"2024-09-10T20:34:27.086228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'12920","reported_seq":13099,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:58.242179+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:34:58.242179+0000","last_peered":"2024-09-10T20:34:58.242179+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:34:58.242179+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31990726,"num_objects":7812,"num_object_clones":0,"num_object_copies":15624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3583,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7812,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12918,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4231},{"shards":"0,2","objects":1064}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.075947+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.075947+0000","last_peered":"2024-09-10T20:34:27.075947+0000","last_clean":"2024-09-10T20:34:27.075947+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.075947+0000","last_undegraded":"2024-09-10T20:34:27.075947+0000","last_fullsized":"2024-09-10T20:34:27.075947+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'12812","reported_seq":12987,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:58.837920+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:34:58.837920+0000","last_peered":"2024-09-10T20:34:58.837920+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:34:58.837920+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31756288,"num_objects":7753,"num_object_clones":0,"num_object_copies":15506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3614,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10640,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1106},{"shards":"2","objects":4141}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'12851","reported_seq":14813,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:58.878388+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:34:58.878388+0000","last_peered":"2024-09-10T20:34:58.878388+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:34:58.878388+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32002048,"num_objects":7813,"num_object_clones":0,"num_object_copies":15626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1623,"num_objects_misplaced":1074,"num_objects_unfound":0,"num_objects_dirty":7813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10637,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":915,"num_bytes_recovered":3706880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1074},{"shards":"2","objects":5116}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'12965","reported_seq":13191,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:58.198525+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:34:58.198525+0000","last_peered":"2024-09-10T20:34:58.198525+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:34:58.198525+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32215040,"num_objects":7865,"num_object_clones":0,"num_object_copies":15730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3606,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9688,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4260},{"shards":"0,2","objects":1099}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'12740","reported_seq":12955,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:58.228182+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:34:58.228182+0000","last_peered":"2024-09-10T20:34:58.228182+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:34:58.228182+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32129024,"num_objects":7844,"num_object_clones":0,"num_object_copies":15688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9434,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4348},{"shards":"0,2","objects":1088}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086748+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086748+0000","last_peered":"2024-09-10T20:34:27.086748+0000","last_clean":"2024-09-10T20:34:27.086748+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086748+0000","last_undegraded":"2024-09-10T20:34:27.086748+0000","last_fullsized":"2024-09-10T20:34:27.086748+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13078","reported_seq":13296,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:58.217268+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:34:58.217268+0000","last_peered":"2024-09-10T20:34:58.217268+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:34:58.217268+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32391168,"num_objects":7908,"num_object_clones":0,"num_object_copies":15816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3674,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9905,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4236},{"shards":"0,2","objects":1130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'12734","reported_seq":19856,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:59.384301+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:34:59.384301+0000","last_peered":"2024-09-10T20:34:59.384301+0000","last_clean":"2024-09-10T20:34:59.384301+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:34:59.384301+0000","last_undegraded":"2024-09-10T20:34:59.384301+0000","last_fullsized":"2024-09-10T20:34:59.384301+0000","mapping_epoch":46,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31875555,"num_objects":7783,"num_object_clones":0,"num_object_copies":15566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7783,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9341,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":513435073,"num_objects":125354,"num_object_clones":0,"num_object_copies":250708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33740,"num_objects_misplaced":1074,"num_objects_unfound":0,"num_objects_dirty":125354,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":170063,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8335,"num_bytes_recovered":32673792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1158660096,"data_stored":1158583305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160767,"ondisk_log_size":160767,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182477,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":788332,"kb_used_data":435996,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93583508,"statfs":{"total":96636764160,"available":95829512192,"internally_reserved":0,"allocated":446459904,"data_stored":446207611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":360771242},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,5,1,1,2,4,0,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986249,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":669456,"kb_used_data":367040,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93702384,"statfs":{"total":96636764160,"available":95951241216,"internally_reserved":0,"allocated":375848960,"data_stored":375596672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309653162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,3,2,1,0,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659196,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646888,"kb_used_data":331992,"kb_used_omap":4,"kb_used_meta":314875,"kb_avail":93724952,"statfs":{"total":96636764160,"available":95974350848,"internally_reserved":0,"allocated":339959808,"data_stored":339645635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":322432686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,3,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":444817408,"data_stored":444813795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":339300352,"data_stored":339234859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":374542336,"data_stored":374534651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:35:04.321 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:35:04.848 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:04.848 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:35:04.880 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2024-09-10T20:35:02.950715+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":513435092,"num_objects":125356,"num_object_clones":0,"num_object_copies":250712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33740,"num_objects_misplaced":1074,"num_objects_unfound":0,"num_objects_dirty":125356,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":170065,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8337,"num_bytes_recovered":32673811,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160770,"ondisk_log_size":160770,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2104676,"kb_used_data":1135028,"kb_used_omap":12,"kb_used_meta":969587,"kb_avail":281010844,"statfs":{"total":289910292480,"available":287755104256,"internally_reserved":0,"allocated":1162268672,"data_stored":1161449918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":992857090},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3,5,7,7,6,7,0,5,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18681856,"num_objects":-4561,"num_object_clones":0,"num_object_copies":-9122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-30,"num_objects_misplaced":-283,"num_objects_unfound":0,"num_objects_dirty":-4561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4561,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":1064960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004677"},"pg_stats":[{"pgid":"2.f","version":"47'12813","reported_seq":12996,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:58.182008+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:34:58.182008+0000","last_peered":"2024-09-10T20:34:58.182008+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:34:58.182008+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32133120,"num_objects":7845,"num_object_clones":0,"num_object_copies":15690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3558,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12813,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4287},{"shards":"0,2","objects":1123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13090","reported_seq":13205,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:58.885877+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:34:58.885877+0000","last_peered":"2024-09-10T20:34:58.885877+0000","last_clean":"2024-09-10T20:34:58.885877+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:34:58.885877+0000","last_undegraded":"2024-09-10T20:34:58.885877+0000","last_fullsized":"2024-09-10T20:34:58.885877+0000","mapping_epoch":35,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32161792,"num_objects":7852,"num_object_clones":0,"num_object_copies":15704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11276,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'12884","reported_seq":20237,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:59.437442+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:34:59.437442+0000","last_peered":"2024-09-10T20:34:59.437442+0000","last_clean":"2024-09-10T20:34:59.437442+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:34:59.437442+0000","last_undegraded":"2024-09-10T20:34:59.437442+0000","last_fullsized":"2024-09-10T20:34:59.437442+0000","mapping_epoch":44,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32075800,"num_objects":7832,"num_object_clones":0,"num_object_copies":15664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7832,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11012,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'12810","reported_seq":12895,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:58.855967+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:34:58.855967+0000","last_peered":"2024-09-10T20:34:58.855967+0000","last_clean":"2024-09-10T20:34:58.855967+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:34:58.855967+0000","last_undegraded":"2024-09-10T20:34:58.855967+0000","last_fullsized":"2024-09-10T20:34:58.855967+0000","mapping_epoch":35,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31924224,"num_objects":7794,"num_object_clones":0,"num_object_copies":15588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7794,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12810,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13025","reported_seq":13155,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:58.191921+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:34:58.191921+0000","last_peered":"2024-09-10T20:34:58.191921+0000","last_clean":"2024-09-10T20:34:58.191921+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:34:58.191921+0000","last_undegraded":"2024-09-10T20:34:58.191921+0000","last_fullsized":"2024-09-10T20:34:58.191921+0000","mapping_epoch":34,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32948224,"num_objects":8044,"num_object_clones":0,"num_object_copies":16088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10021,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'12735","reported_seq":12960,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:58.067019+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:34:58.067019+0000","last_peered":"2024-09-10T20:34:58.067019+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:34:58.067019+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31686656,"num_objects":7736,"num_object_clones":0,"num_object_copies":15472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3574,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9599,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4164},{"shards":"0,2","objects":1117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'12755","reported_seq":13008,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:58.208675+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:34:58.208675+0000","last_peered":"2024-09-10T20:34:58.208675+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:34:58.208675+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32165888,"num_objects":7853,"num_object_clones":0,"num_object_copies":15706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3502,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9350,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4353},{"shards":"0,2","objects":1107}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'12900","reported_seq":13029,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:58.221788+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:34:58.221788+0000","last_peered":"2024-09-10T20:34:58.221788+0000","last_clean":"2024-09-10T20:34:58.221788+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:34:58.221788+0000","last_undegraded":"2024-09-10T20:34:58.221788+0000","last_fullsized":"2024-09-10T20:34:58.221788+0000","mapping_epoch":34,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32096256,"num_objects":7836,"num_object_clones":0,"num_object_copies":15672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7836,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9971,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'12655","reported_seq":12834,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:58.801854+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:34:58.801854+0000","last_peered":"2024-09-10T20:34:58.801854+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:34:58.801854+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31883264,"num_objects":7784,"num_object_clones":0,"num_object_copies":15568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3507,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10648,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1093},{"shards":"2","objects":4279}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086228+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086228+0000","last_peered":"2024-09-10T20:34:27.086228+0000","last_clean":"2024-09-10T20:34:27.086228+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086228+0000","last_undegraded":"2024-09-10T20:34:27.086228+0000","last_fullsized":"2024-09-10T20:34:27.086228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'12920","reported_seq":13099,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:58.242179+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:34:58.242179+0000","last_peered":"2024-09-10T20:34:58.242179+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:34:58.242179+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31990726,"num_objects":7812,"num_object_clones":0,"num_object_copies":15624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3583,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7812,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12918,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4231},{"shards":"0,2","objects":1064}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.075947+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.075947+0000","last_peered":"2024-09-10T20:34:27.075947+0000","last_clean":"2024-09-10T20:34:27.075947+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.075947+0000","last_undegraded":"2024-09-10T20:34:27.075947+0000","last_fullsized":"2024-09-10T20:34:27.075947+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'12812","reported_seq":12987,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:58.837920+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:34:58.837920+0000","last_peered":"2024-09-10T20:34:58.837920+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:34:58.837920+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31756288,"num_objects":7753,"num_object_clones":0,"num_object_copies":15506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3614,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10640,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1106},{"shards":"2","objects":4141}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'12851","reported_seq":14813,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:58.878388+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:34:58.878388+0000","last_peered":"2024-09-10T20:34:58.878388+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:34:58.878388+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32002048,"num_objects":7813,"num_object_clones":0,"num_object_copies":15626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1623,"num_objects_misplaced":1074,"num_objects_unfound":0,"num_objects_dirty":7813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10637,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":915,"num_bytes_recovered":3706880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1074},{"shards":"2","objects":5116}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'12965","reported_seq":13191,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:58.198525+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:34:58.198525+0000","last_peered":"2024-09-10T20:34:58.198525+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:34:58.198525+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32215040,"num_objects":7865,"num_object_clones":0,"num_object_copies":15730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3606,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9688,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4260},{"shards":"0,2","objects":1099}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'12740","reported_seq":12955,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:58.228182+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:34:58.228182+0000","last_peered":"2024-09-10T20:34:58.228182+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:34:58.228182+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32129024,"num_objects":7844,"num_object_clones":0,"num_object_copies":15688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9434,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4348},{"shards":"0,2","objects":1088}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086748+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086748+0000","last_peered":"2024-09-10T20:34:27.086748+0000","last_clean":"2024-09-10T20:34:27.086748+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086748+0000","last_undegraded":"2024-09-10T20:34:27.086748+0000","last_fullsized":"2024-09-10T20:34:27.086748+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13078","reported_seq":13296,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:34:58.217268+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:34:58.217268+0000","last_peered":"2024-09-10T20:34:58.217268+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:34:58.217268+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32391168,"num_objects":7908,"num_object_clones":0,"num_object_copies":15816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3674,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9905,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4236},{"shards":"0,2","objects":1130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'12734","reported_seq":19856,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:59.384301+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:34:59.384301+0000","last_peered":"2024-09-10T20:34:59.384301+0000","last_clean":"2024-09-10T20:34:59.384301+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:34:59.384301+0000","last_undegraded":"2024-09-10T20:34:59.384301+0000","last_fullsized":"2024-09-10T20:34:59.384301+0000","mapping_epoch":46,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31875555,"num_objects":7783,"num_object_clones":0,"num_object_copies":15566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7783,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9341,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":513435073,"num_objects":125354,"num_object_clones":0,"num_object_copies":250708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33740,"num_objects_misplaced":1074,"num_objects_unfound":0,"num_objects_dirty":125354,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":170063,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8335,"num_bytes_recovered":32673792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1158660096,"data_stored":1158583305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160767,"ondisk_log_size":160767,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182477,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":788332,"kb_used_data":435996,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93583508,"statfs":{"total":96636764160,"available":95829512192,"internally_reserved":0,"allocated":446459904,"data_stored":446207611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":360771242},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,5,1,1,2,4,0,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986249,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":669456,"kb_used_data":367040,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93702384,"statfs":{"total":96636764160,"available":95951241216,"internally_reserved":0,"allocated":375848960,"data_stored":375596672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309653162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,3,2,1,0,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659196,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646888,"kb_used_data":331992,"kb_used_omap":4,"kb_used_meta":314875,"kb_avail":93724952,"statfs":{"total":96636764160,"available":95974350848,"internally_reserved":0,"allocated":339959808,"data_stored":339645635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":322432686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,3,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":444817408,"data_stored":444813795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":339300352,"data_stored":339234859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":374542336,"data_stored":374534651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:35:04.881 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:35:04.881 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'12813", 'reported_seq': 12996, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:58.182008+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:34:58.182008+0000', 'last_peered': '2024-09-10T20:34:58.182008+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:34:58.182008+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "36'2800", 'ondisk_log_start': "36'2800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32133120, 'num_objects': 7845, 'num_object_clones': 0, 'num_object_copies': 15690, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3558, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7845, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12813, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 50, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4287}, {'shards': '0,2', 'objects': 1123}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:04.881 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:35:04.881 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'12735", 'reported_seq': 12960, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:58.067019+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:34:58.067019+0000', 'last_peered': '2024-09-10T20:34:58.067019+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:34:58.067019+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31686656, 'num_objects': 7736, 'num_object_clones': 0, 'num_object_copies': 15472, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3574, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7736, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9599, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 42, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4164}, {'shards': '0,2', 'objects': 1117}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:04.881 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:35:04.881 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'12755", 'reported_seq': 13008, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:58.208675+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:34:58.208675+0000', 'last_peered': '2024-09-10T20:34:58.208675+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:34:58.208675+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32165888, 'num_objects': 7853, 'num_object_clones': 0, 'num_object_copies': 15706, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3502, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7853, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9350, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 55, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4353}, {'shards': '0,2', 'objects': 1107}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:04.882 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:35:04.882 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'12655", 'reported_seq': 12834, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:58.801854+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:34:58.801854+0000', 'last_peered': '2024-09-10T20:34:58.801854+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:34:58.801854+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31883264, 'num_objects': 7784, 'num_object_clones': 0, 'num_object_copies': 15568, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3507, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7784, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10648, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 21, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1093}, {'shards': '2', 'objects': 4279}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:35:04.882 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:35:04.882 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'12920", 'reported_seq': 13099, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:58.242179+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:34:58.242179+0000', 'last_peered': '2024-09-10T20:34:58.242179+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:34:58.242179+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31990726, 'num_objects': 7812, 'num_object_clones': 0, 'num_object_copies': 15624, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3583, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7812, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12918, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 36, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4231}, {'shards': '0,2', 'objects': 1064}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:04.882 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:35:04.882 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'12812", 'reported_seq': 12987, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:58.837920+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:34:58.837920+0000', 'last_peered': '2024-09-10T20:34:58.837920+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:34:58.837920+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "36'2800", 'ondisk_log_start': "36'2800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31756288, 'num_objects': 7753, 'num_object_clones': 0, 'num_object_copies': 15506, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3614, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7753, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10640, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 21, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1106}, {'shards': '2', 'objects': 4141}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:35:04.882 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:35:04.882 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'12851", 'reported_seq': 14813, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:58.878388+0000', 'last_change': '2024-09-10T20:34:25.241937+0000', 'last_active': '2024-09-10T20:34:58.878388+0000', 'last_peered': '2024-09-10T20:34:58.878388+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:34:58.878388+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "36'2800", 'ondisk_log_start': "36'2800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32002048, 'num_objects': 7813, 'num_object_clones': 0, 'num_object_copies': 15626, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1623, 'num_objects_misplaced': 1074, 'num_objects_unfound': 0, 'num_objects_dirty': 7813, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10637, 'num_write_kb': 32484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 915, 'num_bytes_recovered': 3706880, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1074}, {'shards': '2', 'objects': 5116}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:35:04.882 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:35:04.882 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'12965", 'reported_seq': 13191, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:58.198525+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:34:58.198525+0000', 'last_peered': '2024-09-10T20:34:58.198525+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:34:58.198525+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32215040, 'num_objects': 7865, 'num_object_clones': 0, 'num_object_copies': 15730, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3606, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7865, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9688, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4260}, {'shards': '0,2', 'objects': 1099}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:04.883 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:35:04.883 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'12740", 'reported_seq': 12955, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:58.228182+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:34:58.228182+0000', 'last_peered': '2024-09-10T20:34:58.228182+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:34:58.228182+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32129024, 'num_objects': 7844, 'num_object_clones': 0, 'num_object_copies': 15688, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7844, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9434, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 36, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4348}, {'shards': '0,2', 'objects': 1088}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:04.883 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:35:04.883 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'13078", 'reported_seq': 13296, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:34:58.217268+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:34:58.217268+0000', 'last_peered': '2024-09-10T20:34:58.217268+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:34:58.217268+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32391168, 'num_objects': 7908, 'num_object_clones': 0, 'num_object_copies': 15816, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3674, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7908, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9905, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 41, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4236}, {'shards': '0,2', 'objects': 1130}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:04.883 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:35:05.558 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:05.558 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33733/246190 objects degraded (13.702%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":417,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":123095,"data_bytes":504174036,"bytes_used":2137726976,"bytes_avail":287772565504,"bytes_total":289910292480,"degraded_objects":33733,"degraded_total":246190,"degraded_ratio":0.13702018765993745,"misplaced_objects":938,"misplaced_total":246190,"misplaced_ratio":0.0038100653966448676,"recovering_objects_per_sec":37,"recovering_bytes_per_sec":133412,"recovering_keys_per_sec":0,"num_objects_recovered":456,"num_bytes_recovered":1601536,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":568},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:35:05.582 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:35:05.582 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:35:06.115 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:06.122 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:35:06.142 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":249,"stamp":"2024-09-10T20:35:04.951159+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":504174036,"num_objects":123095,"num_object_clones":0,"num_object_copies":246190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33733,"num_objects_misplaced":938,"num_objects_unfound":0,"num_objects_dirty":123095,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":172326,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8480,"num_bytes_recovered":33210387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160831,"ondisk_log_size":160831,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2087624,"kb_used_data":1122132,"kb_used_omap":12,"kb_used_meta":965427,"kb_avail":281027896,"statfs":{"total":289910292480,"available":287772565504,"internally_reserved":0,"allocated":1149063168,"data_stored":1148244414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":988597250},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,5,5,12,0,0,0,4,0,0,0,1],"upper_bound":16384},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27942912,"num_objects":-6822,"num_object_clones":0,"num_object_copies":-13644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-37,"num_objects_misplaced":-419,"num_objects_unfound":0,"num_objects_dirty":-6822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6822,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":456,"num_bytes_recovered":1601536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004361"},"pg_stats":[{"pgid":"2.f","version":"47'12949","reported_seq":13138,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:03.243783+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:35:03.243783+0000","last_peered":"2024-09-10T20:35:03.243783+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:35:03.243783+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31576064,"num_objects":7709,"num_object_clones":0,"num_object_copies":15418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3555,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12949,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4154},{"shards":"0,2","objects":987}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13233","reported_seq":13348,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:03.886290+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:35:03.886290+0000","last_peered":"2024-09-10T20:35:03.886290+0000","last_clean":"2024-09-10T20:35:03.886290+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:35:03.886290+0000","last_undegraded":"2024-09-10T20:35:03.886290+0000","last_fullsized":"2024-09-10T20:35:03.886290+0000","mapping_epoch":35,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31576064,"num_objects":7709,"num_object_clones":0,"num_object_copies":15418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11419,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13040","reported_seq":20393,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:04.523913+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:35:04.523913+0000","last_peered":"2024-09-10T20:35:04.523913+0000","last_clean":"2024-09-10T20:35:04.523913+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:35:04.523913+0000","last_undegraded":"2024-09-10T20:35:04.523913+0000","last_fullsized":"2024-09-10T20:35:04.523913+0000","mapping_epoch":44,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31436824,"num_objects":7676,"num_object_clones":0,"num_object_copies":15352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7676,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11168,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'12946","reported_seq":13031,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:03.853323+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:35:03.853323+0000","last_peered":"2024-09-10T20:35:03.853323+0000","last_clean":"2024-09-10T20:35:03.853323+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:35:03.853323+0000","last_undegraded":"2024-09-10T20:35:03.853323+0000","last_fullsized":"2024-09-10T20:35:03.853323+0000","mapping_epoch":35,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31367168,"num_objects":7658,"num_object_clones":0,"num_object_copies":15316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12946,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13177","reported_seq":13308,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:03.212721+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:35:03.212721+0000","last_peered":"2024-09-10T20:35:03.212721+0000","last_clean":"2024-09-10T20:35:03.212721+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:35:03.212721+0000","last_undegraded":"2024-09-10T20:35:03.212721+0000","last_fullsized":"2024-09-10T20:35:03.212721+0000","mapping_epoch":34,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32325632,"num_objects":7892,"num_object_clones":0,"num_object_copies":15784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10173,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'12867","reported_seq":13094,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:03.234029+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:35:03.234029+0000","last_peered":"2024-09-10T20:35:03.234029+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:35:03.234029+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31145984,"num_objects":7604,"num_object_clones":0,"num_object_copies":15208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3573,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9731,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4033},{"shards":"0,2","objects":985}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'12897","reported_seq":13149,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:03.219765+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:35:03.219765+0000","last_peered":"2024-09-10T20:35:03.219765+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:35:03.219765+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31584256,"num_objects":7711,"num_object_clones":0,"num_object_copies":15422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3501,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9492,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4212},{"shards":"0,2","objects":965}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13048","reported_seq":13177,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:03.206472+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:35:03.206472+0000","last_peered":"2024-09-10T20:35:03.206472+0000","last_clean":"2024-09-10T20:35:03.206472+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:35:03.206472+0000","last_undegraded":"2024-09-10T20:35:03.206472+0000","last_fullsized":"2024-09-10T20:35:03.206472+0000","mapping_epoch":34,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31490048,"num_objects":7688,"num_object_clones":0,"num_object_copies":15376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10119,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'12805","reported_seq":12988,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:03.871438+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:35:03.871438+0000","last_peered":"2024-09-10T20:35:03.871438+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:35:03.871438+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31268864,"num_objects":7634,"num_object_clones":0,"num_object_copies":15268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3505,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10798,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":943},{"shards":"2","objects":4131}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086228+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086228+0000","last_peered":"2024-09-10T20:34:27.086228+0000","last_clean":"2024-09-10T20:34:27.086228+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086228+0000","last_undegraded":"2024-09-10T20:34:27.086228+0000","last_fullsized":"2024-09-10T20:34:27.086228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13043","reported_seq":13224,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:03.158183+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:35:03.158183+0000","last_peered":"2024-09-10T20:35:03.158183+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:35:03.158183+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31486918,"num_objects":7689,"num_object_clones":0,"num_object_copies":15378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3583,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13041,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4108},{"shards":"0,2","objects":941}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.075947+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.075947+0000","last_peered":"2024-09-10T20:34:27.075947+0000","last_clean":"2024-09-10T20:34:27.075947+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.075947+0000","last_undegraded":"2024-09-10T20:34:27.075947+0000","last_fullsized":"2024-09-10T20:34:27.075947+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'12934","reported_seq":13117,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:03.809684+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:35:03.809684+0000","last_peered":"2024-09-10T20:35:03.809684+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:35:03.809684+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31256576,"num_objects":7631,"num_object_clones":0,"num_object_copies":15262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3610,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10762,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":984},{"shards":"2","objects":4023}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'12999","reported_seq":15225,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:03.904847+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:35:03.904847+0000","last_peered":"2024-09-10T20:35:03.904847+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:35:03.904847+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31395840,"num_objects":7665,"num_object_clones":0,"num_object_copies":15330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1628,"num_objects_misplaced":938,"num_objects_unfound":0,"num_objects_dirty":7665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10785,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1046,"num_bytes_recovered":4243456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":938},{"shards":"2","objects":5099}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13120","reported_seq":13346,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:03.216555+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:35:03.216555+0000","last_peered":"2024-09-10T20:35:03.216555+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:35:03.216555+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31580160,"num_objects":7710,"num_object_clones":0,"num_object_copies":15420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3606,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9843,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4105},{"shards":"0,2","objects":944}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'12890","reported_seq":13104,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:03.214824+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:35:03.214824+0000","last_peered":"2024-09-10T20:35:03.214824+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:35:03.214824+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31514624,"num_objects":7694,"num_object_clones":0,"num_object_copies":15388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9584,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4198},{"shards":"0,2","objects":938}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086748+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086748+0000","last_peered":"2024-09-10T20:34:27.086748+0000","last_clean":"2024-09-10T20:34:27.086748+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086748+0000","last_undegraded":"2024-09-10T20:34:27.086748+0000","last_fullsized":"2024-09-10T20:34:27.086748+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13225","reported_seq":13445,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:03.148977+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:35:03.148977+0000","last_peered":"2024-09-10T20:35:03.148977+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:35:03.148977+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31789056,"num_objects":7761,"num_object_clones":0,"num_object_copies":15522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3673,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10052,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4090},{"shards":"0,2","objects":983}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'12855","reported_seq":19977,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:04.388869+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:35:04.388869+0000","last_peered":"2024-09-10T20:35:04.388869+0000","last_clean":"2024-09-10T20:35:04.388869+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:35:04.388869+0000","last_undegraded":"2024-09-10T20:35:04.388869+0000","last_fullsized":"2024-09-10T20:35:04.388869+0000","mapping_epoch":46,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31379939,"num_objects":7662,"num_object_clones":0,"num_object_copies":15324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9462,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":504174017,"num_objects":123093,"num_object_clones":0,"num_object_copies":246186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33733,"num_objects_misplaced":938,"num_objects_unfound":0,"num_objects_dirty":123093,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":172324,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8478,"num_bytes_recovered":33210368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1145864192,"data_stored":1145787401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160828,"ondisk_log_size":160828,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182478,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":785668,"kb_used_data":429172,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93586172,"statfs":{"total":96636764160,"available":95832240128,"internally_reserved":0,"allocated":439472128,"data_stored":439219835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,3,2,1,0,0,0,4],"upper_bound":1024},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986250,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":668912,"kb_used_data":362336,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93702928,"statfs":{"total":96636764160,"available":95951798272,"internally_reserved":0,"allocated":371032064,"data_stored":370779776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313913002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,8,0,0,0,0,0,0,0,1],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659197,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":633044,"kb_used_data":330624,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93738796,"statfs":{"total":96636764160,"available":95988527104,"internally_reserved":0,"allocated":338558976,"data_stored":338244803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":309653166},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":437952512,"data_stored":437948899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":338161664,"data_stored":338096171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":369750016,"data_stored":369742331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:35:09.145 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:35:09.706 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:09.706 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:35:09.728 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":251,"stamp":"2024-09-10T20:35:08.952302+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":498968020,"num_objects":121824,"num_object_clones":0,"num_object_copies":243648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33716,"num_objects_misplaced":938,"num_objects_unfound":0,"num_objects_dirty":121824,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":173597,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8497,"num_bytes_recovered":33210387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160702,"ondisk_log_size":160702,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2102872,"kb_used_data":1115172,"kb_used_omap":12,"kb_used_meta":987635,"kb_avail":281012648,"statfs":{"total":289910292480,"available":287756951552,"internally_reserved":0,"allocated":1141936128,"data_stored":1141117374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1011338242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,1,5,3,17,1,0,0,0,0,6,0,1],"upper_bound":16384},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23703552,"num_objects":-5787,"num_object_clones":0,"num_object_copies":-11574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-50,"num_objects_misplaced":-282,"num_objects_unfound":0,"num_objects_dirty":-5787,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5787,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":1134592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003925"},"pg_stats":[{"pgid":"2.f","version":"47'13116","reported_seq":13307,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:08.298029+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:35:08.298029+0000","last_peered":"2024-09-10T20:35:08.298029+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:35:08.298029+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30892032,"num_objects":7542,"num_object_clones":0,"num_object_copies":15084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3554,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13116,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3988},{"shards":"0,2","objects":820}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13233","reported_seq":13348,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:03.886290+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:35:03.886290+0000","last_peered":"2024-09-10T20:35:03.886290+0000","last_clean":"2024-09-10T20:35:03.886290+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:35:03.886290+0000","last_undegraded":"2024-09-10T20:35:03.886290+0000","last_fullsized":"2024-09-10T20:35:03.886290+0000","mapping_epoch":35,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31576064,"num_objects":7709,"num_object_clones":0,"num_object_copies":15418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11419,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13040","reported_seq":20393,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:04.523913+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:35:04.523913+0000","last_peered":"2024-09-10T20:35:04.523913+0000","last_clean":"2024-09-10T20:35:04.523913+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:35:04.523913+0000","last_undegraded":"2024-09-10T20:35:04.523913+0000","last_fullsized":"2024-09-10T20:35:04.523913+0000","mapping_epoch":44,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31436824,"num_objects":7676,"num_object_clones":0,"num_object_copies":15352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7676,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11168,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'12946","reported_seq":13031,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:03.853323+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:35:03.853323+0000","last_peered":"2024-09-10T20:35:03.853323+0000","last_clean":"2024-09-10T20:35:03.853323+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:35:03.853323+0000","last_undegraded":"2024-09-10T20:35:03.853323+0000","last_fullsized":"2024-09-10T20:35:03.853323+0000","mapping_epoch":35,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31367168,"num_objects":7658,"num_object_clones":0,"num_object_copies":15316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12946,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13320","reported_seq":13451,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:08.182901+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:35:08.182901+0000","last_peered":"2024-09-10T20:35:08.182901+0000","last_clean":"2024-09-10T20:35:08.182901+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:35:08.182901+0000","last_undegraded":"2024-09-10T20:35:08.182901+0000","last_fullsized":"2024-09-10T20:35:08.182901+0000","mapping_epoch":34,"log_start":"39'3300","ondisk_log_start":"39'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31739904,"num_objects":7749,"num_object_clones":0,"num_object_copies":15498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13017","reported_seq":13246,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:08.266117+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:35:08.266117+0000","last_peered":"2024-09-10T20:35:08.266117+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:35:08.266117+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30531584,"num_objects":7454,"num_object_clones":0,"num_object_copies":14908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3571,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9881,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3885},{"shards":"0,2","objects":835}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'13040","reported_seq":13301,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:08.268716+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:35:08.268716+0000","last_peered":"2024-09-10T20:35:08.268716+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:35:08.268716+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30998528,"num_objects":7568,"num_object_clones":0,"num_object_copies":15136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9635,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4072},{"shards":"0,2","objects":822}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13190","reported_seq":13319,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:08.163129+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:35:08.163129+0000","last_peered":"2024-09-10T20:35:08.163129+0000","last_clean":"2024-09-10T20:35:08.163129+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:35:08.163129+0000","last_undegraded":"2024-09-10T20:35:08.163129+0000","last_fullsized":"2024-09-10T20:35:08.163129+0000","mapping_epoch":34,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30908416,"num_objects":7546,"num_object_clones":0,"num_object_copies":15092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10261,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'12805","reported_seq":12988,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:03.871438+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:35:03.871438+0000","last_peered":"2024-09-10T20:35:03.871438+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:35:03.871438+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31268864,"num_objects":7634,"num_object_clones":0,"num_object_copies":15268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3505,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10798,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":943},{"shards":"2","objects":4131}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086228+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086228+0000","last_peered":"2024-09-10T20:34:27.086228+0000","last_clean":"2024-09-10T20:34:27.086228+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086228+0000","last_undegraded":"2024-09-10T20:34:27.086228+0000","last_fullsized":"2024-09-10T20:34:27.086228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13166","reported_seq":13353,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:08.233114+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:35:08.233114+0000","last_peered":"2024-09-10T20:35:08.233114+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:35:08.233114+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30983110,"num_objects":7566,"num_object_clones":0,"num_object_copies":15132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3580,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13164,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3988},{"shards":"0,2","objects":818}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.075947+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.075947+0000","last_peered":"2024-09-10T20:34:27.075947+0000","last_clean":"2024-09-10T20:34:27.075947+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.075947+0000","last_undegraded":"2024-09-10T20:34:27.075947+0000","last_fullsized":"2024-09-10T20:34:27.075947+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'12934","reported_seq":13117,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:03.809684+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:35:03.809684+0000","last_peered":"2024-09-10T20:35:03.809684+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:35:03.809684+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31256576,"num_objects":7631,"num_object_clones":0,"num_object_copies":15262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3610,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10762,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":984},{"shards":"2","objects":4023}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'12999","reported_seq":15225,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:03.904847+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:35:03.904847+0000","last_peered":"2024-09-10T20:35:03.904847+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:35:03.904847+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31395840,"num_objects":7665,"num_object_clones":0,"num_object_copies":15330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1628,"num_objects_misplaced":938,"num_objects_unfound":0,"num_objects_dirty":7665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10785,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1046,"num_bytes_recovered":4243456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":938},{"shards":"2","objects":5099}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13244","reported_seq":13472,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:08.242543+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:35:08.242543+0000","last_peered":"2024-09-10T20:35:08.242543+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:35:08.242543+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31072256,"num_objects":7586,"num_object_clones":0,"num_object_copies":15172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3605,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9967,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3982},{"shards":"0,2","objects":820}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13023","reported_seq":13246,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:08.146649+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:35:08.146649+0000","last_peered":"2024-09-10T20:35:08.146649+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:35:08.146649+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30969856,"num_objects":7561,"num_object_clones":0,"num_object_copies":15122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9717,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4069},{"shards":"0,2","objects":805}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086748+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086748+0000","last_peered":"2024-09-10T20:34:27.086748+0000","last_clean":"2024-09-10T20:34:27.086748+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086748+0000","last_undegraded":"2024-09-10T20:34:27.086748+0000","last_fullsized":"2024-09-10T20:34:27.086748+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13371","reported_seq":13595,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:08.256407+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:35:08.256407+0000","last_peered":"2024-09-10T20:35:08.256407+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:35:08.256407+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31191040,"num_objects":7615,"num_object_clones":0,"num_object_copies":15230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3670,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10198,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3947},{"shards":"0,2","objects":837}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'12855","reported_seq":19977,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:04.388869+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:35:04.388869+0000","last_peered":"2024-09-10T20:35:04.388869+0000","last_clean":"2024-09-10T20:35:04.388869+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:35:04.388869+0000","last_undegraded":"2024-09-10T20:35:04.388869+0000","last_fullsized":"2024-09-10T20:35:04.388869+0000","mapping_epoch":46,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31379939,"num_objects":7662,"num_object_clones":0,"num_object_copies":15324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9462,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":498968001,"num_objects":121822,"num_object_clones":0,"num_object_copies":243644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33716,"num_objects_misplaced":938,"num_objects_unfound":0,"num_objects_dirty":121822,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":173595,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8495,"num_bytes_recovered":33210368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1138917376,"data_stored":1138840585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160699,"ondisk_log_size":160699,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182479,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":800916,"kb_used_data":422212,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93570924,"statfs":{"total":96636764160,"available":95816626176,"internally_reserved":0,"allocated":432345088,"data_stored":432092795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":387772074},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,0,3,0,6,1,0,0,0,0,6],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986250,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":668912,"kb_used_data":362336,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93702928,"statfs":{"total":96636764160,"available":95951798272,"internally_reserved":0,"allocated":371032064,"data_stored":370779776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313913002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,8,0,0,0,0,0,0,0,1],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659197,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":633044,"kb_used_data":330624,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93738796,"statfs":{"total":96636764160,"available":95988527104,"internally_reserved":0,"allocated":338558976,"data_stored":338244803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":309653166},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":431005696,"data_stored":431002083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":338161664,"data_stored":338096171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":369750016,"data_stored":369742331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:35:09.730 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:35:10.285 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:10.286 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:35:10.311 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":251,"stamp":"2024-09-10T20:35:08.952302+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":498968020,"num_objects":121824,"num_object_clones":0,"num_object_copies":243648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33716,"num_objects_misplaced":938,"num_objects_unfound":0,"num_objects_dirty":121824,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":173597,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8497,"num_bytes_recovered":33210387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160702,"ondisk_log_size":160702,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2102872,"kb_used_data":1115172,"kb_used_omap":12,"kb_used_meta":987635,"kb_avail":281012648,"statfs":{"total":289910292480,"available":287756951552,"internally_reserved":0,"allocated":1141936128,"data_stored":1141117374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1011338242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,1,5,3,17,1,0,0,0,0,6,0,1],"upper_bound":16384},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23703552,"num_objects":-5787,"num_object_clones":0,"num_object_copies":-11574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-50,"num_objects_misplaced":-282,"num_objects_unfound":0,"num_objects_dirty":-5787,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5787,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":1134592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003925"},"pg_stats":[{"pgid":"2.f","version":"47'13116","reported_seq":13307,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:08.298029+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:35:08.298029+0000","last_peered":"2024-09-10T20:35:08.298029+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:35:08.298029+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30892032,"num_objects":7542,"num_object_clones":0,"num_object_copies":15084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3554,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13116,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3988},{"shards":"0,2","objects":820}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13233","reported_seq":13348,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:03.886290+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:35:03.886290+0000","last_peered":"2024-09-10T20:35:03.886290+0000","last_clean":"2024-09-10T20:35:03.886290+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:35:03.886290+0000","last_undegraded":"2024-09-10T20:35:03.886290+0000","last_fullsized":"2024-09-10T20:35:03.886290+0000","mapping_epoch":35,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31576064,"num_objects":7709,"num_object_clones":0,"num_object_copies":15418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11419,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13040","reported_seq":20393,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:04.523913+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:35:04.523913+0000","last_peered":"2024-09-10T20:35:04.523913+0000","last_clean":"2024-09-10T20:35:04.523913+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:35:04.523913+0000","last_undegraded":"2024-09-10T20:35:04.523913+0000","last_fullsized":"2024-09-10T20:35:04.523913+0000","mapping_epoch":44,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31436824,"num_objects":7676,"num_object_clones":0,"num_object_copies":15352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7676,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11168,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'12946","reported_seq":13031,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:03.853323+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:35:03.853323+0000","last_peered":"2024-09-10T20:35:03.853323+0000","last_clean":"2024-09-10T20:35:03.853323+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:35:03.853323+0000","last_undegraded":"2024-09-10T20:35:03.853323+0000","last_fullsized":"2024-09-10T20:35:03.853323+0000","mapping_epoch":35,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31367168,"num_objects":7658,"num_object_clones":0,"num_object_copies":15316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12946,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13320","reported_seq":13451,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:08.182901+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:35:08.182901+0000","last_peered":"2024-09-10T20:35:08.182901+0000","last_clean":"2024-09-10T20:35:08.182901+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:35:08.182901+0000","last_undegraded":"2024-09-10T20:35:08.182901+0000","last_fullsized":"2024-09-10T20:35:08.182901+0000","mapping_epoch":34,"log_start":"39'3300","ondisk_log_start":"39'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31739904,"num_objects":7749,"num_object_clones":0,"num_object_copies":15498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13017","reported_seq":13246,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:08.266117+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:35:08.266117+0000","last_peered":"2024-09-10T20:35:08.266117+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:35:08.266117+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30531584,"num_objects":7454,"num_object_clones":0,"num_object_copies":14908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3571,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9881,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3885},{"shards":"0,2","objects":835}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'13040","reported_seq":13301,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:08.268716+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:35:08.268716+0000","last_peered":"2024-09-10T20:35:08.268716+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:35:08.268716+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30998528,"num_objects":7568,"num_object_clones":0,"num_object_copies":15136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9635,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4072},{"shards":"0,2","objects":822}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13190","reported_seq":13319,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:08.163129+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:35:08.163129+0000","last_peered":"2024-09-10T20:35:08.163129+0000","last_clean":"2024-09-10T20:35:08.163129+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:35:08.163129+0000","last_undegraded":"2024-09-10T20:35:08.163129+0000","last_fullsized":"2024-09-10T20:35:08.163129+0000","mapping_epoch":34,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30908416,"num_objects":7546,"num_object_clones":0,"num_object_copies":15092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10261,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'12805","reported_seq":12988,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:03.871438+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:35:03.871438+0000","last_peered":"2024-09-10T20:35:03.871438+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:35:03.871438+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31268864,"num_objects":7634,"num_object_clones":0,"num_object_copies":15268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3505,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10798,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":943},{"shards":"2","objects":4131}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086228+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086228+0000","last_peered":"2024-09-10T20:34:27.086228+0000","last_clean":"2024-09-10T20:34:27.086228+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086228+0000","last_undegraded":"2024-09-10T20:34:27.086228+0000","last_fullsized":"2024-09-10T20:34:27.086228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13166","reported_seq":13353,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:08.233114+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:35:08.233114+0000","last_peered":"2024-09-10T20:35:08.233114+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:35:08.233114+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30983110,"num_objects":7566,"num_object_clones":0,"num_object_copies":15132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3580,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13164,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3988},{"shards":"0,2","objects":818}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.075947+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.075947+0000","last_peered":"2024-09-10T20:34:27.075947+0000","last_clean":"2024-09-10T20:34:27.075947+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.075947+0000","last_undegraded":"2024-09-10T20:34:27.075947+0000","last_fullsized":"2024-09-10T20:34:27.075947+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'12934","reported_seq":13117,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:03.809684+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:35:03.809684+0000","last_peered":"2024-09-10T20:35:03.809684+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:35:03.809684+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31256576,"num_objects":7631,"num_object_clones":0,"num_object_copies":15262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3610,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10762,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":984},{"shards":"2","objects":4023}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'12999","reported_seq":15225,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:03.904847+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:35:03.904847+0000","last_peered":"2024-09-10T20:35:03.904847+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:35:03.904847+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31395840,"num_objects":7665,"num_object_clones":0,"num_object_copies":15330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1628,"num_objects_misplaced":938,"num_objects_unfound":0,"num_objects_dirty":7665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10785,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1046,"num_bytes_recovered":4243456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":938},{"shards":"2","objects":5099}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13244","reported_seq":13472,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:08.242543+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:35:08.242543+0000","last_peered":"2024-09-10T20:35:08.242543+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:35:08.242543+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31072256,"num_objects":7586,"num_object_clones":0,"num_object_copies":15172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3605,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9967,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3982},{"shards":"0,2","objects":820}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13023","reported_seq":13246,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:08.146649+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:35:08.146649+0000","last_peered":"2024-09-10T20:35:08.146649+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:35:08.146649+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30969856,"num_objects":7561,"num_object_clones":0,"num_object_copies":15122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9717,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4069},{"shards":"0,2","objects":805}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086748+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086748+0000","last_peered":"2024-09-10T20:34:27.086748+0000","last_clean":"2024-09-10T20:34:27.086748+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086748+0000","last_undegraded":"2024-09-10T20:34:27.086748+0000","last_fullsized":"2024-09-10T20:34:27.086748+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13371","reported_seq":13595,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:08.256407+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:35:08.256407+0000","last_peered":"2024-09-10T20:35:08.256407+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:35:08.256407+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31191040,"num_objects":7615,"num_object_clones":0,"num_object_copies":15230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3670,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10198,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3947},{"shards":"0,2","objects":837}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'12855","reported_seq":19977,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:04.388869+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:35:04.388869+0000","last_peered":"2024-09-10T20:35:04.388869+0000","last_clean":"2024-09-10T20:35:04.388869+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:35:04.388869+0000","last_undegraded":"2024-09-10T20:35:04.388869+0000","last_fullsized":"2024-09-10T20:35:04.388869+0000","mapping_epoch":46,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31379939,"num_objects":7662,"num_object_clones":0,"num_object_copies":15324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9462,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":498968001,"num_objects":121822,"num_object_clones":0,"num_object_copies":243644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33716,"num_objects_misplaced":938,"num_objects_unfound":0,"num_objects_dirty":121822,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":173595,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8495,"num_bytes_recovered":33210368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1138917376,"data_stored":1138840585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160699,"ondisk_log_size":160699,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182479,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":800916,"kb_used_data":422212,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93570924,"statfs":{"total":96636764160,"available":95816626176,"internally_reserved":0,"allocated":432345088,"data_stored":432092795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":387772074},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,0,3,0,6,1,0,0,0,0,6],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986250,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":668912,"kb_used_data":362336,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93702928,"statfs":{"total":96636764160,"available":95951798272,"internally_reserved":0,"allocated":371032064,"data_stored":370779776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313913002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,8,0,0,0,0,0,0,0,1],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659197,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":633044,"kb_used_data":330624,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93738796,"statfs":{"total":96636764160,"available":95988527104,"internally_reserved":0,"allocated":338558976,"data_stored":338244803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":309653166},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":431005696,"data_stored":431002083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":338161664,"data_stored":338096171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":369750016,"data_stored":369742331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:35:10.312 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:35:10.312 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'13116", 'reported_seq': 13307, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:08.298029+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:35:08.298029+0000', 'last_peered': '2024-09-10T20:35:08.298029+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:35:08.298029+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30892032, 'num_objects': 7542, 'num_object_clones': 0, 'num_object_copies': 15084, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3554, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7542, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13116, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 54, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3988}, {'shards': '0,2', 'objects': 820}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:10.313 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:35:10.313 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'13017", 'reported_seq': 13246, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:08.266117+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:35:08.266117+0000', 'last_peered': '2024-09-10T20:35:08.266117+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:35:08.266117+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30531584, 'num_objects': 7454, 'num_object_clones': 0, 'num_object_copies': 14908, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3571, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7454, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9881, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 45, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3885}, {'shards': '0,2', 'objects': 835}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:10.313 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:35:10.313 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'13040", 'reported_seq': 13301, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:08.268716+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:35:08.268716+0000', 'last_peered': '2024-09-10T20:35:08.268716+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:35:08.268716+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30998528, 'num_objects': 7568, 'num_object_clones': 0, 'num_object_copies': 15136, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7568, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9635, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 59, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4072}, {'shards': '0,2', 'objects': 822}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:10.313 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:35:10.313 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'12805", 'reported_seq': 12988, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:03.871438+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:35:03.871438+0000', 'last_peered': '2024-09-10T20:35:03.871438+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:35:03.871438+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "36'2800", 'ondisk_log_start': "36'2800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31268864, 'num_objects': 7634, 'num_object_clones': 0, 'num_object_copies': 15268, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3505, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7634, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10798, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 23, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 943}, {'shards': '2', 'objects': 4131}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:35:10.313 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:35:10.313 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'13166", 'reported_seq': 13353, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:08.233114+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:35:08.233114+0000', 'last_peered': '2024-09-10T20:35:08.233114+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:35:08.233114+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30983110, 'num_objects': 7566, 'num_object_clones': 0, 'num_object_copies': 15132, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3580, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7566, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13164, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 39, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3988}, {'shards': '0,2', 'objects': 818}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:10.314 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:35:10.314 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'12934", 'reported_seq': 13117, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:03.809684+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:35:03.809684+0000', 'last_peered': '2024-09-10T20:35:03.809684+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:35:03.809684+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31256576, 'num_objects': 7631, 'num_object_clones': 0, 'num_object_copies': 15262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3610, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7631, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10762, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 25, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 984}, {'shards': '2', 'objects': 4023}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:35:10.314 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:35:10.314 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'12999", 'reported_seq': 15225, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:03.904847+0000', 'last_change': '2024-09-10T20:34:25.241937+0000', 'last_active': '2024-09-10T20:35:03.904847+0000', 'last_peered': '2024-09-10T20:35:03.904847+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:35:03.904847+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31395840, 'num_objects': 7665, 'num_object_clones': 0, 'num_object_copies': 15330, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1628, 'num_objects_misplaced': 938, 'num_objects_unfound': 0, 'num_objects_dirty': 7665, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10785, 'num_write_kb': 32484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1046, 'num_bytes_recovered': 4243456, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 938}, {'shards': '2', 'objects': 5099}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:35:10.314 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:35:10.314 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'13244", 'reported_seq': 13472, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:08.242543+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:35:08.242543+0000', 'last_peered': '2024-09-10T20:35:08.242543+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:35:08.242543+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31072256, 'num_objects': 7586, 'num_object_clones': 0, 'num_object_copies': 15172, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3605, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7586, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9967, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3982}, {'shards': '0,2', 'objects': 820}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:10.314 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:35:10.314 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'13023", 'reported_seq': 13246, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:08.146649+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:35:08.146649+0000', 'last_peered': '2024-09-10T20:35:08.146649+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:35:08.146649+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30969856, 'num_objects': 7561, 'num_object_clones': 0, 'num_object_copies': 15122, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7561, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9717, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 40, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4069}, {'shards': '0,2', 'objects': 805}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:10.314 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:35:10.314 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'13371", 'reported_seq': 13595, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:08.256407+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:35:08.256407+0000', 'last_peered': '2024-09-10T20:35:08.256407+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:35:08.256407+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "36'3300", 'ondisk_log_start': "36'3300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31191040, 'num_objects': 7615, 'num_object_clones': 0, 'num_object_copies': 15230, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3670, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7615, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10198, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 45, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3947}, {'shards': '0,2', 'objects': 837}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:10.315 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:35:11.001 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:11.002 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33716/243648 objects degraded (13.838%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":422,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":121824,"data_bytes":498968020,"bytes_used":2153340928,"bytes_avail":287756951552,"bytes_total":289910292480,"degraded_objects":33716,"degraded_total":243648,"degraded_ratio":0.13837995797215655,"misplaced_objects":938,"misplaced_total":243648,"misplaced_ratio":0.0038498161281849226,"recovering_objects_per_sec":27,"recovering_bytes_per_sec":94518,"recovering_keys_per_sec":0,"num_objects_recovered":332,"num_bytes_recovered":1134592,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":482},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:35:11.022 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:35:11.023 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:35:11.574 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:11.574 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:35:11.602 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":252,"stamp":"2024-09-10T20:35:10.953060+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":494921172,"num_objects":120836,"num_object_clones":0,"num_object_copies":241672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33713,"num_objects_misplaced":778,"num_objects_unfound":0,"num_objects_dirty":120836,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":174585,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8660,"num_bytes_recovered":33865747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160790,"ondisk_log_size":160790,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2105044,"kb_used_data":1109024,"kb_used_omap":12,"kb_used_meta":995955,"kb_avail":281010476,"statfs":{"total":289910292480,"available":287754727424,"internally_reserved":0,"allocated":1135640576,"data_stored":1134821822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1019857922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,1,5,3,9,1,0,0,0,0,6,2],"upper_bound":8192},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19693568,"num_objects":-4808,"num_object_clones":0,"num_object_copies":-9616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-27,"num_objects_misplaced":-296,"num_objects_unfound":0,"num_objects_dirty":-4808,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4808,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":323,"num_bytes_recovered":1191936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003949"},"pg_stats":[{"pgid":"2.f","version":"47'13116","reported_seq":13307,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:08.298029+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:35:08.298029+0000","last_peered":"2024-09-10T20:35:08.298029+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:35:08.298029+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30892032,"num_objects":7542,"num_object_clones":0,"num_object_copies":15084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3554,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13116,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3988},{"shards":"0,2","objects":820}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13357","reported_seq":13472,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:08.736544+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:35:08.736544+0000","last_peered":"2024-09-10T20:35:08.736544+0000","last_clean":"2024-09-10T20:35:08.736544+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:35:08.736544+0000","last_undegraded":"2024-09-10T20:35:08.736544+0000","last_fullsized":"2024-09-10T20:35:08.736544+0000","mapping_epoch":35,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31068160,"num_objects":7585,"num_object_clones":0,"num_object_copies":15170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11543,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13181","reported_seq":20534,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:09.581557+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:35:09.581557+0000","last_peered":"2024-09-10T20:35:09.581557+0000","last_clean":"2024-09-10T20:35:09.581557+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:35:09.581557+0000","last_undegraded":"2024-09-10T20:35:09.581557+0000","last_fullsized":"2024-09-10T20:35:09.581557+0000","mapping_epoch":44,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30859288,"num_objects":7535,"num_object_clones":0,"num_object_copies":15070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7535,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11309,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'13094","reported_seq":13179,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:08.933684+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:35:08.933684+0000","last_peered":"2024-09-10T20:35:08.933684+0000","last_clean":"2024-09-10T20:35:08.933684+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:35:08.933684+0000","last_undegraded":"2024-09-10T20:35:08.933684+0000","last_fullsized":"2024-09-10T20:35:08.933684+0000","mapping_epoch":35,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30760960,"num_objects":7510,"num_object_clones":0,"num_object_copies":15020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13094,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13320","reported_seq":13451,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:08.182901+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:35:08.182901+0000","last_peered":"2024-09-10T20:35:08.182901+0000","last_clean":"2024-09-10T20:35:08.182901+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:35:08.182901+0000","last_undegraded":"2024-09-10T20:35:08.182901+0000","last_fullsized":"2024-09-10T20:35:08.182901+0000","mapping_epoch":34,"log_start":"39'3300","ondisk_log_start":"39'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31739904,"num_objects":7749,"num_object_clones":0,"num_object_copies":15498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13017","reported_seq":13246,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:08.266117+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:35:08.266117+0000","last_peered":"2024-09-10T20:35:08.266117+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:35:08.266117+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30531584,"num_objects":7454,"num_object_clones":0,"num_object_copies":14908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3571,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9881,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3885},{"shards":"0,2","objects":835}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'13040","reported_seq":13301,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:08.268716+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:35:08.268716+0000","last_peered":"2024-09-10T20:35:08.268716+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:35:08.268716+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30998528,"num_objects":7568,"num_object_clones":0,"num_object_copies":15136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9635,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4072},{"shards":"0,2","objects":822}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13190","reported_seq":13319,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:08.163129+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:35:08.163129+0000","last_peered":"2024-09-10T20:35:08.163129+0000","last_clean":"2024-09-10T20:35:08.163129+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:35:08.163129+0000","last_undegraded":"2024-09-10T20:35:08.163129+0000","last_fullsized":"2024-09-10T20:35:08.163129+0000","mapping_epoch":34,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30908416,"num_objects":7546,"num_object_clones":0,"num_object_copies":15092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10261,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'12948","reported_seq":13137,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:08.906383+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:35:08.906383+0000","last_peered":"2024-09-10T20:35:08.906383+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:35:08.906383+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30683136,"num_objects":7491,"num_object_clones":0,"num_object_copies":14982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3502,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10941,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":800},{"shards":"2","objects":3991}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086228+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086228+0000","last_peered":"2024-09-10T20:34:27.086228+0000","last_clean":"2024-09-10T20:34:27.086228+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086228+0000","last_undegraded":"2024-09-10T20:34:27.086228+0000","last_fullsized":"2024-09-10T20:34:27.086228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13166","reported_seq":13353,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:08.233114+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:35:08.233114+0000","last_peered":"2024-09-10T20:35:08.233114+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:35:08.233114+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30983110,"num_objects":7566,"num_object_clones":0,"num_object_copies":15132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3580,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13164,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3988},{"shards":"0,2","objects":818}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.075947+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.075947+0000","last_peered":"2024-09-10T20:34:27.075947+0000","last_clean":"2024-09-10T20:34:27.075947+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.075947+0000","last_undegraded":"2024-09-10T20:34:27.075947+0000","last_fullsized":"2024-09-10T20:34:27.075947+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13081","reported_seq":13264,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:08.935587+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:35:08.935587+0000","last_peered":"2024-09-10T20:35:08.935587+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:35:08.935587+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30654464,"num_objects":7484,"num_object_clones":0,"num_object_copies":14968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3610,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10909,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":837},{"shards":"2","objects":3876}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13135","reported_seq":15681,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:08.932360+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:35:08.932360+0000","last_peered":"2024-09-10T20:35:08.932360+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:35:08.932360+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30838784,"num_objects":7529,"num_object_clones":0,"num_object_copies":15058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1628,"num_objects_misplaced":778,"num_objects_unfound":0,"num_objects_dirty":7529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10921,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1206,"num_bytes_recovered":4898816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":778},{"shards":"2","objects":5123}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13244","reported_seq":13472,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:08.242543+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:35:08.242543+0000","last_peered":"2024-09-10T20:35:08.242543+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:35:08.242543+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31072256,"num_objects":7586,"num_object_clones":0,"num_object_copies":15172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3605,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9967,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3982},{"shards":"0,2","objects":820}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13023","reported_seq":13246,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:08.146649+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:35:08.146649+0000","last_peered":"2024-09-10T20:35:08.146649+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:35:08.146649+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30969856,"num_objects":7561,"num_object_clones":0,"num_object_copies":15122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9717,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4069},{"shards":"0,2","objects":805}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086748+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086748+0000","last_peered":"2024-09-10T20:34:27.086748+0000","last_clean":"2024-09-10T20:34:27.086748+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086748+0000","last_undegraded":"2024-09-10T20:34:27.086748+0000","last_fullsized":"2024-09-10T20:34:27.086748+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13371","reported_seq":13595,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:08.256407+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:35:08.256407+0000","last_peered":"2024-09-10T20:35:08.256407+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:35:08.256407+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31191040,"num_objects":7615,"num_object_clones":0,"num_object_copies":15230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3670,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10198,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3947},{"shards":"0,2","objects":837}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13004","reported_seq":20126,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:09.568293+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:35:09.568293+0000","last_peered":"2024-09-10T20:35:09.568293+0000","last_clean":"2024-09-10T20:35:09.568293+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:35:09.568293+0000","last_undegraded":"2024-09-10T20:35:09.568293+0000","last_fullsized":"2024-09-10T20:35:09.568293+0000","mapping_epoch":46,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30769635,"num_objects":7513,"num_object_clones":0,"num_object_copies":15026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7513,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9611,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":494921153,"num_objects":120834,"num_object_clones":0,"num_object_copies":241668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33713,"num_objects_misplaced":778,"num_objects_unfound":0,"num_objects_dirty":120834,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":174583,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8658,"num_bytes_recovered":33865728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1133244416,"data_stored":1133167625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160787,"ondisk_log_size":160787,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182479,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":800916,"kb_used_data":422212,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93570924,"statfs":{"total":96636764160,"available":95816626176,"internally_reserved":0,"allocated":432345088,"data_stored":432092795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":387772074},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,0,3,0,6,1,0,0,0,0,6],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986251,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":667808,"kb_used_data":357072,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93704032,"statfs":{"total":96636764160,"available":95952928768,"internally_reserved":0,"allocated":365641728,"data_stored":365389440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318172842},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,0,0,0,0,0,0,0,2],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659198,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":636320,"kb_used_data":329740,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93735520,"statfs":{"total":96636764160,"available":95985172480,"internally_reserved":0,"allocated":337653760,"data_stored":337339587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":313913006},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":431005696,"data_stored":431002083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":337022976,"data_stored":336957483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":365215744,"data_stored":365208059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:35:14.604 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:35:15.177 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:15.177 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:35:15.203 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":254,"stamp":"2024-09-10T20:35:14.954361+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":485643732,"num_objects":118571,"num_object_clones":0,"num_object_copies":237142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33711,"num_objects_misplaced":632,"num_objects_unfound":0,"num_objects_dirty":118571,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":176850,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8808,"num_bytes_recovered":34463763,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2063356,"kb_used_data":1096456,"kb_used_omap":12,"kb_used_meta":966835,"kb_avail":281052164,"statfs":{"total":289910292480,"available":287797415936,"internally_reserved":0,"allocated":1122770944,"data_stored":1121952190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":990039042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,3,4,8,6,1,0,0,0,0,0,6],"upper_bound":8192},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27791360,"num_objects":-6785,"num_object_clones":0,"num_object_copies":-13570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-29,"num_objects_misplaced":-442,"num_objects_unfound":0,"num_objects_dirty":-6785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6785,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":471,"num_bytes_recovered":1789952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003646"},"pg_stats":[{"pgid":"2.f","version":"47'13257","reported_seq":13448,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:13.279302+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:35:13.279302+0000","last_peered":"2024-09-10T20:35:13.279302+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:35:13.279302+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30314496,"num_objects":7401,"num_object_clones":0,"num_object_copies":14802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3554,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13257,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3847},{"shards":"0,2","objects":679}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13514","reported_seq":13629,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:13.867535+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:35:13.867535+0000","last_peered":"2024-09-10T20:35:13.867535+0000","last_clean":"2024-09-10T20:35:13.867535+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:35:13.867535+0000","last_undegraded":"2024-09-10T20:35:13.867535+0000","last_fullsized":"2024-09-10T20:35:13.867535+0000","mapping_epoch":35,"log_start":"39'3500","ondisk_log_start":"39'3500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30425088,"num_objects":7428,"num_object_clones":0,"num_object_copies":14856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11700,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13328","reported_seq":20681,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:14.553354+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:35:14.553354+0000","last_peered":"2024-09-10T20:35:14.553354+0000","last_clean":"2024-09-10T20:35:14.553354+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:35:14.553354+0000","last_undegraded":"2024-09-10T20:35:14.553354+0000","last_fullsized":"2024-09-10T20:35:14.553354+0000","mapping_epoch":44,"log_start":"39'3300","ondisk_log_start":"39'3300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30257176,"num_objects":7388,"num_object_clones":0,"num_object_copies":14776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7388,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11456,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'13242","reported_seq":13327,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:13.919809+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:35:13.919809+0000","last_peered":"2024-09-10T20:35:13.919809+0000","last_clean":"2024-09-10T20:35:13.919809+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:35:13.919809+0000","last_undegraded":"2024-09-10T20:35:13.919809+0000","last_fullsized":"2024-09-10T20:35:13.919809+0000","mapping_epoch":35,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30154752,"num_objects":7362,"num_object_clones":0,"num_object_copies":14724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13242,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13461","reported_seq":13592,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:13.267999+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:35:13.267999+0000","last_peered":"2024-09-10T20:35:13.267999+0000","last_clean":"2024-09-10T20:35:13.267999+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:35:13.267999+0000","last_undegraded":"2024-09-10T20:35:13.267999+0000","last_fullsized":"2024-09-10T20:35:13.267999+0000","mapping_epoch":34,"log_start":"39'3400","ondisk_log_start":"39'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31162368,"num_objects":7608,"num_object_clones":0,"num_object_copies":15216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10457,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13171","reported_seq":13402,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:13.230515+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:35:13.230515+0000","last_peered":"2024-09-10T20:35:13.230515+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:35:13.230515+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29900800,"num_objects":7300,"num_object_clones":0,"num_object_copies":14600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3571,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10035,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3731},{"shards":"0,2","objects":681}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'13184","reported_seq":13445,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:13.277725+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:35:13.277725+0000","last_peered":"2024-09-10T20:35:13.277725+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:35:13.277725+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30408704,"num_objects":7424,"num_object_clones":0,"num_object_copies":14848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7424,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9779,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3928},{"shards":"0,2","objects":678}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13330","reported_seq":13459,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:13.228154+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:35:13.228154+0000","last_peered":"2024-09-10T20:35:13.228154+0000","last_clean":"2024-09-10T20:35:13.228154+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:35:13.228154+0000","last_undegraded":"2024-09-10T20:35:13.228154+0000","last_fullsized":"2024-09-10T20:35:13.228154+0000","mapping_epoch":34,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30334976,"num_objects":7406,"num_object_clones":0,"num_object_copies":14812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10401,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13096","reported_seq":13284,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:13.953440+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:35:13.953440+0000","last_peered":"2024-09-10T20:35:13.953440+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:35:13.953440+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30076928,"num_objects":7343,"num_object_clones":0,"num_object_copies":14686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3502,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11089,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":652},{"shards":"2","objects":3843}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086228+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086228+0000","last_peered":"2024-09-10T20:34:27.086228+0000","last_clean":"2024-09-10T20:34:27.086228+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086228+0000","last_undegraded":"2024-09-10T20:34:27.086228+0000","last_fullsized":"2024-09-10T20:34:27.086228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13288","reported_seq":13476,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:13.289006+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:35:13.289006+0000","last_peered":"2024-09-10T20:35:13.289006+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:35:13.289006+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30483398,"num_objects":7444,"num_object_clones":0,"num_object_copies":14888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3580,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13286,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3866},{"shards":"0,2","objects":696}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13209","reported_seq":13395,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:13.820789+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:35:13.820789+0000","last_peered":"2024-09-10T20:35:13.820789+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:35:13.820789+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30130176,"num_objects":7356,"num_object_clones":0,"num_object_copies":14712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11037,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":709},{"shards":"2","objects":3750}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13277","reported_seq":16114,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:13.923456+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:35:13.923456+0000","last_peered":"2024-09-10T20:35:13.923456+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:35:13.923456+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30257152,"num_objects":7387,"num_object_clones":0,"num_object_copies":14774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1628,"num_objects_misplaced":632,"num_objects_unfound":0,"num_objects_dirty":7387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11063,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1352,"num_bytes_recovered":5496832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":632},{"shards":"2","objects":5127}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13379","reported_seq":13607,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:13.280574+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:35:13.280574+0000","last_peered":"2024-09-10T20:35:13.280574+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:35:13.280574+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30519296,"num_objects":7451,"num_object_clones":0,"num_object_copies":14902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3605,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10102,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3847},{"shards":"0,2","objects":685}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13156","reported_seq":13378,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:13.283582+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:35:13.283582+0000","last_peered":"2024-09-10T20:35:13.283582+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:35:13.283582+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30425088,"num_objects":7428,"num_object_clones":0,"num_object_copies":14856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9850,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3936},{"shards":"0,2","objects":672}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086748+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086748+0000","last_peered":"2024-09-10T20:34:27.086748+0000","last_clean":"2024-09-10T20:34:27.086748+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086748+0000","last_undegraded":"2024-09-10T20:34:27.086748+0000","last_fullsized":"2024-09-10T20:34:27.086748+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13532","reported_seq":13758,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:13.227100+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:35:13.227100+0000","last_peered":"2024-09-10T20:35:13.227100+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:35:13.227100+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"39'3500","ondisk_log_start":"39'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30531584,"num_objects":7454,"num_object_clones":0,"num_object_copies":14908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3670,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10359,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3786},{"shards":"0,2","objects":676}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13128","reported_seq":20250,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:14.550215+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:35:14.550215+0000","last_peered":"2024-09-10T20:35:14.550215+0000","last_clean":"2024-09-10T20:35:14.550215+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:35:14.550215+0000","last_undegraded":"2024-09-10T20:35:14.550215+0000","last_fullsized":"2024-09-10T20:35:14.550215+0000","mapping_epoch":46,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30261731,"num_objects":7389,"num_object_clones":0,"num_object_copies":14778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9735,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":485643713,"num_objects":118569,"num_object_clones":0,"num_object_copies":237138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33711,"num_objects_misplaced":632,"num_objects_unfound":0,"num_objects_dirty":118569,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":176848,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8806,"num_bytes_recovered":34463744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1120403456,"data_stored":1120326665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182480,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":785952,"kb_used_data":415568,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93585888,"statfs":{"total":96636764160,"available":95831949312,"internally_reserved":0,"allocated":425541632,"data_stored":425289339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":379252394},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,3,2,2,0,0,0,0,0,0,6],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986252,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":650700,"kb_used_data":352440,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93721140,"statfs":{"total":96636764160,"available":95970447360,"internally_reserved":0,"allocated":360898560,"data_stored":360646272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305393322},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,4,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659199,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":626704,"kb_used_data":328448,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93745136,"statfs":{"total":96636764160,"available":95995019264,"internally_reserved":0,"allocated":336330752,"data_stored":336016579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":305393326},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,1,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":424026112,"data_stored":424022499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":335917056,"data_stored":335851563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360460288,"data_stored":360452603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:35:15.205 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:35:15.773 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:15.773 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:35:15.797 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":254,"stamp":"2024-09-10T20:35:14.954361+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":485643732,"num_objects":118571,"num_object_clones":0,"num_object_copies":237142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33711,"num_objects_misplaced":632,"num_objects_unfound":0,"num_objects_dirty":118571,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":176850,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8808,"num_bytes_recovered":34463763,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2063356,"kb_used_data":1096456,"kb_used_omap":12,"kb_used_meta":966835,"kb_avail":281052164,"statfs":{"total":289910292480,"available":287797415936,"internally_reserved":0,"allocated":1122770944,"data_stored":1121952190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":990039042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,3,4,8,6,1,0,0,0,0,0,6],"upper_bound":8192},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27791360,"num_objects":-6785,"num_object_clones":0,"num_object_copies":-13570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-29,"num_objects_misplaced":-442,"num_objects_unfound":0,"num_objects_dirty":-6785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6785,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":471,"num_bytes_recovered":1789952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003646"},"pg_stats":[{"pgid":"2.f","version":"47'13257","reported_seq":13448,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:13.279302+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:35:13.279302+0000","last_peered":"2024-09-10T20:35:13.279302+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:35:13.279302+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30314496,"num_objects":7401,"num_object_clones":0,"num_object_copies":14802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3554,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13257,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3847},{"shards":"0,2","objects":679}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13514","reported_seq":13629,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:13.867535+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:35:13.867535+0000","last_peered":"2024-09-10T20:35:13.867535+0000","last_clean":"2024-09-10T20:35:13.867535+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:35:13.867535+0000","last_undegraded":"2024-09-10T20:35:13.867535+0000","last_fullsized":"2024-09-10T20:35:13.867535+0000","mapping_epoch":35,"log_start":"39'3500","ondisk_log_start":"39'3500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30425088,"num_objects":7428,"num_object_clones":0,"num_object_copies":14856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11700,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13328","reported_seq":20681,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:14.553354+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:35:14.553354+0000","last_peered":"2024-09-10T20:35:14.553354+0000","last_clean":"2024-09-10T20:35:14.553354+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:35:14.553354+0000","last_undegraded":"2024-09-10T20:35:14.553354+0000","last_fullsized":"2024-09-10T20:35:14.553354+0000","mapping_epoch":44,"log_start":"39'3300","ondisk_log_start":"39'3300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30257176,"num_objects":7388,"num_object_clones":0,"num_object_copies":14776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7388,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11456,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'13242","reported_seq":13327,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:13.919809+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:35:13.919809+0000","last_peered":"2024-09-10T20:35:13.919809+0000","last_clean":"2024-09-10T20:35:13.919809+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:35:13.919809+0000","last_undegraded":"2024-09-10T20:35:13.919809+0000","last_fullsized":"2024-09-10T20:35:13.919809+0000","mapping_epoch":35,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30154752,"num_objects":7362,"num_object_clones":0,"num_object_copies":14724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13242,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13461","reported_seq":13592,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:13.267999+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:35:13.267999+0000","last_peered":"2024-09-10T20:35:13.267999+0000","last_clean":"2024-09-10T20:35:13.267999+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:35:13.267999+0000","last_undegraded":"2024-09-10T20:35:13.267999+0000","last_fullsized":"2024-09-10T20:35:13.267999+0000","mapping_epoch":34,"log_start":"39'3400","ondisk_log_start":"39'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31162368,"num_objects":7608,"num_object_clones":0,"num_object_copies":15216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10457,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13171","reported_seq":13402,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:13.230515+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:35:13.230515+0000","last_peered":"2024-09-10T20:35:13.230515+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:35:13.230515+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29900800,"num_objects":7300,"num_object_clones":0,"num_object_copies":14600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3571,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10035,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3731},{"shards":"0,2","objects":681}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'13184","reported_seq":13445,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:13.277725+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:35:13.277725+0000","last_peered":"2024-09-10T20:35:13.277725+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:35:13.277725+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30408704,"num_objects":7424,"num_object_clones":0,"num_object_copies":14848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7424,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9779,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3928},{"shards":"0,2","objects":678}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13330","reported_seq":13459,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:13.228154+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:35:13.228154+0000","last_peered":"2024-09-10T20:35:13.228154+0000","last_clean":"2024-09-10T20:35:13.228154+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:35:13.228154+0000","last_undegraded":"2024-09-10T20:35:13.228154+0000","last_fullsized":"2024-09-10T20:35:13.228154+0000","mapping_epoch":34,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30334976,"num_objects":7406,"num_object_clones":0,"num_object_copies":14812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10401,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13096","reported_seq":13284,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:13.953440+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:35:13.953440+0000","last_peered":"2024-09-10T20:35:13.953440+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:35:13.953440+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30076928,"num_objects":7343,"num_object_clones":0,"num_object_copies":14686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3502,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11089,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":652},{"shards":"2","objects":3843}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086228+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086228+0000","last_peered":"2024-09-10T20:34:27.086228+0000","last_clean":"2024-09-10T20:34:27.086228+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086228+0000","last_undegraded":"2024-09-10T20:34:27.086228+0000","last_fullsized":"2024-09-10T20:34:27.086228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13288","reported_seq":13476,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:13.289006+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:35:13.289006+0000","last_peered":"2024-09-10T20:35:13.289006+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:35:13.289006+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30483398,"num_objects":7444,"num_object_clones":0,"num_object_copies":14888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3580,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13286,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3866},{"shards":"0,2","objects":696}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13209","reported_seq":13395,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:13.820789+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:35:13.820789+0000","last_peered":"2024-09-10T20:35:13.820789+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:35:13.820789+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30130176,"num_objects":7356,"num_object_clones":0,"num_object_copies":14712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11037,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":709},{"shards":"2","objects":3750}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13277","reported_seq":16114,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:13.923456+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:35:13.923456+0000","last_peered":"2024-09-10T20:35:13.923456+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:35:13.923456+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30257152,"num_objects":7387,"num_object_clones":0,"num_object_copies":14774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1628,"num_objects_misplaced":632,"num_objects_unfound":0,"num_objects_dirty":7387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11063,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1352,"num_bytes_recovered":5496832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":632},{"shards":"2","objects":5127}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13379","reported_seq":13607,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:13.280574+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:35:13.280574+0000","last_peered":"2024-09-10T20:35:13.280574+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:35:13.280574+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30519296,"num_objects":7451,"num_object_clones":0,"num_object_copies":14902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3605,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10102,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3847},{"shards":"0,2","objects":685}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13156","reported_seq":13378,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:13.283582+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:35:13.283582+0000","last_peered":"2024-09-10T20:35:13.283582+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:35:13.283582+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30425088,"num_objects":7428,"num_object_clones":0,"num_object_copies":14856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9850,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3936},{"shards":"0,2","objects":672}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086748+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086748+0000","last_peered":"2024-09-10T20:34:27.086748+0000","last_clean":"2024-09-10T20:34:27.086748+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086748+0000","last_undegraded":"2024-09-10T20:34:27.086748+0000","last_fullsized":"2024-09-10T20:34:27.086748+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13532","reported_seq":13758,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:13.227100+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:35:13.227100+0000","last_peered":"2024-09-10T20:35:13.227100+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:35:13.227100+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"39'3500","ondisk_log_start":"39'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30531584,"num_objects":7454,"num_object_clones":0,"num_object_copies":14908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3670,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10359,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3786},{"shards":"0,2","objects":676}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13128","reported_seq":20250,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:14.550215+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:35:14.550215+0000","last_peered":"2024-09-10T20:35:14.550215+0000","last_clean":"2024-09-10T20:35:14.550215+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:35:14.550215+0000","last_undegraded":"2024-09-10T20:35:14.550215+0000","last_fullsized":"2024-09-10T20:35:14.550215+0000","mapping_epoch":46,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30261731,"num_objects":7389,"num_object_clones":0,"num_object_copies":14778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9735,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":485643713,"num_objects":118569,"num_object_clones":0,"num_object_copies":237138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33711,"num_objects_misplaced":632,"num_objects_unfound":0,"num_objects_dirty":118569,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":176848,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8806,"num_bytes_recovered":34463744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1120403456,"data_stored":1120326665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182480,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":785952,"kb_used_data":415568,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93585888,"statfs":{"total":96636764160,"available":95831949312,"internally_reserved":0,"allocated":425541632,"data_stored":425289339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":379252394},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,3,2,2,0,0,0,0,0,0,6],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986252,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":650700,"kb_used_data":352440,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93721140,"statfs":{"total":96636764160,"available":95970447360,"internally_reserved":0,"allocated":360898560,"data_stored":360646272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305393322},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,4,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659199,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":626704,"kb_used_data":328448,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93745136,"statfs":{"total":96636764160,"available":95995019264,"internally_reserved":0,"allocated":336330752,"data_stored":336016579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":305393326},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,1,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":424026112,"data_stored":424022499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":335917056,"data_stored":335851563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360460288,"data_stored":360452603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:35:15.798 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:35:15.798 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'13257", 'reported_seq': 13448, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:13.279302+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:35:13.279302+0000', 'last_peered': '2024-09-10T20:35:13.279302+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:35:13.279302+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30314496, 'num_objects': 7401, 'num_object_clones': 0, 'num_object_copies': 14802, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3554, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7401, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13257, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 54, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3847}, {'shards': '0,2', 'objects': 679}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:15.799 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:35:15.799 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'13171", 'reported_seq': 13402, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:13.230515+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:35:13.230515+0000', 'last_peered': '2024-09-10T20:35:13.230515+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:35:13.230515+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29900800, 'num_objects': 7300, 'num_object_clones': 0, 'num_object_copies': 14600, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3571, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7300, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10035, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 45, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3731}, {'shards': '0,2', 'objects': 681}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:15.799 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:35:15.799 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'13184", 'reported_seq': 13445, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:13.277725+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:35:13.277725+0000', 'last_peered': '2024-09-10T20:35:13.277725+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:35:13.277725+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30408704, 'num_objects': 7424, 'num_object_clones': 0, 'num_object_copies': 14848, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7424, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9779, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 59, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3928}, {'shards': '0,2', 'objects': 678}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:15.799 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:35:15.799 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'13096", 'reported_seq': 13284, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:13.953440+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:35:13.953440+0000', 'last_peered': '2024-09-10T20:35:13.953440+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:35:13.953440+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30076928, 'num_objects': 7343, 'num_object_clones': 0, 'num_object_copies': 14686, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3502, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7343, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11089, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 26, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 652}, {'shards': '2', 'objects': 3843}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:35:15.799 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:35:15.799 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'13288", 'reported_seq': 13476, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:13.289006+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:35:13.289006+0000', 'last_peered': '2024-09-10T20:35:13.289006+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:35:13.289006+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30483398, 'num_objects': 7444, 'num_object_clones': 0, 'num_object_copies': 14888, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3580, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7444, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13286, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 39, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3866}, {'shards': '0,2', 'objects': 696}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:15.799 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:35:15.799 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'13209", 'reported_seq': 13395, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:13.820789+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:35:13.820789+0000', 'last_peered': '2024-09-10T20:35:13.820789+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:35:13.820789+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30130176, 'num_objects': 7356, 'num_object_clones': 0, 'num_object_copies': 14712, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7356, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11037, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 709}, {'shards': '2', 'objects': 3750}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:35:15.800 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:35:15.800 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'13277", 'reported_seq': 16114, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:13.923456+0000', 'last_change': '2024-09-10T20:34:25.241937+0000', 'last_active': '2024-09-10T20:35:13.923456+0000', 'last_peered': '2024-09-10T20:35:13.923456+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:35:13.923456+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30257152, 'num_objects': 7387, 'num_object_clones': 0, 'num_object_copies': 14774, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1628, 'num_objects_misplaced': 632, 'num_objects_unfound': 0, 'num_objects_dirty': 7387, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11063, 'num_write_kb': 32484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1352, 'num_bytes_recovered': 5496832, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 632}, {'shards': '2', 'objects': 5127}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:35:15.800 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:35:15.800 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'13379", 'reported_seq': 13607, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:13.280574+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:35:13.280574+0000', 'last_peered': '2024-09-10T20:35:13.280574+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:35:13.280574+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "36'3300", 'ondisk_log_start': "36'3300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30519296, 'num_objects': 7451, 'num_object_clones': 0, 'num_object_copies': 14902, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3605, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7451, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10102, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3847}, {'shards': '0,2', 'objects': 685}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:15.800 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:35:15.800 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'13156", 'reported_seq': 13378, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:13.283582+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:35:13.283582+0000', 'last_peered': '2024-09-10T20:35:13.283582+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:35:13.283582+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30425088, 'num_objects': 7428, 'num_object_clones': 0, 'num_object_copies': 14856, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7428, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9850, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 40, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3936}, {'shards': '0,2', 'objects': 672}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:15.800 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:35:15.800 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'13532", 'reported_seq': 13758, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:13.227100+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:35:13.227100+0000', 'last_peered': '2024-09-10T20:35:13.227100+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:35:13.227100+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "39'3500", 'ondisk_log_start': "39'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30531584, 'num_objects': 7454, 'num_object_clones': 0, 'num_object_copies': 14908, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3670, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7454, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10359, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 45, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3786}, {'shards': '0,2', 'objects': 676}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:15.801 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:35:16.468 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:16.468 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33711/237142 objects degraded (14.216%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":428,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":118571,"data_bytes":485643732,"bytes_used":2112876544,"bytes_avail":287797415936,"bytes_total":289910292480,"degraded_objects":33711,"degraded_total":237142,"degraded_ratio":0.14215533309156539,"misplaced_objects":632,"misplaced_total":237142,"misplaced_ratio":0.0026650698737465317,"recovering_objects_per_sec":39,"recovering_bytes_per_sec":149117,"recovering_keys_per_sec":0,"num_objects_recovered":471,"num_bytes_recovered":1789952,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":565},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:35:16.493 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:35:16.493 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:35:17.023 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:17.027 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:35:17.052 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":255,"stamp":"2024-09-10T20:35:16.954993+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":485643732,"num_objects":118571,"num_object_clones":0,"num_object_copies":237142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33711,"num_objects_misplaced":632,"num_objects_unfound":0,"num_objects_dirty":118571,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":176850,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8808,"num_bytes_recovered":34463763,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2063356,"kb_used_data":1096456,"kb_used_omap":12,"kb_used_meta":966835,"kb_avail":281052164,"statfs":{"total":289910292480,"available":287797415936,"internally_reserved":0,"allocated":1122770944,"data_stored":1121952190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":990039042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,3,4,8,6,1,0,0,0,0,0,6],"upper_bound":8192},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18530304,"num_objects":-4524,"num_object_clones":0,"num_object_copies":-9048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-22,"num_objects_misplaced":-306,"num_objects_unfound":0,"num_objects_dirty":-4524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4524,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":328,"num_bytes_recovered":1253376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003833"},"pg_stats":[{"pgid":"2.f","version":"47'13257","reported_seq":13448,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:13.279302+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:35:13.279302+0000","last_peered":"2024-09-10T20:35:13.279302+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:35:13.279302+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30314496,"num_objects":7401,"num_object_clones":0,"num_object_copies":14802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3554,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13257,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3847},{"shards":"0,2","objects":679}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13514","reported_seq":13629,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:13.867535+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:35:13.867535+0000","last_peered":"2024-09-10T20:35:13.867535+0000","last_clean":"2024-09-10T20:35:13.867535+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:35:13.867535+0000","last_undegraded":"2024-09-10T20:35:13.867535+0000","last_fullsized":"2024-09-10T20:35:13.867535+0000","mapping_epoch":35,"log_start":"39'3500","ondisk_log_start":"39'3500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30425088,"num_objects":7428,"num_object_clones":0,"num_object_copies":14856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11700,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13328","reported_seq":20681,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:14.553354+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:35:14.553354+0000","last_peered":"2024-09-10T20:35:14.553354+0000","last_clean":"2024-09-10T20:35:14.553354+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:35:14.553354+0000","last_undegraded":"2024-09-10T20:35:14.553354+0000","last_fullsized":"2024-09-10T20:35:14.553354+0000","mapping_epoch":44,"log_start":"39'3300","ondisk_log_start":"39'3300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30257176,"num_objects":7388,"num_object_clones":0,"num_object_copies":14776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7388,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11456,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'13242","reported_seq":13327,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:13.919809+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:35:13.919809+0000","last_peered":"2024-09-10T20:35:13.919809+0000","last_clean":"2024-09-10T20:35:13.919809+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:35:13.919809+0000","last_undegraded":"2024-09-10T20:35:13.919809+0000","last_fullsized":"2024-09-10T20:35:13.919809+0000","mapping_epoch":35,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30154752,"num_objects":7362,"num_object_clones":0,"num_object_copies":14724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13242,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13461","reported_seq":13592,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:13.267999+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:35:13.267999+0000","last_peered":"2024-09-10T20:35:13.267999+0000","last_clean":"2024-09-10T20:35:13.267999+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:35:13.267999+0000","last_undegraded":"2024-09-10T20:35:13.267999+0000","last_fullsized":"2024-09-10T20:35:13.267999+0000","mapping_epoch":34,"log_start":"39'3400","ondisk_log_start":"39'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31162368,"num_objects":7608,"num_object_clones":0,"num_object_copies":15216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10457,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13171","reported_seq":13402,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:13.230515+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:35:13.230515+0000","last_peered":"2024-09-10T20:35:13.230515+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:35:13.230515+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29900800,"num_objects":7300,"num_object_clones":0,"num_object_copies":14600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3571,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10035,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3731},{"shards":"0,2","objects":681}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'13184","reported_seq":13445,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:13.277725+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:35:13.277725+0000","last_peered":"2024-09-10T20:35:13.277725+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:35:13.277725+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30408704,"num_objects":7424,"num_object_clones":0,"num_object_copies":14848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7424,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9779,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3928},{"shards":"0,2","objects":678}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13330","reported_seq":13459,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:13.228154+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:35:13.228154+0000","last_peered":"2024-09-10T20:35:13.228154+0000","last_clean":"2024-09-10T20:35:13.228154+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:35:13.228154+0000","last_undegraded":"2024-09-10T20:35:13.228154+0000","last_fullsized":"2024-09-10T20:35:13.228154+0000","mapping_epoch":34,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30334976,"num_objects":7406,"num_object_clones":0,"num_object_copies":14812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10401,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13096","reported_seq":13284,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:13.953440+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:35:13.953440+0000","last_peered":"2024-09-10T20:35:13.953440+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:35:13.953440+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30076928,"num_objects":7343,"num_object_clones":0,"num_object_copies":14686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3502,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11089,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":652},{"shards":"2","objects":3843}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086228+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086228+0000","last_peered":"2024-09-10T20:34:27.086228+0000","last_clean":"2024-09-10T20:34:27.086228+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086228+0000","last_undegraded":"2024-09-10T20:34:27.086228+0000","last_fullsized":"2024-09-10T20:34:27.086228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13288","reported_seq":13476,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:13.289006+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:35:13.289006+0000","last_peered":"2024-09-10T20:35:13.289006+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:35:13.289006+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30483398,"num_objects":7444,"num_object_clones":0,"num_object_copies":14888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3580,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13286,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3866},{"shards":"0,2","objects":696}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13209","reported_seq":13395,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:13.820789+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:35:13.820789+0000","last_peered":"2024-09-10T20:35:13.820789+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:35:13.820789+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30130176,"num_objects":7356,"num_object_clones":0,"num_object_copies":14712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11037,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":709},{"shards":"2","objects":3750}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13277","reported_seq":16114,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:13.923456+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:35:13.923456+0000","last_peered":"2024-09-10T20:35:13.923456+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:35:13.923456+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30257152,"num_objects":7387,"num_object_clones":0,"num_object_copies":14774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1628,"num_objects_misplaced":632,"num_objects_unfound":0,"num_objects_dirty":7387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11063,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1352,"num_bytes_recovered":5496832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":632},{"shards":"2","objects":5127}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13379","reported_seq":13607,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:13.280574+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:35:13.280574+0000","last_peered":"2024-09-10T20:35:13.280574+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:35:13.280574+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30519296,"num_objects":7451,"num_object_clones":0,"num_object_copies":14902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3605,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10102,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3847},{"shards":"0,2","objects":685}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13156","reported_seq":13378,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:13.283582+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:35:13.283582+0000","last_peered":"2024-09-10T20:35:13.283582+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:35:13.283582+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30425088,"num_objects":7428,"num_object_clones":0,"num_object_copies":14856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9850,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3936},{"shards":"0,2","objects":672}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086748+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086748+0000","last_peered":"2024-09-10T20:34:27.086748+0000","last_clean":"2024-09-10T20:34:27.086748+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086748+0000","last_undegraded":"2024-09-10T20:34:27.086748+0000","last_fullsized":"2024-09-10T20:34:27.086748+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13532","reported_seq":13758,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:13.227100+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:35:13.227100+0000","last_peered":"2024-09-10T20:35:13.227100+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:35:13.227100+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"39'3500","ondisk_log_start":"39'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30531584,"num_objects":7454,"num_object_clones":0,"num_object_copies":14908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3670,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10359,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3786},{"shards":"0,2","objects":676}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13128","reported_seq":20250,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:14.550215+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:35:14.550215+0000","last_peered":"2024-09-10T20:35:14.550215+0000","last_clean":"2024-09-10T20:35:14.550215+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:35:14.550215+0000","last_undegraded":"2024-09-10T20:35:14.550215+0000","last_fullsized":"2024-09-10T20:35:14.550215+0000","mapping_epoch":46,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30261731,"num_objects":7389,"num_object_clones":0,"num_object_copies":14778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9735,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":485643713,"num_objects":118569,"num_object_clones":0,"num_object_copies":237138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33711,"num_objects_misplaced":632,"num_objects_unfound":0,"num_objects_dirty":118569,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":176848,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8806,"num_bytes_recovered":34463744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1120403456,"data_stored":1120326665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182480,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":785952,"kb_used_data":415568,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93585888,"statfs":{"total":96636764160,"available":95831949312,"internally_reserved":0,"allocated":425541632,"data_stored":425289339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":379252394},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,3,2,2,0,0,0,0,0,0,6],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986252,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":650700,"kb_used_data":352440,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93721140,"statfs":{"total":96636764160,"available":95970447360,"internally_reserved":0,"allocated":360898560,"data_stored":360646272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305393322},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,4,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659199,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":626704,"kb_used_data":328448,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93745136,"statfs":{"total":96636764160,"available":95995019264,"internally_reserved":0,"allocated":336330752,"data_stored":336016579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":305393326},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,1,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":424026112,"data_stored":424022499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":335917056,"data_stored":335851563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360460288,"data_stored":360452603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:35:20.053 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:35:20.605 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:20.608 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:35:20.627 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":256,"stamp":"2024-09-10T20:35:18.955596+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":480368084,"num_objects":117283,"num_object_clones":0,"num_object_copies":234566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":632,"num_objects_unfound":0,"num_objects_dirty":117283,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":178138,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8820,"num_bytes_recovered":34463763,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160943,"ondisk_log_size":160943,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2060804,"kb_used_data":1089744,"kb_used_omap":12,"kb_used_meta":970995,"kb_avail":281054716,"statfs":{"total":289910292480,"available":287800029184,"internally_reserved":0,"allocated":1115897856,"data_stored":1115079102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":994298882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,0,5,9,7,1,0,0,0,7],"upper_bound":2048},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23805952,"num_objects":-5812,"num_object_clones":0,"num_object_copies":-11624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-34,"num_objects_misplaced":-306,"num_objects_unfound":0,"num_objects_dirty":-5812,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5812,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":1253376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003912"},"pg_stats":[{"pgid":"2.f","version":"47'13398","reported_seq":13595,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:18.247692+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:35:18.247692+0000","last_peered":"2024-09-10T20:35:18.247692+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:35:18.247692+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"39'3300","ondisk_log_start":"39'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29736960,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3551,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13398,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3709},{"shards":"0,2","objects":538}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13514","reported_seq":13629,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:13.867535+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:35:13.867535+0000","last_peered":"2024-09-10T20:35:13.867535+0000","last_clean":"2024-09-10T20:35:13.867535+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:35:13.867535+0000","last_undegraded":"2024-09-10T20:35:13.867535+0000","last_fullsized":"2024-09-10T20:35:13.867535+0000","mapping_epoch":35,"log_start":"39'3500","ondisk_log_start":"39'3500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30425088,"num_objects":7428,"num_object_clones":0,"num_object_copies":14856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11700,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13328","reported_seq":20681,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:14.553354+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:35:14.553354+0000","last_peered":"2024-09-10T20:35:14.553354+0000","last_clean":"2024-09-10T20:35:14.553354+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:35:14.553354+0000","last_undegraded":"2024-09-10T20:35:14.553354+0000","last_fullsized":"2024-09-10T20:35:14.553354+0000","mapping_epoch":44,"log_start":"39'3300","ondisk_log_start":"39'3300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30257176,"num_objects":7388,"num_object_clones":0,"num_object_copies":14776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7388,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11456,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'13242","reported_seq":13327,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:13.919809+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:35:13.919809+0000","last_peered":"2024-09-10T20:35:13.919809+0000","last_clean":"2024-09-10T20:35:13.919809+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:35:13.919809+0000","last_undegraded":"2024-09-10T20:35:13.919809+0000","last_fullsized":"2024-09-10T20:35:13.919809+0000","mapping_epoch":35,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30154752,"num_objects":7362,"num_object_clones":0,"num_object_copies":14724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13242,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13614","reported_seq":13745,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:18.228248+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:35:18.228248+0000","last_peered":"2024-09-10T20:35:18.228248+0000","last_clean":"2024-09-10T20:35:18.228248+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:35:18.228248+0000","last_undegraded":"2024-09-10T20:35:18.228248+0000","last_fullsized":"2024-09-10T20:35:18.228248+0000","mapping_epoch":34,"log_start":"40'3600","ondisk_log_start":"40'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30535680,"num_objects":7455,"num_object_clones":0,"num_object_copies":14910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10610,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13296","reported_seq":13529,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:18.326140+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:35:18.326140+0000","last_peered":"2024-09-10T20:35:18.326140+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:35:18.326140+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29388800,"num_objects":7175,"num_object_clones":0,"num_object_copies":14350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3570,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10160,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3607},{"shards":"0,2","objects":556}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'13296","reported_seq":13559,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:18.274367+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:35:18.274367+0000","last_peered":"2024-09-10T20:35:18.274367+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:35:18.274367+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29949952,"num_objects":7312,"num_object_clones":0,"num_object_copies":14624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3497,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9891,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3817},{"shards":"0,2","objects":566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13474","reported_seq":13603,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:18.339156+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:35:18.339156+0000","last_peered":"2024-09-10T20:35:18.339156+0000","last_clean":"2024-09-10T20:35:18.339156+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:35:18.339156+0000","last_undegraded":"2024-09-10T20:35:18.339156+0000","last_fullsized":"2024-09-10T20:35:18.339156+0000","mapping_epoch":34,"log_start":"39'3400","ondisk_log_start":"39'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29745152,"num_objects":7262,"num_object_clones":0,"num_object_copies":14524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10545,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13096","reported_seq":13284,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:13.953440+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:35:13.953440+0000","last_peered":"2024-09-10T20:35:13.953440+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:35:13.953440+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30076928,"num_objects":7343,"num_object_clones":0,"num_object_copies":14686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3502,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11089,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":652},{"shards":"2","objects":3843}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086228+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086228+0000","last_peered":"2024-09-10T20:34:27.086228+0000","last_clean":"2024-09-10T20:34:27.086228+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086228+0000","last_undegraded":"2024-09-10T20:34:27.086228+0000","last_fullsized":"2024-09-10T20:34:27.086228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13451","reported_seq":13643,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:18.335203+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:35:18.335203+0000","last_peered":"2024-09-10T20:35:18.335203+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:35:18.335203+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"39'3400","ondisk_log_start":"39'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29815750,"num_objects":7281,"num_object_clones":0,"num_object_copies":14562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3577,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13449,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3706},{"shards":"0,2","objects":533}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13209","reported_seq":13395,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:13.820789+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:35:13.820789+0000","last_peered":"2024-09-10T20:35:13.820789+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:35:13.820789+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30130176,"num_objects":7356,"num_object_clones":0,"num_object_copies":14712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11037,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":709},{"shards":"2","objects":3750}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13277","reported_seq":16114,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:13.923456+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:35:13.923456+0000","last_peered":"2024-09-10T20:35:13.923456+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:35:13.923456+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30257152,"num_objects":7387,"num_object_clones":0,"num_object_copies":14774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1628,"num_objects_misplaced":632,"num_objects_unfound":0,"num_objects_dirty":7387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11063,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1352,"num_bytes_recovered":5496832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":632},{"shards":"2","objects":5127}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13539","reported_seq":13771,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:18.287151+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:35:18.287151+0000","last_peered":"2024-09-10T20:35:18.287151+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:35:18.287151+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"39'3500","ondisk_log_start":"39'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29863936,"num_objects":7291,"num_object_clones":0,"num_object_copies":14582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3603,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10262,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3689},{"shards":"0,2","objects":525}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13299","reported_seq":13524,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:18.273067+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:35:18.273067+0000","last_peered":"2024-09-10T20:35:18.273067+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:35:18.273067+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29839360,"num_objects":7285,"num_object_clones":0,"num_object_copies":14570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3494,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9993,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3794},{"shards":"0,2","objects":529}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086748+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086748+0000","last_peered":"2024-09-10T20:34:27.086748+0000","last_clean":"2024-09-10T20:34:27.086748+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086748+0000","last_undegraded":"2024-09-10T20:34:27.086748+0000","last_fullsized":"2024-09-10T20:34:27.086748+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13679","reported_seq":13907,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:18.307869+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:35:18.307869+0000","last_peered":"2024-09-10T20:35:18.307869+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:35:18.307869+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"39'3600","ondisk_log_start":"39'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29929472,"num_objects":7307,"num_object_clones":0,"num_object_copies":14614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3669,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10506,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3640},{"shards":"0,2","objects":529}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13128","reported_seq":20250,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:14.550215+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:35:14.550215+0000","last_peered":"2024-09-10T20:35:14.550215+0000","last_clean":"2024-09-10T20:35:14.550215+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:35:14.550215+0000","last_undegraded":"2024-09-10T20:35:14.550215+0000","last_fullsized":"2024-09-10T20:35:14.550215+0000","mapping_epoch":46,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30261731,"num_objects":7389,"num_object_clones":0,"num_object_copies":14778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9735,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":480368065,"num_objects":117281,"num_object_clones":0,"num_object_copies":234562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":632,"num_objects_unfound":0,"num_objects_dirty":117281,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":178136,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8818,"num_bytes_recovered":34463744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1113587712,"data_stored":1113510921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160940,"ondisk_log_size":160940,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182481,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":783400,"kb_used_data":408856,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93588440,"statfs":{"total":96636764160,"available":95834562560,"internally_reserved":0,"allocated":418668544,"data_stored":418416251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":383512234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,3,0,0,0,0,7],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986252,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":650700,"kb_used_data":352440,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93721140,"statfs":{"total":96636764160,"available":95970447360,"internally_reserved":0,"allocated":360898560,"data_stored":360646272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305393322},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,4,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659199,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":626704,"kb_used_data":328448,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93745136,"statfs":{"total":96636764160,"available":95995019264,"internally_reserved":0,"allocated":336330752,"data_stored":336016579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":305393326},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,1,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":417210368,"data_stored":417206755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":335917056,"data_stored":335851563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360460288,"data_stored":360452603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:35:20.629 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:35:21.182 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:21.183 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:35:21.209 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":257,"stamp":"2024-09-10T20:35:20.956439+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":476280276,"num_objects":116285,"num_object_clones":0,"num_object_copies":232570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":491,"num_objects_unfound":0,"num_objects_dirty":116285,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":179136,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8961,"num_bytes_recovered":35041299,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161041,"ondisk_log_size":161041,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2068520,"kb_used_data":1084980,"kb_used_omap":12,"kb_used_meta":983475,"kb_avail":281047000,"statfs":{"total":289910292480,"available":287792128000,"internally_reserved":0,"allocated":1111019520,"data_stored":1110200766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1007078402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,4,10,6,1,0,0,0,7],"upper_bound":2048},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22687744,"num_objects":-5539,"num_object_clones":0,"num_object_copies":-11078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-17,"num_objects_misplaced":-447,"num_objects_unfound":0,"num_objects_dirty":-5539,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5539,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":464,"num_bytes_recovered":1830912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004137"},"pg_stats":[{"pgid":"2.f","version":"47'13398","reported_seq":13595,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:18.247692+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:35:18.247692+0000","last_peered":"2024-09-10T20:35:18.247692+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:35:18.247692+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"39'3300","ondisk_log_start":"39'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29736960,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3551,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13398,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3709},{"shards":"0,2","objects":538}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13650","reported_seq":13765,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:18.965976+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:35:18.965976+0000","last_peered":"2024-09-10T20:35:18.965976+0000","last_clean":"2024-09-10T20:35:18.965976+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:35:18.965976+0000","last_undegraded":"2024-09-10T20:35:18.965976+0000","last_fullsized":"2024-09-10T20:35:18.965976+0000","mapping_epoch":35,"log_start":"40'3600","ondisk_log_start":"40'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29868032,"num_objects":7292,"num_object_clones":0,"num_object_copies":14584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11836,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13470","reported_seq":20823,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:19.681677+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:35:19.681677+0000","last_peered":"2024-09-10T20:35:19.681677+0000","last_clean":"2024-09-10T20:35:19.681677+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:35:19.681677+0000","last_undegraded":"2024-09-10T20:35:19.681677+0000","last_fullsized":"2024-09-10T20:35:19.681677+0000","mapping_epoch":44,"log_start":"39'3400","ondisk_log_start":"39'3400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29675544,"num_objects":7246,"num_object_clones":0,"num_object_copies":14492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7246,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11598,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'13372","reported_seq":13457,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:18.956446+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:35:18.956446+0000","last_peered":"2024-09-10T20:35:18.956446+0000","last_clean":"2024-09-10T20:35:18.956446+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:35:18.956446+0000","last_undegraded":"2024-09-10T20:35:18.956446+0000","last_fullsized":"2024-09-10T20:35:18.956446+0000","mapping_epoch":35,"log_start":"39'3300","ondisk_log_start":"39'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29622272,"num_objects":7232,"num_object_clones":0,"num_object_copies":14464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7232,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13372,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13614","reported_seq":13745,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:18.228248+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:35:18.228248+0000","last_peered":"2024-09-10T20:35:18.228248+0000","last_clean":"2024-09-10T20:35:18.228248+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:35:18.228248+0000","last_undegraded":"2024-09-10T20:35:18.228248+0000","last_fullsized":"2024-09-10T20:35:18.228248+0000","mapping_epoch":34,"log_start":"40'3600","ondisk_log_start":"40'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30535680,"num_objects":7455,"num_object_clones":0,"num_object_copies":14910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10610,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13296","reported_seq":13529,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:18.326140+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:35:18.326140+0000","last_peered":"2024-09-10T20:35:18.326140+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:35:18.326140+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29388800,"num_objects":7175,"num_object_clones":0,"num_object_copies":14350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3570,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10160,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3607},{"shards":"0,2","objects":556}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'13296","reported_seq":13559,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:18.274367+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:35:18.274367+0000","last_peered":"2024-09-10T20:35:18.274367+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:35:18.274367+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29949952,"num_objects":7312,"num_object_clones":0,"num_object_copies":14624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3497,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9891,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3817},{"shards":"0,2","objects":566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13474","reported_seq":13603,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:18.339156+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:35:18.339156+0000","last_peered":"2024-09-10T20:35:18.339156+0000","last_clean":"2024-09-10T20:35:18.339156+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:35:18.339156+0000","last_undegraded":"2024-09-10T20:35:18.339156+0000","last_fullsized":"2024-09-10T20:35:18.339156+0000","mapping_epoch":34,"log_start":"39'3400","ondisk_log_start":"39'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29745152,"num_objects":7262,"num_object_clones":0,"num_object_copies":14524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10545,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13250","reported_seq":13439,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:18.927095+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:35:18.927095+0000","last_peered":"2024-09-10T20:35:18.927095+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:35:18.927095+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29446144,"num_objects":7189,"num_object_clones":0,"num_object_copies":14378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3502,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11243,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":498},{"shards":"2","objects":3689}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086228+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086228+0000","last_peered":"2024-09-10T20:34:27.086228+0000","last_clean":"2024-09-10T20:34:27.086228+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086228+0000","last_undegraded":"2024-09-10T20:34:27.086228+0000","last_fullsized":"2024-09-10T20:34:27.086228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13451","reported_seq":13643,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:18.335203+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:35:18.335203+0000","last_peered":"2024-09-10T20:35:18.335203+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:35:18.335203+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"39'3400","ondisk_log_start":"39'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29815750,"num_objects":7281,"num_object_clones":0,"num_object_copies":14562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3577,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13449,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3706},{"shards":"0,2","objects":533}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13366","reported_seq":13552,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:18.947158+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:35:18.947158+0000","last_peered":"2024-09-10T20:35:18.947158+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:35:18.947158+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"39'3300","ondisk_log_start":"39'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29487104,"num_objects":7199,"num_object_clones":0,"num_object_copies":14398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11194,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":552},{"shards":"2","objects":3593}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13434","reported_seq":16555,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:18.963834+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:35:18.963834+0000","last_peered":"2024-09-10T20:35:18.963834+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:35:18.963834+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"39'3400","ondisk_log_start":"39'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29614080,"num_objects":7230,"num_object_clones":0,"num_object_copies":14460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1628,"num_objects_misplaced":491,"num_objects_unfound":0,"num_objects_dirty":7230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11220,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1493,"num_bytes_recovered":6074368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":491},{"shards":"2","objects":5111}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13539","reported_seq":13771,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:18.287151+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:35:18.287151+0000","last_peered":"2024-09-10T20:35:18.287151+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:35:18.287151+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"39'3500","ondisk_log_start":"39'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29863936,"num_objects":7291,"num_object_clones":0,"num_object_copies":14582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3603,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10262,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3689},{"shards":"0,2","objects":525}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13299","reported_seq":13524,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:18.273067+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:35:18.273067+0000","last_peered":"2024-09-10T20:35:18.273067+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:35:18.273067+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29839360,"num_objects":7285,"num_object_clones":0,"num_object_copies":14570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3494,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9993,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3794},{"shards":"0,2","objects":529}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086748+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086748+0000","last_peered":"2024-09-10T20:34:27.086748+0000","last_clean":"2024-09-10T20:34:27.086748+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086748+0000","last_undegraded":"2024-09-10T20:34:27.086748+0000","last_fullsized":"2024-09-10T20:34:27.086748+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13679","reported_seq":13907,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:18.307869+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:35:18.307869+0000","last_peered":"2024-09-10T20:35:18.307869+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:35:18.307869+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"39'3600","ondisk_log_start":"39'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29929472,"num_objects":7307,"num_object_clones":0,"num_object_copies":14614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3669,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10506,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3640},{"shards":"0,2","objects":529}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13250","reported_seq":20372,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:19.634974+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:35:19.634974+0000","last_peered":"2024-09-10T20:35:19.634974+0000","last_clean":"2024-09-10T20:35:19.634974+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:35:19.634974+0000","last_undegraded":"2024-09-10T20:35:19.634974+0000","last_fullsized":"2024-09-10T20:35:19.634974+0000","mapping_epoch":46,"log_start":"39'3200","ondisk_log_start":"39'3200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29762019,"num_objects":7267,"num_object_clones":0,"num_object_copies":14534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9857,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":476280257,"num_objects":116283,"num_object_clones":0,"num_object_copies":232566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":491,"num_objects_unfound":0,"num_objects_dirty":116283,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":179134,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8959,"num_bytes_recovered":35041280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1107689472,"data_stored":1107612681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161038,"ondisk_log_size":161038,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182481,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":783400,"kb_used_data":408856,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93588440,"statfs":{"total":96636764160,"available":95834562560,"internally_reserved":0,"allocated":418668544,"data_stored":418416251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":383512234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,3,0,0,0,0,7],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986253,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":651172,"kb_used_data":348756,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93720668,"statfs":{"total":96636764160,"available":95969964032,"internally_reserved":0,"allocated":357126144,"data_stored":356873856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309653162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659200,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":633948,"kb_used_data":327368,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93737892,"statfs":{"total":96636764160,"available":95987601408,"internally_reserved":0,"allocated":335224832,"data_stored":334910659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":313913006},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":417210368,"data_stored":417206755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":334794752,"data_stored":334729259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":355684352,"data_stored":355676667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:35:21.210 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:35:21.210 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'13398", 'reported_seq': 13595, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:18.247692+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:35:18.247692+0000', 'last_peered': '2024-09-10T20:35:18.247692+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:35:18.247692+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "39'3300", 'ondisk_log_start': "39'3300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29736960, 'num_objects': 7260, 'num_object_clones': 0, 'num_object_copies': 14520, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3551, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7260, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13398, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 57, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3709}, {'shards': '0,2', 'objects': 538}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:21.211 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:35:21.211 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'13296", 'reported_seq': 13529, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:18.326140+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:35:18.326140+0000', 'last_peered': '2024-09-10T20:35:18.326140+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:35:18.326140+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29388800, 'num_objects': 7175, 'num_object_clones': 0, 'num_object_copies': 14350, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3570, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7175, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10160, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 46, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3607}, {'shards': '0,2', 'objects': 556}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:21.211 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:35:21.211 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'13296", 'reported_seq': 13559, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:18.274367+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:35:18.274367+0000', 'last_peered': '2024-09-10T20:35:18.274367+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:35:18.274367+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29949952, 'num_objects': 7312, 'num_object_clones': 0, 'num_object_copies': 14624, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3497, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7312, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9891, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 60, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3817}, {'shards': '0,2', 'objects': 566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:21.211 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:35:21.211 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'13250", 'reported_seq': 13439, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:18.927095+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:35:18.927095+0000', 'last_peered': '2024-09-10T20:35:18.927095+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:35:18.927095+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29446144, 'num_objects': 7189, 'num_object_clones': 0, 'num_object_copies': 14378, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3502, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7189, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11243, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 26, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 498}, {'shards': '2', 'objects': 3689}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:35:21.211 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:35:21.211 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'13451", 'reported_seq': 13643, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:18.335203+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:35:18.335203+0000', 'last_peered': '2024-09-10T20:35:18.335203+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:35:18.335203+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "39'3400", 'ondisk_log_start': "39'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29815750, 'num_objects': 7281, 'num_object_clones': 0, 'num_object_copies': 14562, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3577, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7281, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13449, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 42, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3706}, {'shards': '0,2', 'objects': 533}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:21.211 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:35:21.211 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'13366", 'reported_seq': 13552, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:18.947158+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:35:18.947158+0000', 'last_peered': '2024-09-10T20:35:18.947158+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:35:18.947158+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "39'3300", 'ondisk_log_start': "39'3300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29487104, 'num_objects': 7199, 'num_object_clones': 0, 'num_object_copies': 14398, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7199, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11194, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 552}, {'shards': '2', 'objects': 3593}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:35:21.211 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:35:21.212 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'13434", 'reported_seq': 16555, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:18.963834+0000', 'last_change': '2024-09-10T20:34:25.241937+0000', 'last_active': '2024-09-10T20:35:18.963834+0000', 'last_peered': '2024-09-10T20:35:18.963834+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:35:18.963834+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "39'3400", 'ondisk_log_start': "39'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29614080, 'num_objects': 7230, 'num_object_clones': 0, 'num_object_copies': 14460, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1628, 'num_objects_misplaced': 491, 'num_objects_unfound': 0, 'num_objects_dirty': 7230, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11220, 'num_write_kb': 32484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1493, 'num_bytes_recovered': 6074368, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 491}, {'shards': '2', 'objects': 5111}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:35:21.212 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:35:21.212 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'13539", 'reported_seq': 13771, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:18.287151+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:35:18.287151+0000', 'last_peered': '2024-09-10T20:35:18.287151+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:35:18.287151+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "39'3500", 'ondisk_log_start': "39'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29863936, 'num_objects': 7291, 'num_object_clones': 0, 'num_object_copies': 14582, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3603, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7291, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10262, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 46, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3689}, {'shards': '0,2', 'objects': 525}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:21.212 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:35:21.212 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'13299", 'reported_seq': 13524, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:18.273067+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:35:18.273067+0000', 'last_peered': '2024-09-10T20:35:18.273067+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:35:18.273067+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29839360, 'num_objects': 7285, 'num_object_clones': 0, 'num_object_copies': 14570, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3494, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7285, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9993, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 41, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3794}, {'shards': '0,2', 'objects': 529}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:21.212 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:35:21.212 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'13679", 'reported_seq': 13907, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:18.307869+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:35:18.307869+0000', 'last_peered': '2024-09-10T20:35:18.307869+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:35:18.307869+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "39'3600", 'ondisk_log_start': "39'3600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29929472, 'num_objects': 7307, 'num_object_clones': 0, 'num_object_copies': 14614, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3669, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7307, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10506, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 46, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3640}, {'shards': '0,2', 'objects': 529}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:21.212 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:35:21.871 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:21.871 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33699/232570 objects degraded (14.490%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":433,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":116285,"data_bytes":476280276,"bytes_used":2118164480,"bytes_avail":287792128000,"bytes_total":289910292480,"degraded_objects":33699,"degraded_total":232570,"degraded_ratio":0.14489831018618052,"misplaced_objects":491,"misplaced_total":232570,"misplaced_ratio":0.0021111923291912112,"recovering_objects_per_sec":38,"recovering_bytes_per_sec":152523,"recovering_keys_per_sec":0,"num_objects_recovered":464,"num_bytes_recovered":1830912,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":461},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:35:21.895 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:35:21.896 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:35:22.446 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:22.448 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:35:22.475 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":257,"stamp":"2024-09-10T20:35:20.956439+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":476280276,"num_objects":116285,"num_object_clones":0,"num_object_copies":232570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":491,"num_objects_unfound":0,"num_objects_dirty":116285,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":179136,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8961,"num_bytes_recovered":35041299,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161041,"ondisk_log_size":161041,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2068520,"kb_used_data":1084980,"kb_used_omap":12,"kb_used_meta":983475,"kb_avail":281047000,"statfs":{"total":289910292480,"available":287792128000,"internally_reserved":0,"allocated":1111019520,"data_stored":1110200766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1007078402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,4,10,6,1,0,0,0,7],"upper_bound":2048},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22687744,"num_objects":-5539,"num_object_clones":0,"num_object_copies":-11078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-17,"num_objects_misplaced":-447,"num_objects_unfound":0,"num_objects_dirty":-5539,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5539,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":464,"num_bytes_recovered":1830912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004137"},"pg_stats":[{"pgid":"2.f","version":"47'13398","reported_seq":13595,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:18.247692+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:35:18.247692+0000","last_peered":"2024-09-10T20:35:18.247692+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:35:18.247692+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"39'3300","ondisk_log_start":"39'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29736960,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3551,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13398,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3709},{"shards":"0,2","objects":538}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13650","reported_seq":13765,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:18.965976+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:35:18.965976+0000","last_peered":"2024-09-10T20:35:18.965976+0000","last_clean":"2024-09-10T20:35:18.965976+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:35:18.965976+0000","last_undegraded":"2024-09-10T20:35:18.965976+0000","last_fullsized":"2024-09-10T20:35:18.965976+0000","mapping_epoch":35,"log_start":"40'3600","ondisk_log_start":"40'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29868032,"num_objects":7292,"num_object_clones":0,"num_object_copies":14584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11836,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13470","reported_seq":20823,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:19.681677+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:35:19.681677+0000","last_peered":"2024-09-10T20:35:19.681677+0000","last_clean":"2024-09-10T20:35:19.681677+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:35:19.681677+0000","last_undegraded":"2024-09-10T20:35:19.681677+0000","last_fullsized":"2024-09-10T20:35:19.681677+0000","mapping_epoch":44,"log_start":"39'3400","ondisk_log_start":"39'3400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29675544,"num_objects":7246,"num_object_clones":0,"num_object_copies":14492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7246,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11598,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'13372","reported_seq":13457,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:18.956446+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:35:18.956446+0000","last_peered":"2024-09-10T20:35:18.956446+0000","last_clean":"2024-09-10T20:35:18.956446+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:35:18.956446+0000","last_undegraded":"2024-09-10T20:35:18.956446+0000","last_fullsized":"2024-09-10T20:35:18.956446+0000","mapping_epoch":35,"log_start":"39'3300","ondisk_log_start":"39'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29622272,"num_objects":7232,"num_object_clones":0,"num_object_copies":14464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7232,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13372,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13614","reported_seq":13745,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:18.228248+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:35:18.228248+0000","last_peered":"2024-09-10T20:35:18.228248+0000","last_clean":"2024-09-10T20:35:18.228248+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:35:18.228248+0000","last_undegraded":"2024-09-10T20:35:18.228248+0000","last_fullsized":"2024-09-10T20:35:18.228248+0000","mapping_epoch":34,"log_start":"40'3600","ondisk_log_start":"40'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30535680,"num_objects":7455,"num_object_clones":0,"num_object_copies":14910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10610,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13296","reported_seq":13529,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:18.326140+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:35:18.326140+0000","last_peered":"2024-09-10T20:35:18.326140+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:35:18.326140+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29388800,"num_objects":7175,"num_object_clones":0,"num_object_copies":14350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3570,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10160,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3607},{"shards":"0,2","objects":556}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'13296","reported_seq":13559,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:18.274367+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:35:18.274367+0000","last_peered":"2024-09-10T20:35:18.274367+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:35:18.274367+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29949952,"num_objects":7312,"num_object_clones":0,"num_object_copies":14624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3497,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9891,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3817},{"shards":"0,2","objects":566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13474","reported_seq":13603,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:18.339156+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:35:18.339156+0000","last_peered":"2024-09-10T20:35:18.339156+0000","last_clean":"2024-09-10T20:35:18.339156+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:35:18.339156+0000","last_undegraded":"2024-09-10T20:35:18.339156+0000","last_fullsized":"2024-09-10T20:35:18.339156+0000","mapping_epoch":34,"log_start":"39'3400","ondisk_log_start":"39'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29745152,"num_objects":7262,"num_object_clones":0,"num_object_copies":14524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10545,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13250","reported_seq":13439,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:18.927095+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:35:18.927095+0000","last_peered":"2024-09-10T20:35:18.927095+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:35:18.927095+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29446144,"num_objects":7189,"num_object_clones":0,"num_object_copies":14378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3502,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11243,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":498},{"shards":"2","objects":3689}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086228+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086228+0000","last_peered":"2024-09-10T20:34:27.086228+0000","last_clean":"2024-09-10T20:34:27.086228+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086228+0000","last_undegraded":"2024-09-10T20:34:27.086228+0000","last_fullsized":"2024-09-10T20:34:27.086228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13451","reported_seq":13643,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:18.335203+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:35:18.335203+0000","last_peered":"2024-09-10T20:35:18.335203+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:35:18.335203+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"39'3400","ondisk_log_start":"39'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29815750,"num_objects":7281,"num_object_clones":0,"num_object_copies":14562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3577,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13449,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3706},{"shards":"0,2","objects":533}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13366","reported_seq":13552,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:18.947158+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:35:18.947158+0000","last_peered":"2024-09-10T20:35:18.947158+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:35:18.947158+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"39'3300","ondisk_log_start":"39'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29487104,"num_objects":7199,"num_object_clones":0,"num_object_copies":14398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11194,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":552},{"shards":"2","objects":3593}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13434","reported_seq":16555,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:18.963834+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:35:18.963834+0000","last_peered":"2024-09-10T20:35:18.963834+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:35:18.963834+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"39'3400","ondisk_log_start":"39'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29614080,"num_objects":7230,"num_object_clones":0,"num_object_copies":14460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1628,"num_objects_misplaced":491,"num_objects_unfound":0,"num_objects_dirty":7230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11220,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1493,"num_bytes_recovered":6074368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":491},{"shards":"2","objects":5111}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13539","reported_seq":13771,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:18.287151+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:35:18.287151+0000","last_peered":"2024-09-10T20:35:18.287151+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:35:18.287151+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"39'3500","ondisk_log_start":"39'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29863936,"num_objects":7291,"num_object_clones":0,"num_object_copies":14582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3603,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10262,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3689},{"shards":"0,2","objects":525}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13299","reported_seq":13524,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:18.273067+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:35:18.273067+0000","last_peered":"2024-09-10T20:35:18.273067+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:35:18.273067+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29839360,"num_objects":7285,"num_object_clones":0,"num_object_copies":14570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3494,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9993,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3794},{"shards":"0,2","objects":529}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086748+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086748+0000","last_peered":"2024-09-10T20:34:27.086748+0000","last_clean":"2024-09-10T20:34:27.086748+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086748+0000","last_undegraded":"2024-09-10T20:34:27.086748+0000","last_fullsized":"2024-09-10T20:34:27.086748+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13679","reported_seq":13907,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:18.307869+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:35:18.307869+0000","last_peered":"2024-09-10T20:35:18.307869+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:35:18.307869+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"39'3600","ondisk_log_start":"39'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29929472,"num_objects":7307,"num_object_clones":0,"num_object_copies":14614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3669,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10506,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3640},{"shards":"0,2","objects":529}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13250","reported_seq":20372,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:19.634974+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:35:19.634974+0000","last_peered":"2024-09-10T20:35:19.634974+0000","last_clean":"2024-09-10T20:35:19.634974+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:35:19.634974+0000","last_undegraded":"2024-09-10T20:35:19.634974+0000","last_fullsized":"2024-09-10T20:35:19.634974+0000","mapping_epoch":46,"log_start":"39'3200","ondisk_log_start":"39'3200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29762019,"num_objects":7267,"num_object_clones":0,"num_object_copies":14534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9857,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":476280257,"num_objects":116283,"num_object_clones":0,"num_object_copies":232566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":491,"num_objects_unfound":0,"num_objects_dirty":116283,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":179134,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8959,"num_bytes_recovered":35041280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1107689472,"data_stored":1107612681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161038,"ondisk_log_size":161038,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182481,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":783400,"kb_used_data":408856,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93588440,"statfs":{"total":96636764160,"available":95834562560,"internally_reserved":0,"allocated":418668544,"data_stored":418416251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":383512234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,3,0,0,0,0,7],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986253,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":651172,"kb_used_data":348756,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93720668,"statfs":{"total":96636764160,"available":95969964032,"internally_reserved":0,"allocated":357126144,"data_stored":356873856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309653162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659200,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":633948,"kb_used_data":327368,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93737892,"statfs":{"total":96636764160,"available":95987601408,"internally_reserved":0,"allocated":335224832,"data_stored":334910659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":313913006},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":417210368,"data_stored":417206755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":334794752,"data_stored":334729259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":355684352,"data_stored":355676667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:35:25.477 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:35:26.018 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:26.019 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:35:26.042 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2024-09-10T20:35:24.958056+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":466937300,"num_objects":114004,"num_object_clones":0,"num_object_copies":228008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33689,"num_objects_misplaced":318,"num_objects_unfound":0,"num_objects_dirty":114004,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":181417,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9144,"num_bytes_recovered":35749907,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160822,"ondisk_log_size":160822,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2089996,"kb_used_data":1071764,"kb_used_omap":12,"kb_used_meta":1018163,"kb_avail":281025524,"statfs":{"total":289910292480,"available":287770136576,"internally_reserved":0,"allocated":1097486336,"data_stored":1096667582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1042598914},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,5,11,5,6,1,0,0,0,0,7],"upper_bound":8192},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27983872,"num_objects":-6832,"num_object_clones":0,"num_object_copies":-13664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-24,"num_objects_misplaced":-460,"num_objects_unfound":0,"num_objects_dirty":-6832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6832,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":484,"num_bytes_recovered":1884160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004332"},"pg_stats":[{"pgid":"2.f","version":"47'13527","reported_seq":13730,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:23.342898+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:35:23.342898+0000","last_peered":"2024-09-10T20:35:23.342898+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:35:23.342898+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"39'3500","ondisk_log_start":"39'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29208576,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3548,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13527,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3583},{"shards":"0,2","objects":409}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13778","reported_seq":13893,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:23.926383+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:35:23.926383+0000","last_peered":"2024-09-10T20:35:23.926383+0000","last_clean":"2024-09-10T20:35:23.926383+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:35:23.926383+0000","last_undegraded":"2024-09-10T20:35:23.926383+0000","last_fullsized":"2024-09-10T20:35:23.926383+0000","mapping_epoch":35,"log_start":"43'3700","ondisk_log_start":"43'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29343744,"num_objects":7164,"num_object_clones":0,"num_object_copies":14328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11964,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13615","reported_seq":20968,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:24.743723+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:35:24.743723+0000","last_peered":"2024-09-10T20:35:24.743723+0000","last_clean":"2024-09-10T20:35:24.743723+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:35:24.743723+0000","last_undegraded":"2024-09-10T20:35:24.743723+0000","last_fullsized":"2024-09-10T20:35:24.743723+0000","mapping_epoch":44,"log_start":"40'3600","ondisk_log_start":"40'3600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29081624,"num_objects":7101,"num_object_clones":0,"num_object_copies":14202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7101,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11743,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'13529","reported_seq":13614,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:23.978834+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:35:23.978834+0000","last_peered":"2024-09-10T20:35:23.978834+0000","last_clean":"2024-09-10T20:35:23.978834+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:35:23.978834+0000","last_undegraded":"2024-09-10T20:35:23.978834+0000","last_fullsized":"2024-09-10T20:35:23.978834+0000","mapping_epoch":35,"log_start":"40'3500","ondisk_log_start":"40'3500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28979200,"num_objects":7075,"num_object_clones":0,"num_object_copies":14150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13529,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13747","reported_seq":13878,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:23.328169+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:35:23.328169+0000","last_peered":"2024-09-10T20:35:23.328169+0000","last_clean":"2024-09-10T20:35:23.328169+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:35:23.328169+0000","last_undegraded":"2024-09-10T20:35:23.328169+0000","last_fullsized":"2024-09-10T20:35:23.328169+0000","mapping_epoch":34,"log_start":"43'3700","ondisk_log_start":"43'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29990912,"num_objects":7322,"num_object_clones":0,"num_object_copies":14644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10743,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13442","reported_seq":13674,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:23.347923+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:35:23.347923+0000","last_peered":"2024-09-10T20:35:23.347923+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:35:23.347923+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"39'3400","ondisk_log_start":"39'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28790784,"num_objects":7029,"num_object_clones":0,"num_object_copies":14058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3570,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10306,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3461},{"shards":"0,2","objects":410}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'13441","reported_seq":13706,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:23.330694+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:35:23.330694+0000","last_peered":"2024-09-10T20:35:23.330694+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:35:23.330694+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"39'3400","ondisk_log_start":"39'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29356032,"num_objects":7167,"num_object_clones":0,"num_object_copies":14334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3496,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10036,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3673},{"shards":"0,2","objects":421}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13610","reported_seq":13739,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:23.255495+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:35:23.255495+0000","last_peered":"2024-09-10T20:35:23.255495+0000","last_clean":"2024-09-10T20:35:23.255495+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:35:23.255495+0000","last_undegraded":"2024-09-10T20:35:23.255495+0000","last_fullsized":"2024-09-10T20:35:23.255495+0000","mapping_epoch":34,"log_start":"40'3600","ondisk_log_start":"40'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29188096,"num_objects":7126,"num_object_clones":0,"num_object_copies":14252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10681,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13390","reported_seq":13585,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:23.914460+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:35:23.914460+0000","last_peered":"2024-09-10T20:35:23.914460+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:35:23.914460+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"39'3300","ondisk_log_start":"39'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28872704,"num_objects":7049,"num_object_clones":0,"num_object_copies":14098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11383,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":358},{"shards":"2","objects":3552}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086228+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086228+0000","last_peered":"2024-09-10T20:34:27.086228+0000","last_clean":"2024-09-10T20:34:27.086228+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086228+0000","last_undegraded":"2024-09-10T20:34:27.086228+0000","last_fullsized":"2024-09-10T20:34:27.086228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13572","reported_seq":13765,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:23.263644+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:35:23.263644+0000","last_peered":"2024-09-10T20:35:23.263644+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:35:23.263644+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"39'3500","ondisk_log_start":"39'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29320134,"num_objects":7160,"num_object_clones":0,"num_object_copies":14320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3577,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13570,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3585},{"shards":"0,2","objects":412}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13516","reported_seq":13703,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:23.931227+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:35:23.931227+0000","last_peered":"2024-09-10T20:35:23.931227+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:35:23.931227+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"39'3500","ondisk_log_start":"39'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28872704,"num_objects":7049,"num_object_clones":0,"num_object_copies":14098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11344,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":402},{"shards":"2","objects":3443}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13577","reported_seq":17044,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:23.973923+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:35:23.973923+0000","last_peered":"2024-09-10T20:35:23.973923+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:35:23.973923+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"39'3500","ondisk_log_start":"39'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29028352,"num_objects":7087,"num_object_clones":0,"num_object_copies":14174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1628,"num_objects_misplaced":318,"num_objects_unfound":0,"num_objects_dirty":7087,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11363,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1666,"num_bytes_recovered":6782976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":318},{"shards":"2","objects":5141}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13689","reported_seq":13927,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:23.308321+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:35:23.308321+0000","last_peered":"2024-09-10T20:35:23.308321+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:35:23.308321+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"40'3600","ondisk_log_start":"40'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29249536,"num_objects":7141,"num_object_clones":0,"num_object_copies":14282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3600,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7141,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10412,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3542},{"shards":"0,2","objects":375}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13452","reported_seq":13677,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:23.303442+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:35:23.303442+0000","last_peered":"2024-09-10T20:35:23.303442+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:35:23.303442+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"39'3400","ondisk_log_start":"39'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29212672,"num_objects":7132,"num_object_clones":0,"num_object_copies":14264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3494,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10146,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3641},{"shards":"0,2","objects":376}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086748+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086748+0000","last_peered":"2024-09-10T20:34:27.086748+0000","last_clean":"2024-09-10T20:34:27.086748+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086748+0000","last_undegraded":"2024-09-10T20:34:27.086748+0000","last_fullsized":"2024-09-10T20:34:27.086748+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13835","reported_seq":14064,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:23.360109+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:35:23.360109+0000","last_peered":"2024-09-10T20:35:23.360109+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:35:23.360109+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29290496,"num_objects":7151,"num_object_clones":0,"num_object_copies":14302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3669,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7151,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10662,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3484},{"shards":"0,2","objects":373}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13399","reported_seq":20521,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:24.746428+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:35:24.746428+0000","last_peered":"2024-09-10T20:35:24.746428+0000","last_clean":"2024-09-10T20:35:24.746428+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:35:24.746428+0000","last_undegraded":"2024-09-10T20:35:24.746428+0000","last_fullsized":"2024-09-10T20:35:24.746428+0000","mapping_epoch":46,"log_start":"39'3300","ondisk_log_start":"39'3300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29151715,"num_objects":7118,"num_object_clones":0,"num_object_copies":14236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10006,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":466937281,"num_objects":114002,"num_object_clones":0,"num_object_copies":228004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33689,"num_objects_misplaced":318,"num_objects_unfound":0,"num_objects_dirty":114002,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":181415,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9142,"num_bytes_recovered":35749888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1095151616,"data_stored":1095074825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160819,"ondisk_log_size":160819,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182482,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":784976,"kb_used_data":402112,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93586864,"statfs":{"total":96636764160,"available":95832948736,"internally_reserved":0,"allocated":411762688,"data_stored":411510395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,0,4,1,0,0,0,0,7],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986254,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":649856,"kb_used_data":343276,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93721984,"statfs":{"total":96636764160,"available":95971311616,"internally_reserved":0,"allocated":351514624,"data_stored":351262336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313913002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,5,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659201,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":655164,"kb_used_data":326376,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93716676,"statfs":{"total":96636764160,"available":95965876224,"internally_reserved":0,"allocated":334209024,"data_stored":333894851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":336653998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,4,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":410304512,"data_stored":410300899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":333729792,"data_stored":333664299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":351117312,"data_stored":351109627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:35:26.043 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:35:26.576 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:26.577 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:35:26.598 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2024-09-10T20:35:24.958056+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":466937300,"num_objects":114004,"num_object_clones":0,"num_object_copies":228008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33689,"num_objects_misplaced":318,"num_objects_unfound":0,"num_objects_dirty":114004,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":181417,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9144,"num_bytes_recovered":35749907,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160822,"ondisk_log_size":160822,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2089996,"kb_used_data":1071764,"kb_used_omap":12,"kb_used_meta":1018163,"kb_avail":281025524,"statfs":{"total":289910292480,"available":287770136576,"internally_reserved":0,"allocated":1097486336,"data_stored":1096667582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1042598914},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,5,11,5,6,1,0,0,0,0,7],"upper_bound":8192},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27983872,"num_objects":-6832,"num_object_clones":0,"num_object_copies":-13664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-24,"num_objects_misplaced":-460,"num_objects_unfound":0,"num_objects_dirty":-6832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6832,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":484,"num_bytes_recovered":1884160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004332"},"pg_stats":[{"pgid":"2.f","version":"47'13527","reported_seq":13730,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:23.342898+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:35:23.342898+0000","last_peered":"2024-09-10T20:35:23.342898+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:35:23.342898+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"39'3500","ondisk_log_start":"39'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29208576,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3548,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13527,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3583},{"shards":"0,2","objects":409}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13778","reported_seq":13893,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:23.926383+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:35:23.926383+0000","last_peered":"2024-09-10T20:35:23.926383+0000","last_clean":"2024-09-10T20:35:23.926383+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:35:23.926383+0000","last_undegraded":"2024-09-10T20:35:23.926383+0000","last_fullsized":"2024-09-10T20:35:23.926383+0000","mapping_epoch":35,"log_start":"43'3700","ondisk_log_start":"43'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29343744,"num_objects":7164,"num_object_clones":0,"num_object_copies":14328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11964,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13615","reported_seq":20968,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:24.743723+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:35:24.743723+0000","last_peered":"2024-09-10T20:35:24.743723+0000","last_clean":"2024-09-10T20:35:24.743723+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:35:24.743723+0000","last_undegraded":"2024-09-10T20:35:24.743723+0000","last_fullsized":"2024-09-10T20:35:24.743723+0000","mapping_epoch":44,"log_start":"40'3600","ondisk_log_start":"40'3600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29081624,"num_objects":7101,"num_object_clones":0,"num_object_copies":14202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7101,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11743,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'13529","reported_seq":13614,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:23.978834+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:35:23.978834+0000","last_peered":"2024-09-10T20:35:23.978834+0000","last_clean":"2024-09-10T20:35:23.978834+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:35:23.978834+0000","last_undegraded":"2024-09-10T20:35:23.978834+0000","last_fullsized":"2024-09-10T20:35:23.978834+0000","mapping_epoch":35,"log_start":"40'3500","ondisk_log_start":"40'3500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28979200,"num_objects":7075,"num_object_clones":0,"num_object_copies":14150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13529,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13747","reported_seq":13878,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:23.328169+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:35:23.328169+0000","last_peered":"2024-09-10T20:35:23.328169+0000","last_clean":"2024-09-10T20:35:23.328169+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:35:23.328169+0000","last_undegraded":"2024-09-10T20:35:23.328169+0000","last_fullsized":"2024-09-10T20:35:23.328169+0000","mapping_epoch":34,"log_start":"43'3700","ondisk_log_start":"43'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29990912,"num_objects":7322,"num_object_clones":0,"num_object_copies":14644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10743,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13442","reported_seq":13674,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:23.347923+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:35:23.347923+0000","last_peered":"2024-09-10T20:35:23.347923+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:35:23.347923+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"39'3400","ondisk_log_start":"39'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28790784,"num_objects":7029,"num_object_clones":0,"num_object_copies":14058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3570,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10306,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3461},{"shards":"0,2","objects":410}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'13441","reported_seq":13706,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:23.330694+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:35:23.330694+0000","last_peered":"2024-09-10T20:35:23.330694+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:35:23.330694+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"39'3400","ondisk_log_start":"39'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29356032,"num_objects":7167,"num_object_clones":0,"num_object_copies":14334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3496,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10036,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3673},{"shards":"0,2","objects":421}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13610","reported_seq":13739,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:23.255495+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:35:23.255495+0000","last_peered":"2024-09-10T20:35:23.255495+0000","last_clean":"2024-09-10T20:35:23.255495+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:35:23.255495+0000","last_undegraded":"2024-09-10T20:35:23.255495+0000","last_fullsized":"2024-09-10T20:35:23.255495+0000","mapping_epoch":34,"log_start":"40'3600","ondisk_log_start":"40'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29188096,"num_objects":7126,"num_object_clones":0,"num_object_copies":14252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10681,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13390","reported_seq":13585,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:23.914460+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:35:23.914460+0000","last_peered":"2024-09-10T20:35:23.914460+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:35:23.914460+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"39'3300","ondisk_log_start":"39'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28872704,"num_objects":7049,"num_object_clones":0,"num_object_copies":14098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11383,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":358},{"shards":"2","objects":3552}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086228+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086228+0000","last_peered":"2024-09-10T20:34:27.086228+0000","last_clean":"2024-09-10T20:34:27.086228+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086228+0000","last_undegraded":"2024-09-10T20:34:27.086228+0000","last_fullsized":"2024-09-10T20:34:27.086228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13572","reported_seq":13765,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:23.263644+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:35:23.263644+0000","last_peered":"2024-09-10T20:35:23.263644+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:35:23.263644+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"39'3500","ondisk_log_start":"39'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29320134,"num_objects":7160,"num_object_clones":0,"num_object_copies":14320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3577,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13570,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3585},{"shards":"0,2","objects":412}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13516","reported_seq":13703,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:23.931227+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:35:23.931227+0000","last_peered":"2024-09-10T20:35:23.931227+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:35:23.931227+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"39'3500","ondisk_log_start":"39'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28872704,"num_objects":7049,"num_object_clones":0,"num_object_copies":14098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11344,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":402},{"shards":"2","objects":3443}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13577","reported_seq":17044,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:23.973923+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:35:23.973923+0000","last_peered":"2024-09-10T20:35:23.973923+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:35:23.973923+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"39'3500","ondisk_log_start":"39'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29028352,"num_objects":7087,"num_object_clones":0,"num_object_copies":14174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1628,"num_objects_misplaced":318,"num_objects_unfound":0,"num_objects_dirty":7087,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11363,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1666,"num_bytes_recovered":6782976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":318},{"shards":"2","objects":5141}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13689","reported_seq":13927,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:23.308321+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:35:23.308321+0000","last_peered":"2024-09-10T20:35:23.308321+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:35:23.308321+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"40'3600","ondisk_log_start":"40'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29249536,"num_objects":7141,"num_object_clones":0,"num_object_copies":14282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3600,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7141,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10412,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3542},{"shards":"0,2","objects":375}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13452","reported_seq":13677,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:23.303442+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:35:23.303442+0000","last_peered":"2024-09-10T20:35:23.303442+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:35:23.303442+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"39'3400","ondisk_log_start":"39'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29212672,"num_objects":7132,"num_object_clones":0,"num_object_copies":14264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3494,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10146,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3641},{"shards":"0,2","objects":376}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086748+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086748+0000","last_peered":"2024-09-10T20:34:27.086748+0000","last_clean":"2024-09-10T20:34:27.086748+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086748+0000","last_undegraded":"2024-09-10T20:34:27.086748+0000","last_fullsized":"2024-09-10T20:34:27.086748+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13835","reported_seq":14064,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:23.360109+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:35:23.360109+0000","last_peered":"2024-09-10T20:35:23.360109+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:35:23.360109+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29290496,"num_objects":7151,"num_object_clones":0,"num_object_copies":14302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3669,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7151,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10662,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3484},{"shards":"0,2","objects":373}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13399","reported_seq":20521,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:24.746428+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:35:24.746428+0000","last_peered":"2024-09-10T20:35:24.746428+0000","last_clean":"2024-09-10T20:35:24.746428+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:35:24.746428+0000","last_undegraded":"2024-09-10T20:35:24.746428+0000","last_fullsized":"2024-09-10T20:35:24.746428+0000","mapping_epoch":46,"log_start":"39'3300","ondisk_log_start":"39'3300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29151715,"num_objects":7118,"num_object_clones":0,"num_object_copies":14236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10006,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":466937281,"num_objects":114002,"num_object_clones":0,"num_object_copies":228004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33689,"num_objects_misplaced":318,"num_objects_unfound":0,"num_objects_dirty":114002,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":181415,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9142,"num_bytes_recovered":35749888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1095151616,"data_stored":1095074825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160819,"ondisk_log_size":160819,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182482,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":784976,"kb_used_data":402112,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93586864,"statfs":{"total":96636764160,"available":95832948736,"internally_reserved":0,"allocated":411762688,"data_stored":411510395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,0,4,1,0,0,0,0,7],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986254,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":649856,"kb_used_data":343276,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93721984,"statfs":{"total":96636764160,"available":95971311616,"internally_reserved":0,"allocated":351514624,"data_stored":351262336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313913002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,5,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659201,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":655164,"kb_used_data":326376,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93716676,"statfs":{"total":96636764160,"available":95965876224,"internally_reserved":0,"allocated":334209024,"data_stored":333894851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":336653998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,4,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":410304512,"data_stored":410300899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":333729792,"data_stored":333664299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":351117312,"data_stored":351109627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:35:26.599 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:35:26.599 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'13527", 'reported_seq': 13730, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:23.342898+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:35:23.342898+0000', 'last_peered': '2024-09-10T20:35:23.342898+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:35:23.342898+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "39'3500", 'ondisk_log_start': "39'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29208576, 'num_objects': 7131, 'num_object_clones': 0, 'num_object_copies': 14262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3548, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13527, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 60, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3583}, {'shards': '0,2', 'objects': 409}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:26.600 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:35:26.600 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'13442", 'reported_seq': 13674, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:23.347923+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:35:23.347923+0000', 'last_peered': '2024-09-10T20:35:23.347923+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:35:23.347923+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "39'3400", 'ondisk_log_start': "39'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28790784, 'num_objects': 7029, 'num_object_clones': 0, 'num_object_copies': 14058, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3570, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7029, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10306, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 46, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3461}, {'shards': '0,2', 'objects': 410}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:26.600 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:35:26.600 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'13441", 'reported_seq': 13706, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:23.330694+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:35:23.330694+0000', 'last_peered': '2024-09-10T20:35:23.330694+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:35:23.330694+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "39'3400", 'ondisk_log_start': "39'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29356032, 'num_objects': 7167, 'num_object_clones': 0, 'num_object_copies': 14334, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3496, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7167, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10036, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 61, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3673}, {'shards': '0,2', 'objects': 421}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:26.600 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:35:26.600 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'13390", 'reported_seq': 13585, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:23.914460+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:35:23.914460+0000', 'last_peered': '2024-09-10T20:35:23.914460+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:35:23.914460+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "39'3300", 'ondisk_log_start': "39'3300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28872704, 'num_objects': 7049, 'num_object_clones': 0, 'num_object_copies': 14098, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7049, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11383, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 358}, {'shards': '2', 'objects': 3552}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:35:26.600 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:35:26.600 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'13572", 'reported_seq': 13765, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:23.263644+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:35:23.263644+0000', 'last_peered': '2024-09-10T20:35:23.263644+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:35:23.263644+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "39'3500", 'ondisk_log_start': "39'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29320134, 'num_objects': 7160, 'num_object_clones': 0, 'num_object_copies': 14320, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3577, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7160, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13570, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 42, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3585}, {'shards': '0,2', 'objects': 412}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:26.600 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:35:26.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'13516", 'reported_seq': 13703, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:23.931227+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:35:23.931227+0000', 'last_peered': '2024-09-10T20:35:23.931227+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:35:23.931227+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "39'3500", 'ondisk_log_start': "39'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28872704, 'num_objects': 7049, 'num_object_clones': 0, 'num_object_copies': 14098, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7049, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11344, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 402}, {'shards': '2', 'objects': 3443}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:35:26.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:35:26.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'13577", 'reported_seq': 17044, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:23.973923+0000', 'last_change': '2024-09-10T20:34:25.241937+0000', 'last_active': '2024-09-10T20:35:23.973923+0000', 'last_peered': '2024-09-10T20:35:23.973923+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:35:23.973923+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "39'3500", 'ondisk_log_start': "39'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29028352, 'num_objects': 7087, 'num_object_clones': 0, 'num_object_copies': 14174, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1628, 'num_objects_misplaced': 318, 'num_objects_unfound': 0, 'num_objects_dirty': 7087, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11363, 'num_write_kb': 32484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1666, 'num_bytes_recovered': 6782976, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 318}, {'shards': '2', 'objects': 5141}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:35:26.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:35:26.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'13689", 'reported_seq': 13927, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:23.308321+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:35:23.308321+0000', 'last_peered': '2024-09-10T20:35:23.308321+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:35:23.308321+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "40'3600", 'ondisk_log_start': "40'3600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29249536, 'num_objects': 7141, 'num_object_clones': 0, 'num_object_copies': 14282, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3600, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7141, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10412, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 49, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3542}, {'shards': '0,2', 'objects': 375}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:26.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:35:26.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'13452", 'reported_seq': 13677, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:23.303442+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:35:23.303442+0000', 'last_peered': '2024-09-10T20:35:23.303442+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:35:23.303442+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "39'3400", 'ondisk_log_start': "39'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29212672, 'num_objects': 7132, 'num_object_clones': 0, 'num_object_copies': 14264, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3494, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7132, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10146, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 41, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3641}, {'shards': '0,2', 'objects': 376}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:26.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:35:26.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'13835", 'reported_seq': 14064, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:23.360109+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:35:23.360109+0000', 'last_peered': '2024-09-10T20:35:23.360109+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:35:23.360109+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "43'3800", 'ondisk_log_start': "43'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29290496, 'num_objects': 7151, 'num_object_clones': 0, 'num_object_copies': 14302, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3669, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7151, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10662, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 46, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3484}, {'shards': '0,2', 'objects': 373}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:26.602 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:35:27.277 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:27.277 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33689/228008 objects degraded (14.775%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":438,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":114004,"data_bytes":466937300,"bytes_used":2140155904,"bytes_avail":287770136576,"bytes_total":289910292480,"degraded_objects":33689,"degraded_total":228008,"degraded_ratio":0.14775358759341778,"misplaced_objects":318,"misplaced_total":228008,"misplaced_ratio":0.0013946879056875197,"recovering_objects_per_sec":40,"recovering_bytes_per_sec":156956,"recovering_keys_per_sec":0,"num_objects_recovered":484,"num_bytes_recovered":1884160,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":569},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:35:27.307 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:35:27.307 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:35:27.841 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:27.842 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:35:27.867 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":260,"stamp":"2024-09-10T20:35:26.958892+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":466937300,"num_objects":114004,"num_object_clones":0,"num_object_copies":228008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33689,"num_objects_misplaced":318,"num_objects_unfound":0,"num_objects_dirty":114004,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":181417,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9144,"num_bytes_recovered":35749907,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160822,"ondisk_log_size":160822,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2089996,"kb_used_data":1071764,"kb_used_omap":12,"kb_used_meta":1018163,"kb_avail":281025524,"statfs":{"total":289910292480,"available":287770136576,"internally_reserved":0,"allocated":1097486336,"data_stored":1096667582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1042598914},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,5,11,5,6,1,0,0,0,0,7],"upper_bound":8192},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18706432,"num_objects":-4567,"num_object_clones":0,"num_object_copies":-9134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-22,"num_objects_misplaced":-314,"num_objects_unfound":0,"num_objects_dirty":-4567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4567,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":1286144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004531"},"pg_stats":[{"pgid":"2.f","version":"47'13527","reported_seq":13730,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:23.342898+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:35:23.342898+0000","last_peered":"2024-09-10T20:35:23.342898+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:35:23.342898+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"39'3500","ondisk_log_start":"39'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29208576,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3548,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13527,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3583},{"shards":"0,2","objects":409}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13778","reported_seq":13893,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:23.926383+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:35:23.926383+0000","last_peered":"2024-09-10T20:35:23.926383+0000","last_clean":"2024-09-10T20:35:23.926383+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:35:23.926383+0000","last_undegraded":"2024-09-10T20:35:23.926383+0000","last_fullsized":"2024-09-10T20:35:23.926383+0000","mapping_epoch":35,"log_start":"43'3700","ondisk_log_start":"43'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29343744,"num_objects":7164,"num_object_clones":0,"num_object_copies":14328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11964,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13615","reported_seq":20968,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:24.743723+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:35:24.743723+0000","last_peered":"2024-09-10T20:35:24.743723+0000","last_clean":"2024-09-10T20:35:24.743723+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:35:24.743723+0000","last_undegraded":"2024-09-10T20:35:24.743723+0000","last_fullsized":"2024-09-10T20:35:24.743723+0000","mapping_epoch":44,"log_start":"40'3600","ondisk_log_start":"40'3600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29081624,"num_objects":7101,"num_object_clones":0,"num_object_copies":14202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7101,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11743,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'13529","reported_seq":13614,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:23.978834+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:35:23.978834+0000","last_peered":"2024-09-10T20:35:23.978834+0000","last_clean":"2024-09-10T20:35:23.978834+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:35:23.978834+0000","last_undegraded":"2024-09-10T20:35:23.978834+0000","last_fullsized":"2024-09-10T20:35:23.978834+0000","mapping_epoch":35,"log_start":"40'3500","ondisk_log_start":"40'3500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28979200,"num_objects":7075,"num_object_clones":0,"num_object_copies":14150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13529,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13747","reported_seq":13878,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:23.328169+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:35:23.328169+0000","last_peered":"2024-09-10T20:35:23.328169+0000","last_clean":"2024-09-10T20:35:23.328169+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:35:23.328169+0000","last_undegraded":"2024-09-10T20:35:23.328169+0000","last_fullsized":"2024-09-10T20:35:23.328169+0000","mapping_epoch":34,"log_start":"43'3700","ondisk_log_start":"43'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29990912,"num_objects":7322,"num_object_clones":0,"num_object_copies":14644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10743,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13442","reported_seq":13674,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:23.347923+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:35:23.347923+0000","last_peered":"2024-09-10T20:35:23.347923+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:35:23.347923+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"39'3400","ondisk_log_start":"39'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28790784,"num_objects":7029,"num_object_clones":0,"num_object_copies":14058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3570,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10306,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3461},{"shards":"0,2","objects":410}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'13441","reported_seq":13706,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:23.330694+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:35:23.330694+0000","last_peered":"2024-09-10T20:35:23.330694+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:35:23.330694+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"39'3400","ondisk_log_start":"39'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29356032,"num_objects":7167,"num_object_clones":0,"num_object_copies":14334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3496,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10036,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3673},{"shards":"0,2","objects":421}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13610","reported_seq":13739,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:23.255495+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:35:23.255495+0000","last_peered":"2024-09-10T20:35:23.255495+0000","last_clean":"2024-09-10T20:35:23.255495+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:35:23.255495+0000","last_undegraded":"2024-09-10T20:35:23.255495+0000","last_fullsized":"2024-09-10T20:35:23.255495+0000","mapping_epoch":34,"log_start":"40'3600","ondisk_log_start":"40'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29188096,"num_objects":7126,"num_object_clones":0,"num_object_copies":14252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10681,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13390","reported_seq":13585,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:23.914460+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:35:23.914460+0000","last_peered":"2024-09-10T20:35:23.914460+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:35:23.914460+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"39'3300","ondisk_log_start":"39'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28872704,"num_objects":7049,"num_object_clones":0,"num_object_copies":14098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11383,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":358},{"shards":"2","objects":3552}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086228+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086228+0000","last_peered":"2024-09-10T20:34:27.086228+0000","last_clean":"2024-09-10T20:34:27.086228+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086228+0000","last_undegraded":"2024-09-10T20:34:27.086228+0000","last_fullsized":"2024-09-10T20:34:27.086228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13572","reported_seq":13765,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:23.263644+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:35:23.263644+0000","last_peered":"2024-09-10T20:35:23.263644+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:35:23.263644+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"39'3500","ondisk_log_start":"39'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29320134,"num_objects":7160,"num_object_clones":0,"num_object_copies":14320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3577,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13570,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3585},{"shards":"0,2","objects":412}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13516","reported_seq":13703,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:23.931227+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:35:23.931227+0000","last_peered":"2024-09-10T20:35:23.931227+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:35:23.931227+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"39'3500","ondisk_log_start":"39'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28872704,"num_objects":7049,"num_object_clones":0,"num_object_copies":14098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11344,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":402},{"shards":"2","objects":3443}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13577","reported_seq":17044,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:23.973923+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:35:23.973923+0000","last_peered":"2024-09-10T20:35:23.973923+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:35:23.973923+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"39'3500","ondisk_log_start":"39'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29028352,"num_objects":7087,"num_object_clones":0,"num_object_copies":14174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1628,"num_objects_misplaced":318,"num_objects_unfound":0,"num_objects_dirty":7087,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11363,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1666,"num_bytes_recovered":6782976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":318},{"shards":"2","objects":5141}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13689","reported_seq":13927,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:23.308321+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:35:23.308321+0000","last_peered":"2024-09-10T20:35:23.308321+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:35:23.308321+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"40'3600","ondisk_log_start":"40'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29249536,"num_objects":7141,"num_object_clones":0,"num_object_copies":14282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3600,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7141,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10412,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3542},{"shards":"0,2","objects":375}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13452","reported_seq":13677,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:23.303442+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:35:23.303442+0000","last_peered":"2024-09-10T20:35:23.303442+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:35:23.303442+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"39'3400","ondisk_log_start":"39'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29212672,"num_objects":7132,"num_object_clones":0,"num_object_copies":14264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3494,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10146,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3641},{"shards":"0,2","objects":376}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086748+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086748+0000","last_peered":"2024-09-10T20:34:27.086748+0000","last_clean":"2024-09-10T20:34:27.086748+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086748+0000","last_undegraded":"2024-09-10T20:34:27.086748+0000","last_fullsized":"2024-09-10T20:34:27.086748+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13835","reported_seq":14064,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:23.360109+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:35:23.360109+0000","last_peered":"2024-09-10T20:35:23.360109+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:35:23.360109+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29290496,"num_objects":7151,"num_object_clones":0,"num_object_copies":14302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3669,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7151,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10662,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3484},{"shards":"0,2","objects":373}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13399","reported_seq":20521,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:24.746428+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:35:24.746428+0000","last_peered":"2024-09-10T20:35:24.746428+0000","last_clean":"2024-09-10T20:35:24.746428+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:35:24.746428+0000","last_undegraded":"2024-09-10T20:35:24.746428+0000","last_fullsized":"2024-09-10T20:35:24.746428+0000","mapping_epoch":46,"log_start":"39'3300","ondisk_log_start":"39'3300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29151715,"num_objects":7118,"num_object_clones":0,"num_object_copies":14236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10006,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":466937281,"num_objects":114002,"num_object_clones":0,"num_object_copies":228004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33689,"num_objects_misplaced":318,"num_objects_unfound":0,"num_objects_dirty":114002,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":181415,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9142,"num_bytes_recovered":35749888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1095151616,"data_stored":1095074825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160819,"ondisk_log_size":160819,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182482,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":784976,"kb_used_data":402112,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93586864,"statfs":{"total":96636764160,"available":95832948736,"internally_reserved":0,"allocated":411762688,"data_stored":411510395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,0,4,1,0,0,0,0,7],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986254,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":649856,"kb_used_data":343276,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93721984,"statfs":{"total":96636764160,"available":95971311616,"internally_reserved":0,"allocated":351514624,"data_stored":351262336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313913002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,5,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659201,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":655164,"kb_used_data":326376,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93716676,"statfs":{"total":96636764160,"available":95965876224,"internally_reserved":0,"allocated":334209024,"data_stored":333894851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":336653998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,4,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":410304512,"data_stored":410300899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":333729792,"data_stored":333664299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":351117312,"data_stored":351109627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:35:30.869 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:35:31.432 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:31.433 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:35:31.457 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":262,"stamp":"2024-09-10T20:35:30.960112+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":457590228,"num_objects":111722,"num_object_clones":0,"num_object_copies":223444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33664,"num_objects_misplaced":172,"num_objects_unfound":0,"num_objects_dirty":111722,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":183699,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9315,"num_bytes_recovered":36347923,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160904,"ondisk_log_size":160904,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2061864,"kb_used_data":1060280,"kb_used_omap":12,"kb_used_meta":1001523,"kb_avail":281053656,"statfs":{"total":289910292480,"available":287798943744,"internally_reserved":0,"allocated":1085726720,"data_stored":1084907966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1025559554},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,5,8,8,6,2,0,0,0,2],"upper_bound":4096},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22777856,"num_objects":-5561,"num_object_clones":0,"num_object_copies":-11122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-35,"num_objects_misplaced":-460,"num_objects_unfound":0,"num_objects_dirty":-5561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5561,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":495,"num_bytes_recovered":1884160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004515"},"pg_stats":[{"pgid":"2.f","version":"47'13684","reported_seq":13893,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:28.292542+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:35:28.292542+0000","last_peered":"2024-09-10T20:35:28.292542+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:35:28.292542+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"40'3600","ondisk_log_start":"40'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28565504,"num_objects":6974,"num_object_clones":0,"num_object_copies":13948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3545,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6974,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13684,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3429},{"shards":"0,2","objects":252}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13921","reported_seq":14036,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:28.897111+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:35:28.897111+0000","last_peered":"2024-09-10T20:35:28.897111+0000","last_clean":"2024-09-10T20:35:28.897111+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:35:28.897111+0000","last_undegraded":"2024-09-10T20:35:28.897111+0000","last_fullsized":"2024-09-10T20:35:28.897111+0000","mapping_epoch":35,"log_start":"43'3900","ondisk_log_start":"43'3900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28758016,"num_objects":7021,"num_object_clones":0,"num_object_copies":14042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12107,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13773","reported_seq":21126,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:29.772214+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:35:29.772214+0000","last_peered":"2024-09-10T20:35:29.772214+0000","last_clean":"2024-09-10T20:35:29.772214+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:35:29.772214+0000","last_undegraded":"2024-09-10T20:35:29.772214+0000","last_fullsized":"2024-09-10T20:35:29.772214+0000","mapping_epoch":44,"log_start":"43'3700","ondisk_log_start":"43'3700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28434456,"num_objects":6943,"num_object_clones":0,"num_object_copies":13886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6943,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11901,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'13654","reported_seq":13739,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:28.881901+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:35:28.881901+0000","last_peered":"2024-09-10T20:35:28.881901+0000","last_clean":"2024-09-10T20:35:28.881901+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:35:28.881901+0000","last_undegraded":"2024-09-10T20:35:28.881901+0000","last_fullsized":"2024-09-10T20:35:28.881901+0000","mapping_epoch":35,"log_start":"43'3600","ondisk_log_start":"43'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28467200,"num_objects":6950,"num_object_clones":0,"num_object_copies":13900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13654,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13896","reported_seq":14027,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:28.266121+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:35:28.266121+0000","last_peered":"2024-09-10T20:35:28.266121+0000","last_clean":"2024-09-10T20:35:28.266121+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:35:28.266121+0000","last_undegraded":"2024-09-10T20:35:28.266121+0000","last_fullsized":"2024-09-10T20:35:28.266121+0000","mapping_epoch":34,"log_start":"43'3800","ondisk_log_start":"43'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29380608,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10892,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13586","reported_seq":13824,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:28.331035+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:35:28.331035+0000","last_peered":"2024-09-10T20:35:28.331035+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:35:28.331035+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"39'3500","ondisk_log_start":"39'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28200960,"num_objects":6885,"num_object_clones":0,"num_object_copies":13770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3567,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10450,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3320},{"shards":"0,2","objects":266}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'13596","reported_seq":13865,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:28.347016+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:35:28.347016+0000","last_peered":"2024-09-10T20:35:28.347016+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:35:28.347016+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"40'3500","ondisk_log_start":"40'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28721152,"num_objects":7012,"num_object_clones":0,"num_object_copies":14024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3494,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10191,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3520},{"shards":"0,2","objects":266}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13753","reported_seq":13881,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:28.326417+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:35:28.326417+0000","last_peered":"2024-09-10T20:35:28.326417+0000","last_clean":"2024-09-10T20:35:28.326417+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:35:28.326417+0000","last_undegraded":"2024-09-10T20:35:28.326417+0000","last_fullsized":"2024-09-10T20:35:28.326417+0000","mapping_epoch":34,"log_start":"43'3700","ondisk_log_start":"43'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28602368,"num_objects":6983,"num_object_clones":0,"num_object_copies":13966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10824,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13520","reported_seq":13715,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:28.849025+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:35:28.849025+0000","last_peered":"2024-09-10T20:35:28.849025+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:35:28.849025+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"40'3500","ondisk_log_start":"40'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28340224,"num_objects":6919,"num_object_clones":0,"num_object_copies":13838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11513,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":228},{"shards":"2","objects":3422}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086274+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086274+0000","last_peered":"2024-09-10T20:34:27.086274+0000","last_clean":"2024-09-10T20:34:27.086274+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086274+0000","last_undegraded":"2024-09-10T20:34:27.086274+0000","last_fullsized":"2024-09-10T20:34:27.086274+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13729","reported_seq":13932,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:28.378716+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:35:28.378716+0000","last_peered":"2024-09-10T20:35:28.378716+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:35:28.378716+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'3700","ondisk_log_start":"43'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28677062,"num_objects":7003,"num_object_clones":0,"num_object_copies":14006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3572,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13727,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3433},{"shards":"0,2","objects":255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13659","reported_seq":13847,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:28.954453+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:35:28.954453+0000","last_peered":"2024-09-10T20:35:28.954453+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:35:28.954453+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"40'3600","ondisk_log_start":"40'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28286976,"num_objects":6906,"num_object_clones":0,"num_object_copies":13812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11487,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":259},{"shards":"2","objects":3300}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13720","reported_seq":17479,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:28.984982+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:35:28.984982+0000","last_peered":"2024-09-10T20:35:28.984982+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:35:28.984982+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"43'3700","ondisk_log_start":"43'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28442624,"num_objects":6944,"num_object_clones":0,"num_object_copies":13888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1628,"num_objects_misplaced":172,"num_objects_unfound":0,"num_objects_dirty":6944,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11506,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1812,"num_bytes_recovered":7380992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":172},{"shards":"2","objects":5144}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13832","reported_seq":14074,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:28.293080+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:35:28.293080+0000","last_peered":"2024-09-10T20:35:28.293080+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:35:28.293080+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28663808,"num_objects":6998,"num_object_clones":0,"num_object_copies":13996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3598,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10555,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3401},{"shards":"0,2","objects":232}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13586","reported_seq":13817,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:28.268808+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:35:28.268808+0000","last_peered":"2024-09-10T20:35:28.268808+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:35:28.268808+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"40'3500","ondisk_log_start":"40'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28663808,"num_objects":6998,"num_object_clones":0,"num_object_copies":13996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3491,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10280,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3510},{"shards":"0,2","objects":242}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086784+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086784+0000","last_peered":"2024-09-10T20:34:27.086784+0000","last_clean":"2024-09-10T20:34:27.086784+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086784+0000","last_undegraded":"2024-09-10T20:34:27.086784+0000","last_fullsized":"2024-09-10T20:34:27.086784+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13967","reported_seq":14210,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:28.281666+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:35:28.281666+0000","last_peered":"2024-09-10T20:35:28.281666+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:35:28.281666+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'3900","ondisk_log_start":"43'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28749824,"num_objects":7019,"num_object_clones":0,"num_object_copies":14038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3662,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10794,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3359},{"shards":"0,2","objects":241}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13525","reported_seq":20647,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:29.771849+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:35:29.771849+0000","last_peered":"2024-09-10T20:35:29.771849+0000","last_clean":"2024-09-10T20:35:29.771849+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:35:29.771849+0000","last_undegraded":"2024-09-10T20:35:29.771849+0000","last_fullsized":"2024-09-10T20:35:29.771849+0000","mapping_epoch":46,"log_start":"43'3500","ondisk_log_start":"43'3500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28635619,"num_objects":6992,"num_object_clones":0,"num_object_copies":13984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6992,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10132,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":457590209,"num_objects":111720,"num_object_clones":0,"num_object_copies":223440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33664,"num_objects_misplaced":172,"num_objects_unfound":0,"num_objects_dirty":111720,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":183697,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9313,"num_bytes_recovered":36347904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1082310656,"data_stored":1082233865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160901,"ondisk_log_size":160901,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182483,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":765664,"kb_used_data":395280,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93606176,"statfs":{"total":96636764160,"available":95852724224,"internally_reserved":0,"allocated":404766720,"data_stored":404514427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":379252394},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,4,5,2,0,0,0,2],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986255,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":650448,"kb_used_data":339712,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93721392,"statfs":{"total":96636764160,"available":95970705408,"internally_reserved":0,"allocated":347865088,"data_stored":347612800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318172842},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659202,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":645752,"kb_used_data":325288,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93726088,"statfs":{"total":96636764160,"available":95975514112,"internally_reserved":0,"allocated":333094912,"data_stored":332780739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":328134318},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,4,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":403234816,"data_stored":403231203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":332603392,"data_stored":332537899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":346472448,"data_stored":346464763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:35:31.458 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:35:32.013 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:32.013 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:35:32.039 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":262,"stamp":"2024-09-10T20:35:30.960112+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":457590228,"num_objects":111722,"num_object_clones":0,"num_object_copies":223444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33664,"num_objects_misplaced":172,"num_objects_unfound":0,"num_objects_dirty":111722,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":183699,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9315,"num_bytes_recovered":36347923,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160904,"ondisk_log_size":160904,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2061864,"kb_used_data":1060280,"kb_used_omap":12,"kb_used_meta":1001523,"kb_avail":281053656,"statfs":{"total":289910292480,"available":287798943744,"internally_reserved":0,"allocated":1085726720,"data_stored":1084907966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1025559554},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,5,8,8,6,2,0,0,0,2],"upper_bound":4096},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22777856,"num_objects":-5561,"num_object_clones":0,"num_object_copies":-11122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-35,"num_objects_misplaced":-460,"num_objects_unfound":0,"num_objects_dirty":-5561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5561,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":495,"num_bytes_recovered":1884160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004515"},"pg_stats":[{"pgid":"2.f","version":"47'13684","reported_seq":13893,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:28.292542+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:35:28.292542+0000","last_peered":"2024-09-10T20:35:28.292542+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:35:28.292542+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"40'3600","ondisk_log_start":"40'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28565504,"num_objects":6974,"num_object_clones":0,"num_object_copies":13948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3545,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6974,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13684,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3429},{"shards":"0,2","objects":252}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13921","reported_seq":14036,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:28.897111+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:35:28.897111+0000","last_peered":"2024-09-10T20:35:28.897111+0000","last_clean":"2024-09-10T20:35:28.897111+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:35:28.897111+0000","last_undegraded":"2024-09-10T20:35:28.897111+0000","last_fullsized":"2024-09-10T20:35:28.897111+0000","mapping_epoch":35,"log_start":"43'3900","ondisk_log_start":"43'3900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28758016,"num_objects":7021,"num_object_clones":0,"num_object_copies":14042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12107,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13773","reported_seq":21126,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:29.772214+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:35:29.772214+0000","last_peered":"2024-09-10T20:35:29.772214+0000","last_clean":"2024-09-10T20:35:29.772214+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:35:29.772214+0000","last_undegraded":"2024-09-10T20:35:29.772214+0000","last_fullsized":"2024-09-10T20:35:29.772214+0000","mapping_epoch":44,"log_start":"43'3700","ondisk_log_start":"43'3700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28434456,"num_objects":6943,"num_object_clones":0,"num_object_copies":13886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6943,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11901,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'13654","reported_seq":13739,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:28.881901+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:35:28.881901+0000","last_peered":"2024-09-10T20:35:28.881901+0000","last_clean":"2024-09-10T20:35:28.881901+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:35:28.881901+0000","last_undegraded":"2024-09-10T20:35:28.881901+0000","last_fullsized":"2024-09-10T20:35:28.881901+0000","mapping_epoch":35,"log_start":"43'3600","ondisk_log_start":"43'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28467200,"num_objects":6950,"num_object_clones":0,"num_object_copies":13900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13654,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13896","reported_seq":14027,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:28.266121+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:35:28.266121+0000","last_peered":"2024-09-10T20:35:28.266121+0000","last_clean":"2024-09-10T20:35:28.266121+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:35:28.266121+0000","last_undegraded":"2024-09-10T20:35:28.266121+0000","last_fullsized":"2024-09-10T20:35:28.266121+0000","mapping_epoch":34,"log_start":"43'3800","ondisk_log_start":"43'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29380608,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10892,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13586","reported_seq":13824,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:28.331035+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:35:28.331035+0000","last_peered":"2024-09-10T20:35:28.331035+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:35:28.331035+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"39'3500","ondisk_log_start":"39'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28200960,"num_objects":6885,"num_object_clones":0,"num_object_copies":13770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3567,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10450,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3320},{"shards":"0,2","objects":266}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'13596","reported_seq":13865,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:28.347016+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:35:28.347016+0000","last_peered":"2024-09-10T20:35:28.347016+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:35:28.347016+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"40'3500","ondisk_log_start":"40'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28721152,"num_objects":7012,"num_object_clones":0,"num_object_copies":14024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3494,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10191,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3520},{"shards":"0,2","objects":266}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13753","reported_seq":13881,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:28.326417+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:35:28.326417+0000","last_peered":"2024-09-10T20:35:28.326417+0000","last_clean":"2024-09-10T20:35:28.326417+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:35:28.326417+0000","last_undegraded":"2024-09-10T20:35:28.326417+0000","last_fullsized":"2024-09-10T20:35:28.326417+0000","mapping_epoch":34,"log_start":"43'3700","ondisk_log_start":"43'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28602368,"num_objects":6983,"num_object_clones":0,"num_object_copies":13966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10824,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13520","reported_seq":13715,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:28.849025+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:35:28.849025+0000","last_peered":"2024-09-10T20:35:28.849025+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:35:28.849025+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"40'3500","ondisk_log_start":"40'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28340224,"num_objects":6919,"num_object_clones":0,"num_object_copies":13838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11513,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":228},{"shards":"2","objects":3422}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086274+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086274+0000","last_peered":"2024-09-10T20:34:27.086274+0000","last_clean":"2024-09-10T20:34:27.086274+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086274+0000","last_undegraded":"2024-09-10T20:34:27.086274+0000","last_fullsized":"2024-09-10T20:34:27.086274+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13729","reported_seq":13932,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:28.378716+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:35:28.378716+0000","last_peered":"2024-09-10T20:35:28.378716+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:35:28.378716+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'3700","ondisk_log_start":"43'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28677062,"num_objects":7003,"num_object_clones":0,"num_object_copies":14006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3572,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13727,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3433},{"shards":"0,2","objects":255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13659","reported_seq":13847,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:28.954453+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:35:28.954453+0000","last_peered":"2024-09-10T20:35:28.954453+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:35:28.954453+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"40'3600","ondisk_log_start":"40'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28286976,"num_objects":6906,"num_object_clones":0,"num_object_copies":13812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11487,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":259},{"shards":"2","objects":3300}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13720","reported_seq":17479,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:28.984982+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:35:28.984982+0000","last_peered":"2024-09-10T20:35:28.984982+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:35:28.984982+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"43'3700","ondisk_log_start":"43'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28442624,"num_objects":6944,"num_object_clones":0,"num_object_copies":13888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1628,"num_objects_misplaced":172,"num_objects_unfound":0,"num_objects_dirty":6944,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11506,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1812,"num_bytes_recovered":7380992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":172},{"shards":"2","objects":5144}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13832","reported_seq":14074,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:28.293080+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:35:28.293080+0000","last_peered":"2024-09-10T20:35:28.293080+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:35:28.293080+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28663808,"num_objects":6998,"num_object_clones":0,"num_object_copies":13996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3598,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10555,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3401},{"shards":"0,2","objects":232}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13586","reported_seq":13817,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:28.268808+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:35:28.268808+0000","last_peered":"2024-09-10T20:35:28.268808+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:35:28.268808+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"40'3500","ondisk_log_start":"40'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28663808,"num_objects":6998,"num_object_clones":0,"num_object_copies":13996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3491,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10280,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3510},{"shards":"0,2","objects":242}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086784+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086784+0000","last_peered":"2024-09-10T20:34:27.086784+0000","last_clean":"2024-09-10T20:34:27.086784+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086784+0000","last_undegraded":"2024-09-10T20:34:27.086784+0000","last_fullsized":"2024-09-10T20:34:27.086784+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13967","reported_seq":14210,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:28.281666+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:35:28.281666+0000","last_peered":"2024-09-10T20:35:28.281666+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:35:28.281666+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'3900","ondisk_log_start":"43'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28749824,"num_objects":7019,"num_object_clones":0,"num_object_copies":14038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3662,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10794,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3359},{"shards":"0,2","objects":241}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13525","reported_seq":20647,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:29.771849+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:35:29.771849+0000","last_peered":"2024-09-10T20:35:29.771849+0000","last_clean":"2024-09-10T20:35:29.771849+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:35:29.771849+0000","last_undegraded":"2024-09-10T20:35:29.771849+0000","last_fullsized":"2024-09-10T20:35:29.771849+0000","mapping_epoch":46,"log_start":"43'3500","ondisk_log_start":"43'3500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28635619,"num_objects":6992,"num_object_clones":0,"num_object_copies":13984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6992,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10132,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":457590209,"num_objects":111720,"num_object_clones":0,"num_object_copies":223440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33664,"num_objects_misplaced":172,"num_objects_unfound":0,"num_objects_dirty":111720,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":183697,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9313,"num_bytes_recovered":36347904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1082310656,"data_stored":1082233865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160901,"ondisk_log_size":160901,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182483,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":765664,"kb_used_data":395280,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93606176,"statfs":{"total":96636764160,"available":95852724224,"internally_reserved":0,"allocated":404766720,"data_stored":404514427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":379252394},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,4,5,2,0,0,0,2],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986255,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":650448,"kb_used_data":339712,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93721392,"statfs":{"total":96636764160,"available":95970705408,"internally_reserved":0,"allocated":347865088,"data_stored":347612800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318172842},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659202,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":645752,"kb_used_data":325288,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93726088,"statfs":{"total":96636764160,"available":95975514112,"internally_reserved":0,"allocated":333094912,"data_stored":332780739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":328134318},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,4,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":403234816,"data_stored":403231203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":332603392,"data_stored":332537899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":346472448,"data_stored":346464763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:35:32.040 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:35:32.040 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'13684", 'reported_seq': 13893, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:28.292542+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:35:28.292542+0000', 'last_peered': '2024-09-10T20:35:28.292542+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:35:28.292542+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "40'3600", 'ondisk_log_start': "40'3600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28565504, 'num_objects': 6974, 'num_object_clones': 0, 'num_object_copies': 13948, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3545, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6974, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13684, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 63, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3429}, {'shards': '0,2', 'objects': 252}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:32.040 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:35:32.040 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'13586", 'reported_seq': 13824, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:28.331035+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:35:28.331035+0000', 'last_peered': '2024-09-10T20:35:28.331035+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:35:28.331035+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "39'3500", 'ondisk_log_start': "39'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28200960, 'num_objects': 6885, 'num_object_clones': 0, 'num_object_copies': 13770, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3567, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6885, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10450, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 49, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3320}, {'shards': '0,2', 'objects': 266}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:32.041 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:35:32.041 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'13596", 'reported_seq': 13865, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:28.347016+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:35:28.347016+0000', 'last_peered': '2024-09-10T20:35:28.347016+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:35:28.347016+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "40'3500", 'ondisk_log_start': "40'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28721152, 'num_objects': 7012, 'num_object_clones': 0, 'num_object_copies': 14024, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3494, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7012, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10191, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 63, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3520}, {'shards': '0,2', 'objects': 266}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:32.041 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:35:32.041 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'13520", 'reported_seq': 13715, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:28.849025+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:35:28.849025+0000', 'last_peered': '2024-09-10T20:35:28.849025+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:35:28.849025+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "40'3500", 'ondisk_log_start': "40'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28340224, 'num_objects': 6919, 'num_object_clones': 0, 'num_object_copies': 13838, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6919, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11513, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 228}, {'shards': '2', 'objects': 3422}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:35:32.041 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:35:32.041 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'13729", 'reported_seq': 13932, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:28.378716+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:35:28.378716+0000', 'last_peered': '2024-09-10T20:35:28.378716+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:35:28.378716+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "43'3700", 'ondisk_log_start': "43'3700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28677062, 'num_objects': 7003, 'num_object_clones': 0, 'num_object_copies': 14006, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3572, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7003, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13727, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 47, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3433}, {'shards': '0,2', 'objects': 255}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:32.041 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:35:32.041 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'13659", 'reported_seq': 13847, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:28.954453+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:35:28.954453+0000', 'last_peered': '2024-09-10T20:35:28.954453+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:35:28.954453+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "40'3600", 'ondisk_log_start': "40'3600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28286976, 'num_objects': 6906, 'num_object_clones': 0, 'num_object_copies': 13812, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6906, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11487, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 259}, {'shards': '2', 'objects': 3300}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:35:32.041 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:35:32.041 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'13720", 'reported_seq': 17479, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:28.984982+0000', 'last_change': '2024-09-10T20:34:25.241937+0000', 'last_active': '2024-09-10T20:35:28.984982+0000', 'last_peered': '2024-09-10T20:35:28.984982+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:35:28.984982+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "43'3700", 'ondisk_log_start': "43'3700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28442624, 'num_objects': 6944, 'num_object_clones': 0, 'num_object_copies': 13888, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1628, 'num_objects_misplaced': 172, 'num_objects_unfound': 0, 'num_objects_dirty': 6944, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11506, 'num_write_kb': 32484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1812, 'num_bytes_recovered': 7380992, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 172}, {'shards': '2', 'objects': 5144}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:35:32.042 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:35:32.042 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'13832", 'reported_seq': 14074, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:28.293080+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:35:28.293080+0000', 'last_peered': '2024-09-10T20:35:28.293080+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:35:28.293080+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "43'3800", 'ondisk_log_start': "43'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28663808, 'num_objects': 6998, 'num_object_clones': 0, 'num_object_copies': 13996, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3598, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6998, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10555, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3401}, {'shards': '0,2', 'objects': 232}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:32.042 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:35:32.042 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'13586", 'reported_seq': 13817, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:28.268808+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:35:28.268808+0000', 'last_peered': '2024-09-10T20:35:28.268808+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:35:28.268808+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "40'3500", 'ondisk_log_start': "40'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28663808, 'num_objects': 6998, 'num_object_clones': 0, 'num_object_copies': 13996, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3491, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6998, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10280, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3510}, {'shards': '0,2', 'objects': 242}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:32.042 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:35:32.042 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'13967", 'reported_seq': 14210, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:28.281666+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:35:28.281666+0000', 'last_peered': '2024-09-10T20:35:28.281666+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:35:28.281666+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "43'3900", 'ondisk_log_start': "43'3900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28749824, 'num_objects': 7019, 'num_object_clones': 0, 'num_object_copies': 14038, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3662, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7019, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10794, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3359}, {'shards': '0,2', 'objects': 241}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:32.042 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:35:32.718 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:32.718 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33664/223444 objects degraded (15.066%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":444,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":111722,"data_bytes":457590228,"bytes_used":2111348736,"bytes_avail":287798943744,"bytes_total":289910292480,"degraded_objects":33664,"degraded_total":223444,"degraded_ratio":0.15065967311720163,"misplaced_objects":172,"misplaced_total":223444,"misplaced_ratio":0.00076976781654463755,"recovering_objects_per_sec":41,"recovering_bytes_per_sec":156954,"recovering_keys_per_sec":0,"num_objects_recovered":495,"num_bytes_recovered":1884160,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":463},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:35:32.738 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:35:32.739 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:35:33.236 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:33.238 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:35:33.259 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":263,"stamp":"2024-09-10T20:35:32.960911+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":457590228,"num_objects":111722,"num_object_clones":0,"num_object_copies":223444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33664,"num_objects_misplaced":172,"num_objects_unfound":0,"num_objects_dirty":111722,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":183699,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9315,"num_bytes_recovered":36347923,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160904,"ondisk_log_size":160904,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2061864,"kb_used_data":1060280,"kb_used_omap":12,"kb_used_meta":1001523,"kb_avail":281053656,"statfs":{"total":289910292480,"available":287798943744,"internally_reserved":0,"allocated":1085726720,"data_stored":1084907966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1025559554},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,5,8,8,6,2,0,0,0,2],"upper_bound":4096},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18690048,"num_objects":-4563,"num_object_clones":0,"num_object_copies":-9126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-35,"num_objects_misplaced":-319,"num_objects_unfound":0,"num_objects_dirty":-4563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4563,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":354,"num_bytes_recovered":1306624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004471"},"pg_stats":[{"pgid":"2.f","version":"47'13684","reported_seq":13893,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:28.292542+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:35:28.292542+0000","last_peered":"2024-09-10T20:35:28.292542+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:35:28.292542+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"40'3600","ondisk_log_start":"40'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28565504,"num_objects":6974,"num_object_clones":0,"num_object_copies":13948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3545,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6974,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13684,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3429},{"shards":"0,2","objects":252}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13921","reported_seq":14036,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:28.897111+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:35:28.897111+0000","last_peered":"2024-09-10T20:35:28.897111+0000","last_clean":"2024-09-10T20:35:28.897111+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:35:28.897111+0000","last_undegraded":"2024-09-10T20:35:28.897111+0000","last_fullsized":"2024-09-10T20:35:28.897111+0000","mapping_epoch":35,"log_start":"43'3900","ondisk_log_start":"43'3900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28758016,"num_objects":7021,"num_object_clones":0,"num_object_copies":14042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12107,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13773","reported_seq":21126,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:29.772214+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:35:29.772214+0000","last_peered":"2024-09-10T20:35:29.772214+0000","last_clean":"2024-09-10T20:35:29.772214+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:35:29.772214+0000","last_undegraded":"2024-09-10T20:35:29.772214+0000","last_fullsized":"2024-09-10T20:35:29.772214+0000","mapping_epoch":44,"log_start":"43'3700","ondisk_log_start":"43'3700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28434456,"num_objects":6943,"num_object_clones":0,"num_object_copies":13886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6943,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11901,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'13654","reported_seq":13739,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:28.881901+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:35:28.881901+0000","last_peered":"2024-09-10T20:35:28.881901+0000","last_clean":"2024-09-10T20:35:28.881901+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:35:28.881901+0000","last_undegraded":"2024-09-10T20:35:28.881901+0000","last_fullsized":"2024-09-10T20:35:28.881901+0000","mapping_epoch":35,"log_start":"43'3600","ondisk_log_start":"43'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28467200,"num_objects":6950,"num_object_clones":0,"num_object_copies":13900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13654,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13896","reported_seq":14027,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:28.266121+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:35:28.266121+0000","last_peered":"2024-09-10T20:35:28.266121+0000","last_clean":"2024-09-10T20:35:28.266121+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:35:28.266121+0000","last_undegraded":"2024-09-10T20:35:28.266121+0000","last_fullsized":"2024-09-10T20:35:28.266121+0000","mapping_epoch":34,"log_start":"43'3800","ondisk_log_start":"43'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29380608,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10892,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13586","reported_seq":13824,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:28.331035+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:35:28.331035+0000","last_peered":"2024-09-10T20:35:28.331035+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:35:28.331035+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"39'3500","ondisk_log_start":"39'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28200960,"num_objects":6885,"num_object_clones":0,"num_object_copies":13770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3567,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10450,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3320},{"shards":"0,2","objects":266}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'13596","reported_seq":13865,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:28.347016+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:35:28.347016+0000","last_peered":"2024-09-10T20:35:28.347016+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:35:28.347016+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"40'3500","ondisk_log_start":"40'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28721152,"num_objects":7012,"num_object_clones":0,"num_object_copies":14024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3494,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10191,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3520},{"shards":"0,2","objects":266}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13753","reported_seq":13881,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:28.326417+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:35:28.326417+0000","last_peered":"2024-09-10T20:35:28.326417+0000","last_clean":"2024-09-10T20:35:28.326417+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:35:28.326417+0000","last_undegraded":"2024-09-10T20:35:28.326417+0000","last_fullsized":"2024-09-10T20:35:28.326417+0000","mapping_epoch":34,"log_start":"43'3700","ondisk_log_start":"43'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28602368,"num_objects":6983,"num_object_clones":0,"num_object_copies":13966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10824,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13520","reported_seq":13715,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:28.849025+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:35:28.849025+0000","last_peered":"2024-09-10T20:35:28.849025+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:35:28.849025+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"40'3500","ondisk_log_start":"40'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28340224,"num_objects":6919,"num_object_clones":0,"num_object_copies":13838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11513,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":228},{"shards":"2","objects":3422}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086274+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086274+0000","last_peered":"2024-09-10T20:34:27.086274+0000","last_clean":"2024-09-10T20:34:27.086274+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086274+0000","last_undegraded":"2024-09-10T20:34:27.086274+0000","last_fullsized":"2024-09-10T20:34:27.086274+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13729","reported_seq":13932,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:28.378716+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:35:28.378716+0000","last_peered":"2024-09-10T20:35:28.378716+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:35:28.378716+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'3700","ondisk_log_start":"43'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28677062,"num_objects":7003,"num_object_clones":0,"num_object_copies":14006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3572,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13727,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3433},{"shards":"0,2","objects":255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13659","reported_seq":13847,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:28.954453+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:35:28.954453+0000","last_peered":"2024-09-10T20:35:28.954453+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:35:28.954453+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"40'3600","ondisk_log_start":"40'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28286976,"num_objects":6906,"num_object_clones":0,"num_object_copies":13812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11487,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":259},{"shards":"2","objects":3300}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13720","reported_seq":17479,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:28.984982+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:35:28.984982+0000","last_peered":"2024-09-10T20:35:28.984982+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:35:28.984982+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"43'3700","ondisk_log_start":"43'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28442624,"num_objects":6944,"num_object_clones":0,"num_object_copies":13888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1628,"num_objects_misplaced":172,"num_objects_unfound":0,"num_objects_dirty":6944,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11506,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1812,"num_bytes_recovered":7380992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":172},{"shards":"2","objects":5144}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13832","reported_seq":14074,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:28.293080+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:35:28.293080+0000","last_peered":"2024-09-10T20:35:28.293080+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:35:28.293080+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28663808,"num_objects":6998,"num_object_clones":0,"num_object_copies":13996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3598,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10555,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3401},{"shards":"0,2","objects":232}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13586","reported_seq":13817,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:28.268808+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:35:28.268808+0000","last_peered":"2024-09-10T20:35:28.268808+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:35:28.268808+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"40'3500","ondisk_log_start":"40'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28663808,"num_objects":6998,"num_object_clones":0,"num_object_copies":13996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3491,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10280,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3510},{"shards":"0,2","objects":242}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086784+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086784+0000","last_peered":"2024-09-10T20:34:27.086784+0000","last_clean":"2024-09-10T20:34:27.086784+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086784+0000","last_undegraded":"2024-09-10T20:34:27.086784+0000","last_fullsized":"2024-09-10T20:34:27.086784+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13967","reported_seq":14210,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:28.281666+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:35:28.281666+0000","last_peered":"2024-09-10T20:35:28.281666+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:35:28.281666+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'3900","ondisk_log_start":"43'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28749824,"num_objects":7019,"num_object_clones":0,"num_object_copies":14038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3662,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10794,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3359},{"shards":"0,2","objects":241}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13525","reported_seq":20647,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:29.771849+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:35:29.771849+0000","last_peered":"2024-09-10T20:35:29.771849+0000","last_clean":"2024-09-10T20:35:29.771849+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:35:29.771849+0000","last_undegraded":"2024-09-10T20:35:29.771849+0000","last_fullsized":"2024-09-10T20:35:29.771849+0000","mapping_epoch":46,"log_start":"43'3500","ondisk_log_start":"43'3500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28635619,"num_objects":6992,"num_object_clones":0,"num_object_copies":13984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6992,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10132,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":457590209,"num_objects":111720,"num_object_clones":0,"num_object_copies":223440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33664,"num_objects_misplaced":172,"num_objects_unfound":0,"num_objects_dirty":111720,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":183697,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9313,"num_bytes_recovered":36347904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1082310656,"data_stored":1082233865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160901,"ondisk_log_size":160901,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182483,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":765664,"kb_used_data":395280,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93606176,"statfs":{"total":96636764160,"available":95852724224,"internally_reserved":0,"allocated":404766720,"data_stored":404514427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":379252394},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,4,5,2,0,0,0,2],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986255,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":650448,"kb_used_data":339712,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93721392,"statfs":{"total":96636764160,"available":95970705408,"internally_reserved":0,"allocated":347865088,"data_stored":347612800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318172842},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659202,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":645752,"kb_used_data":325288,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93726088,"statfs":{"total":96636764160,"available":95975514112,"internally_reserved":0,"allocated":333094912,"data_stored":332780739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":328134318},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,4,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":403234816,"data_stored":403231203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":332603392,"data_stored":332537899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":346472448,"data_stored":346464763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:35:36.262 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:35:36.820 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:36.822 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:35:36.844 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":264,"stamp":"2024-09-10T20:35:34.961688+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":448202196,"num_objects":109430,"num_object_clones":0,"num_object_copies":218860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33651,"num_objects_misplaced":27,"num_objects_unfound":0,"num_objects_dirty":109430,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":185991,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9473,"num_bytes_recovered":36941843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160896,"ondisk_log_size":160896,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2044876,"kb_used_data":1047452,"kb_used_omap":12,"kb_used_meta":997363,"kb_avail":281070644,"statfs":{"total":289910292480,"available":287816339456,"internally_reserved":0,"allocated":1072590848,"data_stored":1071772094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1021299714},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,3,9,8,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28078080,"num_objects":-6855,"num_object_clones":0,"num_object_copies":-13710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-48,"num_objects_misplaced":-464,"num_objects_unfound":0,"num_objects_dirty":-6855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6855,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":512,"num_bytes_recovered":1900544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004449"},"pg_stats":[{"pgid":"2.f","version":"47'13825","reported_seq":14038,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:33.391568+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:35:33.391568+0000","last_peered":"2024-09-10T20:35:33.391568+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:35:33.391568+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27987968,"num_objects":6833,"num_object_clones":0,"num_object_copies":13666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3543,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13825,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3290},{"shards":"0,2","objects":111}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14041","reported_seq":14156,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:34.017184+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:35:34.017184+0000","last_peered":"2024-09-10T20:35:34.017184+0000","last_clean":"2024-09-10T20:35:34.017184+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:35:34.017184+0000","last_undegraded":"2024-09-10T20:35:34.017184+0000","last_fullsized":"2024-09-10T20:35:34.017184+0000","mapping_epoch":35,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28266496,"num_objects":6901,"num_object_clones":0,"num_object_copies":13802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12227,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13911","reported_seq":21264,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:34.819431+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:35:34.819431+0000","last_peered":"2024-09-10T20:35:34.819431+0000","last_clean":"2024-09-10T20:35:34.819431+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:35:34.819431+0000","last_undegraded":"2024-09-10T20:35:34.819431+0000","last_fullsized":"2024-09-10T20:35:34.819431+0000","mapping_epoch":44,"log_start":"43'3900","ondisk_log_start":"43'3900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27869208,"num_objects":6805,"num_object_clones":0,"num_object_copies":13610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6805,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12039,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'13788","reported_seq":13873,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:34.023696+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:35:34.023696+0000","last_peered":"2024-09-10T20:35:34.023696+0000","last_clean":"2024-09-10T20:35:34.023696+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:35:34.023696+0000","last_undegraded":"2024-09-10T20:35:34.023696+0000","last_fullsized":"2024-09-10T20:35:34.023696+0000","mapping_epoch":35,"log_start":"43'3700","ondisk_log_start":"43'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27918336,"num_objects":6816,"num_object_clones":0,"num_object_copies":13632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6816,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13788,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14066","reported_seq":14197,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:33.381546+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:35:33.381546+0000","last_peered":"2024-09-10T20:35:33.381546+0000","last_clean":"2024-09-10T20:35:33.381546+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:35:33.381546+0000","last_undegraded":"2024-09-10T20:35:33.381546+0000","last_fullsized":"2024-09-10T20:35:33.381546+0000","mapping_epoch":34,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28684288,"num_objects":7003,"num_object_clones":0,"num_object_copies":14006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11062,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13747","reported_seq":13989,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:33.402902+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:35:33.402902+0000","last_peered":"2024-09-10T20:35:33.402902+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:35:33.402902+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'3700","ondisk_log_start":"43'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27541504,"num_objects":6724,"num_object_clones":0,"num_object_copies":13448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3565,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10611,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3161},{"shards":"0,2","objects":105}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'13748","reported_seq":14021,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:33.315546+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:35:33.315546+0000","last_peered":"2024-09-10T20:35:33.315546+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:35:33.315546+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'3700","ondisk_log_start":"43'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28098560,"num_objects":6860,"num_object_clones":0,"num_object_copies":13720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3492,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6860,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10343,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3370},{"shards":"0,2","objects":114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13895","reported_seq":14025,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:33.403457+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:35:33.403457+0000","last_peered":"2024-09-10T20:35:33.403457+0000","last_clean":"2024-09-10T20:35:33.403457+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:35:33.403457+0000","last_undegraded":"2024-09-10T20:35:33.403457+0000","last_fullsized":"2024-09-10T20:35:33.403457+0000","mapping_epoch":34,"log_start":"43'3800","ondisk_log_start":"43'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28020736,"num_objects":6841,"num_object_clones":0,"num_object_copies":13682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10966,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13662","reported_seq":13857,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:33.919754+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:35:33.919754+0000","last_peered":"2024-09-10T20:35:33.919754+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:35:33.919754+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'3600","ondisk_log_start":"43'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27758592,"num_objects":6777,"num_object_clones":0,"num_object_copies":13554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11655,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":86},{"shards":"2","objects":3280}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086274+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086274+0000","last_peered":"2024-09-10T20:34:27.086274+0000","last_clean":"2024-09-10T20:34:27.086274+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086274+0000","last_undegraded":"2024-09-10T20:34:27.086274+0000","last_fullsized":"2024-09-10T20:34:27.086274+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13880","reported_seq":14090,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:33.399256+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:35:33.399256+0000","last_peered":"2024-09-10T20:35:33.399256+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:35:33.399256+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28058566,"num_objects":6852,"num_object_clones":0,"num_object_copies":13704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3569,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13878,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3285},{"shards":"0,2","objects":104}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13813","reported_seq":14001,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:33.913445+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:35:33.913445+0000","last_peered":"2024-09-10T20:35:33.913445+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:35:33.913445+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27656192,"num_objects":6752,"num_object_clones":0,"num_object_copies":13504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11641,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":105},{"shards":"2","objects":3146}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13868","reported_seq":17917,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:34.002761+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:35:34.002761+0000","last_peered":"2024-09-10T20:35:34.002761+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:35:34.002761+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27836416,"num_objects":6796,"num_object_clones":0,"num_object_copies":13592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1628,"num_objects_misplaced":27,"num_objects_unfound":0,"num_objects_dirty":6796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11654,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1957,"num_bytes_recovered":7974912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":27},{"shards":"2","objects":5141}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13962","reported_seq":14206,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:33.392676+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:35:33.392676+0000","last_peered":"2024-09-10T20:35:33.392676+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:35:33.392676+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'3900","ondisk_log_start":"43'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28131328,"num_objects":6868,"num_object_clones":0,"num_object_copies":13736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3597,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10685,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3272},{"shards":"0,2","objects":102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13725","reported_seq":13963,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:33.358822+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:35:33.358822+0000","last_peered":"2024-09-10T20:35:33.358822+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:35:33.358822+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'3700","ondisk_log_start":"43'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28094464,"num_objects":6859,"num_object_clones":0,"num_object_copies":13718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10419,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3374},{"shards":"0,2","objects":103}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086784+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086784+0000","last_peered":"2024-09-10T20:34:27.086784+0000","last_clean":"2024-09-10T20:34:27.086784+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086784+0000","last_undegraded":"2024-09-10T20:34:27.086784+0000","last_fullsized":"2024-09-10T20:34:27.086784+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14095","reported_seq":14338,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:33.339393+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:35:33.339393+0000","last_peered":"2024-09-10T20:35:33.339393+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:35:33.339393+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28225536,"num_objects":6891,"num_object_clones":0,"num_object_copies":13782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3662,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10922,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3231},{"shards":"0,2","objects":113}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13667","reported_seq":20789,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:34.831982+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:35:34.831982+0000","last_peered":"2024-09-10T20:35:34.831982+0000","last_clean":"2024-09-10T20:35:34.831982+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:35:34.831982+0000","last_undegraded":"2024-09-10T20:35:34.831982+0000","last_fullsized":"2024-09-10T20:35:34.831982+0000","mapping_epoch":46,"log_start":"43'3600","ondisk_log_start":"43'3600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28053987,"num_objects":6850,"num_object_clones":0,"num_object_copies":13700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10274,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":448202177,"num_objects":109428,"num_object_clones":0,"num_object_copies":218856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33651,"num_objects_misplaced":27,"num_objects_unfound":0,"num_objects_dirty":109428,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":185989,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9471,"num_bytes_recovered":36941824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1069400064,"data_stored":1069323273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160893,"ondisk_log_size":160893,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182484,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":762908,"kb_used_data":388364,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93608932,"statfs":{"total":96636764160,"available":95855546368,"internally_reserved":0,"allocated":397684736,"data_stored":397432443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":383512234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,1,5,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986256,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":633196,"kb_used_data":334940,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93738644,"statfs":{"total":96636764160,"available":95988371456,"internally_reserved":0,"allocated":342978560,"data_stored":342726272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305393322},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659203,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":648772,"kb_used_data":324148,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93723068,"statfs":{"total":96636764160,"available":95972421632,"internally_reserved":0,"allocated":331927552,"data_stored":331613379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":332394158},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,5,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":396255232,"data_stored":396251619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":331419648,"data_stored":331354155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":341725184,"data_stored":341717499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:35:36.845 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:35:37.399 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:37.399 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:35:37.434 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":265,"stamp":"2024-09-10T20:35:36.962548+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":448202196,"num_objects":109430,"num_object_clones":0,"num_object_copies":218860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33651,"num_objects_misplaced":27,"num_objects_unfound":0,"num_objects_dirty":109430,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":185991,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9473,"num_bytes_recovered":36941843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160896,"ondisk_log_size":160896,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2044876,"kb_used_data":1047452,"kb_used_omap":12,"kb_used_meta":997363,"kb_avail":281070644,"statfs":{"total":289910292480,"available":287816339456,"internally_reserved":0,"allocated":1072590848,"data_stored":1071772094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1021299714},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,3,9,8,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18735104,"num_objects":-4574,"num_object_clones":0,"num_object_copies":-9148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-38,"num_objects_misplaced":-291,"num_objects_unfound":0,"num_objects_dirty":-4574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4574,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":329,"num_bytes_recovered":1191936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004491"},"pg_stats":[{"pgid":"2.f","version":"47'13825","reported_seq":14038,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:33.391568+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:35:33.391568+0000","last_peered":"2024-09-10T20:35:33.391568+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:35:33.391568+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27987968,"num_objects":6833,"num_object_clones":0,"num_object_copies":13666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3543,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13825,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3290},{"shards":"0,2","objects":111}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14041","reported_seq":14156,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:34.017184+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:35:34.017184+0000","last_peered":"2024-09-10T20:35:34.017184+0000","last_clean":"2024-09-10T20:35:34.017184+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:35:34.017184+0000","last_undegraded":"2024-09-10T20:35:34.017184+0000","last_fullsized":"2024-09-10T20:35:34.017184+0000","mapping_epoch":35,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28266496,"num_objects":6901,"num_object_clones":0,"num_object_copies":13802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12227,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13911","reported_seq":21264,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:34.819431+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:35:34.819431+0000","last_peered":"2024-09-10T20:35:34.819431+0000","last_clean":"2024-09-10T20:35:34.819431+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:35:34.819431+0000","last_undegraded":"2024-09-10T20:35:34.819431+0000","last_fullsized":"2024-09-10T20:35:34.819431+0000","mapping_epoch":44,"log_start":"43'3900","ondisk_log_start":"43'3900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27869208,"num_objects":6805,"num_object_clones":0,"num_object_copies":13610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6805,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12039,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'13788","reported_seq":13873,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:34.023696+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:35:34.023696+0000","last_peered":"2024-09-10T20:35:34.023696+0000","last_clean":"2024-09-10T20:35:34.023696+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:35:34.023696+0000","last_undegraded":"2024-09-10T20:35:34.023696+0000","last_fullsized":"2024-09-10T20:35:34.023696+0000","mapping_epoch":35,"log_start":"43'3700","ondisk_log_start":"43'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27918336,"num_objects":6816,"num_object_clones":0,"num_object_copies":13632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6816,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13788,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14066","reported_seq":14197,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:33.381546+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:35:33.381546+0000","last_peered":"2024-09-10T20:35:33.381546+0000","last_clean":"2024-09-10T20:35:33.381546+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:35:33.381546+0000","last_undegraded":"2024-09-10T20:35:33.381546+0000","last_fullsized":"2024-09-10T20:35:33.381546+0000","mapping_epoch":34,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28684288,"num_objects":7003,"num_object_clones":0,"num_object_copies":14006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11062,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13747","reported_seq":13989,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:33.402902+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:35:33.402902+0000","last_peered":"2024-09-10T20:35:33.402902+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:35:33.402902+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'3700","ondisk_log_start":"43'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27541504,"num_objects":6724,"num_object_clones":0,"num_object_copies":13448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3565,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10611,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3161},{"shards":"0,2","objects":105}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'13748","reported_seq":14021,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:33.315546+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:35:33.315546+0000","last_peered":"2024-09-10T20:35:33.315546+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:35:33.315546+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'3700","ondisk_log_start":"43'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28098560,"num_objects":6860,"num_object_clones":0,"num_object_copies":13720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3492,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6860,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10343,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3370},{"shards":"0,2","objects":114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13895","reported_seq":14025,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:33.403457+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:35:33.403457+0000","last_peered":"2024-09-10T20:35:33.403457+0000","last_clean":"2024-09-10T20:35:33.403457+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:35:33.403457+0000","last_undegraded":"2024-09-10T20:35:33.403457+0000","last_fullsized":"2024-09-10T20:35:33.403457+0000","mapping_epoch":34,"log_start":"43'3800","ondisk_log_start":"43'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28020736,"num_objects":6841,"num_object_clones":0,"num_object_copies":13682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10966,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13662","reported_seq":13857,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:33.919754+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:35:33.919754+0000","last_peered":"2024-09-10T20:35:33.919754+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:35:33.919754+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'3600","ondisk_log_start":"43'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27758592,"num_objects":6777,"num_object_clones":0,"num_object_copies":13554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11655,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":86},{"shards":"2","objects":3280}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086274+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086274+0000","last_peered":"2024-09-10T20:34:27.086274+0000","last_clean":"2024-09-10T20:34:27.086274+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086274+0000","last_undegraded":"2024-09-10T20:34:27.086274+0000","last_fullsized":"2024-09-10T20:34:27.086274+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13880","reported_seq":14090,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:33.399256+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:35:33.399256+0000","last_peered":"2024-09-10T20:35:33.399256+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:35:33.399256+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28058566,"num_objects":6852,"num_object_clones":0,"num_object_copies":13704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3569,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13878,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3285},{"shards":"0,2","objects":104}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13813","reported_seq":14001,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:33.913445+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:35:33.913445+0000","last_peered":"2024-09-10T20:35:33.913445+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:35:33.913445+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27656192,"num_objects":6752,"num_object_clones":0,"num_object_copies":13504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11641,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":105},{"shards":"2","objects":3146}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13868","reported_seq":17917,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:34.002761+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:35:34.002761+0000","last_peered":"2024-09-10T20:35:34.002761+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:35:34.002761+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27836416,"num_objects":6796,"num_object_clones":0,"num_object_copies":13592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1628,"num_objects_misplaced":27,"num_objects_unfound":0,"num_objects_dirty":6796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11654,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1957,"num_bytes_recovered":7974912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":27},{"shards":"2","objects":5141}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13962","reported_seq":14206,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:33.392676+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:35:33.392676+0000","last_peered":"2024-09-10T20:35:33.392676+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:35:33.392676+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'3900","ondisk_log_start":"43'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28131328,"num_objects":6868,"num_object_clones":0,"num_object_copies":13736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3597,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10685,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3272},{"shards":"0,2","objects":102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13725","reported_seq":13963,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:33.358822+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:35:33.358822+0000","last_peered":"2024-09-10T20:35:33.358822+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:35:33.358822+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'3700","ondisk_log_start":"43'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28094464,"num_objects":6859,"num_object_clones":0,"num_object_copies":13718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10419,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3374},{"shards":"0,2","objects":103}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086784+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086784+0000","last_peered":"2024-09-10T20:34:27.086784+0000","last_clean":"2024-09-10T20:34:27.086784+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086784+0000","last_undegraded":"2024-09-10T20:34:27.086784+0000","last_fullsized":"2024-09-10T20:34:27.086784+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14095","reported_seq":14338,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:33.339393+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:35:33.339393+0000","last_peered":"2024-09-10T20:35:33.339393+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:35:33.339393+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28225536,"num_objects":6891,"num_object_clones":0,"num_object_copies":13782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3662,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10922,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3231},{"shards":"0,2","objects":113}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13667","reported_seq":20789,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:34.831982+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:35:34.831982+0000","last_peered":"2024-09-10T20:35:34.831982+0000","last_clean":"2024-09-10T20:35:34.831982+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:35:34.831982+0000","last_undegraded":"2024-09-10T20:35:34.831982+0000","last_fullsized":"2024-09-10T20:35:34.831982+0000","mapping_epoch":46,"log_start":"43'3600","ondisk_log_start":"43'3600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28053987,"num_objects":6850,"num_object_clones":0,"num_object_copies":13700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10274,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":448202177,"num_objects":109428,"num_object_clones":0,"num_object_copies":218856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33651,"num_objects_misplaced":27,"num_objects_unfound":0,"num_objects_dirty":109428,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":185989,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9471,"num_bytes_recovered":36941824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1069400064,"data_stored":1069323273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160893,"ondisk_log_size":160893,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182484,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":762908,"kb_used_data":388364,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93608932,"statfs":{"total":96636764160,"available":95855546368,"internally_reserved":0,"allocated":397684736,"data_stored":397432443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":383512234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,1,5,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986256,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":633196,"kb_used_data":334940,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93738644,"statfs":{"total":96636764160,"available":95988371456,"internally_reserved":0,"allocated":342978560,"data_stored":342726272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305393322},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659203,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":648772,"kb_used_data":324148,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93723068,"statfs":{"total":96636764160,"available":95972421632,"internally_reserved":0,"allocated":331927552,"data_stored":331613379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":332394158},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,5,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":396255232,"data_stored":396251619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":331419648,"data_stored":331354155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":341725184,"data_stored":341717499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:35:37.436 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:35:37.436 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'13825", 'reported_seq': 14038, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:33.391568+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:35:33.391568+0000', 'last_peered': '2024-09-10T20:35:33.391568+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:35:33.391568+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "43'3800", 'ondisk_log_start': "43'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27987968, 'num_objects': 6833, 'num_object_clones': 0, 'num_object_copies': 13666, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3543, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6833, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13825, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 65, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3290}, {'shards': '0,2', 'objects': 111}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:37.436 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:35:37.436 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'13747", 'reported_seq': 13989, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:33.402902+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:35:33.402902+0000', 'last_peered': '2024-09-10T20:35:33.402902+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:35:33.402902+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "43'3700", 'ondisk_log_start': "43'3700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27541504, 'num_objects': 6724, 'num_object_clones': 0, 'num_object_copies': 13448, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3565, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6724, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10611, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3161}, {'shards': '0,2', 'objects': 105}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:37.436 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:35:37.436 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'13748", 'reported_seq': 14021, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:33.315546+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:35:33.315546+0000', 'last_peered': '2024-09-10T20:35:33.315546+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:35:33.315546+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "43'3700", 'ondisk_log_start': "43'3700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28098560, 'num_objects': 6860, 'num_object_clones': 0, 'num_object_copies': 13720, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3492, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6860, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10343, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 65, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3370}, {'shards': '0,2', 'objects': 114}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:37.436 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:35:37.437 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'13662", 'reported_seq': 13857, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:33.919754+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:35:33.919754+0000', 'last_peered': '2024-09-10T20:35:33.919754+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:35:33.919754+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "43'3600", 'ondisk_log_start': "43'3600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27758592, 'num_objects': 6777, 'num_object_clones': 0, 'num_object_copies': 13554, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6777, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11655, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 86}, {'shards': '2', 'objects': 3280}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:35:37.437 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:35:37.437 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'13880", 'reported_seq': 14090, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:33.399256+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:35:33.399256+0000', 'last_peered': '2024-09-10T20:35:33.399256+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:35:33.399256+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "43'3800", 'ondisk_log_start': "43'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28058566, 'num_objects': 6852, 'num_object_clones': 0, 'num_object_copies': 13704, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3569, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6852, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13878, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 50, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3285}, {'shards': '0,2', 'objects': 104}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:37.437 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:35:37.437 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'13813", 'reported_seq': 14001, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:33.913445+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:35:33.913445+0000', 'last_peered': '2024-09-10T20:35:33.913445+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:35:33.913445+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "43'3800", 'ondisk_log_start': "43'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27656192, 'num_objects': 6752, 'num_object_clones': 0, 'num_object_copies': 13504, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6752, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11641, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 105}, {'shards': '2', 'objects': 3146}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:35:37.437 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:35:37.437 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'13868", 'reported_seq': 17917, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:34.002761+0000', 'last_change': '2024-09-10T20:34:25.241937+0000', 'last_active': '2024-09-10T20:35:34.002761+0000', 'last_peered': '2024-09-10T20:35:34.002761+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:35:34.002761+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "43'3800", 'ondisk_log_start': "43'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27836416, 'num_objects': 6796, 'num_object_clones': 0, 'num_object_copies': 13592, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1628, 'num_objects_misplaced': 27, 'num_objects_unfound': 0, 'num_objects_dirty': 6796, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11654, 'num_write_kb': 32484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1957, 'num_bytes_recovered': 7974912, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 27}, {'shards': '2', 'objects': 5141}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:35:37.437 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:35:37.437 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'13962", 'reported_seq': 14206, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:33.392676+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:35:33.392676+0000', 'last_peered': '2024-09-10T20:35:33.392676+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:35:33.392676+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "43'3900", 'ondisk_log_start': "43'3900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28131328, 'num_objects': 6868, 'num_object_clones': 0, 'num_object_copies': 13736, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3597, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6868, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10685, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3272}, {'shards': '0,2', 'objects': 102}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:37.437 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:35:37.437 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'13725", 'reported_seq': 13963, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:33.358822+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:35:33.358822+0000', 'last_peered': '2024-09-10T20:35:33.358822+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:35:33.358822+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "43'3700", 'ondisk_log_start': "43'3700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28094464, 'num_objects': 6859, 'num_object_clones': 0, 'num_object_copies': 13718, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6859, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10419, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 47, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3374}, {'shards': '0,2', 'objects': 103}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:37.438 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:35:37.438 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'14095", 'reported_seq': 14338, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:33.339393+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:35:33.339393+0000', 'last_peered': '2024-09-10T20:35:33.339393+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:35:33.339393+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "43'4000", 'ondisk_log_start': "43'4000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28225536, 'num_objects': 6891, 'num_object_clones': 0, 'num_object_copies': 13782, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3662, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6891, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10922, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3231}, {'shards': '0,2', 'objects': 113}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:37.438 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:35:38.084 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:38.084 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33651/218860 objects degraded (15.376%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":449,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":109430,"data_bytes":448202196,"bytes_used":2093953024,"bytes_avail":287816339456,"bytes_total":289910292480,"degraded_objects":33651,"degraded_total":218860,"degraded_ratio":0.15375582564196291,"misplaced_objects":27,"misplaced_total":218860,"misplaced_ratio":0.00012336653568491274,"recovering_objects_per_sec":27,"recovering_bytes_per_sec":99290,"recovering_keys_per_sec":0,"num_objects_recovered":329,"num_bytes_recovered":1191936,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":381},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:35:38.113 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:35:38.113 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:35:38.663 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:38.666 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:35:38.697 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":265,"stamp":"2024-09-10T20:35:36.962548+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":448202196,"num_objects":109430,"num_object_clones":0,"num_object_copies":218860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33651,"num_objects_misplaced":27,"num_objects_unfound":0,"num_objects_dirty":109430,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":185991,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9473,"num_bytes_recovered":36941843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160896,"ondisk_log_size":160896,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2044876,"kb_used_data":1047452,"kb_used_omap":12,"kb_used_meta":997363,"kb_avail":281070644,"statfs":{"total":289910292480,"available":287816339456,"internally_reserved":0,"allocated":1072590848,"data_stored":1071772094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1021299714},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,3,9,8,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18735104,"num_objects":-4574,"num_object_clones":0,"num_object_copies":-9148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-38,"num_objects_misplaced":-291,"num_objects_unfound":0,"num_objects_dirty":-4574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4574,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":329,"num_bytes_recovered":1191936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004491"},"pg_stats":[{"pgid":"2.f","version":"47'13825","reported_seq":14038,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:33.391568+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:35:33.391568+0000","last_peered":"2024-09-10T20:35:33.391568+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:35:33.391568+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27987968,"num_objects":6833,"num_object_clones":0,"num_object_copies":13666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3543,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13825,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3290},{"shards":"0,2","objects":111}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14041","reported_seq":14156,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:34.017184+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:35:34.017184+0000","last_peered":"2024-09-10T20:35:34.017184+0000","last_clean":"2024-09-10T20:35:34.017184+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:35:34.017184+0000","last_undegraded":"2024-09-10T20:35:34.017184+0000","last_fullsized":"2024-09-10T20:35:34.017184+0000","mapping_epoch":35,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28266496,"num_objects":6901,"num_object_clones":0,"num_object_copies":13802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12227,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13911","reported_seq":21264,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:34.819431+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:35:34.819431+0000","last_peered":"2024-09-10T20:35:34.819431+0000","last_clean":"2024-09-10T20:35:34.819431+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:35:34.819431+0000","last_undegraded":"2024-09-10T20:35:34.819431+0000","last_fullsized":"2024-09-10T20:35:34.819431+0000","mapping_epoch":44,"log_start":"43'3900","ondisk_log_start":"43'3900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27869208,"num_objects":6805,"num_object_clones":0,"num_object_copies":13610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6805,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12039,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'13788","reported_seq":13873,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:34.023696+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:35:34.023696+0000","last_peered":"2024-09-10T20:35:34.023696+0000","last_clean":"2024-09-10T20:35:34.023696+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:35:34.023696+0000","last_undegraded":"2024-09-10T20:35:34.023696+0000","last_fullsized":"2024-09-10T20:35:34.023696+0000","mapping_epoch":35,"log_start":"43'3700","ondisk_log_start":"43'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27918336,"num_objects":6816,"num_object_clones":0,"num_object_copies":13632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6816,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13788,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14066","reported_seq":14197,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:33.381546+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:35:33.381546+0000","last_peered":"2024-09-10T20:35:33.381546+0000","last_clean":"2024-09-10T20:35:33.381546+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:35:33.381546+0000","last_undegraded":"2024-09-10T20:35:33.381546+0000","last_fullsized":"2024-09-10T20:35:33.381546+0000","mapping_epoch":34,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28684288,"num_objects":7003,"num_object_clones":0,"num_object_copies":14006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11062,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13747","reported_seq":13989,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:33.402902+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:35:33.402902+0000","last_peered":"2024-09-10T20:35:33.402902+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:35:33.402902+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'3700","ondisk_log_start":"43'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27541504,"num_objects":6724,"num_object_clones":0,"num_object_copies":13448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3565,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10611,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3161},{"shards":"0,2","objects":105}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'13748","reported_seq":14021,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:33.315546+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:35:33.315546+0000","last_peered":"2024-09-10T20:35:33.315546+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:35:33.315546+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'3700","ondisk_log_start":"43'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28098560,"num_objects":6860,"num_object_clones":0,"num_object_copies":13720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3492,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6860,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10343,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3370},{"shards":"0,2","objects":114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13895","reported_seq":14025,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:33.403457+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:35:33.403457+0000","last_peered":"2024-09-10T20:35:33.403457+0000","last_clean":"2024-09-10T20:35:33.403457+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:35:33.403457+0000","last_undegraded":"2024-09-10T20:35:33.403457+0000","last_fullsized":"2024-09-10T20:35:33.403457+0000","mapping_epoch":34,"log_start":"43'3800","ondisk_log_start":"43'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28020736,"num_objects":6841,"num_object_clones":0,"num_object_copies":13682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10966,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13662","reported_seq":13857,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:33.919754+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:35:33.919754+0000","last_peered":"2024-09-10T20:35:33.919754+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:35:33.919754+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'3600","ondisk_log_start":"43'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27758592,"num_objects":6777,"num_object_clones":0,"num_object_copies":13554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11655,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":86},{"shards":"2","objects":3280}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086274+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086274+0000","last_peered":"2024-09-10T20:34:27.086274+0000","last_clean":"2024-09-10T20:34:27.086274+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086274+0000","last_undegraded":"2024-09-10T20:34:27.086274+0000","last_fullsized":"2024-09-10T20:34:27.086274+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13880","reported_seq":14090,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:33.399256+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:35:33.399256+0000","last_peered":"2024-09-10T20:35:33.399256+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:35:33.399256+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28058566,"num_objects":6852,"num_object_clones":0,"num_object_copies":13704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3569,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13878,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3285},{"shards":"0,2","objects":104}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13813","reported_seq":14001,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:33.913445+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:35:33.913445+0000","last_peered":"2024-09-10T20:35:33.913445+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:35:33.913445+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27656192,"num_objects":6752,"num_object_clones":0,"num_object_copies":13504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11641,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":105},{"shards":"2","objects":3146}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13868","reported_seq":17917,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:34.002761+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:35:34.002761+0000","last_peered":"2024-09-10T20:35:34.002761+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:35:34.002761+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27836416,"num_objects":6796,"num_object_clones":0,"num_object_copies":13592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1628,"num_objects_misplaced":27,"num_objects_unfound":0,"num_objects_dirty":6796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11654,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1957,"num_bytes_recovered":7974912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":27},{"shards":"2","objects":5141}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13962","reported_seq":14206,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:33.392676+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:35:33.392676+0000","last_peered":"2024-09-10T20:35:33.392676+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:35:33.392676+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'3900","ondisk_log_start":"43'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28131328,"num_objects":6868,"num_object_clones":0,"num_object_copies":13736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3597,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10685,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3272},{"shards":"0,2","objects":102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13725","reported_seq":13963,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:33.358822+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:35:33.358822+0000","last_peered":"2024-09-10T20:35:33.358822+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:35:33.358822+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'3700","ondisk_log_start":"43'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28094464,"num_objects":6859,"num_object_clones":0,"num_object_copies":13718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10419,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3374},{"shards":"0,2","objects":103}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086784+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086784+0000","last_peered":"2024-09-10T20:34:27.086784+0000","last_clean":"2024-09-10T20:34:27.086784+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086784+0000","last_undegraded":"2024-09-10T20:34:27.086784+0000","last_fullsized":"2024-09-10T20:34:27.086784+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14095","reported_seq":14338,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:33.339393+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:35:33.339393+0000","last_peered":"2024-09-10T20:35:33.339393+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:35:33.339393+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28225536,"num_objects":6891,"num_object_clones":0,"num_object_copies":13782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3662,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10922,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3231},{"shards":"0,2","objects":113}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13667","reported_seq":20789,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:34.831982+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:35:34.831982+0000","last_peered":"2024-09-10T20:35:34.831982+0000","last_clean":"2024-09-10T20:35:34.831982+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:35:34.831982+0000","last_undegraded":"2024-09-10T20:35:34.831982+0000","last_fullsized":"2024-09-10T20:35:34.831982+0000","mapping_epoch":46,"log_start":"43'3600","ondisk_log_start":"43'3600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28053987,"num_objects":6850,"num_object_clones":0,"num_object_copies":13700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10274,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":448202177,"num_objects":109428,"num_object_clones":0,"num_object_copies":218856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33651,"num_objects_misplaced":27,"num_objects_unfound":0,"num_objects_dirty":109428,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":185989,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9471,"num_bytes_recovered":36941824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1069400064,"data_stored":1069323273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160893,"ondisk_log_size":160893,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182484,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":762908,"kb_used_data":388364,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93608932,"statfs":{"total":96636764160,"available":95855546368,"internally_reserved":0,"allocated":397684736,"data_stored":397432443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":383512234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,1,5,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986256,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":633196,"kb_used_data":334940,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93738644,"statfs":{"total":96636764160,"available":95988371456,"internally_reserved":0,"allocated":342978560,"data_stored":342726272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305393322},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659203,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":648772,"kb_used_data":324148,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93723068,"statfs":{"total":96636764160,"available":95972421632,"internally_reserved":0,"allocated":331927552,"data_stored":331613379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":332394158},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,5,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":396255232,"data_stored":396251619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":331419648,"data_stored":331354155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":341725184,"data_stored":341717499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:35:41.700 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:35:42.264 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:42.264 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:35:42.289 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":267,"stamp":"2024-09-10T20:35:40.964227+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":438674900,"num_objects":107104,"num_object_clones":0,"num_object_copies":214208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33485,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107104,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":188317,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9666,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160822,"ondisk_log_size":160822,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2049876,"kb_used_data":1031640,"kb_used_omap":12,"kb_used_meta":1018163,"kb_avail":281065644,"statfs":{"total":289910292480,"available":287811219456,"internally_reserved":0,"allocated":1056399360,"data_stored":1055580606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1042598914},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,5,5,7,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22880256,"num_objects":-5586,"num_object_clones":0,"num_object_copies":-11172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-179,"num_objects_misplaced":-318,"num_objects_unfound":0,"num_objects_dirty":-5586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5586,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":1302528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004771"},"pg_stats":[{"pgid":"2.f","version":"47'13980","reported_seq":14197,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:38.327826+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:35:38.327826+0000","last_peered":"2024-09-10T20:35:38.327826+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:35:38.327826+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'3900","ondisk_log_start":"43'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27353088,"num_objects":6678,"num_object_clones":0,"num_object_copies":13356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6678,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13980,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3137}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14173","reported_seq":14288,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:39.015687+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:35:39.015687+0000","last_peered":"2024-09-10T20:35:39.015687+0000","last_clean":"2024-09-10T20:35:39.015687+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:35:39.015687+0000","last_undegraded":"2024-09-10T20:35:39.015687+0000","last_fullsized":"2024-09-10T20:35:39.015687+0000","mapping_epoch":35,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27725824,"num_objects":6769,"num_object_clones":0,"num_object_copies":13538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12359,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14037","reported_seq":21390,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:39.770893+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:35:39.770893+0000","last_peered":"2024-09-10T20:35:39.770893+0000","last_clean":"2024-09-10T20:35:39.770893+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:35:39.770893+0000","last_undegraded":"2024-09-10T20:35:39.770893+0000","last_fullsized":"2024-09-10T20:35:39.770893+0000","mapping_epoch":44,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27353112,"num_objects":6679,"num_object_clones":0,"num_object_copies":13358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6679,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12165,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'13925","reported_seq":14010,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:39.027856+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:35:39.027856+0000","last_peered":"2024-09-10T20:35:39.027856+0000","last_clean":"2024-09-10T20:35:39.027856+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:35:39.027856+0000","last_undegraded":"2024-09-10T20:35:39.027856+0000","last_fullsized":"2024-09-10T20:35:39.027856+0000","mapping_epoch":35,"log_start":"43'3900","ondisk_log_start":"43'3900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27357184,"num_objects":6679,"num_object_clones":0,"num_object_copies":13358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13925,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14243","reported_seq":14375,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:38.422550+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:35:38.422550+0000","last_peered":"2024-09-10T20:35:38.422550+0000","last_clean":"2024-09-10T20:35:38.422550+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:35:38.422550+0000","last_undegraded":"2024-09-10T20:35:38.422550+0000","last_fullsized":"2024-09-10T20:35:38.422550+0000","mapping_epoch":34,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27959296,"num_objects":6826,"num_object_clones":0,"num_object_copies":13652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11239,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13885","reported_seq":14132,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:38.335264+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:35:38.335264+0000","last_peered":"2024-09-10T20:35:38.335264+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:35:38.335264+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26976256,"num_objects":6586,"num_object_clones":0,"num_object_copies":13172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10749,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3025}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'13908","reported_seq":14189,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:38.432031+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:35:38.432031+0000","last_peered":"2024-09-10T20:35:38.432031+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:35:38.432031+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'3900","ondisk_log_start":"43'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27443200,"num_objects":6700,"num_object_clones":0,"num_object_copies":13400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10503,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3214}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14042","reported_seq":14172,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:38.388372+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:35:38.388372+0000","last_peered":"2024-09-10T20:35:38.388372+0000","last_clean":"2024-09-10T20:35:38.388372+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:35:38.388372+0000","last_undegraded":"2024-09-10T20:35:38.388372+0000","last_fullsized":"2024-09-10T20:35:38.388372+0000","mapping_epoch":34,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27418624,"num_objects":6694,"num_object_clones":0,"num_object_copies":13388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11113,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13822","reported_seq":14014,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:39.009188+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:35:39.009188+0000","last_peered":"2024-09-10T20:35:39.009188+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:35:39.009188+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27103232,"num_objects":6617,"num_object_clones":0,"num_object_copies":13234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11815,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3120}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086274+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086274+0000","last_peered":"2024-09-10T20:34:27.086274+0000","last_clean":"2024-09-10T20:34:27.086274+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086274+0000","last_undegraded":"2024-09-10T20:34:27.086274+0000","last_fullsized":"2024-09-10T20:34:27.086274+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14026","reported_seq":14238,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:38.423295+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:35:38.423295+0000","last_peered":"2024-09-10T20:35:38.423295+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:35:38.423295+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27460550,"num_objects":6706,"num_object_clones":0,"num_object_copies":13412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14024,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3140}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13975","reported_seq":14163,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:38.994658+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:35:38.994658+0000","last_peered":"2024-09-10T20:35:38.994658+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:35:38.994658+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'3900","ondisk_log_start":"43'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26992640,"num_objects":6590,"num_object_clones":0,"num_object_copies":13180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11803,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2984}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14003","reported_seq":18400,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:39.035209+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:35:39.035209+0000","last_peered":"2024-09-10T20:35:39.035209+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:35:39.035209+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27283456,"num_objects":6661,"num_object_clones":0,"num_object_copies":13322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1481,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11789,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2131,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":5180}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14091","reported_seq":14342,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:38.415295+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:35:38.415295+0000","last_peered":"2024-09-10T20:35:38.415295+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:35:38.415295+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27602944,"num_objects":6739,"num_object_clones":0,"num_object_copies":13478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10814,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13862","reported_seq":14102,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:38.355171+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:35:38.355171+0000","last_peered":"2024-09-10T20:35:38.355171+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:35:38.355171+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27533312,"num_objects":6722,"num_object_clones":0,"num_object_copies":13444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10556,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3238}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086784+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086784+0000","last_peered":"2024-09-10T20:34:27.086784+0000","last_clean":"2024-09-10T20:34:27.086784+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086784+0000","last_undegraded":"2024-09-10T20:34:27.086784+0000","last_fullsized":"2024-09-10T20:34:27.086784+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14250","reported_seq":14503,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:38.429313+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:35:38.429313+0000","last_peered":"2024-09-10T20:35:38.429313+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:35:38.429313+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27590656,"num_objects":6736,"num_object_clones":0,"num_object_copies":13472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11077,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3081}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13797","reported_seq":20919,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:39.752848+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:35:39.752848+0000","last_peered":"2024-09-10T20:35:39.752848+0000","last_clean":"2024-09-10T20:35:39.752848+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:35:39.752848+0000","last_undegraded":"2024-09-10T20:35:39.752848+0000","last_fullsized":"2024-09-10T20:35:39.752848+0000","mapping_epoch":46,"log_start":"43'3700","ondisk_log_start":"43'3700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27521507,"num_objects":6720,"num_object_clones":0,"num_object_copies":13440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10404,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":438674881,"num_objects":107102,"num_object_clones":0,"num_object_copies":214204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33485,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107102,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":188315,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9664,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1052557312,"data_stored":1052480521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160819,"ondisk_log_size":160819,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182485,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":764416,"kb_used_data":381548,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93607424,"statfs":{"total":96636764160,"available":95854002176,"internally_reserved":0,"allocated":390705152,"data_stored":390452859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,4,4,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986257,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":632684,"kb_used_data":330264,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93739156,"statfs":{"total":96636764160,"available":95988895744,"internally_reserved":0,"allocated":338190336,"data_stored":337938048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309653162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659204,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":652776,"kb_used_data":319828,"kb_used_omap":4,"kb_used_meta":332923,"kb_avail":93719064,"statfs":{"total":96636764160,"available":95968321536,"internally_reserved":0,"allocated":327503872,"data_stored":327189699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":340913838},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":389124096,"data_stored":389120483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":326574080,"data_stored":326508587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":336859136,"data_stored":336851451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:35:42.291 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:35:42.850 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:42.850 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:35:42.876 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":267,"stamp":"2024-09-10T20:35:40.964227+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":438674900,"num_objects":107104,"num_object_clones":0,"num_object_copies":214208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33485,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107104,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":188317,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9666,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160822,"ondisk_log_size":160822,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2049876,"kb_used_data":1031640,"kb_used_omap":12,"kb_used_meta":1018163,"kb_avail":281065644,"statfs":{"total":289910292480,"available":287811219456,"internally_reserved":0,"allocated":1056399360,"data_stored":1055580606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1042598914},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,5,5,7,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22880256,"num_objects":-5586,"num_object_clones":0,"num_object_copies":-11172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-179,"num_objects_misplaced":-318,"num_objects_unfound":0,"num_objects_dirty":-5586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5586,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":1302528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004771"},"pg_stats":[{"pgid":"2.f","version":"47'13980","reported_seq":14197,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:38.327826+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:35:38.327826+0000","last_peered":"2024-09-10T20:35:38.327826+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:35:38.327826+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'3900","ondisk_log_start":"43'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27353088,"num_objects":6678,"num_object_clones":0,"num_object_copies":13356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6678,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13980,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3137}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14173","reported_seq":14288,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:39.015687+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:35:39.015687+0000","last_peered":"2024-09-10T20:35:39.015687+0000","last_clean":"2024-09-10T20:35:39.015687+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:35:39.015687+0000","last_undegraded":"2024-09-10T20:35:39.015687+0000","last_fullsized":"2024-09-10T20:35:39.015687+0000","mapping_epoch":35,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27725824,"num_objects":6769,"num_object_clones":0,"num_object_copies":13538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12359,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14037","reported_seq":21390,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:39.770893+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:35:39.770893+0000","last_peered":"2024-09-10T20:35:39.770893+0000","last_clean":"2024-09-10T20:35:39.770893+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:35:39.770893+0000","last_undegraded":"2024-09-10T20:35:39.770893+0000","last_fullsized":"2024-09-10T20:35:39.770893+0000","mapping_epoch":44,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27353112,"num_objects":6679,"num_object_clones":0,"num_object_copies":13358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6679,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12165,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'13925","reported_seq":14010,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:39.027856+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:35:39.027856+0000","last_peered":"2024-09-10T20:35:39.027856+0000","last_clean":"2024-09-10T20:35:39.027856+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:35:39.027856+0000","last_undegraded":"2024-09-10T20:35:39.027856+0000","last_fullsized":"2024-09-10T20:35:39.027856+0000","mapping_epoch":35,"log_start":"43'3900","ondisk_log_start":"43'3900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27357184,"num_objects":6679,"num_object_clones":0,"num_object_copies":13358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13925,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14243","reported_seq":14375,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:38.422550+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:35:38.422550+0000","last_peered":"2024-09-10T20:35:38.422550+0000","last_clean":"2024-09-10T20:35:38.422550+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:35:38.422550+0000","last_undegraded":"2024-09-10T20:35:38.422550+0000","last_fullsized":"2024-09-10T20:35:38.422550+0000","mapping_epoch":34,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27959296,"num_objects":6826,"num_object_clones":0,"num_object_copies":13652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11239,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13885","reported_seq":14132,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:38.335264+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:35:38.335264+0000","last_peered":"2024-09-10T20:35:38.335264+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:35:38.335264+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26976256,"num_objects":6586,"num_object_clones":0,"num_object_copies":13172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10749,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3025}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'13908","reported_seq":14189,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:38.432031+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:35:38.432031+0000","last_peered":"2024-09-10T20:35:38.432031+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:35:38.432031+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'3900","ondisk_log_start":"43'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27443200,"num_objects":6700,"num_object_clones":0,"num_object_copies":13400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10503,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3214}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14042","reported_seq":14172,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:38.388372+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:35:38.388372+0000","last_peered":"2024-09-10T20:35:38.388372+0000","last_clean":"2024-09-10T20:35:38.388372+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:35:38.388372+0000","last_undegraded":"2024-09-10T20:35:38.388372+0000","last_fullsized":"2024-09-10T20:35:38.388372+0000","mapping_epoch":34,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27418624,"num_objects":6694,"num_object_clones":0,"num_object_copies":13388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11113,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13822","reported_seq":14014,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:39.009188+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:35:39.009188+0000","last_peered":"2024-09-10T20:35:39.009188+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:35:39.009188+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27103232,"num_objects":6617,"num_object_clones":0,"num_object_copies":13234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11815,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3120}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086274+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086274+0000","last_peered":"2024-09-10T20:34:27.086274+0000","last_clean":"2024-09-10T20:34:27.086274+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086274+0000","last_undegraded":"2024-09-10T20:34:27.086274+0000","last_fullsized":"2024-09-10T20:34:27.086274+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14026","reported_seq":14238,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:38.423295+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:35:38.423295+0000","last_peered":"2024-09-10T20:35:38.423295+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:35:38.423295+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27460550,"num_objects":6706,"num_object_clones":0,"num_object_copies":13412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14024,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3140}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13975","reported_seq":14163,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:38.994658+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:35:38.994658+0000","last_peered":"2024-09-10T20:35:38.994658+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:35:38.994658+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'3900","ondisk_log_start":"43'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26992640,"num_objects":6590,"num_object_clones":0,"num_object_copies":13180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11803,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2984}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14003","reported_seq":18400,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:39.035209+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:35:39.035209+0000","last_peered":"2024-09-10T20:35:39.035209+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:35:39.035209+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27283456,"num_objects":6661,"num_object_clones":0,"num_object_copies":13322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1481,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11789,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2131,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":5180}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14091","reported_seq":14342,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:38.415295+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:35:38.415295+0000","last_peered":"2024-09-10T20:35:38.415295+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:35:38.415295+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27602944,"num_objects":6739,"num_object_clones":0,"num_object_copies":13478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10814,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13862","reported_seq":14102,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:38.355171+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:35:38.355171+0000","last_peered":"2024-09-10T20:35:38.355171+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:35:38.355171+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27533312,"num_objects":6722,"num_object_clones":0,"num_object_copies":13444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10556,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3238}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086784+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086784+0000","last_peered":"2024-09-10T20:34:27.086784+0000","last_clean":"2024-09-10T20:34:27.086784+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086784+0000","last_undegraded":"2024-09-10T20:34:27.086784+0000","last_fullsized":"2024-09-10T20:34:27.086784+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14250","reported_seq":14503,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:38.429313+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:35:38.429313+0000","last_peered":"2024-09-10T20:35:38.429313+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:35:38.429313+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27590656,"num_objects":6736,"num_object_clones":0,"num_object_copies":13472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11077,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3081}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13797","reported_seq":20919,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:39.752848+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:35:39.752848+0000","last_peered":"2024-09-10T20:35:39.752848+0000","last_clean":"2024-09-10T20:35:39.752848+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:35:39.752848+0000","last_undegraded":"2024-09-10T20:35:39.752848+0000","last_fullsized":"2024-09-10T20:35:39.752848+0000","mapping_epoch":46,"log_start":"43'3700","ondisk_log_start":"43'3700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27521507,"num_objects":6720,"num_object_clones":0,"num_object_copies":13440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10404,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":438674881,"num_objects":107102,"num_object_clones":0,"num_object_copies":214204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33485,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107102,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":188315,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9664,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1052557312,"data_stored":1052480521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160819,"ondisk_log_size":160819,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182485,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":764416,"kb_used_data":381548,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93607424,"statfs":{"total":96636764160,"available":95854002176,"internally_reserved":0,"allocated":390705152,"data_stored":390452859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,4,4,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986257,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":632684,"kb_used_data":330264,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93739156,"statfs":{"total":96636764160,"available":95988895744,"internally_reserved":0,"allocated":338190336,"data_stored":337938048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309653162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659204,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":652776,"kb_used_data":319828,"kb_used_omap":4,"kb_used_meta":332923,"kb_avail":93719064,"statfs":{"total":96636764160,"available":95968321536,"internally_reserved":0,"allocated":327503872,"data_stored":327189699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":340913838},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":389124096,"data_stored":389120483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":326574080,"data_stored":326508587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":336859136,"data_stored":336851451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:35:42.877 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:35:42.877 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'13980", 'reported_seq': 14197, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:38.327826+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:35:38.327826+0000', 'last_peered': '2024-09-10T20:35:38.327826+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:35:38.327826+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "43'3900", 'ondisk_log_start': "43'3900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27353088, 'num_objects': 6678, 'num_object_clones': 0, 'num_object_copies': 13356, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6678, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13980, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3137}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:42.877 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:35:42.877 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'13885", 'reported_seq': 14132, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:38.335264+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:35:38.335264+0000', 'last_peered': '2024-09-10T20:35:38.335264+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:35:38.335264+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "43'3800", 'ondisk_log_start': "43'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26976256, 'num_objects': 6586, 'num_object_clones': 0, 'num_object_copies': 13172, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6586, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10749, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3025}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:42.878 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:35:42.878 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'13908", 'reported_seq': 14189, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:38.432031+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:35:38.432031+0000', 'last_peered': '2024-09-10T20:35:38.432031+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:35:38.432031+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "43'3900", 'ondisk_log_start': "43'3900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27443200, 'num_objects': 6700, 'num_object_clones': 0, 'num_object_copies': 13400, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6700, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10503, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3214}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:42.878 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:35:42.878 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'13822", 'reported_seq': 14014, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:39.009188+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:35:39.009188+0000', 'last_peered': '2024-09-10T20:35:39.009188+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:35:39.009188+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "43'3800", 'ondisk_log_start': "43'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27103232, 'num_objects': 6617, 'num_object_clones': 0, 'num_object_copies': 13234, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6617, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11815, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3120}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:35:42.878 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:35:42.878 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'14026", 'reported_seq': 14238, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:38.423295+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:35:38.423295+0000', 'last_peered': '2024-09-10T20:35:38.423295+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:35:38.423295+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "43'4000", 'ondisk_log_start': "43'4000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27460550, 'num_objects': 6706, 'num_object_clones': 0, 'num_object_copies': 13412, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6706, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14024, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3140}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:42.878 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:35:42.878 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'13975", 'reported_seq': 14163, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:38.994658+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:35:38.994658+0000', 'last_peered': '2024-09-10T20:35:38.994658+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:35:38.994658+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "43'3900", 'ondisk_log_start': "43'3900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26992640, 'num_objects': 6590, 'num_object_clones': 0, 'num_object_copies': 13180, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6590, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11803, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2984}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:35:42.878 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:35:42.878 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'14003", 'reported_seq': 18400, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:39.035209+0000', 'last_change': '2024-09-10T20:34:25.241937+0000', 'last_active': '2024-09-10T20:35:39.035209+0000', 'last_peered': '2024-09-10T20:35:39.035209+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:35:39.035209+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "43'4000", 'ondisk_log_start': "43'4000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27283456, 'num_objects': 6661, 'num_object_clones': 0, 'num_object_copies': 13322, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1481, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6661, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11789, 'num_write_kb': 32484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2131, 'num_bytes_recovered': 8085504, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 5180}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:35:42.878 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:35:42.879 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'14091", 'reported_seq': 14342, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:38.415295+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:35:38.415295+0000', 'last_peered': '2024-09-10T20:35:38.415295+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:35:38.415295+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "43'4000", 'ondisk_log_start': "43'4000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27602944, 'num_objects': 6739, 'num_object_clones': 0, 'num_object_copies': 13478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6739, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10814, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:42.879 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:35:42.879 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'13862", 'reported_seq': 14102, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:38.355171+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:35:38.355171+0000', 'last_peered': '2024-09-10T20:35:38.355171+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:35:38.355171+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "43'3800", 'ondisk_log_start': "43'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27533312, 'num_objects': 6722, 'num_object_clones': 0, 'num_object_copies': 13444, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3487, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6722, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10556, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 48, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3238}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:42.879 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:35:42.879 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'14250", 'reported_seq': 14503, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:38.429313+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:35:38.429313+0000', 'last_peered': '2024-09-10T20:35:38.429313+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:35:38.429313+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "43'4200", 'ondisk_log_start': "43'4200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27590656, 'num_objects': 6736, 'num_object_clones': 0, 'num_object_copies': 13472, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6736, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11077, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3081}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:42.879 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:35:43.554 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:43.554 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33485/214208 objects degraded (15.632%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":455,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":107104,"data_bytes":438674900,"bytes_used":2099073024,"bytes_avail":287811219456,"bytes_total":289910292480,"degraded_objects":33485,"degraded_total":214208,"degraded_ratio":0.15632002539587692,"recovering_objects_per_sec":41,"recovering_bytes_per_sec":108500,"recovering_keys_per_sec":0,"num_objects_recovered":497,"num_bytes_recovered":1302528,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":465},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:35:43.580 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:35:43.580 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:35:44.192 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:44.195 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:35:44.221 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":268,"stamp":"2024-09-10T20:35:42.965051+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":438674900,"num_objects":107104,"num_object_clones":0,"num_object_copies":214208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33485,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107104,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":188317,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9666,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160822,"ondisk_log_size":160822,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2049876,"kb_used_data":1031640,"kb_used_omap":12,"kb_used_meta":1018163,"kb_avail":281065644,"statfs":{"total":289910292480,"available":287811219456,"internally_reserved":0,"allocated":1056399360,"data_stored":1055580606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1042598914},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,5,5,7,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18915328,"num_objects":-4618,"num_object_clones":0,"num_object_copies":-9236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-179,"num_objects_misplaced":-172,"num_objects_unfound":0,"num_objects_dirty":-4618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4618,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":351,"num_bytes_recovered":704512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004938"},"pg_stats":[{"pgid":"2.f","version":"47'13980","reported_seq":14197,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:38.327826+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:35:38.327826+0000","last_peered":"2024-09-10T20:35:38.327826+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:35:38.327826+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'3900","ondisk_log_start":"43'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27353088,"num_objects":6678,"num_object_clones":0,"num_object_copies":13356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6678,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13980,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3137}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14173","reported_seq":14288,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:39.015687+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:35:39.015687+0000","last_peered":"2024-09-10T20:35:39.015687+0000","last_clean":"2024-09-10T20:35:39.015687+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:35:39.015687+0000","last_undegraded":"2024-09-10T20:35:39.015687+0000","last_fullsized":"2024-09-10T20:35:39.015687+0000","mapping_epoch":35,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27725824,"num_objects":6769,"num_object_clones":0,"num_object_copies":13538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12359,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14037","reported_seq":21390,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:39.770893+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:35:39.770893+0000","last_peered":"2024-09-10T20:35:39.770893+0000","last_clean":"2024-09-10T20:35:39.770893+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:35:39.770893+0000","last_undegraded":"2024-09-10T20:35:39.770893+0000","last_fullsized":"2024-09-10T20:35:39.770893+0000","mapping_epoch":44,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27353112,"num_objects":6679,"num_object_clones":0,"num_object_copies":13358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6679,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12165,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'13925","reported_seq":14010,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:39.027856+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:35:39.027856+0000","last_peered":"2024-09-10T20:35:39.027856+0000","last_clean":"2024-09-10T20:35:39.027856+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:35:39.027856+0000","last_undegraded":"2024-09-10T20:35:39.027856+0000","last_fullsized":"2024-09-10T20:35:39.027856+0000","mapping_epoch":35,"log_start":"43'3900","ondisk_log_start":"43'3900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27357184,"num_objects":6679,"num_object_clones":0,"num_object_copies":13358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13925,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14243","reported_seq":14375,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:38.422550+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:35:38.422550+0000","last_peered":"2024-09-10T20:35:38.422550+0000","last_clean":"2024-09-10T20:35:38.422550+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:35:38.422550+0000","last_undegraded":"2024-09-10T20:35:38.422550+0000","last_fullsized":"2024-09-10T20:35:38.422550+0000","mapping_epoch":34,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27959296,"num_objects":6826,"num_object_clones":0,"num_object_copies":13652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11239,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13885","reported_seq":14132,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:38.335264+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:35:38.335264+0000","last_peered":"2024-09-10T20:35:38.335264+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:35:38.335264+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26976256,"num_objects":6586,"num_object_clones":0,"num_object_copies":13172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10749,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3025}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'13908","reported_seq":14189,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:38.432031+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:35:38.432031+0000","last_peered":"2024-09-10T20:35:38.432031+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:35:38.432031+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'3900","ondisk_log_start":"43'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27443200,"num_objects":6700,"num_object_clones":0,"num_object_copies":13400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10503,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3214}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14042","reported_seq":14172,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:38.388372+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:35:38.388372+0000","last_peered":"2024-09-10T20:35:38.388372+0000","last_clean":"2024-09-10T20:35:38.388372+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:35:38.388372+0000","last_undegraded":"2024-09-10T20:35:38.388372+0000","last_fullsized":"2024-09-10T20:35:38.388372+0000","mapping_epoch":34,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27418624,"num_objects":6694,"num_object_clones":0,"num_object_copies":13388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11113,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13822","reported_seq":14014,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:39.009188+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:35:39.009188+0000","last_peered":"2024-09-10T20:35:39.009188+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:35:39.009188+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27103232,"num_objects":6617,"num_object_clones":0,"num_object_copies":13234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11815,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3120}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086274+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086274+0000","last_peered":"2024-09-10T20:34:27.086274+0000","last_clean":"2024-09-10T20:34:27.086274+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086274+0000","last_undegraded":"2024-09-10T20:34:27.086274+0000","last_fullsized":"2024-09-10T20:34:27.086274+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14026","reported_seq":14238,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:38.423295+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:35:38.423295+0000","last_peered":"2024-09-10T20:35:38.423295+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:35:38.423295+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27460550,"num_objects":6706,"num_object_clones":0,"num_object_copies":13412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14024,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3140}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13975","reported_seq":14163,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:38.994658+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:35:38.994658+0000","last_peered":"2024-09-10T20:35:38.994658+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:35:38.994658+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'3900","ondisk_log_start":"43'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26992640,"num_objects":6590,"num_object_clones":0,"num_object_copies":13180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11803,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2984}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14003","reported_seq":18400,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:39.035209+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:35:39.035209+0000","last_peered":"2024-09-10T20:35:39.035209+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:35:39.035209+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27283456,"num_objects":6661,"num_object_clones":0,"num_object_copies":13322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1481,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11789,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2131,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":5180}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14091","reported_seq":14342,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:38.415295+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:35:38.415295+0000","last_peered":"2024-09-10T20:35:38.415295+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:35:38.415295+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27602944,"num_objects":6739,"num_object_clones":0,"num_object_copies":13478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10814,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13862","reported_seq":14102,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:38.355171+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:35:38.355171+0000","last_peered":"2024-09-10T20:35:38.355171+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:35:38.355171+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27533312,"num_objects":6722,"num_object_clones":0,"num_object_copies":13444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10556,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3238}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086784+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086784+0000","last_peered":"2024-09-10T20:34:27.086784+0000","last_clean":"2024-09-10T20:34:27.086784+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086784+0000","last_undegraded":"2024-09-10T20:34:27.086784+0000","last_fullsized":"2024-09-10T20:34:27.086784+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14250","reported_seq":14503,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:38.429313+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:35:38.429313+0000","last_peered":"2024-09-10T20:35:38.429313+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:35:38.429313+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27590656,"num_objects":6736,"num_object_clones":0,"num_object_copies":13472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11077,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3081}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13797","reported_seq":20919,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:39.752848+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:35:39.752848+0000","last_peered":"2024-09-10T20:35:39.752848+0000","last_clean":"2024-09-10T20:35:39.752848+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:35:39.752848+0000","last_undegraded":"2024-09-10T20:35:39.752848+0000","last_fullsized":"2024-09-10T20:35:39.752848+0000","mapping_epoch":46,"log_start":"43'3700","ondisk_log_start":"43'3700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27521507,"num_objects":6720,"num_object_clones":0,"num_object_copies":13440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10404,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":438674881,"num_objects":107102,"num_object_clones":0,"num_object_copies":214204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33485,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107102,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":188315,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9664,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1052557312,"data_stored":1052480521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160819,"ondisk_log_size":160819,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182485,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":764416,"kb_used_data":381548,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93607424,"statfs":{"total":96636764160,"available":95854002176,"internally_reserved":0,"allocated":390705152,"data_stored":390452859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,4,4,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986257,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":632684,"kb_used_data":330264,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93739156,"statfs":{"total":96636764160,"available":95988895744,"internally_reserved":0,"allocated":338190336,"data_stored":337938048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309653162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659204,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":652776,"kb_used_data":319828,"kb_used_omap":4,"kb_used_meta":332923,"kb_avail":93719064,"statfs":{"total":96636764160,"available":95968321536,"internally_reserved":0,"allocated":327503872,"data_stored":327189699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":340913838},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":389124096,"data_stored":389120483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":326574080,"data_stored":326508587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":336859136,"data_stored":336851451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:35:47.224 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:35:47.806 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:47.808 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:35:47.828 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":270,"stamp":"2024-09-10T20:35:46.966363+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":428664276,"num_objects":104660,"num_object_clones":0,"num_object_copies":209320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33338,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104660,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":190761,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9813,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160966,"ondisk_log_size":160966,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2009796,"kb_used_data":1012372,"kb_used_omap":12,"kb_used_meta":997363,"kb_avail":281105724,"statfs":{"total":289910292480,"available":287852261376,"internally_reserved":0,"allocated":1036668928,"data_stored":1035850174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1021299714},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,7,6,11,12,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19537920,"num_objects":-4770,"num_object_clones":0,"num_object_copies":-9540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-313,"num_objects_misplaced":-27,"num_objects_unfound":0,"num_objects_dirty":-4770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4770,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":110592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004674"},"pg_stats":[{"pgid":"2.f","version":"47'14141","reported_seq":14358,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:43.475508+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:35:43.475508+0000","last_peered":"2024-09-10T20:35:43.475508+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:35:43.475508+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26693632,"num_objects":6517,"num_object_clones":0,"num_object_copies":13034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14141,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2976}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14333","reported_seq":14448,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:44.011313+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:35:44.011313+0000","last_peered":"2024-09-10T20:35:44.011313+0000","last_clean":"2024-09-10T20:35:44.011313+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:35:44.011313+0000","last_undegraded":"2024-09-10T20:35:44.011313+0000","last_fullsized":"2024-09-10T20:35:44.011313+0000","mapping_epoch":35,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27070464,"num_objects":6609,"num_object_clones":0,"num_object_copies":13218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12519,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14188","reported_seq":21541,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:44.903654+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:35:44.903654+0000","last_peered":"2024-09-10T20:35:44.903654+0000","last_clean":"2024-09-10T20:35:44.903654+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:35:44.903654+0000","last_undegraded":"2024-09-10T20:35:44.903654+0000","last_fullsized":"2024-09-10T20:35:44.903654+0000","mapping_epoch":44,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26734616,"num_objects":6528,"num_object_clones":0,"num_object_copies":13056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6528,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12316,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'14080","reported_seq":14165,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:44.007438+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:35:44.007438+0000","last_peered":"2024-09-10T20:35:44.007438+0000","last_clean":"2024-09-10T20:35:44.007438+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:35:44.007438+0000","last_undegraded":"2024-09-10T20:35:44.007438+0000","last_fullsized":"2024-09-10T20:35:44.007438+0000","mapping_epoch":35,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26722304,"num_objects":6524,"num_object_clones":0,"num_object_copies":13048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14080,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14392","reported_seq":14524,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:43.448092+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:35:43.448092+0000","last_peered":"2024-09-10T20:35:43.448092+0000","last_clean":"2024-09-10T20:35:43.448092+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:35:43.448092+0000","last_undegraded":"2024-09-10T20:35:43.448092+0000","last_fullsized":"2024-09-10T20:35:43.448092+0000","mapping_epoch":34,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27348992,"num_objects":6677,"num_object_clones":0,"num_object_copies":13354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11388,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14036","reported_seq":14284,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:43.458625+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:35:43.458625+0000","last_peered":"2024-09-10T20:35:43.458625+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:35:43.458625+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26357760,"num_objects":6435,"num_object_clones":0,"num_object_copies":12870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10900,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2874}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'14059","reported_seq":14340,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:43.380792+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:35:43.380792+0000","last_peered":"2024-09-10T20:35:43.380792+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:35:43.380792+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26824704,"num_objects":6549,"num_object_clones":0,"num_object_copies":13098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10654,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14194","reported_seq":14324,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:43.237842+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:35:43.237842+0000","last_peered":"2024-09-10T20:35:43.237842+0000","last_clean":"2024-09-10T20:35:43.237842+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:35:43.237842+0000","last_undegraded":"2024-09-10T20:35:43.237842+0000","last_fullsized":"2024-09-10T20:35:43.237842+0000","mapping_epoch":34,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26796032,"num_objects":6542,"num_object_clones":0,"num_object_copies":13084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11265,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13966","reported_seq":14161,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:43.984256+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:35:43.984256+0000","last_peered":"2024-09-10T20:35:43.984256+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:35:43.984256+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'3900","ondisk_log_start":"43'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26513408,"num_objects":6473,"num_object_clones":0,"num_object_copies":12946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11959,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2976}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086274+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086274+0000","last_peered":"2024-09-10T20:34:27.086274+0000","last_clean":"2024-09-10T20:34:27.086274+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086274+0000","last_undegraded":"2024-09-10T20:34:27.086274+0000","last_fullsized":"2024-09-10T20:34:27.086274+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14185","reported_seq":14392,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:43.459304+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:35:43.459304+0000","last_peered":"2024-09-10T20:35:43.459304+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:35:43.459304+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26809286,"num_objects":6547,"num_object_clones":0,"num_object_copies":13094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14183,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2981}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14134","reported_seq":14322,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:44.013203+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:35:44.013203+0000","last_peered":"2024-09-10T20:35:44.013203+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:35:44.013203+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26341376,"num_objects":6431,"num_object_clones":0,"num_object_copies":12862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11962,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2825}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14141","reported_seq":18832,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:44.049383+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:35:44.049383+0000","last_peered":"2024-09-10T20:35:44.049383+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:35:44.049383+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26718208,"num_objects":6523,"num_object_clones":0,"num_object_copies":13046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1334,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11927,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2278,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":5189}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14230","reported_seq":14482,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:43.403597+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:35:43.403597+0000","last_peered":"2024-09-10T20:35:43.403597+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:35:43.403597+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27033600,"num_objects":6600,"num_object_clones":0,"num_object_copies":13200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10953,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3008}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14025","reported_seq":14265,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:43.403218+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:35:43.403218+0000","last_peered":"2024-09-10T20:35:43.403218+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:35:43.403218+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26865664,"num_objects":6559,"num_object_clones":0,"num_object_copies":13118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10719,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3075}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086784+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086784+0000","last_peered":"2024-09-10T20:34:27.086784+0000","last_clean":"2024-09-10T20:34:27.086784+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086784+0000","last_undegraded":"2024-09-10T20:34:27.086784+0000","last_fullsized":"2024-09-10T20:34:27.086784+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14398","reported_seq":14651,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:43.354802+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:35:43.354802+0000","last_peered":"2024-09-10T20:35:43.354802+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:35:43.354802+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26984448,"num_objects":6588,"num_object_clones":0,"num_object_copies":13176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11225,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2933}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13961","reported_seq":21081,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:44.968399+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:35:44.968399+0000","last_peered":"2024-09-10T20:35:44.968399+0000","last_clean":"2024-09-10T20:35:44.968399+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:35:44.968399+0000","last_undegraded":"2024-09-10T20:35:44.968399+0000","last_fullsized":"2024-09-10T20:35:44.968399+0000","mapping_epoch":46,"log_start":"43'3900","ondisk_log_start":"43'3900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26849763,"num_objects":6556,"num_object_clones":0,"num_object_copies":13112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10568,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":428664257,"num_objects":104658,"num_object_clones":0,"num_object_copies":209316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33338,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104658,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":190759,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9811,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1032716288,"data_stored":1032639497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160963,"ondisk_log_size":160963,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182486,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":744680,"kb_used_data":374296,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93627160,"statfs":{"total":96636764160,"available":95874211840,"internally_reserved":0,"allocated":383279104,"data_stored":383026811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":379252394},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,4,4,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986258,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":631916,"kb_used_data":325340,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93739924,"statfs":{"total":96636764160,"available":95989682176,"internally_reserved":0,"allocated":333148160,"data_stored":332895872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313913002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659205,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":633200,"kb_used_data":312736,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93738640,"statfs":{"total":96636764160,"available":95988367360,"internally_reserved":0,"allocated":320241664,"data_stored":319927491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":328134318},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,6,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":381595648,"data_stored":381592035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":319143936,"data_stored":319078443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":331976704,"data_stored":331969019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:35:47.830 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:35:48.359 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:48.359 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:35:48.387 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":270,"stamp":"2024-09-10T20:35:46.966363+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":428664276,"num_objects":104660,"num_object_clones":0,"num_object_copies":209320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33338,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104660,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":190761,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9813,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160966,"ondisk_log_size":160966,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2009796,"kb_used_data":1012372,"kb_used_omap":12,"kb_used_meta":997363,"kb_avail":281105724,"statfs":{"total":289910292480,"available":287852261376,"internally_reserved":0,"allocated":1036668928,"data_stored":1035850174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1021299714},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,7,6,11,12,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19537920,"num_objects":-4770,"num_object_clones":0,"num_object_copies":-9540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-313,"num_objects_misplaced":-27,"num_objects_unfound":0,"num_objects_dirty":-4770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4770,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":110592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004674"},"pg_stats":[{"pgid":"2.f","version":"47'14141","reported_seq":14358,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:43.475508+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:35:43.475508+0000","last_peered":"2024-09-10T20:35:43.475508+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:35:43.475508+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26693632,"num_objects":6517,"num_object_clones":0,"num_object_copies":13034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14141,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2976}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14333","reported_seq":14448,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:44.011313+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:35:44.011313+0000","last_peered":"2024-09-10T20:35:44.011313+0000","last_clean":"2024-09-10T20:35:44.011313+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:35:44.011313+0000","last_undegraded":"2024-09-10T20:35:44.011313+0000","last_fullsized":"2024-09-10T20:35:44.011313+0000","mapping_epoch":35,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27070464,"num_objects":6609,"num_object_clones":0,"num_object_copies":13218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12519,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14188","reported_seq":21541,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:44.903654+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:35:44.903654+0000","last_peered":"2024-09-10T20:35:44.903654+0000","last_clean":"2024-09-10T20:35:44.903654+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:35:44.903654+0000","last_undegraded":"2024-09-10T20:35:44.903654+0000","last_fullsized":"2024-09-10T20:35:44.903654+0000","mapping_epoch":44,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26734616,"num_objects":6528,"num_object_clones":0,"num_object_copies":13056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6528,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12316,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'14080","reported_seq":14165,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:44.007438+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:35:44.007438+0000","last_peered":"2024-09-10T20:35:44.007438+0000","last_clean":"2024-09-10T20:35:44.007438+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:35:44.007438+0000","last_undegraded":"2024-09-10T20:35:44.007438+0000","last_fullsized":"2024-09-10T20:35:44.007438+0000","mapping_epoch":35,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26722304,"num_objects":6524,"num_object_clones":0,"num_object_copies":13048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14080,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14392","reported_seq":14524,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:43.448092+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:35:43.448092+0000","last_peered":"2024-09-10T20:35:43.448092+0000","last_clean":"2024-09-10T20:35:43.448092+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:35:43.448092+0000","last_undegraded":"2024-09-10T20:35:43.448092+0000","last_fullsized":"2024-09-10T20:35:43.448092+0000","mapping_epoch":34,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27348992,"num_objects":6677,"num_object_clones":0,"num_object_copies":13354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11388,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14036","reported_seq":14284,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:43.458625+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:35:43.458625+0000","last_peered":"2024-09-10T20:35:43.458625+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:35:43.458625+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26357760,"num_objects":6435,"num_object_clones":0,"num_object_copies":12870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10900,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2874}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'14059","reported_seq":14340,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:43.380792+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:35:43.380792+0000","last_peered":"2024-09-10T20:35:43.380792+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:35:43.380792+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26824704,"num_objects":6549,"num_object_clones":0,"num_object_copies":13098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10654,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14194","reported_seq":14324,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:43.237842+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:35:43.237842+0000","last_peered":"2024-09-10T20:35:43.237842+0000","last_clean":"2024-09-10T20:35:43.237842+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:35:43.237842+0000","last_undegraded":"2024-09-10T20:35:43.237842+0000","last_fullsized":"2024-09-10T20:35:43.237842+0000","mapping_epoch":34,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26796032,"num_objects":6542,"num_object_clones":0,"num_object_copies":13084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11265,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13966","reported_seq":14161,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:43.984256+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:35:43.984256+0000","last_peered":"2024-09-10T20:35:43.984256+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:35:43.984256+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'3900","ondisk_log_start":"43'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26513408,"num_objects":6473,"num_object_clones":0,"num_object_copies":12946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11959,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2976}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086274+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086274+0000","last_peered":"2024-09-10T20:34:27.086274+0000","last_clean":"2024-09-10T20:34:27.086274+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086274+0000","last_undegraded":"2024-09-10T20:34:27.086274+0000","last_fullsized":"2024-09-10T20:34:27.086274+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14185","reported_seq":14392,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:43.459304+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:35:43.459304+0000","last_peered":"2024-09-10T20:35:43.459304+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:35:43.459304+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26809286,"num_objects":6547,"num_object_clones":0,"num_object_copies":13094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14183,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2981}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14134","reported_seq":14322,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:44.013203+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:35:44.013203+0000","last_peered":"2024-09-10T20:35:44.013203+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:35:44.013203+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26341376,"num_objects":6431,"num_object_clones":0,"num_object_copies":12862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11962,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2825}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14141","reported_seq":18832,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:44.049383+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:35:44.049383+0000","last_peered":"2024-09-10T20:35:44.049383+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:35:44.049383+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26718208,"num_objects":6523,"num_object_clones":0,"num_object_copies":13046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1334,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11927,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2278,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":5189}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14230","reported_seq":14482,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:43.403597+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:35:43.403597+0000","last_peered":"2024-09-10T20:35:43.403597+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:35:43.403597+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27033600,"num_objects":6600,"num_object_clones":0,"num_object_copies":13200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10953,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3008}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14025","reported_seq":14265,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:43.403218+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:35:43.403218+0000","last_peered":"2024-09-10T20:35:43.403218+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:35:43.403218+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26865664,"num_objects":6559,"num_object_clones":0,"num_object_copies":13118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10719,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3075}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086784+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086784+0000","last_peered":"2024-09-10T20:34:27.086784+0000","last_clean":"2024-09-10T20:34:27.086784+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086784+0000","last_undegraded":"2024-09-10T20:34:27.086784+0000","last_fullsized":"2024-09-10T20:34:27.086784+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14398","reported_seq":14651,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:43.354802+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:35:43.354802+0000","last_peered":"2024-09-10T20:35:43.354802+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:35:43.354802+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26984448,"num_objects":6588,"num_object_clones":0,"num_object_copies":13176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11225,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2933}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13961","reported_seq":21081,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:44.968399+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:35:44.968399+0000","last_peered":"2024-09-10T20:35:44.968399+0000","last_clean":"2024-09-10T20:35:44.968399+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:35:44.968399+0000","last_undegraded":"2024-09-10T20:35:44.968399+0000","last_fullsized":"2024-09-10T20:35:44.968399+0000","mapping_epoch":46,"log_start":"43'3900","ondisk_log_start":"43'3900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26849763,"num_objects":6556,"num_object_clones":0,"num_object_copies":13112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10568,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":428664257,"num_objects":104658,"num_object_clones":0,"num_object_copies":209316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33338,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104658,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":190759,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9811,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1032716288,"data_stored":1032639497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160963,"ondisk_log_size":160963,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182486,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":744680,"kb_used_data":374296,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93627160,"statfs":{"total":96636764160,"available":95874211840,"internally_reserved":0,"allocated":383279104,"data_stored":383026811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":379252394},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,4,4,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986258,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":631916,"kb_used_data":325340,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93739924,"statfs":{"total":96636764160,"available":95989682176,"internally_reserved":0,"allocated":333148160,"data_stored":332895872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313913002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659205,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":633200,"kb_used_data":312736,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93738640,"statfs":{"total":96636764160,"available":95988367360,"internally_reserved":0,"allocated":320241664,"data_stored":319927491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":328134318},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,6,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":381595648,"data_stored":381592035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":319143936,"data_stored":319078443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":331976704,"data_stored":331969019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:35:48.389 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:35:48.389 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'14141", 'reported_seq': 14358, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:43.475508+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:35:43.475508+0000', 'last_peered': '2024-09-10T20:35:43.475508+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:35:43.475508+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "43'4100", 'ondisk_log_start': "43'4100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26693632, 'num_objects': 6517, 'num_object_clones': 0, 'num_object_copies': 13034, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6517, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14141, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2976}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:48.389 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:35:48.389 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'14036", 'reported_seq': 14284, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:43.458625+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:35:43.458625+0000', 'last_peered': '2024-09-10T20:35:43.458625+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:35:43.458625+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "43'4000", 'ondisk_log_start': "43'4000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26357760, 'num_objects': 6435, 'num_object_clones': 0, 'num_object_copies': 12870, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6435, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10900, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2874}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:48.389 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:35:48.389 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'14059", 'reported_seq': 14340, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:43.380792+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:35:43.380792+0000', 'last_peered': '2024-09-10T20:35:43.380792+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:35:43.380792+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "43'4000", 'ondisk_log_start': "43'4000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26824704, 'num_objects': 6549, 'num_object_clones': 0, 'num_object_copies': 13098, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6549, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10654, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:48.389 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:35:48.389 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'13966", 'reported_seq': 14161, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:43.984256+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:35:43.984256+0000', 'last_peered': '2024-09-10T20:35:43.984256+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:35:43.984256+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "43'3900", 'ondisk_log_start': "43'3900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26513408, 'num_objects': 6473, 'num_object_clones': 0, 'num_object_copies': 12946, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6473, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11959, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2976}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:35:48.389 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:35:48.389 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'14185", 'reported_seq': 14392, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:43.459304+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:35:43.459304+0000', 'last_peered': '2024-09-10T20:35:43.459304+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:35:43.459304+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "43'4100", 'ondisk_log_start': "43'4100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26809286, 'num_objects': 6547, 'num_object_clones': 0, 'num_object_copies': 13094, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6547, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14183, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2981}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:48.390 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:35:48.390 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'14134", 'reported_seq': 14322, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:44.013203+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:35:44.013203+0000', 'last_peered': '2024-09-10T20:35:44.013203+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:35:44.013203+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "43'4100", 'ondisk_log_start': "43'4100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26341376, 'num_objects': 6431, 'num_object_clones': 0, 'num_object_copies': 12862, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6431, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11962, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2825}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:35:48.390 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:35:48.390 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'14141", 'reported_seq': 18832, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:44.049383+0000', 'last_change': '2024-09-10T20:34:25.241937+0000', 'last_active': '2024-09-10T20:35:44.049383+0000', 'last_peered': '2024-09-10T20:35:44.049383+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:35:44.049383+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "43'4100", 'ondisk_log_start': "43'4100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26718208, 'num_objects': 6523, 'num_object_clones': 0, 'num_object_copies': 13046, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1334, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6523, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11927, 'num_write_kb': 32484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2278, 'num_bytes_recovered': 8085504, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 5189}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:35:48.390 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:35:48.390 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'14230", 'reported_seq': 14482, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:43.403597+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:35:43.403597+0000', 'last_peered': '2024-09-10T20:35:43.403597+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:35:43.403597+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "43'4200", 'ondisk_log_start': "43'4200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27033600, 'num_objects': 6600, 'num_object_clones': 0, 'num_object_copies': 13200, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6600, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10953, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3008}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:48.390 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:35:48.390 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'14025", 'reported_seq': 14265, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:43.403218+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:35:43.403218+0000', 'last_peered': '2024-09-10T20:35:43.403218+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:35:43.403218+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "43'4000", 'ondisk_log_start': "43'4000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26865664, 'num_objects': 6559, 'num_object_clones': 0, 'num_object_copies': 13118, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3487, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6559, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10719, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 48, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3075}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:48.390 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:35:48.390 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'14398", 'reported_seq': 14651, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:43.354802+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:35:43.354802+0000', 'last_peered': '2024-09-10T20:35:43.354802+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:35:43.354802+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "43'4300", 'ondisk_log_start': "43'4300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26984448, 'num_objects': 6588, 'num_object_clones': 0, 'num_object_copies': 13176, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6588, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11225, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2933}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:48.391 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:35:49.081 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:49.082 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33338/209320 objects degraded (15.927%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":460,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":104660,"data_bytes":428664276,"bytes_used":2058031104,"bytes_avail":287852261376,"bytes_total":289910292480,"degraded_objects":33338,"degraded_total":209320,"degraded_ratio":0.15926810624880566,"recovering_objects_per_sec":28,"recovering_bytes_per_sec":9212,"recovering_keys_per_sec":0,"num_objects_recovered":340,"num_bytes_recovered":110592,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":397},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:35:49.110 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:35:49.110 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:35:49.656 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:49.656 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:35:49.678 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":271,"stamp":"2024-09-10T20:35:48.966832+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":422905300,"num_objects":103254,"num_object_clones":0,"num_object_copies":206508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33338,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103254,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":192167,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9813,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160872,"ondisk_log_size":160872,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2006400,"kb_used_data":1004816,"kb_used_omap":12,"kb_used_meta":1001523,"kb_avail":281109120,"statfs":{"total":289910292480,"available":287855738880,"internally_reserved":0,"allocated":1028931584,"data_stored":1028112830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1025559554},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,2,4,6,11,13],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25296896,"num_objects":-6176,"num_object_clones":0,"num_object_copies":-12352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-313,"num_objects_misplaced":-27,"num_objects_unfound":0,"num_objects_dirty":-6176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6176,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":110592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004284"},"pg_stats":[{"pgid":"2.f","version":"47'14291","reported_seq":14508,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:48.454992+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:35:48.454992+0000","last_peered":"2024-09-10T20:35:48.454992+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:35:48.454992+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26079232,"num_objects":6367,"num_object_clones":0,"num_object_copies":12734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14291,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2826}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14333","reported_seq":14448,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:44.011313+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:35:44.011313+0000","last_peered":"2024-09-10T20:35:44.011313+0000","last_clean":"2024-09-10T20:35:44.011313+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:35:44.011313+0000","last_undegraded":"2024-09-10T20:35:44.011313+0000","last_fullsized":"2024-09-10T20:35:44.011313+0000","mapping_epoch":35,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27070464,"num_objects":6609,"num_object_clones":0,"num_object_copies":13218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12519,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14188","reported_seq":21541,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:44.903654+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:35:44.903654+0000","last_peered":"2024-09-10T20:35:44.903654+0000","last_clean":"2024-09-10T20:35:44.903654+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:35:44.903654+0000","last_undegraded":"2024-09-10T20:35:44.903654+0000","last_fullsized":"2024-09-10T20:35:44.903654+0000","mapping_epoch":44,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26734616,"num_objects":6528,"num_object_clones":0,"num_object_copies":13056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6528,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12316,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'14080","reported_seq":14165,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:44.007438+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:35:44.007438+0000","last_peered":"2024-09-10T20:35:44.007438+0000","last_clean":"2024-09-10T20:35:44.007438+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:35:44.007438+0000","last_undegraded":"2024-09-10T20:35:44.007438+0000","last_fullsized":"2024-09-10T20:35:44.007438+0000","mapping_epoch":35,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26722304,"num_objects":6524,"num_object_clones":0,"num_object_copies":13048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14080,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14560","reported_seq":14692,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:48.460163+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:35:48.460163+0000","last_peered":"2024-09-10T20:35:48.460163+0000","last_clean":"2024-09-10T20:35:48.460163+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:35:48.460163+0000","last_undegraded":"2024-09-10T20:35:48.460163+0000","last_fullsized":"2024-09-10T20:35:48.460163+0000","mapping_epoch":34,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26660864,"num_objects":6509,"num_object_clones":0,"num_object_copies":13018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11556,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14201","reported_seq":14446,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:48.477144+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:35:48.477144+0000","last_peered":"2024-09-10T20:35:48.477144+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:35:48.477144+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25681920,"num_objects":6270,"num_object_clones":0,"num_object_copies":12540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11065,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2709}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'14220","reported_seq":14501,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:48.400673+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:35:48.400673+0000","last_peered":"2024-09-10T20:35:48.400673+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:35:48.400673+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26165248,"num_objects":6388,"num_object_clones":0,"num_object_copies":12776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10815,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2902}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14339","reported_seq":14468,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:48.448480+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:35:48.448480+0000","last_peered":"2024-09-10T20:35:48.448480+0000","last_clean":"2024-09-10T20:35:48.448480+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:35:48.448480+0000","last_undegraded":"2024-09-10T20:35:48.448480+0000","last_fullsized":"2024-09-10T20:35:48.448480+0000","mapping_epoch":34,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26202112,"num_objects":6397,"num_object_clones":0,"num_object_copies":12794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11410,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13966","reported_seq":14161,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:43.984256+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:35:43.984256+0000","last_peered":"2024-09-10T20:35:43.984256+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:35:43.984256+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'3900","ondisk_log_start":"43'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26513408,"num_objects":6473,"num_object_clones":0,"num_object_copies":12946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11959,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2976}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086274+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086274+0000","last_peered":"2024-09-10T20:34:27.086274+0000","last_clean":"2024-09-10T20:34:27.086274+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086274+0000","last_undegraded":"2024-09-10T20:34:27.086274+0000","last_fullsized":"2024-09-10T20:34:27.086274+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14341","reported_seq":14553,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:48.446162+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:35:48.446162+0000","last_peered":"2024-09-10T20:35:48.446162+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:35:48.446162+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26170310,"num_objects":6391,"num_object_clones":0,"num_object_copies":12782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14339,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2825}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14134","reported_seq":14322,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:44.013203+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:35:44.013203+0000","last_peered":"2024-09-10T20:35:44.013203+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:35:44.013203+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26341376,"num_objects":6431,"num_object_clones":0,"num_object_copies":12862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11962,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2825}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14141","reported_seq":18832,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:44.049383+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:35:44.049383+0000","last_peered":"2024-09-10T20:35:44.049383+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:35:44.049383+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26718208,"num_objects":6523,"num_object_clones":0,"num_object_copies":13046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1334,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11927,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2278,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":5189}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14360","reported_seq":14611,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:48.386045+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:35:48.386045+0000","last_peered":"2024-09-10T20:35:48.386045+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:35:48.386045+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26501120,"num_objects":6470,"num_object_clones":0,"num_object_copies":12940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11083,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2878}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14183","reported_seq":14422,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:48.503613+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:35:48.503613+0000","last_peered":"2024-09-10T20:35:48.503613+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:35:48.503613+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26218496,"num_objects":6401,"num_object_clones":0,"num_object_copies":12802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10877,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086784+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086784+0000","last_peered":"2024-09-10T20:34:27.086784+0000","last_clean":"2024-09-10T20:34:27.086784+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086784+0000","last_undegraded":"2024-09-10T20:34:27.086784+0000","last_fullsized":"2024-09-10T20:34:27.086784+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14571","reported_seq":14824,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:48.480041+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:35:48.480041+0000","last_peered":"2024-09-10T20:35:48.480041+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:35:48.480041+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26275840,"num_objects":6415,"num_object_clones":0,"num_object_copies":12830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11398,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2760}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13961","reported_seq":21081,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:44.968399+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:35:44.968399+0000","last_peered":"2024-09-10T20:35:44.968399+0000","last_clean":"2024-09-10T20:35:44.968399+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:35:44.968399+0000","last_undegraded":"2024-09-10T20:35:44.968399+0000","last_fullsized":"2024-09-10T20:35:44.968399+0000","mapping_epoch":46,"log_start":"43'3900","ondisk_log_start":"43'3900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26849763,"num_objects":6556,"num_object_clones":0,"num_object_copies":13112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10568,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":422905281,"num_objects":103252,"num_object_clones":0,"num_object_copies":206504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33338,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103252,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":192165,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9811,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1024937984,"data_stored":1024861193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160869,"ondisk_log_size":160869,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182487,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":741284,"kb_used_data":366740,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93630556,"statfs":{"total":96636764160,"available":95877689344,"internally_reserved":0,"allocated":375541760,"data_stored":375289467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":383512234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,0,0,4,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986258,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":631916,"kb_used_data":325340,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93739924,"statfs":{"total":96636764160,"available":95989682176,"internally_reserved":0,"allocated":333148160,"data_stored":332895872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313913002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659205,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":633200,"kb_used_data":312736,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93738640,"statfs":{"total":96636764160,"available":95988367360,"internally_reserved":0,"allocated":320241664,"data_stored":319927491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":328134318},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,6,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":373817344,"data_stored":373813731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":319143936,"data_stored":319078443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":331976704,"data_stored":331969019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:35:52.681 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:35:53.262 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:53.263 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:35:53.284 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":273,"stamp":"2024-09-10T20:35:52.968325+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":418645460,"num_objects":102214,"num_object_clones":0,"num_object_copies":204428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33204,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102214,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":193207,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9947,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160912,"ondisk_log_size":160912,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2024816,"kb_used_data":992700,"kb_used_omap":12,"kb_used_meta":1032051,"kb_avail":281090704,"statfs":{"total":289910292480,"available":287836880896,"internally_reserved":0,"allocated":1016524800,"data_stored":1015706046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1056820226},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,2,1,7,16,14],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20029440,"num_objects":-4890,"num_object_clones":0,"num_object_copies":-9780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-281,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4890,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":281,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004097"},"pg_stats":[{"pgid":"2.f","version":"47'14291","reported_seq":14508,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:48.454992+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:35:48.454992+0000","last_peered":"2024-09-10T20:35:48.454992+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:35:48.454992+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26079232,"num_objects":6367,"num_object_clones":0,"num_object_copies":12734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14291,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2826}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14485","reported_seq":14600,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:48.985583+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:35:48.985583+0000","last_peered":"2024-09-10T20:35:48.985583+0000","last_clean":"2024-09-10T20:35:48.985583+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:35:48.985583+0000","last_undegraded":"2024-09-10T20:35:48.985583+0000","last_fullsized":"2024-09-10T20:35:48.985583+0000","mapping_epoch":35,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26447872,"num_objects":6457,"num_object_clones":0,"num_object_copies":12914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12671,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14343","reported_seq":21696,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:49.999130+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:35:49.999130+0000","last_peered":"2024-09-10T20:35:49.999130+0000","last_clean":"2024-09-10T20:35:49.999130+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:35:49.999130+0000","last_undegraded":"2024-09-10T20:35:49.999130+0000","last_fullsized":"2024-09-10T20:35:49.999130+0000","mapping_epoch":44,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26099736,"num_objects":6373,"num_object_clones":0,"num_object_copies":12746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6373,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12471,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'14217","reported_seq":14302,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:49.043681+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:35:49.043681+0000","last_peered":"2024-09-10T20:35:49.043681+0000","last_clean":"2024-09-10T20:35:49.043681+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:35:49.043681+0000","last_undegraded":"2024-09-10T20:35:49.043681+0000","last_fullsized":"2024-09-10T20:35:49.043681+0000","mapping_epoch":35,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26161152,"num_objects":6387,"num_object_clones":0,"num_object_copies":12774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14217,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14560","reported_seq":14692,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:48.460163+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:35:48.460163+0000","last_peered":"2024-09-10T20:35:48.460163+0000","last_clean":"2024-09-10T20:35:48.460163+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:35:48.460163+0000","last_undegraded":"2024-09-10T20:35:48.460163+0000","last_fullsized":"2024-09-10T20:35:48.460163+0000","mapping_epoch":34,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26660864,"num_objects":6509,"num_object_clones":0,"num_object_copies":13018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11556,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14201","reported_seq":14446,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:48.477144+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:35:48.477144+0000","last_peered":"2024-09-10T20:35:48.477144+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:35:48.477144+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25681920,"num_objects":6270,"num_object_clones":0,"num_object_copies":12540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11065,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2709}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'14220","reported_seq":14501,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:48.400673+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:35:48.400673+0000","last_peered":"2024-09-10T20:35:48.400673+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:35:48.400673+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26165248,"num_objects":6388,"num_object_clones":0,"num_object_copies":12776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10815,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2902}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14339","reported_seq":14468,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:48.448480+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:35:48.448480+0000","last_peered":"2024-09-10T20:35:48.448480+0000","last_clean":"2024-09-10T20:35:48.448480+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:35:48.448480+0000","last_undegraded":"2024-09-10T20:35:48.448480+0000","last_fullsized":"2024-09-10T20:35:48.448480+0000","mapping_epoch":34,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26202112,"num_objects":6397,"num_object_clones":0,"num_object_copies":12794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11410,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14099","reported_seq":14294,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:49.071247+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:35:49.071247+0000","last_peered":"2024-09-10T20:35:49.071247+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:35:49.071247+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25968640,"num_objects":6340,"num_object_clones":0,"num_object_copies":12680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12092,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2843}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086274+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086274+0000","last_peered":"2024-09-10T20:34:27.086274+0000","last_clean":"2024-09-10T20:34:27.086274+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086274+0000","last_undegraded":"2024-09-10T20:34:27.086274+0000","last_fullsized":"2024-09-10T20:34:27.086274+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14341","reported_seq":14553,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:48.446162+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:35:48.446162+0000","last_peered":"2024-09-10T20:35:48.446162+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:35:48.446162+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26170310,"num_objects":6391,"num_object_clones":0,"num_object_copies":12782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14339,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2825}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14271","reported_seq":14457,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:49.062226+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:35:49.062226+0000","last_peered":"2024-09-10T20:35:49.062226+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:35:49.062226+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25780224,"num_objects":6294,"num_object_clones":0,"num_object_copies":12588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12099,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2688}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14295","reported_seq":19253,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:49.049007+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:35:49.049007+0000","last_peered":"2024-09-10T20:35:49.049007+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:35:49.049007+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26087424,"num_objects":6369,"num_object_clones":0,"num_object_copies":12738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1200,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12081,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2412,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":5169}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14360","reported_seq":14611,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:48.386045+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:35:48.386045+0000","last_peered":"2024-09-10T20:35:48.386045+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:35:48.386045+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26501120,"num_objects":6470,"num_object_clones":0,"num_object_copies":12940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11083,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2878}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14183","reported_seq":14422,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:48.503613+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:35:48.503613+0000","last_peered":"2024-09-10T20:35:48.503613+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:35:48.503613+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26218496,"num_objects":6401,"num_object_clones":0,"num_object_copies":12802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10877,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086784+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086784+0000","last_peered":"2024-09-10T20:34:27.086784+0000","last_clean":"2024-09-10T20:34:27.086784+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086784+0000","last_undegraded":"2024-09-10T20:34:27.086784+0000","last_fullsized":"2024-09-10T20:34:27.086784+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14571","reported_seq":14824,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:48.480041+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:35:48.480041+0000","last_peered":"2024-09-10T20:35:48.480041+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:35:48.480041+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26275840,"num_objects":6415,"num_object_clones":0,"num_object_copies":12830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11398,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2760}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14133","reported_seq":21255,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:49.971676+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:35:49.971676+0000","last_peered":"2024-09-10T20:35:49.971676+0000","last_clean":"2024-09-10T20:35:49.971676+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:35:49.971676+0000","last_undegraded":"2024-09-10T20:35:49.971676+0000","last_fullsized":"2024-09-10T20:35:49.971676+0000","mapping_epoch":46,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26145251,"num_objects":6384,"num_object_clones":0,"num_object_copies":12768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10740,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":418645441,"num_objects":102212,"num_object_clones":0,"num_object_copies":204424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33204,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102212,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":193205,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9945,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1012592640,"data_stored":1012515849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160909,"ondisk_log_size":160909,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182487,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":741284,"kb_used_data":366740,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93630556,"statfs":{"total":96636764160,"available":95877689344,"internally_reserved":0,"allocated":375541760,"data_stored":375289467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":383512234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,0,0,4,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986259,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":649300,"kb_used_data":320512,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93722540,"statfs":{"total":96636764160,"available":95971880960,"internally_reserved":0,"allocated":328204288,"data_stored":327952000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336653994},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659206,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":634232,"kb_used_data":305448,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93737608,"statfs":{"total":96636764160,"available":95987310592,"internally_reserved":0,"allocated":312778752,"data_stored":312464579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":336653998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,6,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":373817344,"data_stored":373813731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":311635968,"data_stored":311570475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":327139328,"data_stored":327131643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:35:53.286 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:35:53.862 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:53.863 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:35:53.886 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":273,"stamp":"2024-09-10T20:35:52.968325+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":418645460,"num_objects":102214,"num_object_clones":0,"num_object_copies":204428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33204,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102214,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":193207,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9947,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160912,"ondisk_log_size":160912,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2024816,"kb_used_data":992700,"kb_used_omap":12,"kb_used_meta":1032051,"kb_avail":281090704,"statfs":{"total":289910292480,"available":287836880896,"internally_reserved":0,"allocated":1016524800,"data_stored":1015706046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1056820226},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,2,1,7,16,14],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20029440,"num_objects":-4890,"num_object_clones":0,"num_object_copies":-9780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-281,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4890,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":281,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004097"},"pg_stats":[{"pgid":"2.f","version":"47'14291","reported_seq":14508,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:48.454992+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:35:48.454992+0000","last_peered":"2024-09-10T20:35:48.454992+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:35:48.454992+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26079232,"num_objects":6367,"num_object_clones":0,"num_object_copies":12734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14291,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2826}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14485","reported_seq":14600,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:48.985583+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:35:48.985583+0000","last_peered":"2024-09-10T20:35:48.985583+0000","last_clean":"2024-09-10T20:35:48.985583+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:35:48.985583+0000","last_undegraded":"2024-09-10T20:35:48.985583+0000","last_fullsized":"2024-09-10T20:35:48.985583+0000","mapping_epoch":35,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26447872,"num_objects":6457,"num_object_clones":0,"num_object_copies":12914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12671,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14343","reported_seq":21696,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:49.999130+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:35:49.999130+0000","last_peered":"2024-09-10T20:35:49.999130+0000","last_clean":"2024-09-10T20:35:49.999130+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:35:49.999130+0000","last_undegraded":"2024-09-10T20:35:49.999130+0000","last_fullsized":"2024-09-10T20:35:49.999130+0000","mapping_epoch":44,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26099736,"num_objects":6373,"num_object_clones":0,"num_object_copies":12746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6373,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12471,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'14217","reported_seq":14302,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:49.043681+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:35:49.043681+0000","last_peered":"2024-09-10T20:35:49.043681+0000","last_clean":"2024-09-10T20:35:49.043681+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:35:49.043681+0000","last_undegraded":"2024-09-10T20:35:49.043681+0000","last_fullsized":"2024-09-10T20:35:49.043681+0000","mapping_epoch":35,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26161152,"num_objects":6387,"num_object_clones":0,"num_object_copies":12774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14217,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14560","reported_seq":14692,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:48.460163+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:35:48.460163+0000","last_peered":"2024-09-10T20:35:48.460163+0000","last_clean":"2024-09-10T20:35:48.460163+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:35:48.460163+0000","last_undegraded":"2024-09-10T20:35:48.460163+0000","last_fullsized":"2024-09-10T20:35:48.460163+0000","mapping_epoch":34,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26660864,"num_objects":6509,"num_object_clones":0,"num_object_copies":13018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11556,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14201","reported_seq":14446,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:48.477144+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:35:48.477144+0000","last_peered":"2024-09-10T20:35:48.477144+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:35:48.477144+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25681920,"num_objects":6270,"num_object_clones":0,"num_object_copies":12540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11065,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2709}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'14220","reported_seq":14501,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:48.400673+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:35:48.400673+0000","last_peered":"2024-09-10T20:35:48.400673+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:35:48.400673+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26165248,"num_objects":6388,"num_object_clones":0,"num_object_copies":12776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10815,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2902}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14339","reported_seq":14468,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:48.448480+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:35:48.448480+0000","last_peered":"2024-09-10T20:35:48.448480+0000","last_clean":"2024-09-10T20:35:48.448480+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:35:48.448480+0000","last_undegraded":"2024-09-10T20:35:48.448480+0000","last_fullsized":"2024-09-10T20:35:48.448480+0000","mapping_epoch":34,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26202112,"num_objects":6397,"num_object_clones":0,"num_object_copies":12794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11410,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14099","reported_seq":14294,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:49.071247+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:35:49.071247+0000","last_peered":"2024-09-10T20:35:49.071247+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:35:49.071247+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25968640,"num_objects":6340,"num_object_clones":0,"num_object_copies":12680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12092,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2843}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086274+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086274+0000","last_peered":"2024-09-10T20:34:27.086274+0000","last_clean":"2024-09-10T20:34:27.086274+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086274+0000","last_undegraded":"2024-09-10T20:34:27.086274+0000","last_fullsized":"2024-09-10T20:34:27.086274+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14341","reported_seq":14553,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:48.446162+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:35:48.446162+0000","last_peered":"2024-09-10T20:35:48.446162+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:35:48.446162+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26170310,"num_objects":6391,"num_object_clones":0,"num_object_copies":12782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14339,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2825}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14271","reported_seq":14457,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:49.062226+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:35:49.062226+0000","last_peered":"2024-09-10T20:35:49.062226+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:35:49.062226+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25780224,"num_objects":6294,"num_object_clones":0,"num_object_copies":12588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12099,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2688}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14295","reported_seq":19253,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:49.049007+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:35:49.049007+0000","last_peered":"2024-09-10T20:35:49.049007+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:35:49.049007+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26087424,"num_objects":6369,"num_object_clones":0,"num_object_copies":12738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1200,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12081,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2412,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":5169}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14360","reported_seq":14611,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:48.386045+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:35:48.386045+0000","last_peered":"2024-09-10T20:35:48.386045+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:35:48.386045+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26501120,"num_objects":6470,"num_object_clones":0,"num_object_copies":12940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11083,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2878}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14183","reported_seq":14422,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:48.503613+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:35:48.503613+0000","last_peered":"2024-09-10T20:35:48.503613+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:35:48.503613+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26218496,"num_objects":6401,"num_object_clones":0,"num_object_copies":12802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10877,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086784+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086784+0000","last_peered":"2024-09-10T20:34:27.086784+0000","last_clean":"2024-09-10T20:34:27.086784+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086784+0000","last_undegraded":"2024-09-10T20:34:27.086784+0000","last_fullsized":"2024-09-10T20:34:27.086784+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14571","reported_seq":14824,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:48.480041+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:35:48.480041+0000","last_peered":"2024-09-10T20:35:48.480041+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:35:48.480041+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26275840,"num_objects":6415,"num_object_clones":0,"num_object_copies":12830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11398,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2760}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14133","reported_seq":21255,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:49.971676+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:35:49.971676+0000","last_peered":"2024-09-10T20:35:49.971676+0000","last_clean":"2024-09-10T20:35:49.971676+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:35:49.971676+0000","last_undegraded":"2024-09-10T20:35:49.971676+0000","last_fullsized":"2024-09-10T20:35:49.971676+0000","mapping_epoch":46,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26145251,"num_objects":6384,"num_object_clones":0,"num_object_copies":12768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10740,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":418645441,"num_objects":102212,"num_object_clones":0,"num_object_copies":204424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33204,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102212,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":193205,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9945,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1012592640,"data_stored":1012515849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160909,"ondisk_log_size":160909,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182487,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":741284,"kb_used_data":366740,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93630556,"statfs":{"total":96636764160,"available":95877689344,"internally_reserved":0,"allocated":375541760,"data_stored":375289467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":383512234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,0,0,4,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986259,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":649300,"kb_used_data":320512,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93722540,"statfs":{"total":96636764160,"available":95971880960,"internally_reserved":0,"allocated":328204288,"data_stored":327952000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336653994},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659206,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":634232,"kb_used_data":305448,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93737608,"statfs":{"total":96636764160,"available":95987310592,"internally_reserved":0,"allocated":312778752,"data_stored":312464579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":336653998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,6,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":373817344,"data_stored":373813731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":311635968,"data_stored":311570475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":327139328,"data_stored":327131643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:35:53.887 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:35:53.887 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'14291", 'reported_seq': 14508, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:48.454992+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:35:48.454992+0000', 'last_peered': '2024-09-10T20:35:48.454992+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:35:48.454992+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "43'4200", 'ondisk_log_start': "43'4200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26079232, 'num_objects': 6367, 'num_object_clones': 0, 'num_object_copies': 12734, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6367, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14291, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2826}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:53.887 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:35:53.887 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'14201", 'reported_seq': 14446, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:48.477144+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:35:48.477144+0000', 'last_peered': '2024-09-10T20:35:48.477144+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:35:48.477144+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "43'4200", 'ondisk_log_start': "43'4200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25681920, 'num_objects': 6270, 'num_object_clones': 0, 'num_object_copies': 12540, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6270, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11065, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2709}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:53.887 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:35:53.887 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'14220", 'reported_seq': 14501, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:48.400673+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:35:48.400673+0000', 'last_peered': '2024-09-10T20:35:48.400673+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:35:48.400673+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "43'4200", 'ondisk_log_start': "43'4200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26165248, 'num_objects': 6388, 'num_object_clones': 0, 'num_object_copies': 12776, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6388, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10815, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2902}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:53.888 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:35:53.888 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'14099", 'reported_seq': 14294, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:49.071247+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:35:49.071247+0000', 'last_peered': '2024-09-10T20:35:49.071247+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:35:49.071247+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "43'4000", 'ondisk_log_start': "43'4000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25968640, 'num_objects': 6340, 'num_object_clones': 0, 'num_object_copies': 12680, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6340, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12092, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2843}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:35:53.888 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:35:53.888 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'14341", 'reported_seq': 14553, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:48.446162+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:35:48.446162+0000', 'last_peered': '2024-09-10T20:35:48.446162+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:35:48.446162+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "43'4300", 'ondisk_log_start': "43'4300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26170310, 'num_objects': 6391, 'num_object_clones': 0, 'num_object_copies': 12782, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6391, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14339, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2825}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:53.888 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:35:53.888 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'14271", 'reported_seq': 14457, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:49.062226+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:35:49.062226+0000', 'last_peered': '2024-09-10T20:35:49.062226+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:35:49.062226+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "43'4200", 'ondisk_log_start': "43'4200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25780224, 'num_objects': 6294, 'num_object_clones': 0, 'num_object_copies': 12588, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6294, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12099, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2688}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:35:53.888 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:35:53.888 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'14295", 'reported_seq': 19253, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:49.049007+0000', 'last_change': '2024-09-10T20:34:25.241937+0000', 'last_active': '2024-09-10T20:35:49.049007+0000', 'last_peered': '2024-09-10T20:35:49.049007+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:35:49.049007+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "43'4200", 'ondisk_log_start': "43'4200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26087424, 'num_objects': 6369, 'num_object_clones': 0, 'num_object_copies': 12738, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1200, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6369, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12081, 'num_write_kb': 32484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2412, 'num_bytes_recovered': 8085504, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 5169}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:35:53.888 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:35:53.888 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'14360", 'reported_seq': 14611, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:48.386045+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:35:48.386045+0000', 'last_peered': '2024-09-10T20:35:48.386045+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:35:48.386045+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "43'4300", 'ondisk_log_start': "43'4300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26501120, 'num_objects': 6470, 'num_object_clones': 0, 'num_object_copies': 12940, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6470, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11083, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2878}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:53.889 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:35:53.889 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'14183", 'reported_seq': 14422, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:48.503613+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:35:48.503613+0000', 'last_peered': '2024-09-10T20:35:48.503613+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:35:48.503613+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "43'4100", 'ondisk_log_start': "43'4100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26218496, 'num_objects': 6401, 'num_object_clones': 0, 'num_object_copies': 12802, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3487, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6401, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10877, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 48, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:53.889 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:35:53.889 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'14571", 'reported_seq': 14824, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:48.480041+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:35:48.480041+0000', 'last_peered': '2024-09-10T20:35:48.480041+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:35:48.480041+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "43'4500", 'ondisk_log_start': "43'4500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26275840, 'num_objects': 6415, 'num_object_clones': 0, 'num_object_copies': 12830, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6415, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11398, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2760}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:53.889 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:35:54.585 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:54.585 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33204/204428 objects degraded (16.242%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":466,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":102214,"data_bytes":418645460,"bytes_used":2073411584,"bytes_avail":287836880896,"bytes_total":289910292480,"degraded_objects":33204,"degraded_total":204428,"degraded_ratio":0.1624239340990471,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":281,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":407},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:35:54.617 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:35:54.617 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:35:55.178 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:55.180 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:35:55.210 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":274,"stamp":"2024-09-10T20:35:54.969105+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":409818580,"num_objects":100059,"num_object_clones":0,"num_object_copies":200118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33071,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100059,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":195362,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10080,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2006940,"kb_used_data":978984,"kb_used_omap":12,"kb_used_meta":1027891,"kb_avail":281108580,"statfs":{"total":289910292480,"available":287855185920,"internally_reserved":0,"allocated":1002479616,"data_stored":1001660862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1052560386},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,0,2,2,6,10,15],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28856320,"num_objects":-7045,"num_object_clones":0,"num_object_copies":-14090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-414,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7045,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":414,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004053"},"pg_stats":[{"pgid":"2.f","version":"47'14426","reported_seq":14644,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:53.503466+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:35:53.503466+0000","last_peered":"2024-09-10T20:35:53.503466+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:35:53.503466+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25526272,"num_objects":6232,"num_object_clones":0,"num_object_copies":12464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6232,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14426,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2691}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14664","reported_seq":14779,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:53.946142+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:35:53.946142+0000","last_peered":"2024-09-10T20:35:53.946142+0000","last_clean":"2024-09-10T20:35:53.946142+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:35:53.946142+0000","last_undegraded":"2024-09-10T20:35:53.946142+0000","last_fullsized":"2024-09-10T20:35:53.946142+0000","mapping_epoch":35,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25714688,"num_objects":6278,"num_object_clones":0,"num_object_copies":12556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6278,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12850,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14343","reported_seq":21696,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:49.999130+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:35:49.999130+0000","last_peered":"2024-09-10T20:35:49.999130+0000","last_clean":"2024-09-10T20:35:49.999130+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:35:49.999130+0000","last_undegraded":"2024-09-10T20:35:49.999130+0000","last_fullsized":"2024-09-10T20:35:49.999130+0000","mapping_epoch":44,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26099736,"num_objects":6373,"num_object_clones":0,"num_object_copies":12746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6373,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12471,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'14360","reported_seq":14445,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:54.059918+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:35:54.059918+0000","last_peered":"2024-09-10T20:35:54.059918+0000","last_clean":"2024-09-10T20:35:54.059918+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:35:54.059918+0000","last_undegraded":"2024-09-10T20:35:54.059918+0000","last_fullsized":"2024-09-10T20:35:54.059918+0000","mapping_epoch":35,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25575424,"num_objects":6244,"num_object_clones":0,"num_object_copies":12488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14360,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14721","reported_seq":14853,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:53.426917+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:35:53.426917+0000","last_peered":"2024-09-10T20:35:53.426917+0000","last_clean":"2024-09-10T20:35:53.426917+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:35:53.426917+0000","last_undegraded":"2024-09-10T20:35:53.426917+0000","last_fullsized":"2024-09-10T20:35:53.426917+0000","mapping_epoch":34,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26001408,"num_objects":6348,"num_object_clones":0,"num_object_copies":12696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11717,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14353","reported_seq":14601,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:53.390336+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:35:53.390336+0000","last_peered":"2024-09-10T20:35:53.390336+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:35:53.390336+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25059328,"num_objects":6118,"num_object_clones":0,"num_object_copies":12236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11217,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2557}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'14370","reported_seq":14651,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:53.477950+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:35:53.477950+0000","last_peered":"2024-09-10T20:35:53.477950+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:35:53.477950+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25550848,"num_objects":6238,"num_object_clones":0,"num_object_copies":12476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10965,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2752}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14506","reported_seq":14636,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:53.464044+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:35:53.464044+0000","last_peered":"2024-09-10T20:35:53.464044+0000","last_clean":"2024-09-10T20:35:53.464044+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:35:53.464044+0000","last_undegraded":"2024-09-10T20:35:53.464044+0000","last_fullsized":"2024-09-10T20:35:53.464044+0000","mapping_epoch":34,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25518080,"num_objects":6230,"num_object_clones":0,"num_object_copies":12460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11577,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14264","reported_seq":14457,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:54.076176+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:35:54.076176+0000","last_peered":"2024-09-10T20:35:54.076176+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:35:54.076176+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25292800,"num_objects":6175,"num_object_clones":0,"num_object_copies":12350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12257,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2678}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086274+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086274+0000","last_peered":"2024-09-10T20:34:27.086274+0000","last_clean":"2024-09-10T20:34:27.086274+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086274+0000","last_undegraded":"2024-09-10T20:34:27.086274+0000","last_fullsized":"2024-09-10T20:34:27.086274+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14505","reported_seq":14713,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:53.435998+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:35:53.435998+0000","last_peered":"2024-09-10T20:35:53.435998+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:35:53.435998+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25498566,"num_objects":6227,"num_object_clones":0,"num_object_copies":12454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14503,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2661}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14409","reported_seq":14597,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:54.003418+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:35:54.003418+0000","last_peered":"2024-09-10T20:35:54.003418+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:35:54.003418+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25214976,"num_objects":6156,"num_object_clones":0,"num_object_copies":12312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12237,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2550}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14454","reported_seq":19679,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:54.101587+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:35:54.101587+0000","last_peered":"2024-09-10T20:35:54.101587+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:35:54.101587+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25436160,"num_objects":6210,"num_object_clones":0,"num_object_copies":12420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12240,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2545,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":5143}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14503","reported_seq":14755,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:53.504535+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:35:53.504535+0000","last_peered":"2024-09-10T20:35:53.504535+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:35:53.504535+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25915392,"num_objects":6327,"num_object_clones":0,"num_object_copies":12654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11226,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2735}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14353","reported_seq":14593,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:53.459917+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:35:53.459917+0000","last_peered":"2024-09-10T20:35:53.459917+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:35:53.459917+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25522176,"num_objects":6231,"num_object_clones":0,"num_object_copies":12462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11047,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2747}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086784+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086784+0000","last_peered":"2024-09-10T20:34:27.086784+0000","last_clean":"2024-09-10T20:34:27.086784+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086784+0000","last_undegraded":"2024-09-10T20:34:27.086784+0000","last_fullsized":"2024-09-10T20:34:27.086784+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14700","reported_seq":14952,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:53.466978+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:35:53.466978+0000","last_peered":"2024-09-10T20:35:53.466978+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:35:53.466978+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25747456,"num_objects":6286,"num_object_clones":0,"num_object_copies":12572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11527,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2631}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14133","reported_seq":21255,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:49.971676+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:35:49.971676+0000","last_peered":"2024-09-10T20:35:49.971676+0000","last_clean":"2024-09-10T20:35:49.971676+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:35:49.971676+0000","last_undegraded":"2024-09-10T20:35:49.971676+0000","last_fullsized":"2024-09-10T20:35:49.971676+0000","mapping_epoch":46,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26145251,"num_objects":6384,"num_object_clones":0,"num_object_copies":12768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10740,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":409818561,"num_objects":100057,"num_object_clones":0,"num_object_copies":200114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33071,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100057,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":195360,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10078,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":999976960,"data_stored":999900169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182488,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":741004,"kb_used_data":358136,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93630836,"statfs":{"total":96636764160,"available":95877976064,"internally_reserved":0,"allocated":366731264,"data_stored":366478971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,0,0,4,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986260,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":631704,"kb_used_data":315400,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93740136,"statfs":{"total":96636764160,"available":95989899264,"internally_reserved":0,"allocated":322969600,"data_stored":322717312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323874474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,0,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659206,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":634232,"kb_used_data":305448,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93737608,"statfs":{"total":96636764160,"available":95987310592,"internally_reserved":0,"allocated":312778752,"data_stored":312464579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":336653998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,6,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":366317568,"data_stored":366313955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":311635968,"data_stored":311570475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":322023424,"data_stored":322015739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:35:58.212 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:35:58.781 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:58.782 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:35:58.802 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":275,"stamp":"2024-09-10T20:35:56.971017+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":408581588,"num_objects":99757,"num_object_clones":0,"num_object_copies":199514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33071,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99757,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":195664,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10080,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160769,"ondisk_log_size":160769,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1987216,"kb_used_data":971716,"kb_used_omap":12,"kb_used_meta":1015411,"kb_avail":281128304,"statfs":{"total":289910292480,"available":287875383296,"internally_reserved":0,"allocated":995037184,"data_stored":994218430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1039780866},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,0,2,4,6,6,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21372928,"num_objects":-5218,"num_object_clones":0,"num_object_copies":-10436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-267,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5218,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":267,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005242"},"pg_stats":[{"pgid":"2.f","version":"47'14426","reported_seq":14644,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:53.503466+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:35:53.503466+0000","last_peered":"2024-09-10T20:35:53.503466+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:35:53.503466+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25526272,"num_objects":6232,"num_object_clones":0,"num_object_copies":12464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6232,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14426,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2691}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14664","reported_seq":14779,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:53.946142+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:35:53.946142+0000","last_peered":"2024-09-10T20:35:53.946142+0000","last_clean":"2024-09-10T20:35:53.946142+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:35:53.946142+0000","last_undegraded":"2024-09-10T20:35:53.946142+0000","last_fullsized":"2024-09-10T20:35:53.946142+0000","mapping_epoch":35,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25714688,"num_objects":6278,"num_object_clones":0,"num_object_copies":12556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6278,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12850,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14490","reported_seq":21843,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:55.062246+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:35:55.062246+0000","last_peered":"2024-09-10T20:35:55.062246+0000","last_clean":"2024-09-10T20:35:55.062246+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:35:55.062246+0000","last_undegraded":"2024-09-10T20:35:55.062246+0000","last_fullsized":"2024-09-10T20:35:55.062246+0000","mapping_epoch":44,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25497624,"num_objects":6226,"num_object_clones":0,"num_object_copies":12452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6226,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12618,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'14360","reported_seq":14445,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:54.059918+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:35:54.059918+0000","last_peered":"2024-09-10T20:35:54.059918+0000","last_clean":"2024-09-10T20:35:54.059918+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:35:54.059918+0000","last_undegraded":"2024-09-10T20:35:54.059918+0000","last_fullsized":"2024-09-10T20:35:54.059918+0000","mapping_epoch":35,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25575424,"num_objects":6244,"num_object_clones":0,"num_object_copies":12488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14360,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14721","reported_seq":14853,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:53.426917+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:35:53.426917+0000","last_peered":"2024-09-10T20:35:53.426917+0000","last_clean":"2024-09-10T20:35:53.426917+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:35:53.426917+0000","last_undegraded":"2024-09-10T20:35:53.426917+0000","last_fullsized":"2024-09-10T20:35:53.426917+0000","mapping_epoch":34,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26001408,"num_objects":6348,"num_object_clones":0,"num_object_copies":12696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11717,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14353","reported_seq":14601,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:53.390336+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:35:53.390336+0000","last_peered":"2024-09-10T20:35:53.390336+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:35:53.390336+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25059328,"num_objects":6118,"num_object_clones":0,"num_object_copies":12236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11217,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2557}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'14370","reported_seq":14651,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:53.477950+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:35:53.477950+0000","last_peered":"2024-09-10T20:35:53.477950+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:35:53.477950+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25550848,"num_objects":6238,"num_object_clones":0,"num_object_copies":12476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10965,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2752}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14506","reported_seq":14636,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:53.464044+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:35:53.464044+0000","last_peered":"2024-09-10T20:35:53.464044+0000","last_clean":"2024-09-10T20:35:53.464044+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:35:53.464044+0000","last_undegraded":"2024-09-10T20:35:53.464044+0000","last_fullsized":"2024-09-10T20:35:53.464044+0000","mapping_epoch":34,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25518080,"num_objects":6230,"num_object_clones":0,"num_object_copies":12460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11577,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14264","reported_seq":14457,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:54.076176+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:35:54.076176+0000","last_peered":"2024-09-10T20:35:54.076176+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:35:54.076176+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25292800,"num_objects":6175,"num_object_clones":0,"num_object_copies":12350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12257,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2678}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086274+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086274+0000","last_peered":"2024-09-10T20:34:27.086274+0000","last_clean":"2024-09-10T20:34:27.086274+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086274+0000","last_undegraded":"2024-09-10T20:34:27.086274+0000","last_fullsized":"2024-09-10T20:34:27.086274+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14505","reported_seq":14713,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:53.435998+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:35:53.435998+0000","last_peered":"2024-09-10T20:35:53.435998+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:35:53.435998+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25498566,"num_objects":6227,"num_object_clones":0,"num_object_copies":12454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14503,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2661}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14409","reported_seq":14597,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:54.003418+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:35:54.003418+0000","last_peered":"2024-09-10T20:35:54.003418+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:35:54.003418+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25214976,"num_objects":6156,"num_object_clones":0,"num_object_copies":12312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12237,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2550}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14454","reported_seq":19679,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:54.101587+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:35:54.101587+0000","last_peered":"2024-09-10T20:35:54.101587+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:35:54.101587+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25436160,"num_objects":6210,"num_object_clones":0,"num_object_copies":12420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12240,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2545,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":5143}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14503","reported_seq":14755,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:53.504535+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:35:53.504535+0000","last_peered":"2024-09-10T20:35:53.504535+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:35:53.504535+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25915392,"num_objects":6327,"num_object_clones":0,"num_object_copies":12654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11226,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2735}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14353","reported_seq":14593,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:53.459917+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:35:53.459917+0000","last_peered":"2024-09-10T20:35:53.459917+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:35:53.459917+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25522176,"num_objects":6231,"num_object_clones":0,"num_object_copies":12462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11047,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2747}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086784+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086784+0000","last_peered":"2024-09-10T20:34:27.086784+0000","last_clean":"2024-09-10T20:34:27.086784+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086784+0000","last_undegraded":"2024-09-10T20:34:27.086784+0000","last_fullsized":"2024-09-10T20:34:27.086784+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14700","reported_seq":14952,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:53.466978+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:35:53.466978+0000","last_peered":"2024-09-10T20:35:53.466978+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:35:53.466978+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25747456,"num_objects":6286,"num_object_clones":0,"num_object_copies":12572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11527,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2631}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14288","reported_seq":21410,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:55.002812+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:35:55.002812+0000","last_peered":"2024-09-10T20:35:55.002812+0000","last_clean":"2024-09-10T20:35:55.002812+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:35:55.002812+0000","last_undegraded":"2024-09-10T20:35:55.002812+0000","last_fullsized":"2024-09-10T20:35:55.002812+0000","mapping_epoch":46,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25510371,"num_objects":6229,"num_object_clones":0,"num_object_copies":12458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10895,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":408581569,"num_objects":99755,"num_object_clones":0,"num_object_copies":199510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33071,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99755,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":195662,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10078,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":992563200,"data_stored":992486409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160766,"ondisk_log_size":160766,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182488,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":741004,"kb_used_data":358136,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93630836,"statfs":{"total":96636764160,"available":95877976064,"internally_reserved":0,"allocated":366731264,"data_stored":366478971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,0,0,4,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986260,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":631704,"kb_used_data":315400,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93740136,"statfs":{"total":96636764160,"available":95989899264,"internally_reserved":0,"allocated":322969600,"data_stored":322717312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323874474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,0,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659207,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":614508,"kb_used_data":298180,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93757332,"statfs":{"total":96636764160,"available":96007507968,"internally_reserved":0,"allocated":305336320,"data_stored":305022147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":323874478},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,2,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":366317568,"data_stored":366313955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":304222208,"data_stored":304156715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":322023424,"data_stored":322015739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:35:58.804 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:35:59.407 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:35:59.408 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:35:59.435 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":276,"stamp":"2024-09-10T20:35:58.971592+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":402847188,"num_objects":98357,"num_object_clones":0,"num_object_copies":196714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33071,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98357,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":197064,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10080,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160869,"ondisk_log_size":160869,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1985424,"kb_used_data":964360,"kb_used_omap":12,"kb_used_meta":1020979,"kb_avail":281130096,"statfs":{"total":289910292480,"available":287877218304,"internally_reserved":0,"allocated":987504640,"data_stored":986685886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1045482498},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,8,4,2,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25817088,"num_objects":-6303,"num_object_clones":0,"num_object_copies":-12606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-267,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6303,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":267,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005229"},"pg_stats":[{"pgid":"2.f","version":"47'14568","reported_seq":14786,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:58.533235+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:35:58.533235+0000","last_peered":"2024-09-10T20:35:58.533235+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:35:58.533235+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24944640,"num_objects":6090,"num_object_clones":0,"num_object_copies":12180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14568,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2549}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14664","reported_seq":14779,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:53.946142+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:35:53.946142+0000","last_peered":"2024-09-10T20:35:53.946142+0000","last_clean":"2024-09-10T20:35:53.946142+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:35:53.946142+0000","last_undegraded":"2024-09-10T20:35:53.946142+0000","last_fullsized":"2024-09-10T20:35:53.946142+0000","mapping_epoch":35,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25714688,"num_objects":6278,"num_object_clones":0,"num_object_copies":12556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6278,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12850,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14490","reported_seq":21843,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:55.062246+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:35:55.062246+0000","last_peered":"2024-09-10T20:35:55.062246+0000","last_clean":"2024-09-10T20:35:55.062246+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:35:55.062246+0000","last_undegraded":"2024-09-10T20:35:55.062246+0000","last_fullsized":"2024-09-10T20:35:55.062246+0000","mapping_epoch":44,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25497624,"num_objects":6226,"num_object_clones":0,"num_object_copies":12452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6226,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12618,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'14360","reported_seq":14445,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:54.059918+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:35:54.059918+0000","last_peered":"2024-09-10T20:35:54.059918+0000","last_clean":"2024-09-10T20:35:54.059918+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:35:54.059918+0000","last_undegraded":"2024-09-10T20:35:54.059918+0000","last_fullsized":"2024-09-10T20:35:54.059918+0000","mapping_epoch":35,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25575424,"num_objects":6244,"num_object_clones":0,"num_object_copies":12488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14360,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14877","reported_seq":15009,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:58.503647+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:35:58.503647+0000","last_peered":"2024-09-10T20:35:58.503647+0000","last_clean":"2024-09-10T20:35:58.503647+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:35:58.503647+0000","last_undegraded":"2024-09-10T20:35:58.503647+0000","last_fullsized":"2024-09-10T20:35:58.503647+0000","mapping_epoch":34,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25362432,"num_objects":6192,"num_object_clones":0,"num_object_copies":12384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11873,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14504","reported_seq":14752,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:58.517386+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:35:58.517386+0000","last_peered":"2024-09-10T20:35:58.517386+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:35:58.517386+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24440832,"num_objects":5967,"num_object_clones":0,"num_object_copies":11934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11368,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2406}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'14503","reported_seq":14784,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:58.449961+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:35:58.449961+0000","last_peered":"2024-09-10T20:35:58.449961+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:35:58.449961+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25006080,"num_objects":6105,"num_object_clones":0,"num_object_copies":12210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11098,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2619}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14660","reported_seq":14789,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:58.534929+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:35:58.534929+0000","last_peered":"2024-09-10T20:35:58.534929+0000","last_clean":"2024-09-10T20:35:58.534929+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:35:58.534929+0000","last_undegraded":"2024-09-10T20:35:58.534929+0000","last_fullsized":"2024-09-10T20:35:58.534929+0000","mapping_epoch":34,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24887296,"num_objects":6076,"num_object_clones":0,"num_object_copies":12152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6076,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11731,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14264","reported_seq":14457,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:54.076176+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:35:54.076176+0000","last_peered":"2024-09-10T20:35:54.076176+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:35:54.076176+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25292800,"num_objects":6175,"num_object_clones":0,"num_object_copies":12350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12257,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2678}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086274+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086274+0000","last_peered":"2024-09-10T20:34:27.086274+0000","last_clean":"2024-09-10T20:34:27.086274+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086274+0000","last_undegraded":"2024-09-10T20:34:27.086274+0000","last_fullsized":"2024-09-10T20:34:27.086274+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14676","reported_seq":14888,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:58.517524+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:35:58.517524+0000","last_peered":"2024-09-10T20:35:58.517524+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:35:58.517524+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24798150,"num_objects":6056,"num_object_clones":0,"num_object_copies":12112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6056,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14674,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2490}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14409","reported_seq":14597,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:54.003418+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:35:54.003418+0000","last_peered":"2024-09-10T20:35:54.003418+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:35:54.003418+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25214976,"num_objects":6156,"num_object_clones":0,"num_object_copies":12312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12237,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2550}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14454","reported_seq":19679,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:54.101587+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:35:54.101587+0000","last_peered":"2024-09-10T20:35:54.101587+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:35:54.101587+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25436160,"num_objects":6210,"num_object_clones":0,"num_object_copies":12420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12240,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2545,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":5143}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14681","reported_seq":14932,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:58.544022+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:35:58.544022+0000","last_peered":"2024-09-10T20:35:58.544022+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:35:58.544022+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25186304,"num_objects":6149,"num_object_clones":0,"num_object_copies":12298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11404,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2557}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14515","reported_seq":14755,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:58.534123+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:35:58.534123+0000","last_peered":"2024-09-10T20:35:58.534123+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:35:58.534123+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24858624,"num_objects":6069,"num_object_clones":0,"num_object_copies":12138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11209,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2585}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086784+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086784+0000","last_peered":"2024-09-10T20:34:27.086784+0000","last_clean":"2024-09-10T20:34:27.086784+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086784+0000","last_undegraded":"2024-09-10T20:34:27.086784+0000","last_fullsized":"2024-09-10T20:34:27.086784+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14853","reported_seq":15104,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:58.564291+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:35:58.564291+0000","last_peered":"2024-09-10T20:35:58.564291+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:35:58.564291+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25120768,"num_objects":6133,"num_object_clones":0,"num_object_copies":12266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11680,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2478}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14288","reported_seq":21410,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:55.002812+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:35:55.002812+0000","last_peered":"2024-09-10T20:35:55.002812+0000","last_clean":"2024-09-10T20:35:55.002812+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:35:55.002812+0000","last_undegraded":"2024-09-10T20:35:55.002812+0000","last_fullsized":"2024-09-10T20:35:55.002812+0000","mapping_epoch":46,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25510371,"num_objects":6229,"num_object_clones":0,"num_object_copies":12458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10895,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":402847169,"num_objects":98355,"num_object_clones":0,"num_object_copies":196710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33071,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98355,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":197062,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10078,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":984903680,"data_stored":984826889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160866,"ondisk_log_size":160866,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182489,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":739212,"kb_used_data":350780,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93632628,"statfs":{"total":96636764160,"available":95879811072,"internally_reserved":0,"allocated":359198720,"data_stored":358946427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":397733546},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,2,0,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986260,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":631704,"kb_used_data":315400,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93740136,"statfs":{"total":96636764160,"available":95989899264,"internally_reserved":0,"allocated":322969600,"data_stored":322717312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323874474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,0,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659207,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":614508,"kb_used_data":298180,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93757332,"statfs":{"total":96636764160,"available":96007507968,"internally_reserved":0,"allocated":305336320,"data_stored":305022147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":323874478},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,2,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":358658048,"data_stored":358654435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":304222208,"data_stored":304156715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":322023424,"data_stored":322015739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:35:59.436 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:35:59.436 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'14568", 'reported_seq': 14786, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:58.533235+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:35:58.533235+0000', 'last_peered': '2024-09-10T20:35:58.533235+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:35:58.533235+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "43'4500", 'ondisk_log_start': "43'4500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24944640, 'num_objects': 6090, 'num_object_clones': 0, 'num_object_copies': 12180, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6090, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14568, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2549}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:59.437 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:35:59.437 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'14504", 'reported_seq': 14752, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:58.517386+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:35:58.517386+0000', 'last_peered': '2024-09-10T20:35:58.517386+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:35:58.517386+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "43'4500", 'ondisk_log_start': "43'4500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24440832, 'num_objects': 5967, 'num_object_clones': 0, 'num_object_copies': 11934, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5967, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11368, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2406}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:59.437 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:35:59.437 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'14503", 'reported_seq': 14784, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:58.449961+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:35:58.449961+0000', 'last_peered': '2024-09-10T20:35:58.449961+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:35:58.449961+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "43'4500", 'ondisk_log_start': "43'4500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25006080, 'num_objects': 6105, 'num_object_clones': 0, 'num_object_copies': 12210, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6105, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11098, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2619}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:59.437 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:35:59.437 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'14264", 'reported_seq': 14457, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:54.076176+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:35:54.076176+0000', 'last_peered': '2024-09-10T20:35:54.076176+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:35:54.076176+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "43'4200", 'ondisk_log_start': "43'4200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25292800, 'num_objects': 6175, 'num_object_clones': 0, 'num_object_copies': 12350, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6175, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12257, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2678}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:35:59.437 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:35:59.437 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'14676", 'reported_seq': 14888, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:58.517524+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:35:58.517524+0000', 'last_peered': '2024-09-10T20:35:58.517524+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:35:58.517524+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "43'4600", 'ondisk_log_start': "43'4600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24798150, 'num_objects': 6056, 'num_object_clones': 0, 'num_object_copies': 12112, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6056, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14674, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2490}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:59.437 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:35:59.437 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'14409", 'reported_seq': 14597, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:54.003418+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:35:54.003418+0000', 'last_peered': '2024-09-10T20:35:54.003418+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:35:54.003418+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "43'4400", 'ondisk_log_start': "43'4400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25214976, 'num_objects': 6156, 'num_object_clones': 0, 'num_object_copies': 12312, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6156, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12237, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2550}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:35:59.438 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:35:59.438 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'14454", 'reported_seq': 19679, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:54.101587+0000', 'last_change': '2024-09-10T20:34:25.241937+0000', 'last_active': '2024-09-10T20:35:54.101587+0000', 'last_peered': '2024-09-10T20:35:54.101587+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:35:54.101587+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "43'4400", 'ondisk_log_start': "43'4400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25436160, 'num_objects': 6210, 'num_object_clones': 0, 'num_object_copies': 12420, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6210, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12240, 'num_write_kb': 32484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2545, 'num_bytes_recovered': 8085504, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 5143}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:35:59.438 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:35:59.438 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'14681", 'reported_seq': 14932, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:58.544022+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:35:58.544022+0000', 'last_peered': '2024-09-10T20:35:58.544022+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:35:58.544022+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "43'4600", 'ondisk_log_start': "43'4600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25186304, 'num_objects': 6149, 'num_object_clones': 0, 'num_object_copies': 12298, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6149, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11404, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2557}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:59.438 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:35:59.438 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'14515", 'reported_seq': 14755, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:58.534123+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:35:58.534123+0000', 'last_peered': '2024-09-10T20:35:58.534123+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:35:58.534123+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "43'4500", 'ondisk_log_start': "43'4500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24858624, 'num_objects': 6069, 'num_object_clones': 0, 'num_object_copies': 12138, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3487, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6069, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11209, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 48, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2585}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:59.438 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:35:59.438 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'14853", 'reported_seq': 15104, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:35:58.564291+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:35:58.564291+0000', 'last_peered': '2024-09-10T20:35:58.564291+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:35:58.564291+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25120768, 'num_objects': 6133, 'num_object_clones': 0, 'num_object_copies': 12266, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6133, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11680, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2478}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:35:59.438 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:36:00.149 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:00.149 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33071/196714 objects degraded (16.812%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":471,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":98357,"data_bytes":402847188,"bytes_used":2033074176,"bytes_avail":287877218304,"bytes_total":289910292480,"degraded_objects":33071,"degraded_total":196714,"degraded_ratio":0.16811716502129992,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":267,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":525},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:36:00.176 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:36:00.176 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:36:00.743 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:00.744 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:36:00.773 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":276,"stamp":"2024-09-10T20:35:58.971592+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":402847188,"num_objects":98357,"num_object_clones":0,"num_object_copies":196714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33071,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98357,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":197064,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10080,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160869,"ondisk_log_size":160869,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1985424,"kb_used_data":964360,"kb_used_omap":12,"kb_used_meta":1020979,"kb_avail":281130096,"statfs":{"total":289910292480,"available":287877218304,"internally_reserved":0,"allocated":987504640,"data_stored":986685886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1045482498},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,8,4,2,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25817088,"num_objects":-6303,"num_object_clones":0,"num_object_copies":-12606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-267,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6303,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":267,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005229"},"pg_stats":[{"pgid":"2.f","version":"47'14568","reported_seq":14786,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:58.533235+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:35:58.533235+0000","last_peered":"2024-09-10T20:35:58.533235+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:35:58.533235+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24944640,"num_objects":6090,"num_object_clones":0,"num_object_copies":12180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14568,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2549}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14664","reported_seq":14779,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:53.946142+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:35:53.946142+0000","last_peered":"2024-09-10T20:35:53.946142+0000","last_clean":"2024-09-10T20:35:53.946142+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:35:53.946142+0000","last_undegraded":"2024-09-10T20:35:53.946142+0000","last_fullsized":"2024-09-10T20:35:53.946142+0000","mapping_epoch":35,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25714688,"num_objects":6278,"num_object_clones":0,"num_object_copies":12556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6278,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12850,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14490","reported_seq":21843,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:55.062246+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:35:55.062246+0000","last_peered":"2024-09-10T20:35:55.062246+0000","last_clean":"2024-09-10T20:35:55.062246+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:35:55.062246+0000","last_undegraded":"2024-09-10T20:35:55.062246+0000","last_fullsized":"2024-09-10T20:35:55.062246+0000","mapping_epoch":44,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25497624,"num_objects":6226,"num_object_clones":0,"num_object_copies":12452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6226,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12618,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'14360","reported_seq":14445,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:54.059918+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:35:54.059918+0000","last_peered":"2024-09-10T20:35:54.059918+0000","last_clean":"2024-09-10T20:35:54.059918+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:35:54.059918+0000","last_undegraded":"2024-09-10T20:35:54.059918+0000","last_fullsized":"2024-09-10T20:35:54.059918+0000","mapping_epoch":35,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25575424,"num_objects":6244,"num_object_clones":0,"num_object_copies":12488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14360,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14877","reported_seq":15009,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:58.503647+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:35:58.503647+0000","last_peered":"2024-09-10T20:35:58.503647+0000","last_clean":"2024-09-10T20:35:58.503647+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:35:58.503647+0000","last_undegraded":"2024-09-10T20:35:58.503647+0000","last_fullsized":"2024-09-10T20:35:58.503647+0000","mapping_epoch":34,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25362432,"num_objects":6192,"num_object_clones":0,"num_object_copies":12384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11873,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14504","reported_seq":14752,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:58.517386+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:35:58.517386+0000","last_peered":"2024-09-10T20:35:58.517386+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:35:58.517386+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24440832,"num_objects":5967,"num_object_clones":0,"num_object_copies":11934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11368,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2406}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'14503","reported_seq":14784,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:58.449961+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:35:58.449961+0000","last_peered":"2024-09-10T20:35:58.449961+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:35:58.449961+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25006080,"num_objects":6105,"num_object_clones":0,"num_object_copies":12210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11098,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2619}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14660","reported_seq":14789,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:58.534929+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:35:58.534929+0000","last_peered":"2024-09-10T20:35:58.534929+0000","last_clean":"2024-09-10T20:35:58.534929+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:35:58.534929+0000","last_undegraded":"2024-09-10T20:35:58.534929+0000","last_fullsized":"2024-09-10T20:35:58.534929+0000","mapping_epoch":34,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24887296,"num_objects":6076,"num_object_clones":0,"num_object_copies":12152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6076,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11731,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14264","reported_seq":14457,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:54.076176+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:35:54.076176+0000","last_peered":"2024-09-10T20:35:54.076176+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:35:54.076176+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25292800,"num_objects":6175,"num_object_clones":0,"num_object_copies":12350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12257,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2678}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086274+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086274+0000","last_peered":"2024-09-10T20:34:27.086274+0000","last_clean":"2024-09-10T20:34:27.086274+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086274+0000","last_undegraded":"2024-09-10T20:34:27.086274+0000","last_fullsized":"2024-09-10T20:34:27.086274+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14676","reported_seq":14888,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:58.517524+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:35:58.517524+0000","last_peered":"2024-09-10T20:35:58.517524+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:35:58.517524+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24798150,"num_objects":6056,"num_object_clones":0,"num_object_copies":12112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6056,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14674,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2490}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14409","reported_seq":14597,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:54.003418+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:35:54.003418+0000","last_peered":"2024-09-10T20:35:54.003418+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:35:54.003418+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25214976,"num_objects":6156,"num_object_clones":0,"num_object_copies":12312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12237,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2550}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14454","reported_seq":19679,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:54.101587+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:35:54.101587+0000","last_peered":"2024-09-10T20:35:54.101587+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:35:54.101587+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25436160,"num_objects":6210,"num_object_clones":0,"num_object_copies":12420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12240,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2545,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":5143}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14681","reported_seq":14932,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:58.544022+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:35:58.544022+0000","last_peered":"2024-09-10T20:35:58.544022+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:35:58.544022+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25186304,"num_objects":6149,"num_object_clones":0,"num_object_copies":12298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11404,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2557}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14515","reported_seq":14755,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:58.534123+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:35:58.534123+0000","last_peered":"2024-09-10T20:35:58.534123+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:35:58.534123+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24858624,"num_objects":6069,"num_object_clones":0,"num_object_copies":12138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11209,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2585}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086784+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086784+0000","last_peered":"2024-09-10T20:34:27.086784+0000","last_clean":"2024-09-10T20:34:27.086784+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086784+0000","last_undegraded":"2024-09-10T20:34:27.086784+0000","last_fullsized":"2024-09-10T20:34:27.086784+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14853","reported_seq":15104,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:58.564291+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:35:58.564291+0000","last_peered":"2024-09-10T20:35:58.564291+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:35:58.564291+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25120768,"num_objects":6133,"num_object_clones":0,"num_object_copies":12266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11680,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2478}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14288","reported_seq":21410,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:55.002812+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:35:55.002812+0000","last_peered":"2024-09-10T20:35:55.002812+0000","last_clean":"2024-09-10T20:35:55.002812+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:35:55.002812+0000","last_undegraded":"2024-09-10T20:35:55.002812+0000","last_fullsized":"2024-09-10T20:35:55.002812+0000","mapping_epoch":46,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25510371,"num_objects":6229,"num_object_clones":0,"num_object_copies":12458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10895,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":402847169,"num_objects":98355,"num_object_clones":0,"num_object_copies":196710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33071,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98355,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":197062,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10078,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":984903680,"data_stored":984826889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160866,"ondisk_log_size":160866,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182489,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":739212,"kb_used_data":350780,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93632628,"statfs":{"total":96636764160,"available":95879811072,"internally_reserved":0,"allocated":359198720,"data_stored":358946427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":397733546},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,2,0,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986260,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":631704,"kb_used_data":315400,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93740136,"statfs":{"total":96636764160,"available":95989899264,"internally_reserved":0,"allocated":322969600,"data_stored":322717312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323874474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,0,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659207,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":614508,"kb_used_data":298180,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93757332,"statfs":{"total":96636764160,"available":96007507968,"internally_reserved":0,"allocated":305336320,"data_stored":305022147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":323874478},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,2,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":358658048,"data_stored":358654435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":304222208,"data_stored":304156715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":322023424,"data_stored":322015739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:36:03.775 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:36:04.355 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:04.357 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:36:04.378 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":278,"stamp":"2024-09-10T20:36:02.973084+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":398435796,"num_objects":97280,"num_object_clones":0,"num_object_copies":194560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97280,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":198141,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10231,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160746,"ondisk_log_size":160746,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2003860,"kb_used_data":952276,"kb_used_omap":12,"kb_used_meta":1051507,"kb_avail":281111660,"statfs":{"total":289910292480,"available":287858339840,"internally_reserved":0,"allocated":975130624,"data_stored":974311870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1076743170},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,4,7,13,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20209664,"num_objects":-4934,"num_object_clones":0,"num_object_copies":-9868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-284,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4934,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005580"},"pg_stats":[{"pgid":"2.f","version":"47'14568","reported_seq":14786,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:58.533235+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:35:58.533235+0000","last_peered":"2024-09-10T20:35:58.533235+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:35:58.533235+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24944640,"num_objects":6090,"num_object_clones":0,"num_object_copies":12180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14568,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2549}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14814","reported_seq":14929,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:59.105077+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:35:59.105077+0000","last_peered":"2024-09-10T20:35:59.105077+0000","last_clean":"2024-09-10T20:35:59.105077+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:35:59.105077+0000","last_undegraded":"2024-09-10T20:35:59.105077+0000","last_fullsized":"2024-09-10T20:35:59.105077+0000","mapping_epoch":35,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25100288,"num_objects":6128,"num_object_clones":0,"num_object_copies":12256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13000,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14646","reported_seq":21999,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:00.132798+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:36:00.132798+0000","last_peered":"2024-09-10T20:36:00.132798+0000","last_clean":"2024-09-10T20:36:00.132798+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:36:00.132798+0000","last_undegraded":"2024-09-10T20:36:00.132798+0000","last_fullsized":"2024-09-10T20:36:00.132798+0000","mapping_epoch":44,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24858648,"num_objects":6070,"num_object_clones":0,"num_object_copies":12140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6070,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12774,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'14516","reported_seq":14602,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:59.100150+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:35:59.100150+0000","last_peered":"2024-09-10T20:35:59.100150+0000","last_clean":"2024-09-10T20:35:59.100150+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:35:59.100150+0000","last_undegraded":"2024-09-10T20:35:59.100150+0000","last_fullsized":"2024-09-10T20:35:59.100150+0000","mapping_epoch":35,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24936448,"num_objects":6088,"num_object_clones":0,"num_object_copies":12176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6088,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14516,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14877","reported_seq":15009,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:58.503647+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:35:58.503647+0000","last_peered":"2024-09-10T20:35:58.503647+0000","last_clean":"2024-09-10T20:35:58.503647+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:35:58.503647+0000","last_undegraded":"2024-09-10T20:35:58.503647+0000","last_fullsized":"2024-09-10T20:35:58.503647+0000","mapping_epoch":34,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25362432,"num_objects":6192,"num_object_clones":0,"num_object_copies":12384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11873,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14504","reported_seq":14752,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:58.517386+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:35:58.517386+0000","last_peered":"2024-09-10T20:35:58.517386+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:35:58.517386+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24440832,"num_objects":5967,"num_object_clones":0,"num_object_copies":11934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11368,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2406}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'14503","reported_seq":14784,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:58.449961+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:35:58.449961+0000","last_peered":"2024-09-10T20:35:58.449961+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:35:58.449961+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25006080,"num_objects":6105,"num_object_clones":0,"num_object_copies":12210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11098,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2619}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14660","reported_seq":14789,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:35:58.534929+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:35:58.534929+0000","last_peered":"2024-09-10T20:35:58.534929+0000","last_clean":"2024-09-10T20:35:58.534929+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:35:58.534929+0000","last_undegraded":"2024-09-10T20:35:58.534929+0000","last_fullsized":"2024-09-10T20:35:58.534929+0000","mapping_epoch":34,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24887296,"num_objects":6076,"num_object_clones":0,"num_object_copies":12152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6076,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11731,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14405","reported_seq":14601,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:59.112082+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:35:59.112082+0000","last_peered":"2024-09-10T20:35:59.112082+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:35:59.112082+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24715264,"num_objects":6034,"num_object_clones":0,"num_object_copies":12068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12398,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2537}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086274+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086274+0000","last_peered":"2024-09-10T20:34:27.086274+0000","last_clean":"2024-09-10T20:34:27.086274+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086274+0000","last_undegraded":"2024-09-10T20:34:27.086274+0000","last_fullsized":"2024-09-10T20:34:27.086274+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14676","reported_seq":14888,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:58.517524+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:35:58.517524+0000","last_peered":"2024-09-10T20:35:58.517524+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:35:58.517524+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24798150,"num_objects":6056,"num_object_clones":0,"num_object_copies":12112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6056,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14674,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2490}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14560","reported_seq":14748,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:59.088203+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:35:59.088203+0000","last_peered":"2024-09-10T20:35:59.088203+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:35:59.088203+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24596480,"num_objects":6005,"num_object_clones":0,"num_object_copies":12010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12388,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2399}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14599","reported_seq":20126,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:59.108116+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:35:59.108116+0000","last_peered":"2024-09-10T20:35:59.108116+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:35:59.108116+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24842240,"num_objects":6065,"num_object_clones":0,"num_object_copies":12130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":916,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12385,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2696,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":5149}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14681","reported_seq":14932,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:58.544022+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:35:58.544022+0000","last_peered":"2024-09-10T20:35:58.544022+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:35:58.544022+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25186304,"num_objects":6149,"num_object_clones":0,"num_object_copies":12298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11404,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2557}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14515","reported_seq":14755,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:58.534123+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:35:58.534123+0000","last_peered":"2024-09-10T20:35:58.534123+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:35:58.534123+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24858624,"num_objects":6069,"num_object_clones":0,"num_object_copies":12138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11209,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2585}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086784+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086784+0000","last_peered":"2024-09-10T20:34:27.086784+0000","last_clean":"2024-09-10T20:34:27.086784+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086784+0000","last_undegraded":"2024-09-10T20:34:27.086784+0000","last_fullsized":"2024-09-10T20:34:27.086784+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14853","reported_seq":15104,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:35:58.564291+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:35:58.564291+0000","last_peered":"2024-09-10T20:35:58.564291+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:35:58.564291+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25120768,"num_objects":6133,"num_object_clones":0,"num_object_copies":12266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11680,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2478}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14466","reported_seq":21587,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:00.072149+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:36:00.072149+0000","last_peered":"2024-09-10T20:36:00.072149+0000","last_clean":"2024-09-10T20:36:00.072149+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:36:00.072149+0000","last_undegraded":"2024-09-10T20:36:00.072149+0000","last_fullsized":"2024-09-10T20:36:00.072149+0000","mapping_epoch":46,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24781283,"num_objects":6051,"num_object_clones":0,"num_object_copies":12102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11073,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":398435777,"num_objects":97278,"num_object_clones":0,"num_object_copies":194556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97278,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":198139,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10229,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":972345344,"data_stored":972268553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160743,"ondisk_log_size":160743,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182489,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":739212,"kb_used_data":350780,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93632628,"statfs":{"total":96636764160,"available":95879811072,"internally_reserved":0,"allocated":359198720,"data_stored":358946427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":397733546},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,2,0,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986261,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":631096,"kb_used_data":310632,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93740744,"statfs":{"total":96636764160,"available":95990521856,"internally_reserved":0,"allocated":318087168,"data_stored":317834880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328134314},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659208,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":633552,"kb_used_data":290864,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93738288,"statfs":{"total":96636764160,"available":95988006912,"internally_reserved":0,"allocated":297844736,"data_stored":297530563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,11,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":358658048,"data_stored":358654435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":296624128,"data_stored":296558635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":317063168,"data_stored":317055483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:36:04.380 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:36:04.984 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:04.985 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:36:05.013 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":279,"stamp":"2024-09-10T20:36:04.973733+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":389821908,"num_objects":95177,"num_object_clones":0,"num_object_copies":190354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32754,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95177,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":200244,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10397,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160649,"ondisk_log_size":160649,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2003864,"kb_used_data":939784,"kb_used_omap":12,"kb_used_meta":1063987,"kb_avail":281111656,"statfs":{"total":289910292480,"available":287858335744,"internally_reserved":0,"allocated":962338816,"data_stored":961520062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1089522690},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,6,2,5,13,4,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28823552,"num_objects":-7037,"num_object_clones":0,"num_object_copies":-14074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-450,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7037,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":450,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005408"},"pg_stats":[{"pgid":"2.f","version":"47'14703","reported_seq":14920,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:03.578751+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:36:03.578751+0000","last_peered":"2024-09-10T20:36:03.578751+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:36:03.578751+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24391680,"num_objects":5955,"num_object_clones":0,"num_object_copies":11910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14703,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2414}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14976","reported_seq":15091,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:04.092702+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:36:04.092702+0000","last_peered":"2024-09-10T20:36:04.092702+0000","last_clean":"2024-09-10T20:36:04.092702+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:36:04.092702+0000","last_undegraded":"2024-09-10T20:36:04.092702+0000","last_fullsized":"2024-09-10T20:36:04.092702+0000","mapping_epoch":35,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24436736,"num_objects":5966,"num_object_clones":0,"num_object_copies":11932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13162,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14646","reported_seq":21999,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:00.132798+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:36:00.132798+0000","last_peered":"2024-09-10T20:36:00.132798+0000","last_clean":"2024-09-10T20:36:00.132798+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:36:00.132798+0000","last_undegraded":"2024-09-10T20:36:00.132798+0000","last_fullsized":"2024-09-10T20:36:00.132798+0000","mapping_epoch":44,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24858648,"num_objects":6070,"num_object_clones":0,"num_object_copies":12140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6070,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12774,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'14674","reported_seq":14760,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:04.028125+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:36:04.028125+0000","last_peered":"2024-09-10T20:36:04.028125+0000","last_clean":"2024-09-10T20:36:04.028125+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:36:04.028125+0000","last_undegraded":"2024-09-10T20:36:04.028125+0000","last_fullsized":"2024-09-10T20:36:04.028125+0000","mapping_epoch":35,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24289280,"num_objects":5930,"num_object_clones":0,"num_object_copies":11860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14674,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15029","reported_seq":15161,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:03.540565+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:36:03.540565+0000","last_peered":"2024-09-10T20:36:03.540565+0000","last_clean":"2024-09-10T20:36:03.540565+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:36:03.540565+0000","last_undegraded":"2024-09-10T20:36:03.540565+0000","last_fullsized":"2024-09-10T20:36:03.540565+0000","mapping_epoch":34,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24739840,"num_objects":6040,"num_object_clones":0,"num_object_copies":12080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12025,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14674","reported_seq":14921,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:03.561061+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:36:03.561061+0000","last_peered":"2024-09-10T20:36:03.561061+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:36:03.561061+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23744512,"num_objects":5797,"num_object_clones":0,"num_object_copies":11594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11538,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2236}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'14650","reported_seq":14930,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:03.580863+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:36:03.580863+0000","last_peered":"2024-09-10T20:36:03.580863+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:36:03.580863+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24403968,"num_objects":5958,"num_object_clones":0,"num_object_copies":11916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11245,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2472}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14815","reported_seq":14945,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:03.569949+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:36:03.569949+0000","last_peered":"2024-09-10T20:36:03.569949+0000","last_clean":"2024-09-10T20:36:03.569949+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:36:03.569949+0000","last_undegraded":"2024-09-10T20:36:03.569949+0000","last_fullsized":"2024-09-10T20:36:03.569949+0000","mapping_epoch":34,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24252416,"num_objects":5921,"num_object_clones":0,"num_object_copies":11842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11886,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14534","reported_seq":14730,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:03.986837+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:36:03.986837+0000","last_peered":"2024-09-10T20:36:03.986837+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:36:03.986837+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24186880,"num_objects":5905,"num_object_clones":0,"num_object_copies":11810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12527,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2408}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086274+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086274+0000","last_peered":"2024-09-10T20:34:27.086274+0000","last_clean":"2024-09-10T20:34:27.086274+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086274+0000","last_undegraded":"2024-09-10T20:34:27.086274+0000","last_fullsized":"2024-09-10T20:34:27.086274+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14824","reported_seq":15036,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:03.556559+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:36:03.556559+0000","last_peered":"2024-09-10T20:36:03.556559+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:36:03.556559+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24191942,"num_objects":5908,"num_object_clones":0,"num_object_copies":11816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14822,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2342}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14709","reported_seq":14897,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:04.007779+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:36:04.007779+0000","last_peered":"2024-09-10T20:36:04.007779+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:36:04.007779+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23986176,"num_objects":5856,"num_object_clones":0,"num_object_copies":11712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12537,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2250}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14744","reported_seq":20603,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:04.102927+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:36:04.102927+0000","last_peered":"2024-09-10T20:36:04.102927+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:36:04.102927+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24248320,"num_objects":5920,"num_object_clones":0,"num_object_copies":11840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":750,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12530,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2862,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":5170}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14831","reported_seq":15083,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:03.524364+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:36:03.524364+0000","last_peered":"2024-09-10T20:36:03.524364+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:36:03.524364+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24571904,"num_objects":5999,"num_object_clones":0,"num_object_copies":11998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11554,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2407}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14655","reported_seq":14895,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:03.556003+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:36:03.556003+0000","last_peered":"2024-09-10T20:36:03.556003+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:36:03.556003+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24285184,"num_objects":5929,"num_object_clones":0,"num_object_copies":11858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11349,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2445}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086784+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086784+0000","last_peered":"2024-09-10T20:34:27.086784+0000","last_clean":"2024-09-10T20:34:27.086784+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086784+0000","last_undegraded":"2024-09-10T20:34:27.086784+0000","last_fullsized":"2024-09-10T20:34:27.086784+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15016","reported_seq":15269,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:03.470187+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:36:03.470187+0000","last_peered":"2024-09-10T20:36:03.470187+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:36:03.470187+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24453120,"num_objects":5970,"num_object_clones":0,"num_object_copies":11940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11843,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2315}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14466","reported_seq":21587,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:00.072149+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:36:00.072149+0000","last_peered":"2024-09-10T20:36:00.072149+0000","last_clean":"2024-09-10T20:36:00.072149+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:36:00.072149+0000","last_undegraded":"2024-09-10T20:36:00.072149+0000","last_fullsized":"2024-09-10T20:36:00.072149+0000","mapping_epoch":46,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24781283,"num_objects":6051,"num_object_clones":0,"num_object_copies":12102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11073,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":389821889,"num_objects":95175,"num_object_clones":0,"num_object_copies":190350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32754,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95175,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":200242,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10395,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":959770624,"data_stored":959693833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160646,"ondisk_log_size":160646,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182490,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":739956,"kb_used_data":343192,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93631884,"statfs":{"total":96636764160,"available":95879049216,"internally_reserved":0,"allocated":351428608,"data_stored":351176315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,0,2,1,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986262,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":630356,"kb_used_data":305728,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93741484,"statfs":{"total":96636764160,"available":95991279616,"internally_reserved":0,"allocated":313065472,"data_stored":312813184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332394154},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,1,1,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659208,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":633552,"kb_used_data":290864,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93738288,"statfs":{"total":96636764160,"available":95988006912,"internally_reserved":0,"allocated":297844736,"data_stored":297530563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,11,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":350986240,"data_stored":350982627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":296624128,"data_stored":296558635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":312160256,"data_stored":312152571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:36:05.015 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:36:05.015 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'14703", 'reported_seq': 14920, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:03.578751+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:36:03.578751+0000', 'last_peered': '2024-09-10T20:36:03.578751+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:36:03.578751+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "43'4700", 'ondisk_log_start': "43'4700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24391680, 'num_objects': 5955, 'num_object_clones': 0, 'num_object_copies': 11910, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5955, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14703, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2414}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:05.015 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:36:05.015 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'14674", 'reported_seq': 14921, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:03.561061+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:36:03.561061+0000', 'last_peered': '2024-09-10T20:36:03.561061+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:36:03.561061+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "43'4600", 'ondisk_log_start': "43'4600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23744512, 'num_objects': 5797, 'num_object_clones': 0, 'num_object_copies': 11594, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5797, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11538, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2236}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:05.015 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:36:05.015 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'14650", 'reported_seq': 14930, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:03.580863+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:36:03.580863+0000', 'last_peered': '2024-09-10T20:36:03.580863+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:36:03.580863+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "43'4600", 'ondisk_log_start': "43'4600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24403968, 'num_objects': 5958, 'num_object_clones': 0, 'num_object_copies': 11916, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5958, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11245, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2472}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:05.015 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:36:05.015 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'14534", 'reported_seq': 14730, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:03.986837+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:36:03.986837+0000', 'last_peered': '2024-09-10T20:36:03.986837+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:36:03.986837+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "43'4500", 'ondisk_log_start': "43'4500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24186880, 'num_objects': 5905, 'num_object_clones': 0, 'num_object_copies': 11810, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5905, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12527, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2408}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:36:05.015 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:36:05.016 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'14824", 'reported_seq': 15036, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:03.556559+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:36:03.556559+0000', 'last_peered': '2024-09-10T20:36:03.556559+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:36:03.556559+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24191942, 'num_objects': 5908, 'num_object_clones': 0, 'num_object_copies': 11816, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5908, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14822, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2342}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:05.016 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:36:05.016 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'14709", 'reported_seq': 14897, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:04.007779+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:36:04.007779+0000', 'last_peered': '2024-09-10T20:36:04.007779+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:36:04.007779+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "43'4700", 'ondisk_log_start': "43'4700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23986176, 'num_objects': 5856, 'num_object_clones': 0, 'num_object_copies': 11712, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5856, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12537, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2250}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:36:05.016 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:36:05.016 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'14744", 'reported_seq': 20603, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:04.102927+0000', 'last_change': '2024-09-10T20:34:25.241937+0000', 'last_active': '2024-09-10T20:36:04.102927+0000', 'last_peered': '2024-09-10T20:36:04.102927+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:36:04.102927+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "43'4700", 'ondisk_log_start': "43'4700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24248320, 'num_objects': 5920, 'num_object_clones': 0, 'num_object_copies': 11840, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 750, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5920, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12530, 'num_write_kb': 32484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2862, 'num_bytes_recovered': 8085504, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 5170}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:36:05.016 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:36:05.016 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'14831", 'reported_seq': 15083, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:03.524364+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:36:03.524364+0000', 'last_peered': '2024-09-10T20:36:03.524364+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:36:03.524364+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24571904, 'num_objects': 5999, 'num_object_clones': 0, 'num_object_copies': 11998, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5999, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11554, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2407}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:05.016 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:36:05.016 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'14655", 'reported_seq': 14895, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:03.556003+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:36:03.556003+0000', 'last_peered': '2024-09-10T20:36:03.556003+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:36:03.556003+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "43'4600", 'ondisk_log_start': "43'4600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24285184, 'num_objects': 5929, 'num_object_clones': 0, 'num_object_copies': 11858, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3487, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5929, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11349, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 48, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2445}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:05.016 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:36:05.017 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'15016", 'reported_seq': 15269, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:03.470187+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:36:03.470187+0000', 'last_peered': '2024-09-10T20:36:03.470187+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:36:03.470187+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "43'5000", 'ondisk_log_start': "43'5000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24453120, 'num_objects': 5970, 'num_object_clones': 0, 'num_object_copies': 11940, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5970, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11843, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2315}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:05.017 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:36:05.731 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:05.731 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32920/194560 objects degraded (16.920%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":477,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":97280,"data_bytes":398435796,"bytes_used":2051952640,"bytes_avail":287858339840,"bytes_total":289910292480,"degraded_objects":32920,"degraded_total":194560,"degraded_ratio":0.16920230263157893,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":284,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":410},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:36:05.763 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:36:05.763 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:36:06.328 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:06.330 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:36:06.350 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":279,"stamp":"2024-09-10T20:36:04.973733+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":389821908,"num_objects":95177,"num_object_clones":0,"num_object_copies":190354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32754,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95177,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":200244,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10397,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160649,"ondisk_log_size":160649,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2003864,"kb_used_data":939784,"kb_used_omap":12,"kb_used_meta":1063987,"kb_avail":281111656,"statfs":{"total":289910292480,"available":287858335744,"internally_reserved":0,"allocated":962338816,"data_stored":961520062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1089522690},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,6,2,5,13,4,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28823552,"num_objects":-7037,"num_object_clones":0,"num_object_copies":-14074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-450,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7037,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":450,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005408"},"pg_stats":[{"pgid":"2.f","version":"47'14703","reported_seq":14920,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:03.578751+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:36:03.578751+0000","last_peered":"2024-09-10T20:36:03.578751+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:36:03.578751+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24391680,"num_objects":5955,"num_object_clones":0,"num_object_copies":11910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14703,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2414}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14976","reported_seq":15091,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:04.092702+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:36:04.092702+0000","last_peered":"2024-09-10T20:36:04.092702+0000","last_clean":"2024-09-10T20:36:04.092702+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:36:04.092702+0000","last_undegraded":"2024-09-10T20:36:04.092702+0000","last_fullsized":"2024-09-10T20:36:04.092702+0000","mapping_epoch":35,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24436736,"num_objects":5966,"num_object_clones":0,"num_object_copies":11932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13162,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14646","reported_seq":21999,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:00.132798+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:36:00.132798+0000","last_peered":"2024-09-10T20:36:00.132798+0000","last_clean":"2024-09-10T20:36:00.132798+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:36:00.132798+0000","last_undegraded":"2024-09-10T20:36:00.132798+0000","last_fullsized":"2024-09-10T20:36:00.132798+0000","mapping_epoch":44,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24858648,"num_objects":6070,"num_object_clones":0,"num_object_copies":12140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6070,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12774,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'14674","reported_seq":14760,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:04.028125+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:36:04.028125+0000","last_peered":"2024-09-10T20:36:04.028125+0000","last_clean":"2024-09-10T20:36:04.028125+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:36:04.028125+0000","last_undegraded":"2024-09-10T20:36:04.028125+0000","last_fullsized":"2024-09-10T20:36:04.028125+0000","mapping_epoch":35,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24289280,"num_objects":5930,"num_object_clones":0,"num_object_copies":11860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14674,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15029","reported_seq":15161,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:03.540565+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:36:03.540565+0000","last_peered":"2024-09-10T20:36:03.540565+0000","last_clean":"2024-09-10T20:36:03.540565+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:36:03.540565+0000","last_undegraded":"2024-09-10T20:36:03.540565+0000","last_fullsized":"2024-09-10T20:36:03.540565+0000","mapping_epoch":34,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24739840,"num_objects":6040,"num_object_clones":0,"num_object_copies":12080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12025,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14674","reported_seq":14921,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:03.561061+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:36:03.561061+0000","last_peered":"2024-09-10T20:36:03.561061+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:36:03.561061+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23744512,"num_objects":5797,"num_object_clones":0,"num_object_copies":11594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11538,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2236}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'14650","reported_seq":14930,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:03.580863+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:36:03.580863+0000","last_peered":"2024-09-10T20:36:03.580863+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:36:03.580863+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24403968,"num_objects":5958,"num_object_clones":0,"num_object_copies":11916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11245,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2472}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14815","reported_seq":14945,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:03.569949+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:36:03.569949+0000","last_peered":"2024-09-10T20:36:03.569949+0000","last_clean":"2024-09-10T20:36:03.569949+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:36:03.569949+0000","last_undegraded":"2024-09-10T20:36:03.569949+0000","last_fullsized":"2024-09-10T20:36:03.569949+0000","mapping_epoch":34,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24252416,"num_objects":5921,"num_object_clones":0,"num_object_copies":11842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11886,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080443+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080443+0000","last_peered":"2024-09-10T20:34:27.080443+0000","last_clean":"2024-09-10T20:34:27.080443+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080443+0000","last_undegraded":"2024-09-10T20:34:27.080443+0000","last_fullsized":"2024-09-10T20:34:27.080443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14534","reported_seq":14730,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:03.986837+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:36:03.986837+0000","last_peered":"2024-09-10T20:36:03.986837+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:36:03.986837+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24186880,"num_objects":5905,"num_object_clones":0,"num_object_copies":11810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12527,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2408}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086274+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086274+0000","last_peered":"2024-09-10T20:34:27.086274+0000","last_clean":"2024-09-10T20:34:27.086274+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086274+0000","last_undegraded":"2024-09-10T20:34:27.086274+0000","last_fullsized":"2024-09-10T20:34:27.086274+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14824","reported_seq":15036,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:03.556559+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:36:03.556559+0000","last_peered":"2024-09-10T20:36:03.556559+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:36:03.556559+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24191942,"num_objects":5908,"num_object_clones":0,"num_object_copies":11816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14822,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2342}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14709","reported_seq":14897,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:04.007779+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:36:04.007779+0000","last_peered":"2024-09-10T20:36:04.007779+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:36:04.007779+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23986176,"num_objects":5856,"num_object_clones":0,"num_object_copies":11712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12537,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2250}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14744","reported_seq":20603,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:04.102927+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:36:04.102927+0000","last_peered":"2024-09-10T20:36:04.102927+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:36:04.102927+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24248320,"num_objects":5920,"num_object_clones":0,"num_object_copies":11840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":750,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12530,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2862,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":5170}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14831","reported_seq":15083,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:03.524364+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:36:03.524364+0000","last_peered":"2024-09-10T20:36:03.524364+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:36:03.524364+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24571904,"num_objects":5999,"num_object_clones":0,"num_object_copies":11998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11554,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2407}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14655","reported_seq":14895,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:03.556003+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:36:03.556003+0000","last_peered":"2024-09-10T20:36:03.556003+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:36:03.556003+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24285184,"num_objects":5929,"num_object_clones":0,"num_object_copies":11858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11349,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2445}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086784+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086784+0000","last_peered":"2024-09-10T20:34:27.086784+0000","last_clean":"2024-09-10T20:34:27.086784+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086784+0000","last_undegraded":"2024-09-10T20:34:27.086784+0000","last_fullsized":"2024-09-10T20:34:27.086784+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15016","reported_seq":15269,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:03.470187+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:36:03.470187+0000","last_peered":"2024-09-10T20:36:03.470187+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:36:03.470187+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24453120,"num_objects":5970,"num_object_clones":0,"num_object_copies":11940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11843,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2315}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14466","reported_seq":21587,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:00.072149+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:36:00.072149+0000","last_peered":"2024-09-10T20:36:00.072149+0000","last_clean":"2024-09-10T20:36:00.072149+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:36:00.072149+0000","last_undegraded":"2024-09-10T20:36:00.072149+0000","last_fullsized":"2024-09-10T20:36:00.072149+0000","mapping_epoch":46,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24781283,"num_objects":6051,"num_object_clones":0,"num_object_copies":12102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11073,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":389821889,"num_objects":95175,"num_object_clones":0,"num_object_copies":190350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32754,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95175,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":200242,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10395,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":959770624,"data_stored":959693833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160646,"ondisk_log_size":160646,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182490,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":739956,"kb_used_data":343192,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93631884,"statfs":{"total":96636764160,"available":95879049216,"internally_reserved":0,"allocated":351428608,"data_stored":351176315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,0,2,1,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986262,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":630356,"kb_used_data":305728,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93741484,"statfs":{"total":96636764160,"available":95991279616,"internally_reserved":0,"allocated":313065472,"data_stored":312813184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332394154},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,1,1,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659208,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":633552,"kb_used_data":290864,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93738288,"statfs":{"total":96636764160,"available":95988006912,"internally_reserved":0,"allocated":297844736,"data_stored":297530563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,11,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":350986240,"data_stored":350982627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":296624128,"data_stored":296558635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":312160256,"data_stored":312152571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:36:09.352 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:36:09.888 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:09.891 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:36:09.914 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":281,"stamp":"2024-09-10T20:36:08.975203+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":382985684,"num_objects":93508,"num_object_clones":0,"num_object_copies":187016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32754,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93508,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":201913,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10397,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160918,"ondisk_log_size":160918,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1997612,"kb_used_data":925236,"kb_used_omap":12,"kb_used_meta":1072307,"kb_avail":281117908,"statfs":{"total":289910292480,"available":287864737792,"internally_reserved":0,"allocated":947441664,"data_stored":946622910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1098042370},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,7,3,8,11,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25595904,"num_objects":-6249,"num_object_clones":0,"num_object_copies":-12498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-317,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6249,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":317,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004186"},"pg_stats":[{"pgid":"2.f","version":"47'14853","reported_seq":15071,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:08.647361+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:36:08.647361+0000","last_peered":"2024-09-10T20:36:08.647361+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:36:08.647361+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23777280,"num_objects":5805,"num_object_clones":0,"num_object_copies":11610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14853,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2264}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14976","reported_seq":15091,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:04.092702+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:36:04.092702+0000","last_peered":"2024-09-10T20:36:04.092702+0000","last_clean":"2024-09-10T20:36:04.092702+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:36:04.092702+0000","last_undegraded":"2024-09-10T20:36:04.092702+0000","last_fullsized":"2024-09-10T20:36:04.092702+0000","mapping_epoch":35,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24436736,"num_objects":5966,"num_object_clones":0,"num_object_copies":11932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13162,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14791","reported_seq":22143,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:05.188777+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:36:05.188777+0000","last_peered":"2024-09-10T20:36:05.188777+0000","last_clean":"2024-09-10T20:36:05.188777+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:36:05.188777+0000","last_undegraded":"2024-09-10T20:36:05.188777+0000","last_fullsized":"2024-09-10T20:36:05.188777+0000","mapping_epoch":44,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24264728,"num_objects":5925,"num_object_clones":0,"num_object_copies":11850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5925,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12919,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'14674","reported_seq":14760,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:04.028125+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:36:04.028125+0000","last_peered":"2024-09-10T20:36:04.028125+0000","last_clean":"2024-09-10T20:36:04.028125+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:36:04.028125+0000","last_undegraded":"2024-09-10T20:36:04.028125+0000","last_fullsized":"2024-09-10T20:36:04.028125+0000","mapping_epoch":35,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24289280,"num_objects":5930,"num_object_clones":0,"num_object_copies":11860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14674,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15179","reported_seq":15311,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:08.567866+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:36:08.567866+0000","last_peered":"2024-09-10T20:36:08.567866+0000","last_clean":"2024-09-10T20:36:08.567866+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:36:08.567866+0000","last_undegraded":"2024-09-10T20:36:08.567866+0000","last_fullsized":"2024-09-10T20:36:08.567866+0000","mapping_epoch":34,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24125440,"num_objects":5890,"num_object_clones":0,"num_object_copies":11780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12175,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14841","reported_seq":15087,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:08.605013+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:36:08.605013+0000","last_peered":"2024-09-10T20:36:08.605013+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:36:08.605013+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23060480,"num_objects":5630,"num_object_clones":0,"num_object_copies":11260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11705,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2069}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'14805","reported_seq":15082,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:08.623177+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:36:08.623177+0000","last_peered":"2024-09-10T20:36:08.623177+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:36:08.623177+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23769088,"num_objects":5803,"num_object_clones":0,"num_object_copies":11606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11400,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2317}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14974","reported_seq":15104,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:08.556977+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:36:08.556977+0000","last_peered":"2024-09-10T20:36:08.556977+0000","last_clean":"2024-09-10T20:36:08.556977+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:36:08.556977+0000","last_undegraded":"2024-09-10T20:36:08.556977+0000","last_fullsized":"2024-09-10T20:36:08.556977+0000","mapping_epoch":34,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23601152,"num_objects":5762,"num_object_clones":0,"num_object_copies":11524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12045,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080479+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080479+0000","last_peered":"2024-09-10T20:34:27.080479+0000","last_clean":"2024-09-10T20:34:27.080479+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080479+0000","last_undegraded":"2024-09-10T20:34:27.080479+0000","last_fullsized":"2024-09-10T20:34:27.080479+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14534","reported_seq":14730,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:03.986837+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:36:03.986837+0000","last_peered":"2024-09-10T20:36:03.986837+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:36:03.986837+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24186880,"num_objects":5905,"num_object_clones":0,"num_object_copies":11810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12527,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2408}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086274+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086274+0000","last_peered":"2024-09-10T20:34:27.086274+0000","last_clean":"2024-09-10T20:34:27.086274+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086274+0000","last_undegraded":"2024-09-10T20:34:27.086274+0000","last_fullsized":"2024-09-10T20:34:27.086274+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14955","reported_seq":15167,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:08.584350+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:36:08.584350+0000","last_peered":"2024-09-10T20:36:08.584350+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:36:08.584350+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23655366,"num_objects":5777,"num_object_clones":0,"num_object_copies":11554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14953,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2211}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14709","reported_seq":14897,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:04.007779+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:36:04.007779+0000","last_peered":"2024-09-10T20:36:04.007779+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:36:04.007779+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23986176,"num_objects":5856,"num_object_clones":0,"num_object_copies":11712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12537,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2250}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14744","reported_seq":20603,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:04.102927+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:36:04.102927+0000","last_peered":"2024-09-10T20:36:04.102927+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:36:04.102927+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24248320,"num_objects":5920,"num_object_clones":0,"num_object_copies":11840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":750,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12530,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2862,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":5170}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14977","reported_seq":15228,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:08.605792+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:36:08.605792+0000","last_peered":"2024-09-10T20:36:08.605792+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:36:08.605792+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23973888,"num_objects":5853,"num_object_clones":0,"num_object_copies":11706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11700,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2261}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14798","reported_seq":15038,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:08.611931+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:36:08.611931+0000","last_peered":"2024-09-10T20:36:08.611931+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:36:08.611931+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23699456,"num_objects":5786,"num_object_clones":0,"num_object_copies":11572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11492,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2302}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086784+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086784+0000","last_peered":"2024-09-10T20:34:27.086784+0000","last_clean":"2024-09-10T20:34:27.086784+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086784+0000","last_undegraded":"2024-09-10T20:34:27.086784+0000","last_fullsized":"2024-09-10T20:34:27.086784+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15175","reported_seq":15428,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:08.566015+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:36:08.566015+0000","last_peered":"2024-09-10T20:36:08.566015+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:36:08.566015+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23801856,"num_objects":5811,"num_object_clones":0,"num_object_copies":11622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12002,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2156}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14630","reported_seq":21752,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:05.136591+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:36:05.136591+0000","last_peered":"2024-09-10T20:36:05.136591+0000","last_clean":"2024-09-10T20:36:05.136591+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:36:05.136591+0000","last_undegraded":"2024-09-10T20:36:05.136591+0000","last_fullsized":"2024-09-10T20:36:05.136591+0000","mapping_epoch":46,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24109539,"num_objects":5887,"num_object_clones":0,"num_object_copies":11774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11237,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":382985665,"num_objects":93506,"num_object_clones":0,"num_object_copies":187012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32754,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93506,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":201911,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10395,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":944738304,"data_stored":944661513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160915,"ondisk_log_size":160915,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182491,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":736836,"kb_used_data":335924,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93635004,"statfs":{"total":96636764160,"available":95882244096,"internally_reserved":0,"allocated":343986176,"data_stored":343733883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":410513066},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,3,6,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986262,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":630356,"kb_used_data":305728,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93741484,"statfs":{"total":96636764160,"available":95991279616,"internally_reserved":0,"allocated":313065472,"data_stored":312813184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332394154},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,1,1,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659209,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":630420,"kb_used_data":283584,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93741420,"statfs":{"total":96636764160,"available":95991214080,"internally_reserved":0,"allocated":290390016,"data_stored":290075843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355135150},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,4,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":343445504,"data_stored":343441891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":289132544,"data_stored":289067051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":312160256,"data_stored":312152571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:36:09.915 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:36:10.470 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:10.471 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:36:10.496 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":281,"stamp":"2024-09-10T20:36:08.975203+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":382985684,"num_objects":93508,"num_object_clones":0,"num_object_copies":187016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32754,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93508,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":201913,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10397,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160918,"ondisk_log_size":160918,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1997612,"kb_used_data":925236,"kb_used_omap":12,"kb_used_meta":1072307,"kb_avail":281117908,"statfs":{"total":289910292480,"available":287864737792,"internally_reserved":0,"allocated":947441664,"data_stored":946622910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1098042370},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,7,3,8,11,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25595904,"num_objects":-6249,"num_object_clones":0,"num_object_copies":-12498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-317,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6249,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":317,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004186"},"pg_stats":[{"pgid":"2.f","version":"47'14853","reported_seq":15071,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:08.647361+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:36:08.647361+0000","last_peered":"2024-09-10T20:36:08.647361+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:36:08.647361+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23777280,"num_objects":5805,"num_object_clones":0,"num_object_copies":11610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14853,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2264}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14976","reported_seq":15091,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:04.092702+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:36:04.092702+0000","last_peered":"2024-09-10T20:36:04.092702+0000","last_clean":"2024-09-10T20:36:04.092702+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:36:04.092702+0000","last_undegraded":"2024-09-10T20:36:04.092702+0000","last_fullsized":"2024-09-10T20:36:04.092702+0000","mapping_epoch":35,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24436736,"num_objects":5966,"num_object_clones":0,"num_object_copies":11932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13162,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14791","reported_seq":22143,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:05.188777+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:36:05.188777+0000","last_peered":"2024-09-10T20:36:05.188777+0000","last_clean":"2024-09-10T20:36:05.188777+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:36:05.188777+0000","last_undegraded":"2024-09-10T20:36:05.188777+0000","last_fullsized":"2024-09-10T20:36:05.188777+0000","mapping_epoch":44,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24264728,"num_objects":5925,"num_object_clones":0,"num_object_copies":11850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5925,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12919,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'14674","reported_seq":14760,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:04.028125+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:36:04.028125+0000","last_peered":"2024-09-10T20:36:04.028125+0000","last_clean":"2024-09-10T20:36:04.028125+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:36:04.028125+0000","last_undegraded":"2024-09-10T20:36:04.028125+0000","last_fullsized":"2024-09-10T20:36:04.028125+0000","mapping_epoch":35,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24289280,"num_objects":5930,"num_object_clones":0,"num_object_copies":11860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14674,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15179","reported_seq":15311,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:08.567866+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:36:08.567866+0000","last_peered":"2024-09-10T20:36:08.567866+0000","last_clean":"2024-09-10T20:36:08.567866+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:36:08.567866+0000","last_undegraded":"2024-09-10T20:36:08.567866+0000","last_fullsized":"2024-09-10T20:36:08.567866+0000","mapping_epoch":34,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24125440,"num_objects":5890,"num_object_clones":0,"num_object_copies":11780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12175,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14841","reported_seq":15087,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:08.605013+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:36:08.605013+0000","last_peered":"2024-09-10T20:36:08.605013+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:36:08.605013+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23060480,"num_objects":5630,"num_object_clones":0,"num_object_copies":11260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11705,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2069}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'14805","reported_seq":15082,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:08.623177+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:36:08.623177+0000","last_peered":"2024-09-10T20:36:08.623177+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:36:08.623177+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23769088,"num_objects":5803,"num_object_clones":0,"num_object_copies":11606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11400,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2317}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14974","reported_seq":15104,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:08.556977+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:36:08.556977+0000","last_peered":"2024-09-10T20:36:08.556977+0000","last_clean":"2024-09-10T20:36:08.556977+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:36:08.556977+0000","last_undegraded":"2024-09-10T20:36:08.556977+0000","last_fullsized":"2024-09-10T20:36:08.556977+0000","mapping_epoch":34,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23601152,"num_objects":5762,"num_object_clones":0,"num_object_copies":11524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12045,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080479+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080479+0000","last_peered":"2024-09-10T20:34:27.080479+0000","last_clean":"2024-09-10T20:34:27.080479+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080479+0000","last_undegraded":"2024-09-10T20:34:27.080479+0000","last_fullsized":"2024-09-10T20:34:27.080479+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14534","reported_seq":14730,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:03.986837+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:36:03.986837+0000","last_peered":"2024-09-10T20:36:03.986837+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:36:03.986837+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24186880,"num_objects":5905,"num_object_clones":0,"num_object_copies":11810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12527,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2408}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086274+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086274+0000","last_peered":"2024-09-10T20:34:27.086274+0000","last_clean":"2024-09-10T20:34:27.086274+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086274+0000","last_undegraded":"2024-09-10T20:34:27.086274+0000","last_fullsized":"2024-09-10T20:34:27.086274+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14955","reported_seq":15167,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:08.584350+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:36:08.584350+0000","last_peered":"2024-09-10T20:36:08.584350+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:36:08.584350+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23655366,"num_objects":5777,"num_object_clones":0,"num_object_copies":11554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14953,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2211}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14709","reported_seq":14897,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:04.007779+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:36:04.007779+0000","last_peered":"2024-09-10T20:36:04.007779+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:36:04.007779+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23986176,"num_objects":5856,"num_object_clones":0,"num_object_copies":11712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12537,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2250}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14744","reported_seq":20603,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:04.102927+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:36:04.102927+0000","last_peered":"2024-09-10T20:36:04.102927+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:36:04.102927+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24248320,"num_objects":5920,"num_object_clones":0,"num_object_copies":11840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":750,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12530,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2862,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":5170}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14977","reported_seq":15228,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:08.605792+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:36:08.605792+0000","last_peered":"2024-09-10T20:36:08.605792+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:36:08.605792+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23973888,"num_objects":5853,"num_object_clones":0,"num_object_copies":11706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11700,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2261}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14798","reported_seq":15038,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:08.611931+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:36:08.611931+0000","last_peered":"2024-09-10T20:36:08.611931+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:36:08.611931+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23699456,"num_objects":5786,"num_object_clones":0,"num_object_copies":11572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11492,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2302}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086784+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086784+0000","last_peered":"2024-09-10T20:34:27.086784+0000","last_clean":"2024-09-10T20:34:27.086784+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086784+0000","last_undegraded":"2024-09-10T20:34:27.086784+0000","last_fullsized":"2024-09-10T20:34:27.086784+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15175","reported_seq":15428,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:08.566015+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:36:08.566015+0000","last_peered":"2024-09-10T20:36:08.566015+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:36:08.566015+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23801856,"num_objects":5811,"num_object_clones":0,"num_object_copies":11622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12002,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2156}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14630","reported_seq":21752,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:05.136591+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:36:05.136591+0000","last_peered":"2024-09-10T20:36:05.136591+0000","last_clean":"2024-09-10T20:36:05.136591+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:36:05.136591+0000","last_undegraded":"2024-09-10T20:36:05.136591+0000","last_fullsized":"2024-09-10T20:36:05.136591+0000","mapping_epoch":46,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24109539,"num_objects":5887,"num_object_clones":0,"num_object_copies":11774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11237,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":382985665,"num_objects":93506,"num_object_clones":0,"num_object_copies":187012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32754,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93506,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":201911,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10395,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":944738304,"data_stored":944661513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160915,"ondisk_log_size":160915,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182491,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":736836,"kb_used_data":335924,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93635004,"statfs":{"total":96636764160,"available":95882244096,"internally_reserved":0,"allocated":343986176,"data_stored":343733883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":410513066},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,3,6,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986262,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":630356,"kb_used_data":305728,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93741484,"statfs":{"total":96636764160,"available":95991279616,"internally_reserved":0,"allocated":313065472,"data_stored":312813184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332394154},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,1,1,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659209,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":630420,"kb_used_data":283584,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93741420,"statfs":{"total":96636764160,"available":95991214080,"internally_reserved":0,"allocated":290390016,"data_stored":290075843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355135150},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,4,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":343445504,"data_stored":343441891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":289132544,"data_stored":289067051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":312160256,"data_stored":312152571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:36:10.497 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:36:10.498 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'14853", 'reported_seq': 15071, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:08.647361+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:36:08.647361+0000', 'last_peered': '2024-09-10T20:36:08.647361+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:36:08.647361+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23777280, 'num_objects': 5805, 'num_object_clones': 0, 'num_object_copies': 11610, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5805, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14853, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2264}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:10.498 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:36:10.498 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'14841", 'reported_seq': 15087, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:08.605013+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:36:08.605013+0000', 'last_peered': '2024-09-10T20:36:08.605013+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:36:08.605013+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23060480, 'num_objects': 5630, 'num_object_clones': 0, 'num_object_copies': 11260, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5630, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11705, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2069}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:10.498 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:36:10.498 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'14805", 'reported_seq': 15082, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:08.623177+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:36:08.623177+0000', 'last_peered': '2024-09-10T20:36:08.623177+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:36:08.623177+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23769088, 'num_objects': 5803, 'num_object_clones': 0, 'num_object_copies': 11606, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5803, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11400, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2317}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:10.498 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:36:10.498 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'14534", 'reported_seq': 14730, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:03.986837+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:36:03.986837+0000', 'last_peered': '2024-09-10T20:36:03.986837+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:36:03.986837+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "43'4500", 'ondisk_log_start': "43'4500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24186880, 'num_objects': 5905, 'num_object_clones': 0, 'num_object_copies': 11810, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5905, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12527, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2408}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:36:10.498 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:36:10.498 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'14955", 'reported_seq': 15167, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:08.584350+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:36:08.584350+0000', 'last_peered': '2024-09-10T20:36:08.584350+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:36:08.584350+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "43'4900", 'ondisk_log_start': "43'4900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23655366, 'num_objects': 5777, 'num_object_clones': 0, 'num_object_copies': 11554, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5777, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14953, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2211}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:10.499 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:36:10.499 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'14709", 'reported_seq': 14897, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:04.007779+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:36:04.007779+0000', 'last_peered': '2024-09-10T20:36:04.007779+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:36:04.007779+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "43'4700", 'ondisk_log_start': "43'4700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23986176, 'num_objects': 5856, 'num_object_clones': 0, 'num_object_copies': 11712, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5856, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12537, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2250}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:36:10.499 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:36:10.499 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'14744", 'reported_seq': 20603, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:04.102927+0000', 'last_change': '2024-09-10T20:34:25.241937+0000', 'last_active': '2024-09-10T20:36:04.102927+0000', 'last_peered': '2024-09-10T20:36:04.102927+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:36:04.102927+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "43'4700", 'ondisk_log_start': "43'4700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24248320, 'num_objects': 5920, 'num_object_clones': 0, 'num_object_copies': 11840, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 750, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5920, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12530, 'num_write_kb': 32484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2862, 'num_bytes_recovered': 8085504, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 5170}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:36:10.499 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:36:10.499 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'14977", 'reported_seq': 15228, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:08.605792+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:36:08.605792+0000', 'last_peered': '2024-09-10T20:36:08.605792+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:36:08.605792+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "43'4900", 'ondisk_log_start': "43'4900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23973888, 'num_objects': 5853, 'num_object_clones': 0, 'num_object_copies': 11706, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5853, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11700, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2261}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:10.499 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:36:10.499 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'14798", 'reported_seq': 15038, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:08.611931+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:36:08.611931+0000', 'last_peered': '2024-09-10T20:36:08.611931+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:36:08.611931+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "43'4700", 'ondisk_log_start': "43'4700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23699456, 'num_objects': 5786, 'num_object_clones': 0, 'num_object_copies': 11572, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3487, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5786, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11492, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 48, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2302}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:10.499 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:36:10.499 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'15175", 'reported_seq': 15428, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:08.566015+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:36:08.566015+0000', 'last_peered': '2024-09-10T20:36:08.566015+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:36:08.566015+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "43'5100", 'ondisk_log_start': "43'5100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23801856, 'num_objects': 5811, 'num_object_clones': 0, 'num_object_copies': 11622, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5811, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12002, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2156}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:10.500 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:36:11.200 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:11.201 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32614/184798 objects degraded (17.648%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":482,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":92399,"data_bytes":378443220,"bytes_used":2028863488,"bytes_avail":287881428992,"bytes_total":289910292480,"degraded_objects":32614,"degraded_total":184798,"degraded_ratio":0.17648459398911243,"recovering_objects_per_sec":38,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":457,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":496},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:36:11.228 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:36:11.228 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:36:11.773 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:11.775 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:36:11.804 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":282,"stamp":"2024-09-10T20:36:10.975960+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":378443220,"num_objects":92399,"num_object_clones":0,"num_object_copies":184798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32614,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92399,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":203022,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10537,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160827,"ondisk_log_size":160827,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1981312,"kb_used_data":913084,"kb_used_omap":12,"kb_used_meta":1068147,"kb_avail":281134208,"statfs":{"total":289910292480,"available":287881428992,"internally_reserved":0,"allocated":934998016,"data_stored":934179262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1093782530},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,2,12,10,13,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24403968,"num_objects":-5958,"num_object_clones":0,"num_object_copies":-11916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-457,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5958,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":457,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004367"},"pg_stats":[{"pgid":"2.f","version":"47'14853","reported_seq":15071,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:08.647361+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:36:08.647361+0000","last_peered":"2024-09-10T20:36:08.647361+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:36:08.647361+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23777280,"num_objects":5805,"num_object_clones":0,"num_object_copies":11610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14853,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2264}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15141","reported_seq":15257,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:09.081331+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:36:09.081331+0000","last_peered":"2024-09-10T20:36:09.081331+0000","last_clean":"2024-09-10T20:36:09.081331+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:36:09.081331+0000","last_undegraded":"2024-09-10T20:36:09.081331+0000","last_fullsized":"2024-09-10T20:36:09.081331+0000","mapping_epoch":35,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23760896,"num_objects":5801,"num_object_clones":0,"num_object_copies":11602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5801,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13327,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14949","reported_seq":22302,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:10.257917+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:36:10.257917+0000","last_peered":"2024-09-10T20:36:10.257917+0000","last_clean":"2024-09-10T20:36:10.257917+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:36:10.257917+0000","last_undegraded":"2024-09-10T20:36:10.257917+0000","last_fullsized":"2024-09-10T20:36:10.257917+0000","mapping_epoch":44,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23617560,"num_objects":5767,"num_object_clones":0,"num_object_copies":11534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5767,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13077,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'14823","reported_seq":14909,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:09.101492+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:36:09.101492+0000","last_peered":"2024-09-10T20:36:09.101492+0000","last_clean":"2024-09-10T20:36:09.101492+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:36:09.101492+0000","last_undegraded":"2024-09-10T20:36:09.101492+0000","last_fullsized":"2024-09-10T20:36:09.101492+0000","mapping_epoch":35,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23678976,"num_objects":5781,"num_object_clones":0,"num_object_copies":11562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14823,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15179","reported_seq":15311,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:08.567866+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:36:08.567866+0000","last_peered":"2024-09-10T20:36:08.567866+0000","last_clean":"2024-09-10T20:36:08.567866+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:36:08.567866+0000","last_undegraded":"2024-09-10T20:36:08.567866+0000","last_fullsized":"2024-09-10T20:36:08.567866+0000","mapping_epoch":34,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24125440,"num_objects":5890,"num_object_clones":0,"num_object_copies":11780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12175,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14841","reported_seq":15087,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:08.605013+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:36:08.605013+0000","last_peered":"2024-09-10T20:36:08.605013+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:36:08.605013+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23060480,"num_objects":5630,"num_object_clones":0,"num_object_copies":11260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11705,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2069}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'14805","reported_seq":15082,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:08.623177+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:36:08.623177+0000","last_peered":"2024-09-10T20:36:08.623177+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:36:08.623177+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23769088,"num_objects":5803,"num_object_clones":0,"num_object_copies":11606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11400,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2317}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14974","reported_seq":15104,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:08.556977+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:36:08.556977+0000","last_peered":"2024-09-10T20:36:08.556977+0000","last_clean":"2024-09-10T20:36:08.556977+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:36:08.556977+0000","last_undegraded":"2024-09-10T20:36:08.556977+0000","last_fullsized":"2024-09-10T20:36:08.556977+0000","mapping_epoch":34,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23601152,"num_objects":5762,"num_object_clones":0,"num_object_copies":11524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12045,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080479+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080479+0000","last_peered":"2024-09-10T20:34:27.080479+0000","last_clean":"2024-09-10T20:34:27.080479+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080479+0000","last_undegraded":"2024-09-10T20:34:27.080479+0000","last_fullsized":"2024-09-10T20:34:27.080479+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14685","reported_seq":14881,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:09.104329+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:36:09.104329+0000","last_peered":"2024-09-10T20:36:09.104329+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:36:09.104329+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23568384,"num_objects":5754,"num_object_clones":0,"num_object_copies":11508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12678,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2257}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086274+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086274+0000","last_peered":"2024-09-10T20:34:27.086274+0000","last_clean":"2024-09-10T20:34:27.086274+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086274+0000","last_undegraded":"2024-09-10T20:34:27.086274+0000","last_fullsized":"2024-09-10T20:34:27.086274+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14955","reported_seq":15167,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:08.584350+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:36:08.584350+0000","last_peered":"2024-09-10T20:36:08.584350+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:36:08.584350+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23655366,"num_objects":5777,"num_object_clones":0,"num_object_copies":11554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14953,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2211}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14852","reported_seq":15039,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:09.095722+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:36:09.095722+0000","last_peered":"2024-09-10T20:36:09.095722+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:36:09.095722+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23400448,"num_objects":5713,"num_object_clones":0,"num_object_copies":11426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12680,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2107}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14908","reported_seq":21047,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:09.132882+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:36:09.132882+0000","last_peered":"2024-09-10T20:36:09.132882+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:36:09.132882+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23576576,"num_objects":5756,"num_object_clones":0,"num_object_copies":11512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":610,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12694,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3002,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":5146}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14977","reported_seq":15228,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:08.605792+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:36:08.605792+0000","last_peered":"2024-09-10T20:36:08.605792+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:36:08.605792+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23973888,"num_objects":5853,"num_object_clones":0,"num_object_copies":11706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11700,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2261}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14798","reported_seq":15038,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:08.611931+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:36:08.611931+0000","last_peered":"2024-09-10T20:36:08.611931+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:36:08.611931+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23699456,"num_objects":5786,"num_object_clones":0,"num_object_copies":11572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11492,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2302}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086784+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086784+0000","last_peered":"2024-09-10T20:34:27.086784+0000","last_clean":"2024-09-10T20:34:27.086784+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086784+0000","last_undegraded":"2024-09-10T20:34:27.086784+0000","last_fullsized":"2024-09-10T20:34:27.086784+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15175","reported_seq":15428,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:08.566015+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:36:08.566015+0000","last_peered":"2024-09-10T20:36:08.566015+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:36:08.566015+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23801856,"num_objects":5811,"num_object_clones":0,"num_object_copies":11622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12002,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2156}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14809","reported_seq":21931,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:10.166208+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:36:10.166208+0000","last_peered":"2024-09-10T20:36:10.166208+0000","last_clean":"2024-09-10T20:36:10.166208+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:36:10.166208+0000","last_undegraded":"2024-09-10T20:36:10.166208+0000","last_fullsized":"2024-09-10T20:36:10.166208+0000","mapping_epoch":46,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23376355,"num_objects":5708,"num_object_clones":0,"num_object_copies":11416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11416,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":378443201,"num_objects":92397,"num_object_clones":0,"num_object_copies":184794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32614,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92397,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":203020,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10535,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":931991552,"data_stored":931914761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160824,"ondisk_log_size":160824,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182491,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":736836,"kb_used_data":335924,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93635004,"statfs":{"total":96636764160,"available":95882244096,"internally_reserved":0,"allocated":343986176,"data_stored":343733883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":410513066},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,3,6,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986263,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":629752,"kb_used_data":300964,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93742088,"statfs":{"total":96636764160,"available":95991898112,"internally_reserved":0,"allocated":308187136,"data_stored":307934848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336653994},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659210,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":614724,"kb_used_data":276196,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93757116,"statfs":{"total":96636764160,"available":96007286784,"internally_reserved":0,"allocated":282824704,"data_stored":282510531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":346615470},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,6,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":343445504,"data_stored":343441891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":281497600,"data_stored":281432107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":307048448,"data_stored":307040763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:36:14.806 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:36:15.352 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:15.354 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:36:15.372 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":284,"stamp":"2024-09-10T20:36:14.977666+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":369657300,"num_objects":90254,"num_object_clones":0,"num_object_copies":180508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90254,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":205167,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10695,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160672,"ondisk_log_size":160672,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1948220,"kb_used_data":900788,"kb_used_omap":12,"kb_used_meta":1047347,"kb_avail":281167300,"statfs":{"total":289910292480,"available":287915315200,"internally_reserved":0,"allocated":922406912,"data_stored":921588158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1072483330},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,2,3,11,7,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28778496,"num_objects":-7026,"num_object_clones":0,"num_object_copies":-14052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-464,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7026,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":464,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004582"},"pg_stats":[{"pgid":"2.f","version":"47'15020","reported_seq":15236,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:13.675886+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:36:13.675886+0000","last_peered":"2024-09-10T20:36:13.675886+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:36:13.675886+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23093248,"num_objects":5638,"num_object_clones":0,"num_object_copies":11276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15020,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2097}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15319","reported_seq":15434,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:14.163490+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:36:14.163490+0000","last_peered":"2024-09-10T20:36:14.163490+0000","last_clean":"2024-09-10T20:36:14.163490+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:36:14.163490+0000","last_undegraded":"2024-09-10T20:36:14.163490+0000","last_fullsized":"2024-09-10T20:36:14.163490+0000","mapping_epoch":35,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23031808,"num_objects":5623,"num_object_clones":0,"num_object_copies":11246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13505,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14949","reported_seq":22302,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:10.257917+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:36:10.257917+0000","last_peered":"2024-09-10T20:36:10.257917+0000","last_clean":"2024-09-10T20:36:10.257917+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:36:10.257917+0000","last_undegraded":"2024-09-10T20:36:10.257917+0000","last_fullsized":"2024-09-10T20:36:10.257917+0000","mapping_epoch":44,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23617560,"num_objects":5767,"num_object_clones":0,"num_object_copies":11534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5767,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13077,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'14967","reported_seq":15053,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:14.143038+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:36:14.143038+0000","last_peered":"2024-09-10T20:36:14.143038+0000","last_clean":"2024-09-10T20:36:14.143038+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:36:14.143038+0000","last_undegraded":"2024-09-10T20:36:14.143038+0000","last_fullsized":"2024-09-10T20:36:14.143038+0000","mapping_epoch":35,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23089152,"num_objects":5637,"num_object_clones":0,"num_object_copies":11274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14967,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15338","reported_seq":15470,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:13.613677+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:36:13.613677+0000","last_peered":"2024-09-10T20:36:13.613677+0000","last_clean":"2024-09-10T20:36:13.613677+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:36:13.613677+0000","last_undegraded":"2024-09-10T20:36:13.613677+0000","last_fullsized":"2024-09-10T20:36:13.613677+0000","mapping_epoch":34,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23474176,"num_objects":5731,"num_object_clones":0,"num_object_copies":11462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12334,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14986","reported_seq":15234,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:13.641896+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:36:13.641896+0000","last_peered":"2024-09-10T20:36:13.641896+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:36:13.641896+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22466560,"num_objects":5485,"num_object_clones":0,"num_object_copies":10970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11850,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1924}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'14961","reported_seq":15242,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:13.593690+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:36:13.593690+0000","last_peered":"2024-09-10T20:36:13.593690+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:36:13.593690+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23130112,"num_objects":5647,"num_object_clones":0,"num_object_copies":11294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5647,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11556,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15134","reported_seq":15264,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:13.596834+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:36:13.596834+0000","last_peered":"2024-09-10T20:36:13.596834+0000","last_clean":"2024-09-10T20:36:13.596834+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:36:13.596834+0000","last_undegraded":"2024-09-10T20:36:13.596834+0000","last_fullsized":"2024-09-10T20:36:13.596834+0000","mapping_epoch":34,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22945792,"num_objects":5602,"num_object_clones":0,"num_object_copies":11204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12205,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080479+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080479+0000","last_peered":"2024-09-10T20:34:27.080479+0000","last_clean":"2024-09-10T20:34:27.080479+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080479+0000","last_undegraded":"2024-09-10T20:34:27.080479+0000","last_fullsized":"2024-09-10T20:34:27.080479+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14824","reported_seq":15020,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:14.105577+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:36:14.105577+0000","last_peered":"2024-09-10T20:36:14.105577+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:36:14.105577+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22999040,"num_objects":5615,"num_object_clones":0,"num_object_copies":11230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12817,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2118}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086274+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086274+0000","last_peered":"2024-09-10T20:34:27.086274+0000","last_clean":"2024-09-10T20:34:27.086274+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086274+0000","last_undegraded":"2024-09-10T20:34:27.086274+0000","last_fullsized":"2024-09-10T20:34:27.086274+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15106","reported_seq":15318,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:13.579620+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:36:13.579620+0000","last_peered":"2024-09-10T20:36:13.579620+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:36:13.579620+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23036870,"num_objects":5626,"num_object_clones":0,"num_object_copies":11252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15104,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2060}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14970","reported_seq":15153,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:14.089729+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:36:14.089729+0000","last_peered":"2024-09-10T20:36:14.089729+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:36:14.089729+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22917120,"num_objects":5595,"num_object_clones":0,"num_object_copies":11190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12798,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1989}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15072","reported_seq":21527,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:14.145873+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:36:14.145873+0000","last_peered":"2024-09-10T20:36:14.145873+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:36:14.145873+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22904832,"num_objects":5592,"num_object_clones":0,"num_object_copies":11184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":452,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12858,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3160,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":5140}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15121","reported_seq":15373,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:13.512535+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:36:13.512535+0000","last_peered":"2024-09-10T20:36:13.512535+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:36:13.512535+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23384064,"num_objects":5709,"num_object_clones":0,"num_object_copies":11418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11844,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14956","reported_seq":15196,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:13.658579+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:36:13.658579+0000","last_peered":"2024-09-10T20:36:13.658579+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:36:13.658579+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23052288,"num_objects":5628,"num_object_clones":0,"num_object_copies":11256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11650,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2144}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086784+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086784+0000","last_peered":"2024-09-10T20:34:27.086784+0000","last_clean":"2024-09-10T20:34:27.086784+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086784+0000","last_undegraded":"2024-09-10T20:34:27.086784+0000","last_fullsized":"2024-09-10T20:34:27.086784+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15337","reported_seq":15590,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:13.647690+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:36:13.647690+0000","last_peered":"2024-09-10T20:36:13.647690+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:36:13.647690+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23138304,"num_objects":5649,"num_object_clones":0,"num_object_copies":11298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12164,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1994}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14809","reported_seq":21931,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:10.166208+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:36:10.166208+0000","last_peered":"2024-09-10T20:36:10.166208+0000","last_clean":"2024-09-10T20:36:10.166208+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:36:10.166208+0000","last_undegraded":"2024-09-10T20:36:10.166208+0000","last_fullsized":"2024-09-10T20:36:10.166208+0000","mapping_epoch":46,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23376355,"num_objects":5708,"num_object_clones":0,"num_object_copies":11416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11416,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":369657281,"num_objects":90252,"num_object_clones":0,"num_object_copies":180504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90252,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":205165,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10693,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":919474176,"data_stored":919397385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160669,"ondisk_log_size":160669,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182492,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":720964,"kb_used_data":328372,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93650876,"statfs":{"total":96636764160,"available":95898497024,"internally_reserved":0,"allocated":336252928,"data_stored":336000635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":401993386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,2,4,2,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986264,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612532,"kb_used_data":296220,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93759308,"statfs":{"total":96636764160,"available":96009531392,"internally_reserved":0,"allocated":303329280,"data_stored":303076992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323874474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659210,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":614724,"kb_used_data":276196,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93757116,"statfs":{"total":96636764160,"available":96007286784,"internally_reserved":0,"allocated":282824704,"data_stored":282510531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":346615470},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,6,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":335777792,"data_stored":335774179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":281497600,"data_stored":281432107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":302198784,"data_stored":302191099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:36:15.374 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:36:15.926 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:15.929 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:36:15.948 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":284,"stamp":"2024-09-10T20:36:14.977666+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":369657300,"num_objects":90254,"num_object_clones":0,"num_object_copies":180508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90254,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":205167,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10695,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160672,"ondisk_log_size":160672,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1948220,"kb_used_data":900788,"kb_used_omap":12,"kb_used_meta":1047347,"kb_avail":281167300,"statfs":{"total":289910292480,"available":287915315200,"internally_reserved":0,"allocated":922406912,"data_stored":921588158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1072483330},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,2,3,11,7,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28778496,"num_objects":-7026,"num_object_clones":0,"num_object_copies":-14052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-464,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7026,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":464,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004582"},"pg_stats":[{"pgid":"2.f","version":"47'15020","reported_seq":15236,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:13.675886+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:36:13.675886+0000","last_peered":"2024-09-10T20:36:13.675886+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:36:13.675886+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23093248,"num_objects":5638,"num_object_clones":0,"num_object_copies":11276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15020,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2097}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15319","reported_seq":15434,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:14.163490+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:36:14.163490+0000","last_peered":"2024-09-10T20:36:14.163490+0000","last_clean":"2024-09-10T20:36:14.163490+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:36:14.163490+0000","last_undegraded":"2024-09-10T20:36:14.163490+0000","last_fullsized":"2024-09-10T20:36:14.163490+0000","mapping_epoch":35,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23031808,"num_objects":5623,"num_object_clones":0,"num_object_copies":11246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13505,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14949","reported_seq":22302,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:10.257917+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:36:10.257917+0000","last_peered":"2024-09-10T20:36:10.257917+0000","last_clean":"2024-09-10T20:36:10.257917+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:36:10.257917+0000","last_undegraded":"2024-09-10T20:36:10.257917+0000","last_fullsized":"2024-09-10T20:36:10.257917+0000","mapping_epoch":44,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23617560,"num_objects":5767,"num_object_clones":0,"num_object_copies":11534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5767,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13077,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'14967","reported_seq":15053,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:14.143038+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:36:14.143038+0000","last_peered":"2024-09-10T20:36:14.143038+0000","last_clean":"2024-09-10T20:36:14.143038+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:36:14.143038+0000","last_undegraded":"2024-09-10T20:36:14.143038+0000","last_fullsized":"2024-09-10T20:36:14.143038+0000","mapping_epoch":35,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23089152,"num_objects":5637,"num_object_clones":0,"num_object_copies":11274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14967,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15338","reported_seq":15470,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:13.613677+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:36:13.613677+0000","last_peered":"2024-09-10T20:36:13.613677+0000","last_clean":"2024-09-10T20:36:13.613677+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:36:13.613677+0000","last_undegraded":"2024-09-10T20:36:13.613677+0000","last_fullsized":"2024-09-10T20:36:13.613677+0000","mapping_epoch":34,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23474176,"num_objects":5731,"num_object_clones":0,"num_object_copies":11462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12334,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14986","reported_seq":15234,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:13.641896+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:36:13.641896+0000","last_peered":"2024-09-10T20:36:13.641896+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:36:13.641896+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22466560,"num_objects":5485,"num_object_clones":0,"num_object_copies":10970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11850,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1924}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'14961","reported_seq":15242,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:13.593690+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:36:13.593690+0000","last_peered":"2024-09-10T20:36:13.593690+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:36:13.593690+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23130112,"num_objects":5647,"num_object_clones":0,"num_object_copies":11294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5647,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11556,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15134","reported_seq":15264,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:13.596834+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:36:13.596834+0000","last_peered":"2024-09-10T20:36:13.596834+0000","last_clean":"2024-09-10T20:36:13.596834+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:36:13.596834+0000","last_undegraded":"2024-09-10T20:36:13.596834+0000","last_fullsized":"2024-09-10T20:36:13.596834+0000","mapping_epoch":34,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22945792,"num_objects":5602,"num_object_clones":0,"num_object_copies":11204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12205,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080479+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080479+0000","last_peered":"2024-09-10T20:34:27.080479+0000","last_clean":"2024-09-10T20:34:27.080479+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080479+0000","last_undegraded":"2024-09-10T20:34:27.080479+0000","last_fullsized":"2024-09-10T20:34:27.080479+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14824","reported_seq":15020,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:14.105577+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:36:14.105577+0000","last_peered":"2024-09-10T20:36:14.105577+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:36:14.105577+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22999040,"num_objects":5615,"num_object_clones":0,"num_object_copies":11230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12817,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2118}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086274+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086274+0000","last_peered":"2024-09-10T20:34:27.086274+0000","last_clean":"2024-09-10T20:34:27.086274+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086274+0000","last_undegraded":"2024-09-10T20:34:27.086274+0000","last_fullsized":"2024-09-10T20:34:27.086274+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15106","reported_seq":15318,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:13.579620+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:36:13.579620+0000","last_peered":"2024-09-10T20:36:13.579620+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:36:13.579620+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23036870,"num_objects":5626,"num_object_clones":0,"num_object_copies":11252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15104,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2060}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14970","reported_seq":15153,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:14.089729+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:36:14.089729+0000","last_peered":"2024-09-10T20:36:14.089729+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:36:14.089729+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22917120,"num_objects":5595,"num_object_clones":0,"num_object_copies":11190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12798,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1989}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15072","reported_seq":21527,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:14.145873+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:36:14.145873+0000","last_peered":"2024-09-10T20:36:14.145873+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:36:14.145873+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22904832,"num_objects":5592,"num_object_clones":0,"num_object_copies":11184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":452,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12858,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3160,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":5140}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15121","reported_seq":15373,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:13.512535+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:36:13.512535+0000","last_peered":"2024-09-10T20:36:13.512535+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:36:13.512535+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23384064,"num_objects":5709,"num_object_clones":0,"num_object_copies":11418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11844,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14956","reported_seq":15196,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:13.658579+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:36:13.658579+0000","last_peered":"2024-09-10T20:36:13.658579+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:36:13.658579+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23052288,"num_objects":5628,"num_object_clones":0,"num_object_copies":11256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11650,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2144}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086784+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086784+0000","last_peered":"2024-09-10T20:34:27.086784+0000","last_clean":"2024-09-10T20:34:27.086784+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086784+0000","last_undegraded":"2024-09-10T20:34:27.086784+0000","last_fullsized":"2024-09-10T20:34:27.086784+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15337","reported_seq":15590,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:13.647690+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:36:13.647690+0000","last_peered":"2024-09-10T20:36:13.647690+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:36:13.647690+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23138304,"num_objects":5649,"num_object_clones":0,"num_object_copies":11298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12164,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1994}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14809","reported_seq":21931,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:10.166208+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:36:10.166208+0000","last_peered":"2024-09-10T20:36:10.166208+0000","last_clean":"2024-09-10T20:36:10.166208+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:36:10.166208+0000","last_undegraded":"2024-09-10T20:36:10.166208+0000","last_fullsized":"2024-09-10T20:36:10.166208+0000","mapping_epoch":46,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23376355,"num_objects":5708,"num_object_clones":0,"num_object_copies":11416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11416,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":369657281,"num_objects":90252,"num_object_clones":0,"num_object_copies":180504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90252,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":205165,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10693,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":919474176,"data_stored":919397385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160669,"ondisk_log_size":160669,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182492,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":720964,"kb_used_data":328372,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93650876,"statfs":{"total":96636764160,"available":95898497024,"internally_reserved":0,"allocated":336252928,"data_stored":336000635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":401993386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,2,4,2,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986264,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612532,"kb_used_data":296220,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93759308,"statfs":{"total":96636764160,"available":96009531392,"internally_reserved":0,"allocated":303329280,"data_stored":303076992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323874474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659210,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":614724,"kb_used_data":276196,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93757116,"statfs":{"total":96636764160,"available":96007286784,"internally_reserved":0,"allocated":282824704,"data_stored":282510531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":346615470},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,6,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":335777792,"data_stored":335774179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":281497600,"data_stored":281432107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":302198784,"data_stored":302191099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:36:15.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:36:15.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'15020", 'reported_seq': 15236, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:13.675886+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:36:13.675886+0000', 'last_peered': '2024-09-10T20:36:13.675886+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:36:13.675886+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "43'5000", 'ondisk_log_start': "43'5000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23093248, 'num_objects': 5638, 'num_object_clones': 0, 'num_object_copies': 11276, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5638, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15020, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2097}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:15.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:36:15.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'14986", 'reported_seq': 15234, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:13.641896+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:36:13.641896+0000', 'last_peered': '2024-09-10T20:36:13.641896+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:36:13.641896+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "43'4900", 'ondisk_log_start': "43'4900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22466560, 'num_objects': 5485, 'num_object_clones': 0, 'num_object_copies': 10970, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5485, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11850, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1924}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:15.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:36:15.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'14961", 'reported_seq': 15242, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:13.593690+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:36:13.593690+0000', 'last_peered': '2024-09-10T20:36:13.593690+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:36:13.593690+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "43'4900", 'ondisk_log_start': "43'4900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23130112, 'num_objects': 5647, 'num_object_clones': 0, 'num_object_copies': 11294, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5647, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11556, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:15.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:36:15.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'14824", 'reported_seq': 15020, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:14.105577+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:36:14.105577+0000', 'last_peered': '2024-09-10T20:36:14.105577+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:36:14.105577+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22999040, 'num_objects': 5615, 'num_object_clones': 0, 'num_object_copies': 11230, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5615, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12817, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2118}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:36:15.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:36:15.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'15106", 'reported_seq': 15318, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:13.579620+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:36:13.579620+0000', 'last_peered': '2024-09-10T20:36:13.579620+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:36:13.579620+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "43'5100", 'ondisk_log_start': "43'5100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23036870, 'num_objects': 5626, 'num_object_clones': 0, 'num_object_copies': 11252, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5626, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15104, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2060}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:15.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:36:15.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'14970", 'reported_seq': 15153, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:14.089729+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:36:14.089729+0000', 'last_peered': '2024-09-10T20:36:14.089729+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:36:14.089729+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "43'4900", 'ondisk_log_start': "43'4900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22917120, 'num_objects': 5595, 'num_object_clones': 0, 'num_object_copies': 11190, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5595, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12798, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1989}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:36:15.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:36:15.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'15072", 'reported_seq': 21527, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:14.145873+0000', 'last_change': '2024-09-10T20:34:25.241937+0000', 'last_active': '2024-09-10T20:36:14.145873+0000', 'last_peered': '2024-09-10T20:36:14.145873+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:36:14.145873+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "43'5000", 'ondisk_log_start': "43'5000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22904832, 'num_objects': 5592, 'num_object_clones': 0, 'num_object_copies': 11184, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 452, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5592, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12858, 'num_write_kb': 32484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3160, 'num_bytes_recovered': 8085504, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 5140}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:36:15.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:36:15.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'15121", 'reported_seq': 15373, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:13.512535+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:36:13.512535+0000', 'last_peered': '2024-09-10T20:36:13.512535+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:36:13.512535+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "43'5100", 'ondisk_log_start': "43'5100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23384064, 'num_objects': 5709, 'num_object_clones': 0, 'num_object_copies': 11418, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5709, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11844, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2117}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:15.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:36:15.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'14956", 'reported_seq': 15196, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:13.658579+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:36:13.658579+0000', 'last_peered': '2024-09-10T20:36:13.658579+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:36:13.658579+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "43'4900", 'ondisk_log_start': "43'4900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23052288, 'num_objects': 5628, 'num_object_clones': 0, 'num_object_copies': 11256, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3487, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5628, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11650, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 48, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2144}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:15.952 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:36:15.952 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'15337", 'reported_seq': 15590, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:13.647690+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:36:13.647690+0000', 'last_peered': '2024-09-10T20:36:13.647690+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:36:13.647690+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "43'5300", 'ondisk_log_start': "43'5300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23138304, 'num_objects': 5649, 'num_object_clones': 0, 'num_object_copies': 11298, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5649, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12164, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1994}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:15.952 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:36:16.666 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:16.666 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32456/180508 objects degraded (17.980%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":488,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":90254,"data_bytes":369657300,"bytes_used":1994977280,"bytes_avail":287915315200,"bytes_total":289910292480,"degraded_objects":32456,"degraded_total":180508,"degraded_ratio":0.17980366521151417,"recovering_objects_per_sec":38,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":464,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":585},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:36:16.693 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:36:16.693 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:36:17.246 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:17.248 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:36:17.272 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":285,"stamp":"2024-09-10T20:36:16.978364+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":368387540,"num_objects":89944,"num_object_clones":0,"num_object_copies":179888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89944,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":205477,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10695,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160782,"ondisk_log_size":160782,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1944932,"kb_used_data":893356,"kb_used_omap":12,"kb_used_meta":1051507,"kb_avail":281170588,"statfs":{"total":289910292480,"available":287918682112,"internally_reserved":0,"allocated":914796544,"data_stored":913977790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1076743170},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,2,5,7,8,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21434368,"num_objects":-5233,"num_object_clones":0,"num_object_copies":-10466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5233,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":298,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004631"},"pg_stats":[{"pgid":"2.f","version":"47'15020","reported_seq":15236,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:13.675886+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:36:13.675886+0000","last_peered":"2024-09-10T20:36:13.675886+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:36:13.675886+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23093248,"num_objects":5638,"num_object_clones":0,"num_object_copies":11276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15020,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2097}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15319","reported_seq":15434,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:14.163490+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:36:14.163490+0000","last_peered":"2024-09-10T20:36:14.163490+0000","last_clean":"2024-09-10T20:36:14.163490+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:36:14.163490+0000","last_undegraded":"2024-09-10T20:36:14.163490+0000","last_fullsized":"2024-09-10T20:36:14.163490+0000","mapping_epoch":35,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23031808,"num_objects":5623,"num_object_clones":0,"num_object_copies":11246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13505,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15093","reported_seq":22446,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:15.348187+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:36:15.348187+0000","last_peered":"2024-09-10T20:36:15.348187+0000","last_clean":"2024-09-10T20:36:15.348187+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:36:15.348187+0000","last_undegraded":"2024-09-10T20:36:15.348187+0000","last_fullsized":"2024-09-10T20:36:15.348187+0000","mapping_epoch":44,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23027736,"num_objects":5623,"num_object_clones":0,"num_object_copies":11246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5623,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13221,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'14967","reported_seq":15053,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:14.143038+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:36:14.143038+0000","last_peered":"2024-09-10T20:36:14.143038+0000","last_clean":"2024-09-10T20:36:14.143038+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:36:14.143038+0000","last_undegraded":"2024-09-10T20:36:14.143038+0000","last_fullsized":"2024-09-10T20:36:14.143038+0000","mapping_epoch":35,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23089152,"num_objects":5637,"num_object_clones":0,"num_object_copies":11274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14967,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15338","reported_seq":15470,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:13.613677+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:36:13.613677+0000","last_peered":"2024-09-10T20:36:13.613677+0000","last_clean":"2024-09-10T20:36:13.613677+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:36:13.613677+0000","last_undegraded":"2024-09-10T20:36:13.613677+0000","last_fullsized":"2024-09-10T20:36:13.613677+0000","mapping_epoch":34,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23474176,"num_objects":5731,"num_object_clones":0,"num_object_copies":11462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12334,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14986","reported_seq":15234,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:13.641896+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:36:13.641896+0000","last_peered":"2024-09-10T20:36:13.641896+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:36:13.641896+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22466560,"num_objects":5485,"num_object_clones":0,"num_object_copies":10970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11850,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1924}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'14961","reported_seq":15242,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:13.593690+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:36:13.593690+0000","last_peered":"2024-09-10T20:36:13.593690+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:36:13.593690+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23130112,"num_objects":5647,"num_object_clones":0,"num_object_copies":11294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5647,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11556,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15134","reported_seq":15264,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:13.596834+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:36:13.596834+0000","last_peered":"2024-09-10T20:36:13.596834+0000","last_clean":"2024-09-10T20:36:13.596834+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:36:13.596834+0000","last_undegraded":"2024-09-10T20:36:13.596834+0000","last_fullsized":"2024-09-10T20:36:13.596834+0000","mapping_epoch":34,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22945792,"num_objects":5602,"num_object_clones":0,"num_object_copies":11204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12205,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080479+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080479+0000","last_peered":"2024-09-10T20:34:27.080479+0000","last_clean":"2024-09-10T20:34:27.080479+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080479+0000","last_undegraded":"2024-09-10T20:34:27.080479+0000","last_fullsized":"2024-09-10T20:34:27.080479+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14824","reported_seq":15020,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:14.105577+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:36:14.105577+0000","last_peered":"2024-09-10T20:36:14.105577+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:36:14.105577+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22999040,"num_objects":5615,"num_object_clones":0,"num_object_copies":11230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12817,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2118}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086274+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086274+0000","last_peered":"2024-09-10T20:34:27.086274+0000","last_clean":"2024-09-10T20:34:27.086274+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086274+0000","last_undegraded":"2024-09-10T20:34:27.086274+0000","last_fullsized":"2024-09-10T20:34:27.086274+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15106","reported_seq":15318,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:13.579620+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:36:13.579620+0000","last_peered":"2024-09-10T20:36:13.579620+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:36:13.579620+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23036870,"num_objects":5626,"num_object_clones":0,"num_object_copies":11252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15104,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2060}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14970","reported_seq":15153,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:14.089729+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:36:14.089729+0000","last_peered":"2024-09-10T20:36:14.089729+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:36:14.089729+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22917120,"num_objects":5595,"num_object_clones":0,"num_object_copies":11190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12798,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1989}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15072","reported_seq":21527,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:14.145873+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:36:14.145873+0000","last_peered":"2024-09-10T20:36:14.145873+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:36:14.145873+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22904832,"num_objects":5592,"num_object_clones":0,"num_object_copies":11184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":452,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12858,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3160,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":5140}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15121","reported_seq":15373,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:13.512535+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:36:13.512535+0000","last_peered":"2024-09-10T20:36:13.512535+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:36:13.512535+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23384064,"num_objects":5709,"num_object_clones":0,"num_object_copies":11418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11844,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14956","reported_seq":15196,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:13.658579+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:36:13.658579+0000","last_peered":"2024-09-10T20:36:13.658579+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:36:13.658579+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23052288,"num_objects":5628,"num_object_clones":0,"num_object_copies":11256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11650,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2144}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086784+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086784+0000","last_peered":"2024-09-10T20:34:27.086784+0000","last_clean":"2024-09-10T20:34:27.086784+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086784+0000","last_undegraded":"2024-09-10T20:34:27.086784+0000","last_fullsized":"2024-09-10T20:34:27.086784+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15337","reported_seq":15590,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:13.647690+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:36:13.647690+0000","last_peered":"2024-09-10T20:36:13.647690+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:36:13.647690+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23138304,"num_objects":5649,"num_object_clones":0,"num_object_copies":11298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12164,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1994}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14975","reported_seq":22097,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:15.373413+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:36:15.373413+0000","last_peered":"2024-09-10T20:36:15.373413+0000","last_clean":"2024-09-10T20:36:15.373413+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:36:15.373413+0000","last_undegraded":"2024-09-10T20:36:15.373413+0000","last_fullsized":"2024-09-10T20:36:15.373413+0000","mapping_epoch":46,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22696419,"num_objects":5542,"num_object_clones":0,"num_object_copies":11084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11582,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":368387521,"num_objects":89942,"num_object_clones":0,"num_object_copies":179884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89942,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":205475,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10693,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":911884288,"data_stored":911807497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160779,"ondisk_log_size":160779,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182492,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":720964,"kb_used_data":328372,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93650876,"statfs":{"total":96636764160,"available":95898497024,"internally_reserved":0,"allocated":336252928,"data_stored":336000635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":401993386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,2,4,2,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986264,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612532,"kb_used_data":296220,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93759308,"statfs":{"total":96636764160,"available":96009531392,"internally_reserved":0,"allocated":303329280,"data_stored":303076992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323874474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659211,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":611436,"kb_used_data":268764,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93760404,"statfs":{"total":96636764160,"available":96010653696,"internally_reserved":0,"allocated":275214336,"data_stored":274900163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,2,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":335777792,"data_stored":335774179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":273907712,"data_stored":273842219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":302198784,"data_stored":302191099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:36:20.274 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:36:20.858 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:20.860 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:36:20.878 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2024-09-10T20:36:18.979167+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":362956244,"num_objects":88618,"num_object_clones":0,"num_object_copies":177236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88618,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":206803,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10695,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160808,"ondisk_log_size":160808,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1941948,"kb_used_data":886208,"kb_used_omap":12,"kb_used_meta":1055667,"kb_avail":281173572,"statfs":{"total":289910292480,"available":287921737728,"internally_reserved":0,"allocated":907476992,"data_stored":906658238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1081003010},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,6,6,13,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25600000,"num_objects":-6250,"num_object_clones":0,"num_object_copies":-12500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6250,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":298,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004783"},"pg_stats":[{"pgid":"2.f","version":"47'15155","reported_seq":15371,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:18.636440+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:36:18.636440+0000","last_peered":"2024-09-10T20:36:18.636440+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:36:18.636440+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22540288,"num_objects":5503,"num_object_clones":0,"num_object_copies":11006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15155,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1962}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15319","reported_seq":15434,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:14.163490+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:36:14.163490+0000","last_peered":"2024-09-10T20:36:14.163490+0000","last_clean":"2024-09-10T20:36:14.163490+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:36:14.163490+0000","last_undegraded":"2024-09-10T20:36:14.163490+0000","last_fullsized":"2024-09-10T20:36:14.163490+0000","mapping_epoch":35,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23031808,"num_objects":5623,"num_object_clones":0,"num_object_copies":11246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13505,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15093","reported_seq":22446,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:15.348187+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:36:15.348187+0000","last_peered":"2024-09-10T20:36:15.348187+0000","last_clean":"2024-09-10T20:36:15.348187+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:36:15.348187+0000","last_undegraded":"2024-09-10T20:36:15.348187+0000","last_fullsized":"2024-09-10T20:36:15.348187+0000","mapping_epoch":44,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23027736,"num_objects":5623,"num_object_clones":0,"num_object_copies":11246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5623,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13221,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'14967","reported_seq":15053,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:14.143038+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:36:14.143038+0000","last_peered":"2024-09-10T20:36:14.143038+0000","last_clean":"2024-09-10T20:36:14.143038+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:36:14.143038+0000","last_undegraded":"2024-09-10T20:36:14.143038+0000","last_fullsized":"2024-09-10T20:36:14.143038+0000","mapping_epoch":35,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23089152,"num_objects":5637,"num_object_clones":0,"num_object_copies":11274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14967,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15475","reported_seq":15607,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:18.732143+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:36:18.732143+0000","last_peered":"2024-09-10T20:36:18.732143+0000","last_clean":"2024-09-10T20:36:18.732143+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:36:18.732143+0000","last_undegraded":"2024-09-10T20:36:18.732143+0000","last_fullsized":"2024-09-10T20:36:18.732143+0000","mapping_epoch":34,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22913024,"num_objects":5594,"num_object_clones":0,"num_object_copies":11188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5594,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12471,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15135","reported_seq":15383,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:18.616463+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:36:18.616463+0000","last_peered":"2024-09-10T20:36:18.616463+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:36:18.616463+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21856256,"num_objects":5336,"num_object_clones":0,"num_object_copies":10672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11999,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1775}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15103","reported_seq":15384,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:18.695447+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:36:18.695447+0000","last_peered":"2024-09-10T20:36:18.695447+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:36:18.695447+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22548480,"num_objects":5505,"num_object_clones":0,"num_object_copies":11010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11698,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2019}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15302","reported_seq":15432,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:18.697226+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:36:18.697226+0000","last_peered":"2024-09-10T20:36:18.697226+0000","last_clean":"2024-09-10T20:36:18.697226+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:36:18.697226+0000","last_undegraded":"2024-09-10T20:36:18.697226+0000","last_fullsized":"2024-09-10T20:36:18.697226+0000","mapping_epoch":34,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22257664,"num_objects":5434,"num_object_clones":0,"num_object_copies":10868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12373,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080479+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080479+0000","last_peered":"2024-09-10T20:34:27.080479+0000","last_clean":"2024-09-10T20:34:27.080479+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080479+0000","last_undegraded":"2024-09-10T20:34:27.080479+0000","last_fullsized":"2024-09-10T20:34:27.080479+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14824","reported_seq":15020,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:14.105577+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:36:14.105577+0000","last_peered":"2024-09-10T20:36:14.105577+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:36:14.105577+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22999040,"num_objects":5615,"num_object_clones":0,"num_object_copies":11230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12817,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2118}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086274+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086274+0000","last_peered":"2024-09-10T20:34:27.086274+0000","last_clean":"2024-09-10T20:34:27.086274+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086274+0000","last_undegraded":"2024-09-10T20:34:27.086274+0000","last_fullsized":"2024-09-10T20:34:27.086274+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15245","reported_seq":15457,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:18.616446+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:36:18.616446+0000","last_peered":"2024-09-10T20:36:18.616446+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:36:18.616446+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22467526,"num_objects":5487,"num_object_clones":0,"num_object_copies":10974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15243,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1921}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14970","reported_seq":15153,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:14.089729+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:36:14.089729+0000","last_peered":"2024-09-10T20:36:14.089729+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:36:14.089729+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22917120,"num_objects":5595,"num_object_clones":0,"num_object_copies":11190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12798,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1989}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079292+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079292+0000","last_peered":"2024-09-10T20:34:27.079292+0000","last_clean":"2024-09-10T20:34:27.079292+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079292+0000","last_undegraded":"2024-09-10T20:34:27.079292+0000","last_fullsized":"2024-09-10T20:34:27.079292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15072","reported_seq":21527,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:14.145873+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:36:14.145873+0000","last_peered":"2024-09-10T20:36:14.145873+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:36:14.145873+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22904832,"num_objects":5592,"num_object_clones":0,"num_object_copies":11184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":452,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12858,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3160,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":5140}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15274","reported_seq":15526,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:18.558991+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:36:18.558991+0000","last_peered":"2024-09-10T20:36:18.558991+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:36:18.558991+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22757376,"num_objects":5556,"num_object_clones":0,"num_object_copies":11112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11997,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1964}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079951+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079951+0000","last_peered":"2024-09-10T20:34:27.079951+0000","last_clean":"2024-09-10T20:34:27.079951+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079951+0000","last_undegraded":"2024-09-10T20:34:27.079951+0000","last_fullsized":"2024-09-10T20:34:27.079951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15117","reported_seq":15357,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:18.718277+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:36:18.718277+0000","last_peered":"2024-09-10T20:36:18.718277+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:36:18.718277+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22392832,"num_objects":5467,"num_object_clones":0,"num_object_copies":10934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11811,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1983}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086784+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086784+0000","last_peered":"2024-09-10T20:34:27.086784+0000","last_clean":"2024-09-10T20:34:27.086784+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086784+0000","last_undegraded":"2024-09-10T20:34:27.086784+0000","last_fullsized":"2024-09-10T20:34:27.086784+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15479","reported_seq":15732,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:18.683589+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:36:18.683589+0000","last_peered":"2024-09-10T20:36:18.683589+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:36:18.683589+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22556672,"num_objects":5507,"num_object_clones":0,"num_object_copies":11014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12306,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1852}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14975","reported_seq":22097,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:15.373413+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:36:15.373413+0000","last_peered":"2024-09-10T20:36:15.373413+0000","last_clean":"2024-09-10T20:36:15.373413+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:36:15.373413+0000","last_undegraded":"2024-09-10T20:36:15.373413+0000","last_fullsized":"2024-09-10T20:36:15.373413+0000","mapping_epoch":46,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22696419,"num_objects":5542,"num_object_clones":0,"num_object_copies":11084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11582,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":362956225,"num_objects":88616,"num_object_clones":0,"num_object_copies":177232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88616,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":206801,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10693,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":904577024,"data_stored":904500233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160805,"ondisk_log_size":160805,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182493,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":717980,"kb_used_data":321224,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93653860,"statfs":{"total":96636764160,"available":95901552640,"internally_reserved":0,"allocated":328933376,"data_stored":328681083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,7,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986264,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612532,"kb_used_data":296220,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93759308,"statfs":{"total":96636764160,"available":96009531392,"internally_reserved":0,"allocated":303329280,"data_stored":303076992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323874474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659211,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":611436,"kb_used_data":268764,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93760404,"statfs":{"total":96636764160,"available":96010653696,"internally_reserved":0,"allocated":275214336,"data_stored":274900163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,2,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":328470528,"data_stored":328466915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":273907712,"data_stored":273842219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":302198784,"data_stored":302191099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:36:20.882 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:36:21.442 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:21.444 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:36:21.478 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":287,"stamp":"2024-09-10T20:36:20.979970+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":358847956,"num_objects":87615,"num_object_clones":0,"num_object_copies":175230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32318,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87615,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":207806,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10833,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160611,"ondisk_log_size":160611,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1926164,"kb_used_data":874576,"kb_used_omap":12,"kb_used_meta":1051507,"kb_avail":281189356,"statfs":{"total":289910292480,"available":287937900544,"internally_reserved":0,"allocated":895565824,"data_stored":894747070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1076743170},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,6,9,17,10,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24137728,"num_objects":-5893,"num_object_clones":0,"num_object_copies":-11786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-436,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5893,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":436,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004766"},"pg_stats":[{"pgid":"2.f","version":"47'15155","reported_seq":15371,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:18.636440+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:36:18.636440+0000","last_peered":"2024-09-10T20:36:18.636440+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:36:18.636440+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22540288,"num_objects":5503,"num_object_clones":0,"num_object_copies":11006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15155,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1962}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15466","reported_seq":15582,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:19.162327+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:36:19.162327+0000","last_peered":"2024-09-10T20:36:19.162327+0000","last_clean":"2024-09-10T20:36:19.162327+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:36:19.162327+0000","last_undegraded":"2024-09-10T20:36:19.162327+0000","last_fullsized":"2024-09-10T20:36:19.162327+0000","mapping_epoch":35,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22429696,"num_objects":5476,"num_object_clones":0,"num_object_copies":10952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13652,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15221","reported_seq":22573,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:20.434251+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:36:20.434251+0000","last_peered":"2024-09-10T20:36:20.434251+0000","last_clean":"2024-09-10T20:36:20.434251+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:36:20.434251+0000","last_undegraded":"2024-09-10T20:36:20.434251+0000","last_fullsized":"2024-09-10T20:36:20.434251+0000","mapping_epoch":44,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22503448,"num_objects":5495,"num_object_clones":0,"num_object_copies":10990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5495,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13349,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'15103","reported_seq":15188,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:19.158061+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:36:19.158061+0000","last_peered":"2024-09-10T20:36:19.158061+0000","last_clean":"2024-09-10T20:36:19.158061+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:36:19.158061+0000","last_undegraded":"2024-09-10T20:36:19.158061+0000","last_fullsized":"2024-09-10T20:36:19.158061+0000","mapping_epoch":35,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22532096,"num_objects":5501,"num_object_clones":0,"num_object_copies":11002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15103,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15475","reported_seq":15607,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:18.732143+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:36:18.732143+0000","last_peered":"2024-09-10T20:36:18.732143+0000","last_clean":"2024-09-10T20:36:18.732143+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:36:18.732143+0000","last_undegraded":"2024-09-10T20:36:18.732143+0000","last_fullsized":"2024-09-10T20:36:18.732143+0000","mapping_epoch":34,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22913024,"num_objects":5594,"num_object_clones":0,"num_object_copies":11188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5594,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12471,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15135","reported_seq":15383,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:18.616463+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:36:18.616463+0000","last_peered":"2024-09-10T20:36:18.616463+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:36:18.616463+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21856256,"num_objects":5336,"num_object_clones":0,"num_object_copies":10672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11999,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1775}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15103","reported_seq":15384,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:18.695447+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:36:18.695447+0000","last_peered":"2024-09-10T20:36:18.695447+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:36:18.695447+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22548480,"num_objects":5505,"num_object_clones":0,"num_object_copies":11010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11698,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2019}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15302","reported_seq":15432,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:18.697226+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:36:18.697226+0000","last_peered":"2024-09-10T20:36:18.697226+0000","last_clean":"2024-09-10T20:36:18.697226+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:36:18.697226+0000","last_undegraded":"2024-09-10T20:36:18.697226+0000","last_fullsized":"2024-09-10T20:36:18.697226+0000","mapping_epoch":34,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22257664,"num_objects":5434,"num_object_clones":0,"num_object_copies":10868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12373,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080479+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080479+0000","last_peered":"2024-09-10T20:34:27.080479+0000","last_clean":"2024-09-10T20:34:27.080479+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080479+0000","last_undegraded":"2024-09-10T20:34:27.080479+0000","last_fullsized":"2024-09-10T20:34:27.080479+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14967","reported_seq":15163,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:19.160779+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:36:19.160779+0000","last_peered":"2024-09-10T20:36:19.160779+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:36:19.160779+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22413312,"num_objects":5472,"num_object_clones":0,"num_object_copies":10944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12960,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1975}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086274+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086274+0000","last_peered":"2024-09-10T20:34:27.086274+0000","last_clean":"2024-09-10T20:34:27.086274+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086274+0000","last_undegraded":"2024-09-10T20:34:27.086274+0000","last_fullsized":"2024-09-10T20:34:27.086274+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15245","reported_seq":15457,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:18.616446+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:36:18.616446+0000","last_peered":"2024-09-10T20:36:18.616446+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:36:18.616446+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22467526,"num_objects":5487,"num_object_clones":0,"num_object_copies":10974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15243,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1921}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15106","reported_seq":15293,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:19.118129+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:36:19.118129+0000","last_peered":"2024-09-10T20:36:19.118129+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:36:19.118129+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22360064,"num_objects":5459,"num_object_clones":0,"num_object_copies":10918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5459,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12934,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1853}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079338+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079338+0000","last_peered":"2024-09-10T20:34:27.079338+0000","last_clean":"2024-09-10T20:34:27.079338+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079338+0000","last_undegraded":"2024-09-10T20:34:27.079338+0000","last_fullsized":"2024-09-10T20:34:27.079338+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15227","reported_seq":21958,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:19.136957+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:36:19.136957+0000","last_peered":"2024-09-10T20:36:19.136957+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:36:19.136957+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22269952,"num_objects":5437,"num_object_clones":0,"num_object_copies":10874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":314,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13013,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3298,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":5123}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15274","reported_seq":15526,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:18.558991+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:36:18.558991+0000","last_peered":"2024-09-10T20:36:18.558991+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:36:18.558991+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22757376,"num_objects":5556,"num_object_clones":0,"num_object_copies":11112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11997,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1964}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079988+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079988+0000","last_peered":"2024-09-10T20:34:27.079988+0000","last_clean":"2024-09-10T20:34:27.079988+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079988+0000","last_undegraded":"2024-09-10T20:34:27.079988+0000","last_fullsized":"2024-09-10T20:34:27.079988+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15117","reported_seq":15357,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:18.718277+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:36:18.718277+0000","last_peered":"2024-09-10T20:36:18.718277+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:36:18.718277+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22392832,"num_objects":5467,"num_object_clones":0,"num_object_copies":10934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11811,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1983}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086784+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086784+0000","last_peered":"2024-09-10T20:34:27.086784+0000","last_clean":"2024-09-10T20:34:27.086784+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086784+0000","last_undegraded":"2024-09-10T20:34:27.086784+0000","last_fullsized":"2024-09-10T20:34:27.086784+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15479","reported_seq":15732,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:18.683589+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:36:18.683589+0000","last_peered":"2024-09-10T20:36:18.683589+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:36:18.683589+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22556672,"num_objects":5507,"num_object_clones":0,"num_object_copies":11014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12306,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1852}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15133","reported_seq":22255,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:20.381021+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:36:20.381021+0000","last_peered":"2024-09-10T20:36:20.381021+0000","last_clean":"2024-09-10T20:36:20.381021+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:36:20.381021+0000","last_undegraded":"2024-09-10T20:36:20.381021+0000","last_fullsized":"2024-09-10T20:36:20.381021+0000","mapping_epoch":46,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22049251,"num_objects":5384,"num_object_clones":0,"num_object_copies":10768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11740,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":358847937,"num_objects":87613,"num_object_clones":0,"num_object_copies":175226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32318,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87613,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":207804,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10831,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":892657664,"data_stored":892580873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160608,"ondisk_log_size":160608,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182493,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":717980,"kb_used_data":321224,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93653860,"statfs":{"total":96636764160,"available":95901552640,"internally_reserved":0,"allocated":328933376,"data_stored":328681083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,7,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986265,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612008,"kb_used_data":291544,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93759832,"statfs":{"total":96636764160,"available":96010067968,"internally_reserved":0,"allocated":298541056,"data_stored":298288768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328134314},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,5,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659212,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":596176,"kb_used_data":261808,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93775664,"statfs":{"total":96636764160,"available":96026279936,"internally_reserved":0,"allocated":268091392,"data_stored":267777219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":342355630},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,1,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":328470528,"data_stored":328466915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":266727424,"data_stored":266661931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":297459712,"data_stored":297452027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:36:21.480 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:36:21.480 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'15155", 'reported_seq': 15371, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:18.636440+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:36:18.636440+0000', 'last_peered': '2024-09-10T20:36:18.636440+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:36:18.636440+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "43'5100", 'ondisk_log_start': "43'5100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22540288, 'num_objects': 5503, 'num_object_clones': 0, 'num_object_copies': 11006, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5503, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15155, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1962}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:21.480 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:36:21.480 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'15135", 'reported_seq': 15383, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:18.616463+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:36:18.616463+0000', 'last_peered': '2024-09-10T20:36:18.616463+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:36:18.616463+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "43'5100", 'ondisk_log_start': "43'5100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21856256, 'num_objects': 5336, 'num_object_clones': 0, 'num_object_copies': 10672, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5336, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11999, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1775}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:21.480 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:36:21.481 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'15103", 'reported_seq': 15384, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:18.695447+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:36:18.695447+0000', 'last_peered': '2024-09-10T20:36:18.695447+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:36:18.695447+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "43'5100", 'ondisk_log_start': "43'5100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22548480, 'num_objects': 5505, 'num_object_clones': 0, 'num_object_copies': 11010, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5505, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11698, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2019}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:21.481 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:36:21.481 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'14967", 'reported_seq': 15163, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:19.160779+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:36:19.160779+0000', 'last_peered': '2024-09-10T20:36:19.160779+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:36:19.160779+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "43'4900", 'ondisk_log_start': "43'4900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22413312, 'num_objects': 5472, 'num_object_clones': 0, 'num_object_copies': 10944, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5472, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12960, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1975}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:36:21.481 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:36:21.481 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'15245", 'reported_seq': 15457, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:18.616446+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:36:18.616446+0000', 'last_peered': '2024-09-10T20:36:18.616446+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:36:18.616446+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22467526, 'num_objects': 5487, 'num_object_clones': 0, 'num_object_copies': 10974, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5487, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15243, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1921}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:21.481 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:36:21.482 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'15106", 'reported_seq': 15293, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:19.118129+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:36:19.118129+0000', 'last_peered': '2024-09-10T20:36:19.118129+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:36:19.118129+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "43'5100", 'ondisk_log_start': "43'5100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22360064, 'num_objects': 5459, 'num_object_clones': 0, 'num_object_copies': 10918, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5459, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12934, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1853}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:36:21.482 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:36:21.482 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'15227", 'reported_seq': 21958, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:19.136957+0000', 'last_change': '2024-09-10T20:34:25.241937+0000', 'last_active': '2024-09-10T20:36:19.136957+0000', 'last_peered': '2024-09-10T20:36:19.136957+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:36:19.136957+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22269952, 'num_objects': 5437, 'num_object_clones': 0, 'num_object_copies': 10874, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 314, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5437, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13013, 'num_write_kb': 32484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3298, 'num_bytes_recovered': 8085504, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 5123}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:36:21.482 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:36:21.483 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'15274", 'reported_seq': 15526, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:18.558991+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:36:18.558991+0000', 'last_peered': '2024-09-10T20:36:18.558991+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:36:18.558991+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22757376, 'num_objects': 5556, 'num_object_clones': 0, 'num_object_copies': 11112, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5556, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11997, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1964}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:21.483 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:36:21.483 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'15117", 'reported_seq': 15357, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:18.718277+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:36:18.718277+0000', 'last_peered': '2024-09-10T20:36:18.718277+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:36:18.718277+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "43'5100", 'ondisk_log_start': "43'5100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22392832, 'num_objects': 5467, 'num_object_clones': 0, 'num_object_copies': 10934, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3487, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5467, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11811, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 48, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1983}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:21.483 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:36:21.483 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'15479", 'reported_seq': 15732, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:18.683589+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:36:18.683589+0000', 'last_peered': '2024-09-10T20:36:18.683589+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:36:18.683589+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "43'5400", 'ondisk_log_start': "43'5400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22556672, 'num_objects': 5507, 'num_object_clones': 0, 'num_object_copies': 11014, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5507, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12306, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1852}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:21.484 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:36:22.161 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:22.162 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32318/175230 objects degraded (18.443%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":493,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":87615,"data_bytes":358847956,"bytes_used":1972391936,"bytes_avail":287937900544,"bytes_total":289910292480,"degraded_objects":32318,"degraded_total":175230,"degraded_ratio":0.18443188951663531,"recovering_objects_per_sec":36,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":436,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":490},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:36:22.182 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:36:22.182 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:36:22.778 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:22.778 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:36:22.809 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":287,"stamp":"2024-09-10T20:36:20.979970+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":358847956,"num_objects":87615,"num_object_clones":0,"num_object_copies":175230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32318,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87615,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":207806,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10833,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160611,"ondisk_log_size":160611,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1926164,"kb_used_data":874576,"kb_used_omap":12,"kb_used_meta":1051507,"kb_avail":281189356,"statfs":{"total":289910292480,"available":287937900544,"internally_reserved":0,"allocated":895565824,"data_stored":894747070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1076743170},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,6,9,17,10,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24137728,"num_objects":-5893,"num_object_clones":0,"num_object_copies":-11786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-436,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5893,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":436,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004766"},"pg_stats":[{"pgid":"2.f","version":"47'15155","reported_seq":15371,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:18.636440+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:36:18.636440+0000","last_peered":"2024-09-10T20:36:18.636440+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:36:18.636440+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22540288,"num_objects":5503,"num_object_clones":0,"num_object_copies":11006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15155,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1962}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15466","reported_seq":15582,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:19.162327+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:36:19.162327+0000","last_peered":"2024-09-10T20:36:19.162327+0000","last_clean":"2024-09-10T20:36:19.162327+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:36:19.162327+0000","last_undegraded":"2024-09-10T20:36:19.162327+0000","last_fullsized":"2024-09-10T20:36:19.162327+0000","mapping_epoch":35,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22429696,"num_objects":5476,"num_object_clones":0,"num_object_copies":10952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13652,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15221","reported_seq":22573,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:20.434251+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:36:20.434251+0000","last_peered":"2024-09-10T20:36:20.434251+0000","last_clean":"2024-09-10T20:36:20.434251+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:36:20.434251+0000","last_undegraded":"2024-09-10T20:36:20.434251+0000","last_fullsized":"2024-09-10T20:36:20.434251+0000","mapping_epoch":44,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22503448,"num_objects":5495,"num_object_clones":0,"num_object_copies":10990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5495,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13349,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'15103","reported_seq":15188,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:19.158061+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:36:19.158061+0000","last_peered":"2024-09-10T20:36:19.158061+0000","last_clean":"2024-09-10T20:36:19.158061+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:36:19.158061+0000","last_undegraded":"2024-09-10T20:36:19.158061+0000","last_fullsized":"2024-09-10T20:36:19.158061+0000","mapping_epoch":35,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22532096,"num_objects":5501,"num_object_clones":0,"num_object_copies":11002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15103,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15475","reported_seq":15607,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:18.732143+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:36:18.732143+0000","last_peered":"2024-09-10T20:36:18.732143+0000","last_clean":"2024-09-10T20:36:18.732143+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:36:18.732143+0000","last_undegraded":"2024-09-10T20:36:18.732143+0000","last_fullsized":"2024-09-10T20:36:18.732143+0000","mapping_epoch":34,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22913024,"num_objects":5594,"num_object_clones":0,"num_object_copies":11188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5594,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12471,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15135","reported_seq":15383,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:18.616463+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:36:18.616463+0000","last_peered":"2024-09-10T20:36:18.616463+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:36:18.616463+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21856256,"num_objects":5336,"num_object_clones":0,"num_object_copies":10672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11999,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1775}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15103","reported_seq":15384,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:18.695447+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:36:18.695447+0000","last_peered":"2024-09-10T20:36:18.695447+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:36:18.695447+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22548480,"num_objects":5505,"num_object_clones":0,"num_object_copies":11010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11698,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2019}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15302","reported_seq":15432,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:18.697226+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:36:18.697226+0000","last_peered":"2024-09-10T20:36:18.697226+0000","last_clean":"2024-09-10T20:36:18.697226+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:36:18.697226+0000","last_undegraded":"2024-09-10T20:36:18.697226+0000","last_fullsized":"2024-09-10T20:36:18.697226+0000","mapping_epoch":34,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22257664,"num_objects":5434,"num_object_clones":0,"num_object_copies":10868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12373,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080479+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080479+0000","last_peered":"2024-09-10T20:34:27.080479+0000","last_clean":"2024-09-10T20:34:27.080479+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080479+0000","last_undegraded":"2024-09-10T20:34:27.080479+0000","last_fullsized":"2024-09-10T20:34:27.080479+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14967","reported_seq":15163,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:19.160779+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:36:19.160779+0000","last_peered":"2024-09-10T20:36:19.160779+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:36:19.160779+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22413312,"num_objects":5472,"num_object_clones":0,"num_object_copies":10944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12960,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1975}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086274+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086274+0000","last_peered":"2024-09-10T20:34:27.086274+0000","last_clean":"2024-09-10T20:34:27.086274+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086274+0000","last_undegraded":"2024-09-10T20:34:27.086274+0000","last_fullsized":"2024-09-10T20:34:27.086274+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15245","reported_seq":15457,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:18.616446+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:36:18.616446+0000","last_peered":"2024-09-10T20:36:18.616446+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:36:18.616446+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22467526,"num_objects":5487,"num_object_clones":0,"num_object_copies":10974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15243,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1921}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15106","reported_seq":15293,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:19.118129+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:36:19.118129+0000","last_peered":"2024-09-10T20:36:19.118129+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:36:19.118129+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22360064,"num_objects":5459,"num_object_clones":0,"num_object_copies":10918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5459,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12934,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1853}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079338+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079338+0000","last_peered":"2024-09-10T20:34:27.079338+0000","last_clean":"2024-09-10T20:34:27.079338+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079338+0000","last_undegraded":"2024-09-10T20:34:27.079338+0000","last_fullsized":"2024-09-10T20:34:27.079338+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15227","reported_seq":21958,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:19.136957+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:36:19.136957+0000","last_peered":"2024-09-10T20:36:19.136957+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:36:19.136957+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22269952,"num_objects":5437,"num_object_clones":0,"num_object_copies":10874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":314,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13013,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3298,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":5123}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15274","reported_seq":15526,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:18.558991+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:36:18.558991+0000","last_peered":"2024-09-10T20:36:18.558991+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:36:18.558991+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22757376,"num_objects":5556,"num_object_clones":0,"num_object_copies":11112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11997,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1964}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079988+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079988+0000","last_peered":"2024-09-10T20:34:27.079988+0000","last_clean":"2024-09-10T20:34:27.079988+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079988+0000","last_undegraded":"2024-09-10T20:34:27.079988+0000","last_fullsized":"2024-09-10T20:34:27.079988+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15117","reported_seq":15357,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:18.718277+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:36:18.718277+0000","last_peered":"2024-09-10T20:36:18.718277+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:36:18.718277+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22392832,"num_objects":5467,"num_object_clones":0,"num_object_copies":10934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11811,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1983}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086784+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086784+0000","last_peered":"2024-09-10T20:34:27.086784+0000","last_clean":"2024-09-10T20:34:27.086784+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086784+0000","last_undegraded":"2024-09-10T20:34:27.086784+0000","last_fullsized":"2024-09-10T20:34:27.086784+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15479","reported_seq":15732,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:18.683589+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:36:18.683589+0000","last_peered":"2024-09-10T20:36:18.683589+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:36:18.683589+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22556672,"num_objects":5507,"num_object_clones":0,"num_object_copies":11014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12306,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1852}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15133","reported_seq":22255,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:20.381021+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:36:20.381021+0000","last_peered":"2024-09-10T20:36:20.381021+0000","last_clean":"2024-09-10T20:36:20.381021+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:36:20.381021+0000","last_undegraded":"2024-09-10T20:36:20.381021+0000","last_fullsized":"2024-09-10T20:36:20.381021+0000","mapping_epoch":46,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22049251,"num_objects":5384,"num_object_clones":0,"num_object_copies":10768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11740,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":358847937,"num_objects":87613,"num_object_clones":0,"num_object_copies":175226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32318,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87613,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":207804,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10831,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":892657664,"data_stored":892580873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160608,"ondisk_log_size":160608,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182493,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":717980,"kb_used_data":321224,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93653860,"statfs":{"total":96636764160,"available":95901552640,"internally_reserved":0,"allocated":328933376,"data_stored":328681083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,7,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986265,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612008,"kb_used_data":291544,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93759832,"statfs":{"total":96636764160,"available":96010067968,"internally_reserved":0,"allocated":298541056,"data_stored":298288768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328134314},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,5,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659212,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":596176,"kb_used_data":261808,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93775664,"statfs":{"total":96636764160,"available":96026279936,"internally_reserved":0,"allocated":268091392,"data_stored":267777219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":342355630},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,1,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":328470528,"data_stored":328466915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":266727424,"data_stored":266661931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":297459712,"data_stored":297452027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:36:25.811 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:36:26.383 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:26.386 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:36:26.409 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-09-10T20:36:24.981513+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":350131668,"num_objects":85487,"num_object_clones":0,"num_object_copies":170974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85487,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":209934,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10995,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160739,"ondisk_log_size":160739,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1922532,"kb_used_data":862608,"kb_used_omap":12,"kb_used_meta":1059827,"kb_avail":281192988,"statfs":{"total":289910292480,"available":287941619712,"internally_reserved":0,"allocated":883310592,"data_stored":882491838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1085262850},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,5,5,14,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28311552,"num_objects":-6912,"num_object_clones":0,"num_object_copies":-13824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6912,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":458,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004639"},"pg_stats":[{"pgid":"2.f","version":"47'15310","reported_seq":15528,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:23.737244+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:36:23.737244+0000","last_peered":"2024-09-10T20:36:23.737244+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:36:23.737244+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21905408,"num_objects":5348,"num_object_clones":0,"num_object_copies":10696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15310,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1807}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15631","reported_seq":15747,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:24.152880+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:36:24.152880+0000","last_peered":"2024-09-10T20:36:24.152880+0000","last_clean":"2024-09-10T20:36:24.152880+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:36:24.152880+0000","last_undegraded":"2024-09-10T20:36:24.152880+0000","last_fullsized":"2024-09-10T20:36:24.152880+0000","mapping_epoch":35,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21753856,"num_objects":5311,"num_object_clones":0,"num_object_copies":10622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13817,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15221","reported_seq":22573,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:20.434251+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:36:20.434251+0000","last_peered":"2024-09-10T20:36:20.434251+0000","last_clean":"2024-09-10T20:36:20.434251+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:36:20.434251+0000","last_undegraded":"2024-09-10T20:36:20.434251+0000","last_fullsized":"2024-09-10T20:36:20.434251+0000","mapping_epoch":44,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22503448,"num_objects":5495,"num_object_clones":0,"num_object_copies":10990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5495,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13349,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'15253","reported_seq":15339,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:24.023805+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:36:24.023805+0000","last_peered":"2024-09-10T20:36:24.023805+0000","last_clean":"2024-09-10T20:36:24.023805+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:36:24.023805+0000","last_undegraded":"2024-09-10T20:36:24.023805+0000","last_fullsized":"2024-09-10T20:36:24.023805+0000","mapping_epoch":35,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21917696,"num_objects":5351,"num_object_clones":0,"num_object_copies":10702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15253,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15632","reported_seq":15762,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:23.723300+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:36:23.723300+0000","last_peered":"2024-09-10T20:36:23.723300+0000","last_clean":"2024-09-10T20:36:23.723300+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:36:23.723300+0000","last_undegraded":"2024-09-10T20:36:23.723300+0000","last_fullsized":"2024-09-10T20:36:23.723300+0000","mapping_epoch":34,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22269952,"num_objects":5437,"num_object_clones":0,"num_object_copies":10874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12628,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15294","reported_seq":15541,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:23.731423+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:36:23.731423+0000","last_peered":"2024-09-10T20:36:23.731423+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:36:23.731423+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21204992,"num_objects":5177,"num_object_clones":0,"num_object_copies":10354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12158,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1616}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15236","reported_seq":15514,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:23.694263+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:36:23.694263+0000","last_peered":"2024-09-10T20:36:23.694263+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:36:23.694263+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22003712,"num_objects":5372,"num_object_clones":0,"num_object_copies":10744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11831,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1886}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15441","reported_seq":15571,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:23.743017+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:36:23.743017+0000","last_peered":"2024-09-10T20:36:23.743017+0000","last_clean":"2024-09-10T20:36:23.743017+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:36:23.743017+0000","last_undegraded":"2024-09-10T20:36:23.743017+0000","last_fullsized":"2024-09-10T20:36:23.743017+0000","mapping_epoch":34,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21688320,"num_objects":5295,"num_object_clones":0,"num_object_copies":10590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12512,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080479+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080479+0000","last_peered":"2024-09-10T20:34:27.080479+0000","last_clean":"2024-09-10T20:34:27.080479+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080479+0000","last_undegraded":"2024-09-10T20:34:27.080479+0000","last_fullsized":"2024-09-10T20:34:27.080479+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15104","reported_seq":15300,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:24.184981+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:36:24.184981+0000","last_peered":"2024-09-10T20:36:24.184981+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:36:24.184981+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21852160,"num_objects":5335,"num_object_clones":0,"num_object_copies":10670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13097,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1838}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086274+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086274+0000","last_peered":"2024-09-10T20:34:27.086274+0000","last_clean":"2024-09-10T20:34:27.086274+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086274+0000","last_undegraded":"2024-09-10T20:34:27.086274+0000","last_fullsized":"2024-09-10T20:34:27.086274+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15397","reported_seq":15608,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:23.766564+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:36:23.766564+0000","last_peered":"2024-09-10T20:36:23.766564+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:36:23.766564+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21844934,"num_objects":5335,"num_object_clones":0,"num_object_copies":10670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15395,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1769}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15252","reported_seq":15440,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:24.098457+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:36:24.098457+0000","last_peered":"2024-09-10T20:36:24.098457+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:36:24.098457+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21762048,"num_objects":5313,"num_object_clones":0,"num_object_copies":10626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13080,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1707}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079338+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079338+0000","last_peered":"2024-09-10T20:34:27.079338+0000","last_clean":"2024-09-10T20:34:27.079338+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079338+0000","last_undegraded":"2024-09-10T20:34:27.079338+0000","last_fullsized":"2024-09-10T20:34:27.079338+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15391","reported_seq":22446,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:24.186759+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:36:24.186759+0000","last_peered":"2024-09-10T20:36:24.186759+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:36:24.186759+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21598208,"num_objects":5273,"num_object_clones":0,"num_object_copies":10546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":152,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13177,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3460,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":5121}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15453","reported_seq":15701,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:23.672890+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:36:23.672890+0000","last_peered":"2024-09-10T20:36:23.672890+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:36:23.672890+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22024192,"num_objects":5377,"num_object_clones":0,"num_object_copies":10754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12176,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1785}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079988+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079988+0000","last_peered":"2024-09-10T20:34:27.079988+0000","last_clean":"2024-09-10T20:34:27.079988+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079988+0000","last_undegraded":"2024-09-10T20:34:27.079988+0000","last_fullsized":"2024-09-10T20:34:27.079988+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15259","reported_seq":15499,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:23.678907+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:36:23.678907+0000","last_peered":"2024-09-10T20:36:23.678907+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:36:23.678907+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21811200,"num_objects":5325,"num_object_clones":0,"num_object_copies":10650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11953,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1841}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086784+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086784+0000","last_peered":"2024-09-10T20:34:27.086784+0000","last_clean":"2024-09-10T20:34:27.086784+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086784+0000","last_undegraded":"2024-09-10T20:34:27.086784+0000","last_fullsized":"2024-09-10T20:34:27.086784+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15629","reported_seq":15882,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:23.743827+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:36:23.743827+0000","last_peered":"2024-09-10T20:36:23.743827+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:36:23.743827+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21942272,"num_objects":5357,"num_object_clones":0,"num_object_copies":10714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12456,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1702}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15133","reported_seq":22255,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:20.381021+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:36:20.381021+0000","last_peered":"2024-09-10T20:36:20.381021+0000","last_clean":"2024-09-10T20:36:20.381021+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:36:20.381021+0000","last_undegraded":"2024-09-10T20:36:20.381021+0000","last_fullsized":"2024-09-10T20:36:20.381021+0000","mapping_epoch":46,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22049251,"num_objects":5384,"num_object_clones":0,"num_object_copies":10768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11740,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":350131649,"num_objects":85485,"num_object_clones":0,"num_object_copies":170970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85485,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":209932,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10993,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":880205824,"data_stored":880129033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160736,"ondisk_log_size":160736,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182494,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":714952,"kb_used_data":314024,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93656888,"statfs":{"total":96636764160,"available":95904653312,"internally_reserved":0,"allocated":321560576,"data_stored":321308283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":410513066},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,0,8,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986266,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":611404,"kb_used_data":286776,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93760436,"statfs":{"total":96636764160,"available":96010686464,"internally_reserved":0,"allocated":293658624,"data_stored":293406336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332394154},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659212,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":596176,"kb_used_data":261808,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93775664,"statfs":{"total":96636764160,"available":96026279936,"internally_reserved":0,"allocated":268091392,"data_stored":267777219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":342355630},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,1,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":320991232,"data_stored":320987619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":266727424,"data_stored":266661931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":292487168,"data_stored":292479483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:36:26.410 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:36:26.973 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:26.975 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:36:26.993 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-09-10T20:36:24.981513+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":350131668,"num_objects":85487,"num_object_clones":0,"num_object_copies":170974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85487,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":209934,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10995,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160739,"ondisk_log_size":160739,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1922532,"kb_used_data":862608,"kb_used_omap":12,"kb_used_meta":1059827,"kb_avail":281192988,"statfs":{"total":289910292480,"available":287941619712,"internally_reserved":0,"allocated":883310592,"data_stored":882491838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1085262850},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,5,5,14,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28311552,"num_objects":-6912,"num_object_clones":0,"num_object_copies":-13824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6912,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":458,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004639"},"pg_stats":[{"pgid":"2.f","version":"47'15310","reported_seq":15528,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:23.737244+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:36:23.737244+0000","last_peered":"2024-09-10T20:36:23.737244+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:36:23.737244+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21905408,"num_objects":5348,"num_object_clones":0,"num_object_copies":10696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15310,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1807}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15631","reported_seq":15747,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:24.152880+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:36:24.152880+0000","last_peered":"2024-09-10T20:36:24.152880+0000","last_clean":"2024-09-10T20:36:24.152880+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:36:24.152880+0000","last_undegraded":"2024-09-10T20:36:24.152880+0000","last_fullsized":"2024-09-10T20:36:24.152880+0000","mapping_epoch":35,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21753856,"num_objects":5311,"num_object_clones":0,"num_object_copies":10622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13817,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15221","reported_seq":22573,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:20.434251+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:36:20.434251+0000","last_peered":"2024-09-10T20:36:20.434251+0000","last_clean":"2024-09-10T20:36:20.434251+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:36:20.434251+0000","last_undegraded":"2024-09-10T20:36:20.434251+0000","last_fullsized":"2024-09-10T20:36:20.434251+0000","mapping_epoch":44,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22503448,"num_objects":5495,"num_object_clones":0,"num_object_copies":10990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5495,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13349,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'15253","reported_seq":15339,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:24.023805+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:36:24.023805+0000","last_peered":"2024-09-10T20:36:24.023805+0000","last_clean":"2024-09-10T20:36:24.023805+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:36:24.023805+0000","last_undegraded":"2024-09-10T20:36:24.023805+0000","last_fullsized":"2024-09-10T20:36:24.023805+0000","mapping_epoch":35,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21917696,"num_objects":5351,"num_object_clones":0,"num_object_copies":10702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15253,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15632","reported_seq":15762,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:23.723300+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:36:23.723300+0000","last_peered":"2024-09-10T20:36:23.723300+0000","last_clean":"2024-09-10T20:36:23.723300+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:36:23.723300+0000","last_undegraded":"2024-09-10T20:36:23.723300+0000","last_fullsized":"2024-09-10T20:36:23.723300+0000","mapping_epoch":34,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22269952,"num_objects":5437,"num_object_clones":0,"num_object_copies":10874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12628,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15294","reported_seq":15541,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:23.731423+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:36:23.731423+0000","last_peered":"2024-09-10T20:36:23.731423+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:36:23.731423+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21204992,"num_objects":5177,"num_object_clones":0,"num_object_copies":10354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12158,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1616}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15236","reported_seq":15514,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:23.694263+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:36:23.694263+0000","last_peered":"2024-09-10T20:36:23.694263+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:36:23.694263+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22003712,"num_objects":5372,"num_object_clones":0,"num_object_copies":10744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11831,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1886}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15441","reported_seq":15571,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:23.743017+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:36:23.743017+0000","last_peered":"2024-09-10T20:36:23.743017+0000","last_clean":"2024-09-10T20:36:23.743017+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:36:23.743017+0000","last_undegraded":"2024-09-10T20:36:23.743017+0000","last_fullsized":"2024-09-10T20:36:23.743017+0000","mapping_epoch":34,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21688320,"num_objects":5295,"num_object_clones":0,"num_object_copies":10590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12512,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080479+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080479+0000","last_peered":"2024-09-10T20:34:27.080479+0000","last_clean":"2024-09-10T20:34:27.080479+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080479+0000","last_undegraded":"2024-09-10T20:34:27.080479+0000","last_fullsized":"2024-09-10T20:34:27.080479+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15104","reported_seq":15300,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:24.184981+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:36:24.184981+0000","last_peered":"2024-09-10T20:36:24.184981+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:36:24.184981+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21852160,"num_objects":5335,"num_object_clones":0,"num_object_copies":10670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13097,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1838}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086274+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086274+0000","last_peered":"2024-09-10T20:34:27.086274+0000","last_clean":"2024-09-10T20:34:27.086274+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086274+0000","last_undegraded":"2024-09-10T20:34:27.086274+0000","last_fullsized":"2024-09-10T20:34:27.086274+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15397","reported_seq":15608,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:23.766564+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:36:23.766564+0000","last_peered":"2024-09-10T20:36:23.766564+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:36:23.766564+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21844934,"num_objects":5335,"num_object_clones":0,"num_object_copies":10670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15395,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1769}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15252","reported_seq":15440,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:24.098457+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:36:24.098457+0000","last_peered":"2024-09-10T20:36:24.098457+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:36:24.098457+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21762048,"num_objects":5313,"num_object_clones":0,"num_object_copies":10626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13080,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1707}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079338+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079338+0000","last_peered":"2024-09-10T20:34:27.079338+0000","last_clean":"2024-09-10T20:34:27.079338+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079338+0000","last_undegraded":"2024-09-10T20:34:27.079338+0000","last_fullsized":"2024-09-10T20:34:27.079338+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15391","reported_seq":22446,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:24.186759+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:36:24.186759+0000","last_peered":"2024-09-10T20:36:24.186759+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:36:24.186759+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21598208,"num_objects":5273,"num_object_clones":0,"num_object_copies":10546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":152,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13177,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3460,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":5121}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15453","reported_seq":15701,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:23.672890+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:36:23.672890+0000","last_peered":"2024-09-10T20:36:23.672890+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:36:23.672890+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22024192,"num_objects":5377,"num_object_clones":0,"num_object_copies":10754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12176,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1785}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079988+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079988+0000","last_peered":"2024-09-10T20:34:27.079988+0000","last_clean":"2024-09-10T20:34:27.079988+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079988+0000","last_undegraded":"2024-09-10T20:34:27.079988+0000","last_fullsized":"2024-09-10T20:34:27.079988+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15259","reported_seq":15499,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:23.678907+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:36:23.678907+0000","last_peered":"2024-09-10T20:36:23.678907+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:36:23.678907+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21811200,"num_objects":5325,"num_object_clones":0,"num_object_copies":10650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11953,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1841}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086784+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086784+0000","last_peered":"2024-09-10T20:34:27.086784+0000","last_clean":"2024-09-10T20:34:27.086784+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086784+0000","last_undegraded":"2024-09-10T20:34:27.086784+0000","last_fullsized":"2024-09-10T20:34:27.086784+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15629","reported_seq":15882,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:23.743827+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:36:23.743827+0000","last_peered":"2024-09-10T20:36:23.743827+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:36:23.743827+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21942272,"num_objects":5357,"num_object_clones":0,"num_object_copies":10714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12456,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1702}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080340+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080340+0000","last_peered":"2024-09-10T20:34:27.080340+0000","last_clean":"2024-09-10T20:34:27.080340+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080340+0000","last_undegraded":"2024-09-10T20:34:27.080340+0000","last_fullsized":"2024-09-10T20:34:27.080340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15133","reported_seq":22255,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:20.381021+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:36:20.381021+0000","last_peered":"2024-09-10T20:36:20.381021+0000","last_clean":"2024-09-10T20:36:20.381021+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:36:20.381021+0000","last_undegraded":"2024-09-10T20:36:20.381021+0000","last_fullsized":"2024-09-10T20:36:20.381021+0000","mapping_epoch":46,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22049251,"num_objects":5384,"num_object_clones":0,"num_object_copies":10768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11740,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":350131649,"num_objects":85485,"num_object_clones":0,"num_object_copies":170970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85485,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":209932,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10993,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":880205824,"data_stored":880129033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160736,"ondisk_log_size":160736,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182494,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":714952,"kb_used_data":314024,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93656888,"statfs":{"total":96636764160,"available":95904653312,"internally_reserved":0,"allocated":321560576,"data_stored":321308283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":410513066},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,0,8,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986266,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":611404,"kb_used_data":286776,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93760436,"statfs":{"total":96636764160,"available":96010686464,"internally_reserved":0,"allocated":293658624,"data_stored":293406336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332394154},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659212,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":596176,"kb_used_data":261808,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93775664,"statfs":{"total":96636764160,"available":96026279936,"internally_reserved":0,"allocated":268091392,"data_stored":267777219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":342355630},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,1,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":320991232,"data_stored":320987619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":266727424,"data_stored":266661931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":292487168,"data_stored":292479483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:36:26.994 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:36:26.995 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'15310", 'reported_seq': 15528, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:23.737244+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:36:23.737244+0000', 'last_peered': '2024-09-10T20:36:23.737244+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:36:23.737244+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "43'5300", 'ondisk_log_start': "43'5300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21905408, 'num_objects': 5348, 'num_object_clones': 0, 'num_object_copies': 10696, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5348, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15310, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1807}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:26.995 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:36:26.995 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'15294", 'reported_seq': 15541, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:23.731423+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:36:23.731423+0000', 'last_peered': '2024-09-10T20:36:23.731423+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:36:23.731423+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21204992, 'num_objects': 5177, 'num_object_clones': 0, 'num_object_copies': 10354, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5177, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12158, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1616}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:26.995 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:36:26.995 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'15236", 'reported_seq': 15514, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:23.694263+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:36:23.694263+0000', 'last_peered': '2024-09-10T20:36:23.694263+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:36:23.694263+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22003712, 'num_objects': 5372, 'num_object_clones': 0, 'num_object_copies': 10744, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5372, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11831, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1886}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:26.995 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:36:26.995 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'15104", 'reported_seq': 15300, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:24.184981+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:36:24.184981+0000', 'last_peered': '2024-09-10T20:36:24.184981+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:36:24.184981+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "43'5100", 'ondisk_log_start': "43'5100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21852160, 'num_objects': 5335, 'num_object_clones': 0, 'num_object_copies': 10670, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5335, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13097, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1838}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:36:26.996 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:36:26.996 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'15397", 'reported_seq': 15608, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:23.766564+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:36:23.766564+0000', 'last_peered': '2024-09-10T20:36:23.766564+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:36:23.766564+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "43'5300", 'ondisk_log_start': "43'5300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21844934, 'num_objects': 5335, 'num_object_clones': 0, 'num_object_copies': 10670, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5335, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15395, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1769}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:26.996 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:36:26.996 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'15252", 'reported_seq': 15440, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:24.098457+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:36:24.098457+0000', 'last_peered': '2024-09-10T20:36:24.098457+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:36:24.098457+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21762048, 'num_objects': 5313, 'num_object_clones': 0, 'num_object_copies': 10626, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5313, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13080, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1707}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:36:26.996 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:36:26.996 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'15391", 'reported_seq': 22446, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:24.186759+0000', 'last_change': '2024-09-10T20:34:25.241937+0000', 'last_active': '2024-09-10T20:36:24.186759+0000', 'last_peered': '2024-09-10T20:36:24.186759+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:36:24.186759+0000', 'last_undegraded': '2024-09-10T20:30:05.347418+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "43'5300", 'ondisk_log_start': "43'5300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21598208, 'num_objects': 5273, 'num_object_clones': 0, 'num_object_copies': 10546, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 152, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5273, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13177, 'num_write_kb': 32484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3460, 'num_bytes_recovered': 8085504, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 5121}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:36:26.996 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:36:26.997 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'15453", 'reported_seq': 15701, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:23.672890+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:36:23.672890+0000', 'last_peered': '2024-09-10T20:36:23.672890+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:36:23.672890+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "43'5400", 'ondisk_log_start': "43'5400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22024192, 'num_objects': 5377, 'num_object_clones': 0, 'num_object_copies': 10754, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5377, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12176, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1785}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:26.997 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:36:26.997 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'15259", 'reported_seq': 15499, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:23.678907+0000', 'last_change': '2024-09-10T20:30:05.709270+0000', 'last_active': '2024-09-10T20:36:23.678907+0000', 'last_peered': '2024-09-10T20:36:23.678907+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:36:23.678907+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21811200, 'num_objects': 5325, 'num_object_clones': 0, 'num_object_copies': 10650, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3487, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5325, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11953, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 48, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1841}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:26.997 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:36:26.997 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'15629", 'reported_seq': 15882, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:23.743827+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:36:23.743827+0000', 'last_peered': '2024-09-10T20:36:23.743827+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:36:23.743827+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "43'5600", 'ondisk_log_start': "43'5600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21942272, 'num_objects': 5357, 'num_object_clones': 0, 'num_object_copies': 10714, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5357, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12456, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1702}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:26.997 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:36:27.669 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:27.670 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32156/170340 objects degraded (18.878%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":499,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":85170,"data_bytes":348833236,"bytes_used":1965498368,"bytes_avail":287944794112,"bytes_total":289910292480,"degraded_objects":32156,"degraded_total":170340,"degraded_ratio":0.1887753903956792,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":423},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:36:27.695 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:36:27.695 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:36:28.244 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:28.245 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:36:28.269 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2024-09-10T20:36:26.982132+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":348833236,"num_objects":85170,"num_object_clones":0,"num_object_copies":170340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85170,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":210251,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10995,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160856,"ondisk_log_size":160856,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1919432,"kb_used_data":855360,"kb_used_omap":12,"kb_used_meta":1063987,"kb_avail":281196088,"statfs":{"total":289910292480,"available":287944794112,"internally_reserved":0,"allocated":875888640,"data_stored":875069886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1089522690},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,4,9,12,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20824064,"num_objects":-5084,"num_object_clones":0,"num_object_copies":-10168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-300,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5084,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004466"},"pg_stats":[{"pgid":"2.f","version":"47'15310","reported_seq":15528,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:23.737244+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:36:23.737244+0000","last_peered":"2024-09-10T20:36:23.737244+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:36:23.737244+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21905408,"num_objects":5348,"num_object_clones":0,"num_object_copies":10696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15310,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1807}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15631","reported_seq":15747,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:24.152880+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:36:24.152880+0000","last_peered":"2024-09-10T20:36:24.152880+0000","last_clean":"2024-09-10T20:36:24.152880+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:36:24.152880+0000","last_undegraded":"2024-09-10T20:36:24.152880+0000","last_fullsized":"2024-09-10T20:36:24.152880+0000","mapping_epoch":35,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21753856,"num_objects":5311,"num_object_clones":0,"num_object_copies":10622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13817,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15393","reported_seq":22746,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:25.512526+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:36:25.512526+0000","last_peered":"2024-09-10T20:36:25.512526+0000","last_clean":"2024-09-10T20:36:25.512526+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:36:25.512526+0000","last_undegraded":"2024-09-10T20:36:25.512526+0000","last_fullsized":"2024-09-10T20:36:25.512526+0000","mapping_epoch":44,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21798936,"num_objects":5323,"num_object_clones":0,"num_object_copies":10646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5323,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13521,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'15253","reported_seq":15339,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:24.023805+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:36:24.023805+0000","last_peered":"2024-09-10T20:36:24.023805+0000","last_clean":"2024-09-10T20:36:24.023805+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:36:24.023805+0000","last_undegraded":"2024-09-10T20:36:24.023805+0000","last_fullsized":"2024-09-10T20:36:24.023805+0000","mapping_epoch":35,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21917696,"num_objects":5351,"num_object_clones":0,"num_object_copies":10702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15253,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15632","reported_seq":15762,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:23.723300+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:36:23.723300+0000","last_peered":"2024-09-10T20:36:23.723300+0000","last_clean":"2024-09-10T20:36:23.723300+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:36:23.723300+0000","last_undegraded":"2024-09-10T20:36:23.723300+0000","last_fullsized":"2024-09-10T20:36:23.723300+0000","mapping_epoch":34,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22269952,"num_objects":5437,"num_object_clones":0,"num_object_copies":10874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12628,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15294","reported_seq":15541,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:23.731423+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:36:23.731423+0000","last_peered":"2024-09-10T20:36:23.731423+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:36:23.731423+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21204992,"num_objects":5177,"num_object_clones":0,"num_object_copies":10354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12158,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1616}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15236","reported_seq":15514,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:23.694263+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:36:23.694263+0000","last_peered":"2024-09-10T20:36:23.694263+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:36:23.694263+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22003712,"num_objects":5372,"num_object_clones":0,"num_object_copies":10744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11831,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1886}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15441","reported_seq":15571,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:23.743017+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:36:23.743017+0000","last_peered":"2024-09-10T20:36:23.743017+0000","last_clean":"2024-09-10T20:36:23.743017+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:36:23.743017+0000","last_undegraded":"2024-09-10T20:36:23.743017+0000","last_fullsized":"2024-09-10T20:36:23.743017+0000","mapping_epoch":34,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21688320,"num_objects":5295,"num_object_clones":0,"num_object_copies":10590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12512,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080479+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:34:27.080479+0000","last_peered":"2024-09-10T20:34:27.080479+0000","last_clean":"2024-09-10T20:34:27.080479+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:34:27.080479+0000","last_undegraded":"2024-09-10T20:34:27.080479+0000","last_fullsized":"2024-09-10T20:34:27.080479+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15104","reported_seq":15300,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:24.184981+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:36:24.184981+0000","last_peered":"2024-09-10T20:36:24.184981+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:36:24.184981+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21852160,"num_objects":5335,"num_object_clones":0,"num_object_copies":10670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13097,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1838}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086274+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086274+0000","last_peered":"2024-09-10T20:34:27.086274+0000","last_clean":"2024-09-10T20:34:27.086274+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086274+0000","last_undegraded":"2024-09-10T20:34:27.086274+0000","last_fullsized":"2024-09-10T20:34:27.086274+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15397","reported_seq":15608,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:23.766564+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:36:23.766564+0000","last_peered":"2024-09-10T20:36:23.766564+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:36:23.766564+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21844934,"num_objects":5335,"num_object_clones":0,"num_object_copies":10670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15395,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1769}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15252","reported_seq":15440,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:24.098457+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:36:24.098457+0000","last_peered":"2024-09-10T20:36:24.098457+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:36:24.098457+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21762048,"num_objects":5313,"num_object_clones":0,"num_object_copies":10626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13080,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1707}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079338+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:34:27.079338+0000","last_peered":"2024-09-10T20:34:27.079338+0000","last_clean":"2024-09-10T20:34:27.079338+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:34:27.079338+0000","last_undegraded":"2024-09-10T20:34:27.079338+0000","last_fullsized":"2024-09-10T20:34:27.079338+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15391","reported_seq":22446,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:24.186759+0000","last_change":"2024-09-10T20:34:25.241937+0000","last_active":"2024-09-10T20:36:24.186759+0000","last_peered":"2024-09-10T20:36:24.186759+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:36:24.186759+0000","last_undegraded":"2024-09-10T20:30:05.347418+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21598208,"num_objects":5273,"num_object_clones":0,"num_object_copies":10546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":152,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13177,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3460,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":5121}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15453","reported_seq":15701,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:23.672890+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:36:23.672890+0000","last_peered":"2024-09-10T20:36:23.672890+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:36:23.672890+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22024192,"num_objects":5377,"num_object_clones":0,"num_object_copies":10754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12176,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1785}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.079988+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:34:27.079988+0000","last_peered":"2024-09-10T20:34:27.079988+0000","last_clean":"2024-09-10T20:34:27.079988+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:34:27.079988+0000","last_undegraded":"2024-09-10T20:34:27.079988+0000","last_fullsized":"2024-09-10T20:34:27.079988+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15259","reported_seq":15499,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:23.678907+0000","last_change":"2024-09-10T20:30:05.709270+0000","last_active":"2024-09-10T20:36:23.678907+0000","last_peered":"2024-09-10T20:36:23.678907+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:36:23.678907+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21811200,"num_objects":5325,"num_object_clones":0,"num_object_copies":10650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11953,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1841}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086784+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086784+0000","last_peered":"2024-09-10T20:34:27.086784+0000","last_clean":"2024-09-10T20:34:27.086784+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086784+0000","last_undegraded":"2024-09-10T20:34:27.086784+0000","last_fullsized":"2024-09-10T20:34:27.086784+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15629","reported_seq":15882,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:23.743827+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:36:23.743827+0000","last_peered":"2024-09-10T20:36:23.743827+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:36:23.743827+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21942272,"num_objects":5357,"num_object_clones":0,"num_object_copies":10714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12456,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1702}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080384+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:34:27.080384+0000","last_peered":"2024-09-10T20:34:27.080384+0000","last_clean":"2024-09-10T20:34:27.080384+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:34:27.080384+0000","last_undegraded":"2024-09-10T20:34:27.080384+0000","last_fullsized":"2024-09-10T20:34:27.080384+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.080059+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:34:27.080059+0000","last_peered":"2024-09-10T20:34:27.080059+0000","last_clean":"2024-09-10T20:34:27.080059+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:34:27.080059+0000","last_undegraded":"2024-09-10T20:34:27.080059+0000","last_fullsized":"2024-09-10T20:34:27.080059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15278","reported_seq":22399,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:25.510891+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:36:25.510891+0000","last_peered":"2024-09-10T20:36:25.510891+0000","last_clean":"2024-09-10T20:36:25.510891+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:36:25.510891+0000","last_undegraded":"2024-09-10T20:36:25.510891+0000","last_fullsized":"2024-09-10T20:36:25.510891+0000","mapping_epoch":46,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21455331,"num_objects":5239,"num_object_clones":0,"num_object_copies":10478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11885,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":348833217,"num_objects":85168,"num_object_clones":0,"num_object_copies":170336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85168,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":210249,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10993,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":872550400,"data_stored":872473609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160853,"ondisk_log_size":160853,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182494,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":714952,"kb_used_data":314024,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93656888,"statfs":{"total":96636764160,"available":95904653312,"internally_reserved":0,"allocated":321560576,"data_stored":321308283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":410513066},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,0,8,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986266,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":611404,"kb_used_data":286776,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93760436,"statfs":{"total":96636764160,"available":96010686464,"internally_reserved":0,"allocated":293658624,"data_stored":293406336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332394154},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659213,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":593076,"kb_used_data":254560,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93778764,"statfs":{"total":96636764160,"available":96029454336,"internally_reserved":0,"allocated":260669440,"data_stored":260355267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":346615470},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,5,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":320991232,"data_stored":320987619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":259072000,"data_stored":259006507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":292487168,"data_stored":292479483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:36:31.272 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:36:31.825 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:31.828 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:36:31.852 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-09-10T20:36:30.983603+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":338998740,"num_objects":82769,"num_object_clones":0,"num_object_copies":165538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32001,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82769,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":212652,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11148,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160857,"ondisk_log_size":160857,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1905656,"kb_used_data":837436,"kb_used_omap":12,"kb_used_meta":1068147,"kb_avail":281209864,"statfs":{"total":289910292480,"available":287958900736,"internally_reserved":0,"allocated":857534464,"data_stored":856721732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1093782530},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,4,3,13,8,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19849216,"num_objects":-4846,"num_object_clones":0,"num_object_copies":-9692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-317,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4846,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002856"},"pg_stats":[{"pgid":"2.f","version":"47'15474","reported_seq":15692,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:28.816687+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:36:28.816687+0000","last_peered":"2024-09-10T20:36:28.816687+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:36:28.816687+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21233664,"num_objects":5184,"num_object_clones":0,"num_object_copies":10368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15474,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1643}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15755","reported_seq":15871,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:29.033703+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:36:29.033703+0000","last_peered":"2024-09-10T20:36:29.033703+0000","last_clean":"2024-09-10T20:36:29.033703+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:36:29.033703+0000","last_undegraded":"2024-09-10T20:36:29.033703+0000","last_fullsized":"2024-09-10T20:36:29.033703+0000","mapping_epoch":35,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21245952,"num_objects":5187,"num_object_clones":0,"num_object_copies":10374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13941,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'15542","reported_seq":22899,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.538925+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:36:30.538925+0000","last_peered":"2024-09-10T20:36:30.538925+0000","last_clean":"2024-09-10T20:36:30.538925+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:36:30.538925+0000","last_undegraded":"2024-09-10T20:36:30.538925+0000","last_fullsized":"2024-09-10T20:36:30.538925+0000","mapping_epoch":44,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21188632,"num_objects":5174,"num_object_clones":0,"num_object_copies":10348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5174,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13670,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'15394","reported_seq":15480,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:29.116351+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:36:29.116351+0000","last_peered":"2024-09-10T20:36:29.116351+0000","last_clean":"2024-09-10T20:36:29.116351+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:36:29.116351+0000","last_undegraded":"2024-09-10T20:36:29.116351+0000","last_fullsized":"2024-09-10T20:36:29.116351+0000","mapping_epoch":35,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21340160,"num_objects":5210,"num_object_clones":0,"num_object_copies":10420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15394,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15773","reported_seq":15905,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:28.812332+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:36:28.812332+0000","last_peered":"2024-09-10T20:36:28.812332+0000","last_clean":"2024-09-10T20:36:28.812332+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:36:28.812332+0000","last_undegraded":"2024-09-10T20:36:28.812332+0000","last_fullsized":"2024-09-10T20:36:28.812332+0000","mapping_epoch":34,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21692416,"num_objects":5296,"num_object_clones":0,"num_object_copies":10592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12769,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15431","reported_seq":15679,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:28.624148+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:36:28.624148+0000","last_peered":"2024-09-10T20:36:28.624148+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:36:28.624148+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20643840,"num_objects":5040,"num_object_clones":0,"num_object_copies":10080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12295,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1479}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15394","reported_seq":15675,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:28.821063+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:36:28.821063+0000","last_peered":"2024-09-10T20:36:28.821063+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:36:28.821063+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21356544,"num_objects":5214,"num_object_clones":0,"num_object_copies":10428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11989,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1728}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15600","reported_seq":15730,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:28.758421+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:36:28.758421+0000","last_peered":"2024-09-10T20:36:28.758421+0000","last_clean":"2024-09-10T20:36:28.758421+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:36:28.758421+0000","last_undegraded":"2024-09-10T20:36:28.758421+0000","last_fullsized":"2024-09-10T20:36:28.758421+0000","mapping_epoch":34,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21037056,"num_objects":5136,"num_object_clones":0,"num_object_copies":10272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12671,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15234","reported_seq":15430,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:29.157150+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:36:29.157150+0000","last_peered":"2024-09-10T20:36:29.157150+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:36:29.157150+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21319680,"num_objects":5205,"num_object_clones":0,"num_object_copies":10410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13227,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1708}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086274+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086274+0000","last_peered":"2024-09-10T20:34:27.086274+0000","last_clean":"2024-09-10T20:34:27.086274+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086274+0000","last_undegraded":"2024-09-10T20:34:27.086274+0000","last_fullsized":"2024-09-10T20:34:27.086274+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15561","reported_seq":15768,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:28.829615+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:36:28.829615+0000","last_peered":"2024-09-10T20:36:28.829615+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:36:28.829615+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21173190,"num_objects":5171,"num_object_clones":0,"num_object_copies":10342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15559,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1605}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15420","reported_seq":15608,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:29.158159+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:36:29.158159+0000","last_peered":"2024-09-10T20:36:29.158159+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:36:29.158159+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21073920,"num_objects":5145,"num_object_clones":0,"num_object_copies":10290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13248,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1539}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15548","reported_seq":22908,"reported_epoch":47,"state":"active+undersized+remapped","last_fresh":"2024-09-10T20:36:29.147428+0000","last_change":"2024-09-10T20:36:28.760528+0000","last_active":"2024-09-10T20:36:29.147428+0000","last_peered":"2024-09-10T20:36:29.147428+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:36:29.147428+0000","last_undegraded":"2024-09-10T20:36:29.147428+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20955136,"num_objects":5116,"num_object_clones":0,"num_object_copies":10232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13334,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"2","objects":5116}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15602","reported_seq":15854,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:28.737803+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:36:28.737803+0000","last_peered":"2024-09-10T20:36:28.737803+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:36:28.737803+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21413888,"num_objects":5228,"num_object_clones":0,"num_object_copies":10456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12325,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1636}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15423","reported_seq":15666,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:28.819980+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:36:28.819980+0000","last_peered":"2024-09-10T20:36:28.819980+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:36:28.819980+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21139456,"num_objects":5161,"num_object_clones":0,"num_object_copies":10322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3484,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5161,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12117,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1677}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086784+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086784+0000","last_peered":"2024-09-10T20:34:27.086784+0000","last_clean":"2024-09-10T20:34:27.086784+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086784+0000","last_undegraded":"2024-09-10T20:34:27.086784+0000","last_fullsized":"2024-09-10T20:34:27.086784+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15785","reported_seq":16037,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:28.794037+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:36:28.794037+0000","last_peered":"2024-09-10T20:36:28.794037+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:36:28.794037+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21303296,"num_objects":5201,"num_object_clones":0,"num_object_copies":10402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12612,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1546}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317586+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317586+0000","last_peered":"2024-09-10T20:36:30.317586+0000","last_clean":"2024-09-10T20:36:30.317586+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317586+0000","last_undegraded":"2024-09-10T20:36:30.317586+0000","last_fullsized":"2024-09-10T20:36:30.317586+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'15418","reported_seq":22544,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.551556+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:36:30.551556+0000","last_peered":"2024-09-10T20:36:30.551556+0000","last_clean":"2024-09-10T20:36:30.551556+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:36:30.551556+0000","last_undegraded":"2024-09-10T20:36:30.551556+0000","last_fullsized":"2024-09-10T20:36:30.551556+0000","mapping_epoch":46,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20881891,"num_objects":5099,"num_object_clones":0,"num_object_copies":10198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5099,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12025,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":338998721,"num_objects":82767,"num_object_clones":0,"num_object_copies":165534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32001,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82767,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":212650,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11146,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":853405696,"data_stored":853339744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160854,"ondisk_log_size":160854,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182495,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":700700,"kb_used_data":308108,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93671140,"statfs":{"total":96636764160,"available":95919247360,"internally_reserved":0,"allocated":315502592,"data_stored":315250299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":401993386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,0,8,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986267,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610760,"kb_used_data":281976,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93761080,"statfs":{"total":96636764160,"available":96011345920,"internally_reserved":0,"allocated":288743424,"data_stored":288491136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336653994},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,1,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659214,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594196,"kb_used_data":247352,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93777644,"statfs":{"total":96636764160,"available":96028307456,"internally_reserved":0,"allocated":253288448,"data_stored":252980297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355135150},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,4,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":313536512,"data_stored":313532899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":252227584,"data_stored":252172930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":287641600,"data_stored":287633915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:36:31.854 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:36:32.414 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:32.414 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:36:32.436 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-09-10T20:36:30.983603+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":338998740,"num_objects":82769,"num_object_clones":0,"num_object_copies":165538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32001,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82769,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":212652,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11148,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160857,"ondisk_log_size":160857,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1905656,"kb_used_data":837436,"kb_used_omap":12,"kb_used_meta":1068147,"kb_avail":281209864,"statfs":{"total":289910292480,"available":287958900736,"internally_reserved":0,"allocated":857534464,"data_stored":856721732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1093782530},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,4,3,13,8,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19849216,"num_objects":-4846,"num_object_clones":0,"num_object_copies":-9692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-317,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4846,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002856"},"pg_stats":[{"pgid":"2.f","version":"47'15474","reported_seq":15692,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:28.816687+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:36:28.816687+0000","last_peered":"2024-09-10T20:36:28.816687+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:36:28.816687+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21233664,"num_objects":5184,"num_object_clones":0,"num_object_copies":10368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15474,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1643}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15755","reported_seq":15871,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:29.033703+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:36:29.033703+0000","last_peered":"2024-09-10T20:36:29.033703+0000","last_clean":"2024-09-10T20:36:29.033703+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:36:29.033703+0000","last_undegraded":"2024-09-10T20:36:29.033703+0000","last_fullsized":"2024-09-10T20:36:29.033703+0000","mapping_epoch":35,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21245952,"num_objects":5187,"num_object_clones":0,"num_object_copies":10374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13941,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'15542","reported_seq":22899,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.538925+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:36:30.538925+0000","last_peered":"2024-09-10T20:36:30.538925+0000","last_clean":"2024-09-10T20:36:30.538925+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:36:30.538925+0000","last_undegraded":"2024-09-10T20:36:30.538925+0000","last_fullsized":"2024-09-10T20:36:30.538925+0000","mapping_epoch":44,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21188632,"num_objects":5174,"num_object_clones":0,"num_object_copies":10348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5174,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13670,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'15394","reported_seq":15480,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:29.116351+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:36:29.116351+0000","last_peered":"2024-09-10T20:36:29.116351+0000","last_clean":"2024-09-10T20:36:29.116351+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:36:29.116351+0000","last_undegraded":"2024-09-10T20:36:29.116351+0000","last_fullsized":"2024-09-10T20:36:29.116351+0000","mapping_epoch":35,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21340160,"num_objects":5210,"num_object_clones":0,"num_object_copies":10420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15394,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15773","reported_seq":15905,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:28.812332+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:36:28.812332+0000","last_peered":"2024-09-10T20:36:28.812332+0000","last_clean":"2024-09-10T20:36:28.812332+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:36:28.812332+0000","last_undegraded":"2024-09-10T20:36:28.812332+0000","last_fullsized":"2024-09-10T20:36:28.812332+0000","mapping_epoch":34,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21692416,"num_objects":5296,"num_object_clones":0,"num_object_copies":10592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12769,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15431","reported_seq":15679,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:28.624148+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:36:28.624148+0000","last_peered":"2024-09-10T20:36:28.624148+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:36:28.624148+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20643840,"num_objects":5040,"num_object_clones":0,"num_object_copies":10080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12295,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1479}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15394","reported_seq":15675,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:28.821063+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:36:28.821063+0000","last_peered":"2024-09-10T20:36:28.821063+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:36:28.821063+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21356544,"num_objects":5214,"num_object_clones":0,"num_object_copies":10428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11989,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1728}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15600","reported_seq":15730,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:28.758421+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:36:28.758421+0000","last_peered":"2024-09-10T20:36:28.758421+0000","last_clean":"2024-09-10T20:36:28.758421+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:36:28.758421+0000","last_undegraded":"2024-09-10T20:36:28.758421+0000","last_fullsized":"2024-09-10T20:36:28.758421+0000","mapping_epoch":34,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21037056,"num_objects":5136,"num_object_clones":0,"num_object_copies":10272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12671,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15234","reported_seq":15430,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:29.157150+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:36:29.157150+0000","last_peered":"2024-09-10T20:36:29.157150+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:36:29.157150+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21319680,"num_objects":5205,"num_object_clones":0,"num_object_copies":10410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13227,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1708}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086274+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086274+0000","last_peered":"2024-09-10T20:34:27.086274+0000","last_clean":"2024-09-10T20:34:27.086274+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086274+0000","last_undegraded":"2024-09-10T20:34:27.086274+0000","last_fullsized":"2024-09-10T20:34:27.086274+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15561","reported_seq":15768,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:28.829615+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:36:28.829615+0000","last_peered":"2024-09-10T20:36:28.829615+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:36:28.829615+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21173190,"num_objects":5171,"num_object_clones":0,"num_object_copies":10342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15559,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1605}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15420","reported_seq":15608,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:29.158159+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:36:29.158159+0000","last_peered":"2024-09-10T20:36:29.158159+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:36:29.158159+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21073920,"num_objects":5145,"num_object_clones":0,"num_object_copies":10290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13248,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1539}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15548","reported_seq":22908,"reported_epoch":47,"state":"active+undersized+remapped","last_fresh":"2024-09-10T20:36:29.147428+0000","last_change":"2024-09-10T20:36:28.760528+0000","last_active":"2024-09-10T20:36:29.147428+0000","last_peered":"2024-09-10T20:36:29.147428+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:36:29.147428+0000","last_undegraded":"2024-09-10T20:36:29.147428+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20955136,"num_objects":5116,"num_object_clones":0,"num_object_copies":10232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13334,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"2","objects":5116}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15602","reported_seq":15854,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:28.737803+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:36:28.737803+0000","last_peered":"2024-09-10T20:36:28.737803+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:36:28.737803+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21413888,"num_objects":5228,"num_object_clones":0,"num_object_copies":10456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12325,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1636}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15423","reported_seq":15666,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:28.819980+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:36:28.819980+0000","last_peered":"2024-09-10T20:36:28.819980+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:36:28.819980+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21139456,"num_objects":5161,"num_object_clones":0,"num_object_copies":10322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3484,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5161,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12117,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1677}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086784+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086784+0000","last_peered":"2024-09-10T20:34:27.086784+0000","last_clean":"2024-09-10T20:34:27.086784+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086784+0000","last_undegraded":"2024-09-10T20:34:27.086784+0000","last_fullsized":"2024-09-10T20:34:27.086784+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15785","reported_seq":16037,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:28.794037+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:36:28.794037+0000","last_peered":"2024-09-10T20:36:28.794037+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:36:28.794037+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21303296,"num_objects":5201,"num_object_clones":0,"num_object_copies":10402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12612,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1546}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317586+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317586+0000","last_peered":"2024-09-10T20:36:30.317586+0000","last_clean":"2024-09-10T20:36:30.317586+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317586+0000","last_undegraded":"2024-09-10T20:36:30.317586+0000","last_fullsized":"2024-09-10T20:36:30.317586+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'15418","reported_seq":22544,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.551556+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:36:30.551556+0000","last_peered":"2024-09-10T20:36:30.551556+0000","last_clean":"2024-09-10T20:36:30.551556+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:36:30.551556+0000","last_undegraded":"2024-09-10T20:36:30.551556+0000","last_fullsized":"2024-09-10T20:36:30.551556+0000","mapping_epoch":46,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20881891,"num_objects":5099,"num_object_clones":0,"num_object_copies":10198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5099,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12025,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":338998721,"num_objects":82767,"num_object_clones":0,"num_object_copies":165534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32001,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82767,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":212650,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11146,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":853405696,"data_stored":853339744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160854,"ondisk_log_size":160854,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182495,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":700700,"kb_used_data":308108,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93671140,"statfs":{"total":96636764160,"available":95919247360,"internally_reserved":0,"allocated":315502592,"data_stored":315250299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":401993386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,0,8,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986267,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610760,"kb_used_data":281976,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93761080,"statfs":{"total":96636764160,"available":96011345920,"internally_reserved":0,"allocated":288743424,"data_stored":288491136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336653994},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,1,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659214,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594196,"kb_used_data":247352,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93777644,"statfs":{"total":96636764160,"available":96028307456,"internally_reserved":0,"allocated":253288448,"data_stored":252980297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355135150},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,4,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":313536512,"data_stored":313532899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":252227584,"data_stored":252172930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":287641600,"data_stored":287633915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:36:32.438 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:36:32.438 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'15474", 'reported_seq': 15692, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:28.816687+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:36:28.816687+0000', 'last_peered': '2024-09-10T20:36:28.816687+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:36:28.816687+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "43'5400", 'ondisk_log_start': "43'5400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21233664, 'num_objects': 5184, 'num_object_clones': 0, 'num_object_copies': 10368, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5184, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15474, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1643}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:32.438 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:36:32.438 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'15431", 'reported_seq': 15679, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:28.624148+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:36:28.624148+0000', 'last_peered': '2024-09-10T20:36:28.624148+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:36:28.624148+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "43'5400", 'ondisk_log_start': "43'5400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20643840, 'num_objects': 5040, 'num_object_clones': 0, 'num_object_copies': 10080, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5040, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12295, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1479}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:32.438 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:36:32.438 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'15394", 'reported_seq': 15675, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:28.821063+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:36:28.821063+0000', 'last_peered': '2024-09-10T20:36:28.821063+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:36:28.821063+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "43'5300", 'ondisk_log_start': "43'5300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21356544, 'num_objects': 5214, 'num_object_clones': 0, 'num_object_copies': 10428, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5214, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11989, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1728}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:32.438 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:36:32.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'15234", 'reported_seq': 15430, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:29.157150+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:36:29.157150+0000', 'last_peered': '2024-09-10T20:36:29.157150+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:36:29.157150+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21319680, 'num_objects': 5205, 'num_object_clones': 0, 'num_object_copies': 10410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5205, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13227, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1708}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:36:32.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:36:32.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'15561", 'reported_seq': 15768, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:28.829615+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:36:28.829615+0000', 'last_peered': '2024-09-10T20:36:28.829615+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:36:28.829615+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "43'5500", 'ondisk_log_start': "43'5500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21173190, 'num_objects': 5171, 'num_object_clones': 0, 'num_object_copies': 10342, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5171, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15559, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1605}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:32.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:36:32.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'15420", 'reported_seq': 15608, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:29.158159+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:36:29.158159+0000', 'last_peered': '2024-09-10T20:36:29.158159+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:36:29.158159+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "43'5400", 'ondisk_log_start': "43'5400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21073920, 'num_objects': 5145, 'num_object_clones': 0, 'num_object_copies': 10290, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5145, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13248, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1539}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:36:32.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T20:36:32.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'15548", 'reported_seq': 22908, 'reported_epoch': 47, 'state': 'active+undersized+remapped', 'last_fresh': '2024-09-10T20:36:29.147428+0000', 'last_change': '2024-09-10T20:36:28.760528+0000', 'last_active': '2024-09-10T20:36:29.147428+0000', 'last_peered': '2024-09-10T20:36:29.147428+0000', 'last_clean': '2024-09-10T20:30:03.302216+0000', 'last_became_active': '2024-09-10T20:30:05.639072+0000', 'last_became_peered': '2024-09-10T20:30:05.639072+0000', 'last_unstale': '2024-09-10T20:36:29.147428+0000', 'last_undegraded': '2024-09-10T20:36:29.147428+0000', 'last_fullsized': '2024-09-10T20:30:05.311985+0000', 'mapping_epoch': 42, 'log_start': "43'5500", 'ondisk_log_start': "43'5500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2246", 'last_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:45.018072+0000', 'objects_scrubbed': 2208, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 423, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20955136, 'num_objects': 5116, 'num_object_clones': 0, 'num_object_copies': 10232, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5116, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13334, 'num_write_kb': 32484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3612, 'num_bytes_recovered': 8085504, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0', '1'], 'object_location_counts': [{'shards': '2', 'objects': 5116}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:36:32.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:36:32.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'15602", 'reported_seq': 15854, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:28.737803+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:36:28.737803+0000', 'last_peered': '2024-09-10T20:36:28.737803+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:36:28.737803+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "43'5600", 'ondisk_log_start': "43'5600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21413888, 'num_objects': 5228, 'num_object_clones': 0, 'num_object_copies': 10456, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5228, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12325, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1636}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:32.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:36:32.440 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'15423", 'reported_seq': 15666, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:28.819980+0000', 'last_change': '2024-09-10T20:36:28.778006+0000', 'last_active': '2024-09-10T20:36:28.819980+0000', 'last_peered': '2024-09-10T20:36:28.819980+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:36:28.819980+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "43'5400", 'ondisk_log_start': "43'5400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21139456, 'num_objects': 5161, 'num_object_clones': 0, 'num_object_copies': 10322, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3484, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5161, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12117, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 49, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1677}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:32.440 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:36:32.440 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'15785", 'reported_seq': 16037, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:28.794037+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:36:28.794037+0000', 'last_peered': '2024-09-10T20:36:28.794037+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:36:28.794037+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "43'5700", 'ondisk_log_start': "43'5700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21303296, 'num_objects': 5201, 'num_object_clones': 0, 'num_object_copies': 10402, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5201, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12612, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1546}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:32.440 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:36:33.137 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:33.137 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32001/165538 objects degraded (19.332%), 9 pgs degraded, 10 pgs undersized","count":19},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":504,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":82769,"data_bytes":338998740,"bytes_used":1951391744,"bytes_avail":287958900736,"bytes_total":289910292480,"degraded_objects":32001,"degraded_total":165538,"degraded_ratio":0.19331513006077156,"recovering_objects_per_sec":39,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":315,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":605},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:36:33.168 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:36:33.168 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:36:33.742 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:33.745 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:36:33.769 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2024-09-10T20:36:32.984314+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":338998740,"num_objects":82769,"num_object_clones":0,"num_object_copies":165538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32001,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82769,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":212652,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11148,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160857,"ondisk_log_size":160857,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1905656,"kb_used_data":837436,"kb_used_omap":12,"kb_used_meta":1068147,"kb_avail":281209864,"statfs":{"total":289910292480,"available":287958900736,"internally_reserved":0,"allocated":857534464,"data_stored":856721732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1093782530},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,4,3,13,8,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11132928,"num_objects":-2718,"num_object_clones":0,"num_object_copies":-5436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2718,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002801"},"pg_stats":[{"pgid":"2.f","version":"47'15474","reported_seq":15692,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:28.816687+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:36:28.816687+0000","last_peered":"2024-09-10T20:36:28.816687+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:36:28.816687+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21233664,"num_objects":5184,"num_object_clones":0,"num_object_copies":10368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15474,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1643}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15755","reported_seq":15871,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:29.033703+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:36:29.033703+0000","last_peered":"2024-09-10T20:36:29.033703+0000","last_clean":"2024-09-10T20:36:29.033703+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:36:29.033703+0000","last_undegraded":"2024-09-10T20:36:29.033703+0000","last_fullsized":"2024-09-10T20:36:29.033703+0000","mapping_epoch":35,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21245952,"num_objects":5187,"num_object_clones":0,"num_object_copies":10374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13941,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'15542","reported_seq":22899,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.538925+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:36:30.538925+0000","last_peered":"2024-09-10T20:36:30.538925+0000","last_clean":"2024-09-10T20:36:30.538925+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:36:30.538925+0000","last_undegraded":"2024-09-10T20:36:30.538925+0000","last_fullsized":"2024-09-10T20:36:30.538925+0000","mapping_epoch":44,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21188632,"num_objects":5174,"num_object_clones":0,"num_object_copies":10348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5174,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13670,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'15394","reported_seq":15480,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:29.116351+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:36:29.116351+0000","last_peered":"2024-09-10T20:36:29.116351+0000","last_clean":"2024-09-10T20:36:29.116351+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:36:29.116351+0000","last_undegraded":"2024-09-10T20:36:29.116351+0000","last_fullsized":"2024-09-10T20:36:29.116351+0000","mapping_epoch":35,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21340160,"num_objects":5210,"num_object_clones":0,"num_object_copies":10420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15394,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15773","reported_seq":15905,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:28.812332+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:36:28.812332+0000","last_peered":"2024-09-10T20:36:28.812332+0000","last_clean":"2024-09-10T20:36:28.812332+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:36:28.812332+0000","last_undegraded":"2024-09-10T20:36:28.812332+0000","last_fullsized":"2024-09-10T20:36:28.812332+0000","mapping_epoch":34,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21692416,"num_objects":5296,"num_object_clones":0,"num_object_copies":10592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12769,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15431","reported_seq":15679,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:28.624148+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:36:28.624148+0000","last_peered":"2024-09-10T20:36:28.624148+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:36:28.624148+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20643840,"num_objects":5040,"num_object_clones":0,"num_object_copies":10080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12295,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1479}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15394","reported_seq":15675,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:28.821063+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:36:28.821063+0000","last_peered":"2024-09-10T20:36:28.821063+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:36:28.821063+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21356544,"num_objects":5214,"num_object_clones":0,"num_object_copies":10428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11989,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1728}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15600","reported_seq":15730,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:36:28.758421+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:36:28.758421+0000","last_peered":"2024-09-10T20:36:28.758421+0000","last_clean":"2024-09-10T20:36:28.758421+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:36:28.758421+0000","last_undegraded":"2024-09-10T20:36:28.758421+0000","last_fullsized":"2024-09-10T20:36:28.758421+0000","mapping_epoch":34,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21037056,"num_objects":5136,"num_object_clones":0,"num_object_copies":10272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12671,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15234","reported_seq":15430,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:29.157150+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:36:29.157150+0000","last_peered":"2024-09-10T20:36:29.157150+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:36:29.157150+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21319680,"num_objects":5205,"num_object_clones":0,"num_object_copies":10410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13227,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1708}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086274+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:34:27.086274+0000","last_peered":"2024-09-10T20:34:27.086274+0000","last_clean":"2024-09-10T20:34:27.086274+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:34:27.086274+0000","last_undegraded":"2024-09-10T20:34:27.086274+0000","last_fullsized":"2024-09-10T20:34:27.086274+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15561","reported_seq":15768,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:28.829615+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:36:28.829615+0000","last_peered":"2024-09-10T20:36:28.829615+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:36:28.829615+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21173190,"num_objects":5171,"num_object_clones":0,"num_object_copies":10342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15559,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1605}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.076007+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:34:27.076007+0000","last_peered":"2024-09-10T20:34:27.076007+0000","last_clean":"2024-09-10T20:34:27.076007+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:34:27.076007+0000","last_undegraded":"2024-09-10T20:34:27.076007+0000","last_fullsized":"2024-09-10T20:34:27.076007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15420","reported_seq":15608,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:29.158159+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:36:29.158159+0000","last_peered":"2024-09-10T20:36:29.158159+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:36:29.158159+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21073920,"num_objects":5145,"num_object_clones":0,"num_object_copies":10290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13248,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1539}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15548","reported_seq":22908,"reported_epoch":47,"state":"active+undersized+remapped","last_fresh":"2024-09-10T20:36:29.147428+0000","last_change":"2024-09-10T20:36:28.760528+0000","last_active":"2024-09-10T20:36:29.147428+0000","last_peered":"2024-09-10T20:36:29.147428+0000","last_clean":"2024-09-10T20:30:03.302216+0000","last_became_active":"2024-09-10T20:30:05.639072+0000","last_became_peered":"2024-09-10T20:30:05.639072+0000","last_unstale":"2024-09-10T20:36:29.147428+0000","last_undegraded":"2024-09-10T20:36:29.147428+0000","last_fullsized":"2024-09-10T20:30:05.311985+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20955136,"num_objects":5116,"num_object_clones":0,"num_object_copies":10232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13334,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"2","objects":5116}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15602","reported_seq":15854,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:28.737803+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:36:28.737803+0000","last_peered":"2024-09-10T20:36:28.737803+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:36:28.737803+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21413888,"num_objects":5228,"num_object_clones":0,"num_object_copies":10456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12325,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1636}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15423","reported_seq":15666,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:28.819980+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:36:28.819980+0000","last_peered":"2024-09-10T20:36:28.819980+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:36:28.819980+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21139456,"num_objects":5161,"num_object_clones":0,"num_object_copies":10322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3484,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5161,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12117,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1677}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T20:34:27.086784+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:34:27.086784+0000","last_peered":"2024-09-10T20:34:27.086784+0000","last_clean":"2024-09-10T20:34:27.086784+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:34:27.086784+0000","last_undegraded":"2024-09-10T20:34:27.086784+0000","last_fullsized":"2024-09-10T20:34:27.086784+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15785","reported_seq":16037,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:28.794037+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:36:28.794037+0000","last_peered":"2024-09-10T20:36:28.794037+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:36:28.794037+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21303296,"num_objects":5201,"num_object_clones":0,"num_object_copies":10402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12612,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1546}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317586+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317586+0000","last_peered":"2024-09-10T20:36:30.317586+0000","last_clean":"2024-09-10T20:36:30.317586+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317586+0000","last_undegraded":"2024-09-10T20:36:30.317586+0000","last_fullsized":"2024-09-10T20:36:30.317586+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'15418","reported_seq":22544,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.551556+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:36:30.551556+0000","last_peered":"2024-09-10T20:36:30.551556+0000","last_clean":"2024-09-10T20:36:30.551556+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:36:30.551556+0000","last_undegraded":"2024-09-10T20:36:30.551556+0000","last_fullsized":"2024-09-10T20:36:30.551556+0000","mapping_epoch":46,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20881891,"num_objects":5099,"num_object_clones":0,"num_object_copies":10198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5099,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12025,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":338998721,"num_objects":82767,"num_object_clones":0,"num_object_copies":165534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32001,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82767,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":212650,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11146,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":853405696,"data_stored":853339744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160854,"ondisk_log_size":160854,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182495,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":700700,"kb_used_data":308108,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93671140,"statfs":{"total":96636764160,"available":95919247360,"internally_reserved":0,"allocated":315502592,"data_stored":315250299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":401993386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,0,8,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986267,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610760,"kb_used_data":281976,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93761080,"statfs":{"total":96636764160,"available":96011345920,"internally_reserved":0,"allocated":288743424,"data_stored":288491136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336653994},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,1,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659214,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594196,"kb_used_data":247352,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93777644,"statfs":{"total":96636764160,"available":96028307456,"internally_reserved":0,"allocated":253288448,"data_stored":252980297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355135150},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,4,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":313536512,"data_stored":313532899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":252227584,"data_stored":252172930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":287641600,"data_stored":287633915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:36:36.771 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:36:37.206 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:37.207 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:36:37.219 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":297,"stamp":"2024-09-10T20:36:36.985462+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":329967060,"num_objects":80564,"num_object_clones":0,"num_object_copies":161128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31849,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80564,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":214857,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11302,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160762,"ondisk_log_size":160762,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1865716,"kb_used_data":818304,"kb_used_omap":12,"kb_used_meta":1047347,"kb_avail":281249804,"statfs":{"total":289910292480,"available":287999799296,"internally_reserved":0,"allocated":837943296,"data_stored":837106416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1072483330},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,2,4,7,7,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-13164544,"num_objects":-3214,"num_object_clones":0,"num_object_copies":-6428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-304,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-3214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3214,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002710"},"pg_stats":[{"pgid":"2.f","version":"49'15606","reported_seq":15828,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:33.785944+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:36:33.785944+0000","last_peered":"2024-09-10T20:36:33.785944+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:36:33.785944+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20692992,"num_objects":5052,"num_object_clones":0,"num_object_copies":10104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15606,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1511}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'15885","reported_seq":16005,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:34.162389+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:36:34.162389+0000","last_peered":"2024-09-10T20:36:34.162389+0000","last_clean":"2024-09-10T20:36:34.162389+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:36:34.162389+0000","last_undegraded":"2024-09-10T20:36:34.162389+0000","last_fullsized":"2024-09-10T20:36:34.162389+0000","mapping_epoch":35,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20713472,"num_objects":5057,"num_object_clones":0,"num_object_copies":10114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14071,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'15692","reported_seq":23049,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:35.536824+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:36:35.536824+0000","last_peered":"2024-09-10T20:36:35.536824+0000","last_clean":"2024-09-10T20:36:35.536824+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:36:35.536824+0000","last_undegraded":"2024-09-10T20:36:35.536824+0000","last_fullsized":"2024-09-10T20:36:35.536824+0000","mapping_epoch":44,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20574232,"num_objects":5024,"num_object_clones":0,"num_object_copies":10048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5024,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13820,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'15534","reported_seq":15624,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:34.153898+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:36:34.153898+0000","last_peered":"2024-09-10T20:36:34.153898+0000","last_clean":"2024-09-10T20:36:34.153898+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:36:34.153898+0000","last_undegraded":"2024-09-10T20:36:34.153898+0000","last_fullsized":"2024-09-10T20:36:34.153898+0000","mapping_epoch":35,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20766720,"num_objects":5070,"num_object_clones":0,"num_object_copies":10140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5070,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15534,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'15914","reported_seq":16050,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:33.773842+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:36:33.773842+0000","last_peered":"2024-09-10T20:36:33.773842+0000","last_clean":"2024-09-10T20:36:33.773842+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:36:33.773842+0000","last_undegraded":"2024-09-10T20:36:33.773842+0000","last_fullsized":"2024-09-10T20:36:33.773842+0000","mapping_epoch":34,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21114880,"num_objects":5155,"num_object_clones":0,"num_object_copies":10310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12910,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'15570","reported_seq":15822,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:33.812627+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:36:33.812627+0000","last_peered":"2024-09-10T20:36:33.812627+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:36:33.812627+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20074496,"num_objects":4901,"num_object_clones":0,"num_object_copies":9802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12434,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1340}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'15518","reported_seq":15804,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:33.848840+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:36:33.848840+0000","last_peered":"2024-09-10T20:36:33.848840+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:36:33.848840+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20848640,"num_objects":5090,"num_object_clones":0,"num_object_copies":10180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12113,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1604}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'15735","reported_seq":15869,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:33.814940+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:36:33.814940+0000","last_peered":"2024-09-10T20:36:33.814940+0000","last_clean":"2024-09-10T20:36:33.814940+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:36:33.814940+0000","last_undegraded":"2024-09-10T20:36:33.814940+0000","last_fullsized":"2024-09-10T20:36:33.814940+0000","mapping_epoch":34,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20484096,"num_objects":5001,"num_object_clones":0,"num_object_copies":10002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12806,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'15355","reported_seq":15555,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:34.156277+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:36:34.156277+0000","last_peered":"2024-09-10T20:36:34.156277+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:36:34.156277+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20824064,"num_objects":5084,"num_object_clones":0,"num_object_copies":10168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13348,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1587}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318026+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318026+0000","last_peered":"2024-09-10T20:36:30.318026+0000","last_clean":"2024-09-10T20:36:30.318026+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318026+0000","last_undegraded":"2024-09-10T20:36:30.318026+0000","last_fullsized":"2024-09-10T20:36:30.318026+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'15704","reported_seq":15919,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:33.809739+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:36:33.809739+0000","last_peered":"2024-09-10T20:36:33.809739+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:36:33.809739+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20587462,"num_objects":5028,"num_object_clones":0,"num_object_copies":10056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15702,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1462}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315256+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315256+0000","last_peered":"2024-09-10T20:36:30.315256+0000","last_clean":"2024-09-10T20:36:30.315256+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315256+0000","last_undegraded":"2024-09-10T20:36:30.315256+0000","last_fullsized":"2024-09-10T20:36:30.315256+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'15554","reported_seq":15746,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:34.165945+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:36:34.165945+0000","last_peered":"2024-09-10T20:36:34.165945+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:36:34.165945+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20525056,"num_objects":5011,"num_object_clones":0,"num_object_copies":10022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13382,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1405}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'15673","reported_seq":23043,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:34.088430+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:36:34.088430+0000","last_peered":"2024-09-10T20:36:34.088430+0000","last_clean":"2024-09-10T20:36:34.088430+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:36:34.088430+0000","last_undegraded":"2024-09-10T20:36:34.088430+0000","last_fullsized":"2024-09-10T20:36:34.088430+0000","mapping_epoch":48,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20443136,"num_objects":4991,"num_object_clones":0,"num_object_copies":9982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13459,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'15744","reported_seq":16001,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:33.784834+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:36:33.784834+0000","last_peered":"2024-09-10T20:36:33.784834+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:36:33.784834+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20832256,"num_objects":5086,"num_object_clones":0,"num_object_copies":10172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12467,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1494}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'15575","reported_seq":16130,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:33.835883+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:36:33.835883+0000","last_peered":"2024-09-10T20:36:33.835883+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:36:33.835883+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20516864,"num_objects":5009,"num_object_clones":0,"num_object_copies":10018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3332,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12269,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1677}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319607+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319607+0000","last_peered":"2024-09-10T20:36:30.319607+0000","last_clean":"2024-09-10T20:36:30.319607+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319607+0000","last_undegraded":"2024-09-10T20:36:30.319607+0000","last_fullsized":"2024-09-10T20:36:30.319607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'15928","reported_seq":16185,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:33.765606+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:36:33.765606+0000","last_peered":"2024-09-10T20:36:33.765606+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:36:33.765606+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20717568,"num_objects":5058,"num_object_clones":0,"num_object_copies":10116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12755,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1403}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'15572","reported_seq":22699,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:35.506805+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:36:35.506805+0000","last_peered":"2024-09-10T20:36:35.506805+0000","last_clean":"2024-09-10T20:36:35.506805+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:36:35.506805+0000","last_undegraded":"2024-09-10T20:36:35.506805+0000","last_fullsized":"2024-09-10T20:36:35.506805+0000","mapping_epoch":46,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20251107,"num_objects":4945,"num_object_clones":0,"num_object_copies":9890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12179,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":329967041,"num_objects":80562,"num_object_clones":0,"num_object_copies":161124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31849,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80562,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":214855,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11300,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":834801664,"data_stored":834735712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160759,"ondisk_log_size":160759,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182496,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":696788,"kb_used_data":300036,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93675052,"statfs":{"total":96636764160,"available":95923253248,"internally_reserved":0,"allocated":307236864,"data_stored":306974916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,2,2,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986268,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":593952,"kb_used_data":277644,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93777888,"statfs":{"total":96636764160,"available":96028557312,"internally_reserved":0,"allocated":284307456,"data_stored":284045513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323874474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659215,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574976,"kb_used_data":240624,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93796864,"statfs":{"total":96636764160,"available":96047988736,"internally_reserved":0,"allocated":246398976,"data_stored":246085987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":342355630},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,2,3,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":306769920,"data_stored":306766307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":244674560,"data_stored":244619906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":283357184,"data_stored":283349499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:36:37.221 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:36:37.756 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:37.759 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:36:37.789 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":297,"stamp":"2024-09-10T20:36:36.985462+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":329967060,"num_objects":80564,"num_object_clones":0,"num_object_copies":161128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31849,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80564,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":214857,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11302,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160762,"ondisk_log_size":160762,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1865716,"kb_used_data":818304,"kb_used_omap":12,"kb_used_meta":1047347,"kb_avail":281249804,"statfs":{"total":289910292480,"available":287999799296,"internally_reserved":0,"allocated":837943296,"data_stored":837106416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1072483330},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,2,4,7,7,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-13164544,"num_objects":-3214,"num_object_clones":0,"num_object_copies":-6428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-304,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-3214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3214,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002710"},"pg_stats":[{"pgid":"2.f","version":"49'15606","reported_seq":15828,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:33.785944+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:36:33.785944+0000","last_peered":"2024-09-10T20:36:33.785944+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:36:33.785944+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20692992,"num_objects":5052,"num_object_clones":0,"num_object_copies":10104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15606,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1511}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'15885","reported_seq":16005,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:34.162389+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:36:34.162389+0000","last_peered":"2024-09-10T20:36:34.162389+0000","last_clean":"2024-09-10T20:36:34.162389+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:36:34.162389+0000","last_undegraded":"2024-09-10T20:36:34.162389+0000","last_fullsized":"2024-09-10T20:36:34.162389+0000","mapping_epoch":35,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20713472,"num_objects":5057,"num_object_clones":0,"num_object_copies":10114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14071,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'15692","reported_seq":23049,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:35.536824+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:36:35.536824+0000","last_peered":"2024-09-10T20:36:35.536824+0000","last_clean":"2024-09-10T20:36:35.536824+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:36:35.536824+0000","last_undegraded":"2024-09-10T20:36:35.536824+0000","last_fullsized":"2024-09-10T20:36:35.536824+0000","mapping_epoch":44,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20574232,"num_objects":5024,"num_object_clones":0,"num_object_copies":10048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5024,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13820,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'15534","reported_seq":15624,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:34.153898+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:36:34.153898+0000","last_peered":"2024-09-10T20:36:34.153898+0000","last_clean":"2024-09-10T20:36:34.153898+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:36:34.153898+0000","last_undegraded":"2024-09-10T20:36:34.153898+0000","last_fullsized":"2024-09-10T20:36:34.153898+0000","mapping_epoch":35,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20766720,"num_objects":5070,"num_object_clones":0,"num_object_copies":10140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5070,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15534,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'15914","reported_seq":16050,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:33.773842+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:36:33.773842+0000","last_peered":"2024-09-10T20:36:33.773842+0000","last_clean":"2024-09-10T20:36:33.773842+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:36:33.773842+0000","last_undegraded":"2024-09-10T20:36:33.773842+0000","last_fullsized":"2024-09-10T20:36:33.773842+0000","mapping_epoch":34,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21114880,"num_objects":5155,"num_object_clones":0,"num_object_copies":10310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12910,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'15570","reported_seq":15822,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:33.812627+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:36:33.812627+0000","last_peered":"2024-09-10T20:36:33.812627+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:36:33.812627+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20074496,"num_objects":4901,"num_object_clones":0,"num_object_copies":9802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12434,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1340}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'15518","reported_seq":15804,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:33.848840+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:36:33.848840+0000","last_peered":"2024-09-10T20:36:33.848840+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:36:33.848840+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20848640,"num_objects":5090,"num_object_clones":0,"num_object_copies":10180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12113,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1604}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'15735","reported_seq":15869,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:33.814940+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:36:33.814940+0000","last_peered":"2024-09-10T20:36:33.814940+0000","last_clean":"2024-09-10T20:36:33.814940+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:36:33.814940+0000","last_undegraded":"2024-09-10T20:36:33.814940+0000","last_fullsized":"2024-09-10T20:36:33.814940+0000","mapping_epoch":34,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20484096,"num_objects":5001,"num_object_clones":0,"num_object_copies":10002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12806,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'15355","reported_seq":15555,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:34.156277+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:36:34.156277+0000","last_peered":"2024-09-10T20:36:34.156277+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:36:34.156277+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20824064,"num_objects":5084,"num_object_clones":0,"num_object_copies":10168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13348,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1587}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318026+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318026+0000","last_peered":"2024-09-10T20:36:30.318026+0000","last_clean":"2024-09-10T20:36:30.318026+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318026+0000","last_undegraded":"2024-09-10T20:36:30.318026+0000","last_fullsized":"2024-09-10T20:36:30.318026+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'15704","reported_seq":15919,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:33.809739+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:36:33.809739+0000","last_peered":"2024-09-10T20:36:33.809739+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:36:33.809739+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20587462,"num_objects":5028,"num_object_clones":0,"num_object_copies":10056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15702,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1462}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315256+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315256+0000","last_peered":"2024-09-10T20:36:30.315256+0000","last_clean":"2024-09-10T20:36:30.315256+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315256+0000","last_undegraded":"2024-09-10T20:36:30.315256+0000","last_fullsized":"2024-09-10T20:36:30.315256+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'15554","reported_seq":15746,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:34.165945+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:36:34.165945+0000","last_peered":"2024-09-10T20:36:34.165945+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:36:34.165945+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20525056,"num_objects":5011,"num_object_clones":0,"num_object_copies":10022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13382,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1405}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'15673","reported_seq":23043,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:34.088430+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:36:34.088430+0000","last_peered":"2024-09-10T20:36:34.088430+0000","last_clean":"2024-09-10T20:36:34.088430+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:36:34.088430+0000","last_undegraded":"2024-09-10T20:36:34.088430+0000","last_fullsized":"2024-09-10T20:36:34.088430+0000","mapping_epoch":48,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20443136,"num_objects":4991,"num_object_clones":0,"num_object_copies":9982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13459,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'15744","reported_seq":16001,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:33.784834+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:36:33.784834+0000","last_peered":"2024-09-10T20:36:33.784834+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:36:33.784834+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20832256,"num_objects":5086,"num_object_clones":0,"num_object_copies":10172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12467,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1494}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'15575","reported_seq":16130,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:33.835883+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:36:33.835883+0000","last_peered":"2024-09-10T20:36:33.835883+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:36:33.835883+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20516864,"num_objects":5009,"num_object_clones":0,"num_object_copies":10018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3332,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12269,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1677}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319607+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319607+0000","last_peered":"2024-09-10T20:36:30.319607+0000","last_clean":"2024-09-10T20:36:30.319607+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319607+0000","last_undegraded":"2024-09-10T20:36:30.319607+0000","last_fullsized":"2024-09-10T20:36:30.319607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'15928","reported_seq":16185,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:33.765606+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:36:33.765606+0000","last_peered":"2024-09-10T20:36:33.765606+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:36:33.765606+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20717568,"num_objects":5058,"num_object_clones":0,"num_object_copies":10116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12755,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1403}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'15572","reported_seq":22699,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:35.506805+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:36:35.506805+0000","last_peered":"2024-09-10T20:36:35.506805+0000","last_clean":"2024-09-10T20:36:35.506805+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:36:35.506805+0000","last_undegraded":"2024-09-10T20:36:35.506805+0000","last_fullsized":"2024-09-10T20:36:35.506805+0000","mapping_epoch":46,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20251107,"num_objects":4945,"num_object_clones":0,"num_object_copies":9890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12179,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":329967041,"num_objects":80562,"num_object_clones":0,"num_object_copies":161124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31849,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80562,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":214855,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11300,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":834801664,"data_stored":834735712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160759,"ondisk_log_size":160759,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182496,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":696788,"kb_used_data":300036,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93675052,"statfs":{"total":96636764160,"available":95923253248,"internally_reserved":0,"allocated":307236864,"data_stored":306974916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,2,2,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986268,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":593952,"kb_used_data":277644,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93777888,"statfs":{"total":96636764160,"available":96028557312,"internally_reserved":0,"allocated":284307456,"data_stored":284045513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323874474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659215,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574976,"kb_used_data":240624,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93796864,"statfs":{"total":96636764160,"available":96047988736,"internally_reserved":0,"allocated":246398976,"data_stored":246085987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":342355630},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,2,3,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":306769920,"data_stored":306766307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":244674560,"data_stored":244619906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":283357184,"data_stored":283349499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:36:37.791 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:36:37.791 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'15606", 'reported_seq': 15828, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:33.785944+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:36:33.785944+0000', 'last_peered': '2024-09-10T20:36:33.785944+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:36:33.785944+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "43'5600", 'ondisk_log_start': "43'5600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20692992, 'num_objects': 5052, 'num_object_clones': 0, 'num_object_copies': 10104, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5052, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15606, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1511}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:37.791 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:36:37.791 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'15570", 'reported_seq': 15822, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:33.812627+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:36:33.812627+0000', 'last_peered': '2024-09-10T20:36:33.812627+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:36:33.812627+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "43'5500", 'ondisk_log_start': "43'5500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20074496, 'num_objects': 4901, 'num_object_clones': 0, 'num_object_copies': 9802, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4901, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12434, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1340}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:37.791 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:36:37.791 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'15518", 'reported_seq': 15804, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:33.848840+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:36:33.848840+0000', 'last_peered': '2024-09-10T20:36:33.848840+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:36:33.848840+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "43'5500", 'ondisk_log_start': "43'5500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20848640, 'num_objects': 5090, 'num_object_clones': 0, 'num_object_copies': 10180, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5090, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12113, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1604}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:37.791 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:36:37.791 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'15355", 'reported_seq': 15555, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:34.156277+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:36:34.156277+0000', 'last_peered': '2024-09-10T20:36:34.156277+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:36:34.156277+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "43'5300", 'ondisk_log_start': "43'5300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20824064, 'num_objects': 5084, 'num_object_clones': 0, 'num_object_copies': 10168, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5084, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13348, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1587}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:36:37.792 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:36:37.792 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'15704", 'reported_seq': 15919, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:33.809739+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:36:33.809739+0000', 'last_peered': '2024-09-10T20:36:33.809739+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:36:33.809739+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "43'5700", 'ondisk_log_start': "43'5700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20587462, 'num_objects': 5028, 'num_object_clones': 0, 'num_object_copies': 10056, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5028, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15702, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1462}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:37.792 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:36:37.792 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'15554", 'reported_seq': 15746, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:34.165945+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:36:34.165945+0000', 'last_peered': '2024-09-10T20:36:34.165945+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:36:34.165945+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "43'5500", 'ondisk_log_start': "43'5500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20525056, 'num_objects': 5011, 'num_object_clones': 0, 'num_object_copies': 10022, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5011, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13382, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1405}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:36:37.792 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:36:37.792 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'15744", 'reported_seq': 16001, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:33.784834+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:36:33.784834+0000', 'last_peered': '2024-09-10T20:36:33.784834+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:36:33.784834+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "43'5700", 'ondisk_log_start': "43'5700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20832256, 'num_objects': 5086, 'num_object_clones': 0, 'num_object_copies': 10172, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5086, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12467, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1494}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:37.792 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:36:37.792 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'15575", 'reported_seq': 16130, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:33.835883+0000', 'last_change': '2024-09-10T20:36:28.778006+0000', 'last_active': '2024-09-10T20:36:33.835883+0000', 'last_peered': '2024-09-10T20:36:33.835883+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:36:33.835883+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "43'5500", 'ondisk_log_start': "43'5500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20516864, 'num_objects': 5009, 'num_object_clones': 0, 'num_object_copies': 10018, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3332, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5009, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12269, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 203, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1677}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:37.792 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:36:37.792 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'15928", 'reported_seq': 16185, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:33.765606+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:36:33.765606+0000', 'last_peered': '2024-09-10T20:36:33.765606+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:36:33.765606+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "43'5900", 'ondisk_log_start': "43'5900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20717568, 'num_objects': 5058, 'num_object_clones': 0, 'num_object_copies': 10116, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5058, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12755, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1403}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:37.793 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:36:38.474 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:38.474 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31849/161128 objects degraded (19.766%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":510,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":80564,"data_bytes":329967060,"bytes_used":1910493184,"bytes_avail":287999799296,"bytes_total":289910292480,"degraded_objects":31849,"degraded_total":161128,"degraded_ratio":0.19766272776922694,"recovering_objects_per_sec":38,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":306,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":401},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:36:38.497 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:36:38.498 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:36:39.090 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:39.092 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:36:39.114 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":298,"stamp":"2024-09-10T20:36:38.986106+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":324412884,"num_objects":79208,"num_object_clones":0,"num_object_copies":158416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31735,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79208,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":216213,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11416,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160918,"ondisk_log_size":160918,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1862736,"kb_used_data":811160,"kb_used_omap":12,"kb_used_meta":1051507,"kb_avail":281252784,"statfs":{"total":289910292480,"available":288002850816,"internally_reserved":0,"allocated":830627840,"data_stored":829790960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1076743170},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,11,0,3,7,7,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-15769600,"num_objects":-3850,"num_object_clones":0,"num_object_copies":-7700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-266,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-3850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3850,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.676623"},"pg_stats":[{"pgid":"2.f","version":"49'15768","reported_seq":15990,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:38.810240+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:36:38.810240+0000","last_peered":"2024-09-10T20:36:38.810240+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:36:38.810240+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20029440,"num_objects":4890,"num_object_clones":0,"num_object_copies":9780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15768,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1349}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'15885","reported_seq":16005,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:34.162389+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:36:34.162389+0000","last_peered":"2024-09-10T20:36:34.162389+0000","last_clean":"2024-09-10T20:36:34.162389+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:36:34.162389+0000","last_undegraded":"2024-09-10T20:36:34.162389+0000","last_fullsized":"2024-09-10T20:36:34.162389+0000","mapping_epoch":35,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20713472,"num_objects":5057,"num_object_clones":0,"num_object_copies":10114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14071,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'15692","reported_seq":23049,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:35.536824+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:36:35.536824+0000","last_peered":"2024-09-10T20:36:35.536824+0000","last_clean":"2024-09-10T20:36:35.536824+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:36:35.536824+0000","last_undegraded":"2024-09-10T20:36:35.536824+0000","last_fullsized":"2024-09-10T20:36:35.536824+0000","mapping_epoch":44,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20574232,"num_objects":5024,"num_object_clones":0,"num_object_copies":10048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5024,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13820,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'15534","reported_seq":15624,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:34.153898+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:36:34.153898+0000","last_peered":"2024-09-10T20:36:34.153898+0000","last_clean":"2024-09-10T20:36:34.153898+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:36:34.153898+0000","last_undegraded":"2024-09-10T20:36:34.153898+0000","last_fullsized":"2024-09-10T20:36:34.153898+0000","mapping_epoch":35,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20766720,"num_objects":5070,"num_object_clones":0,"num_object_copies":10140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5070,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15534,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'16068","reported_seq":16204,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:38.809408+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:36:38.809408+0000","last_peered":"2024-09-10T20:36:38.809408+0000","last_clean":"2024-09-10T20:36:38.809408+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:36:38.809408+0000","last_undegraded":"2024-09-10T20:36:38.809408+0000","last_fullsized":"2024-09-10T20:36:38.809408+0000","mapping_epoch":34,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20484096,"num_objects":5001,"num_object_clones":0,"num_object_copies":10002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13064,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'15709","reported_seq":15961,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:38.773202+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:36:38.773202+0000","last_peered":"2024-09-10T20:36:38.773202+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:36:38.773202+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19505152,"num_objects":4762,"num_object_clones":0,"num_object_copies":9524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12573,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1201}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'15680","reported_seq":15966,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:38.819212+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:36:38.819212+0000","last_peered":"2024-09-10T20:36:38.819212+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:36:38.819212+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20185088,"num_objects":4928,"num_object_clones":0,"num_object_copies":9856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12275,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1442}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'15861","reported_seq":15994,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:38.813875+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:36:38.813875+0000","last_peered":"2024-09-10T20:36:38.813875+0000","last_clean":"2024-09-10T20:36:38.813875+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:36:38.813875+0000","last_undegraded":"2024-09-10T20:36:38.813875+0000","last_fullsized":"2024-09-10T20:36:38.813875+0000","mapping_epoch":34,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19968000,"num_objects":4875,"num_object_clones":0,"num_object_copies":9750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12932,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'15355","reported_seq":15555,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:34.156277+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:36:34.156277+0000","last_peered":"2024-09-10T20:36:34.156277+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:36:34.156277+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20824064,"num_objects":5084,"num_object_clones":0,"num_object_copies":10168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13348,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1587}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318026+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318026+0000","last_peered":"2024-09-10T20:36:30.318026+0000","last_clean":"2024-09-10T20:36:30.318026+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318026+0000","last_undegraded":"2024-09-10T20:36:30.318026+0000","last_fullsized":"2024-09-10T20:36:30.318026+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'15849","reported_seq":16065,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:38.852697+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:36:38.852697+0000","last_peered":"2024-09-10T20:36:38.852697+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:36:38.852697+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19993542,"num_objects":4883,"num_object_clones":0,"num_object_copies":9766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15847,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1317}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315256+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315256+0000","last_peered":"2024-09-10T20:36:30.315256+0000","last_clean":"2024-09-10T20:36:30.315256+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315256+0000","last_undegraded":"2024-09-10T20:36:30.315256+0000","last_fullsized":"2024-09-10T20:36:30.315256+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'15554","reported_seq":15746,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:34.165945+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:36:34.165945+0000","last_peered":"2024-09-10T20:36:34.165945+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:36:34.165945+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20525056,"num_objects":5011,"num_object_clones":0,"num_object_copies":10022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13382,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1405}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'15673","reported_seq":23043,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:34.088430+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:36:34.088430+0000","last_peered":"2024-09-10T20:36:34.088430+0000","last_clean":"2024-09-10T20:36:34.088430+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:36:34.088430+0000","last_undegraded":"2024-09-10T20:36:34.088430+0000","last_fullsized":"2024-09-10T20:36:34.088430+0000","mapping_epoch":48,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20443136,"num_objects":4991,"num_object_clones":0,"num_object_copies":9982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13459,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'15906","reported_seq":16162,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:38.870298+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:36:38.870298+0000","last_peered":"2024-09-10T20:36:38.870298+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:36:38.870298+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20168704,"num_objects":4924,"num_object_clones":0,"num_object_copies":9848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12629,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1332}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'15731","reported_seq":16514,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:38.822275+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:36:38.822275+0000","last_peered":"2024-09-10T20:36:38.822275+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:36:38.822275+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19877888,"num_objects":4853,"num_object_clones":0,"num_object_copies":9706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3218,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12425,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":317,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1635}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319607+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319607+0000","last_peered":"2024-09-10T20:36:30.319607+0000","last_clean":"2024-09-10T20:36:30.319607+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319607+0000","last_undegraded":"2024-09-10T20:36:30.319607+0000","last_fullsized":"2024-09-10T20:36:30.319607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'16078","reported_seq":16335,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:38.809864+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:36:38.809864+0000","last_peered":"2024-09-10T20:36:38.809864+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:36:38.809864+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20103168,"num_objects":4908,"num_object_clones":0,"num_object_copies":9816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12905,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1253}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'15572","reported_seq":22699,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:35.506805+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:36:35.506805+0000","last_peered":"2024-09-10T20:36:35.506805+0000","last_clean":"2024-09-10T20:36:35.506805+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:36:35.506805+0000","last_undegraded":"2024-09-10T20:36:35.506805+0000","last_fullsized":"2024-09-10T20:36:35.506805+0000","mapping_epoch":46,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20251107,"num_objects":4945,"num_object_clones":0,"num_object_copies":9890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12179,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":324412865,"num_objects":79206,"num_object_clones":0,"num_object_copies":158412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31735,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79206,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":216211,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11414,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":827490304,"data_stored":827424352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160915,"ondisk_log_size":160915,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182497,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":693808,"kb_used_data":292892,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93678032,"statfs":{"total":96636764160,"available":95926304768,"internally_reserved":0,"allocated":299921408,"data_stored":299659460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":410513066},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,10,0,0,2,2,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986268,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":593952,"kb_used_data":277644,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93777888,"statfs":{"total":96636764160,"available":96028557312,"internally_reserved":0,"allocated":284307456,"data_stored":284045513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323874474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659215,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574976,"kb_used_data":240624,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93796864,"statfs":{"total":96636764160,"available":96047988736,"internally_reserved":0,"allocated":246398976,"data_stored":246085987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":342355630},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,2,3,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":299458560,"data_stored":299454947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":244674560,"data_stored":244619906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":283357184,"data_stored":283349499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:36:42.117 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:36:42.669 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:42.672 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:36:42.699 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2024-09-10T20:36:40.986917+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":320337364,"num_objects":78213,"num_object_clones":0,"num_object_copies":156426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31735,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78213,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":217208,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11416,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160813,"ondisk_log_size":160813,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1881712,"kb_used_data":799604,"kb_used_omap":12,"kb_used_meta":1082035,"kb_avail":281233808,"statfs":{"total":289910292480,"available":287983419392,"internally_reserved":0,"allocated":818794496,"data_stored":817957616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1108003842},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,10,2,3,8,7,9,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19845120,"num_objects":-4845,"num_object_clones":0,"num_object_copies":-9690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-266,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4845,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.673128"},"pg_stats":[{"pgid":"2.f","version":"49'15768","reported_seq":15990,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:38.810240+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:36:38.810240+0000","last_peered":"2024-09-10T20:36:38.810240+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:36:38.810240+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20029440,"num_objects":4890,"num_object_clones":0,"num_object_copies":9780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15768,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1349}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'16029","reported_seq":16149,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:39.161357+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:36:39.161357+0000","last_peered":"2024-09-10T20:36:39.161357+0000","last_clean":"2024-09-10T20:36:39.161357+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:36:39.161357+0000","last_undegraded":"2024-09-10T20:36:39.161357+0000","last_fullsized":"2024-09-10T20:36:39.161357+0000","mapping_epoch":35,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20123648,"num_objects":4913,"num_object_clones":0,"num_object_copies":9826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14215,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'15873","reported_seq":23230,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:40.587732+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:36:40.587732+0000","last_peered":"2024-09-10T20:36:40.587732+0000","last_clean":"2024-09-10T20:36:40.587732+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:36:40.587732+0000","last_undegraded":"2024-09-10T20:36:40.587732+0000","last_fullsized":"2024-09-10T20:36:40.587732+0000","mapping_epoch":44,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19832856,"num_objects":4843,"num_object_clones":0,"num_object_copies":9686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4843,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14001,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'15664","reported_seq":15754,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:39.078967+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:36:39.078967+0000","last_peered":"2024-09-10T20:36:39.078967+0000","last_clean":"2024-09-10T20:36:39.078967+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:36:39.078967+0000","last_undegraded":"2024-09-10T20:36:39.078967+0000","last_fullsized":"2024-09-10T20:36:39.078967+0000","mapping_epoch":35,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20234240,"num_objects":4940,"num_object_clones":0,"num_object_copies":9880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15664,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'16068","reported_seq":16204,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:38.809408+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:36:38.809408+0000","last_peered":"2024-09-10T20:36:38.809408+0000","last_clean":"2024-09-10T20:36:38.809408+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:36:38.809408+0000","last_undegraded":"2024-09-10T20:36:38.809408+0000","last_fullsized":"2024-09-10T20:36:38.809408+0000","mapping_epoch":34,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20484096,"num_objects":5001,"num_object_clones":0,"num_object_copies":10002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13064,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'15709","reported_seq":15961,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:38.773202+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:36:38.773202+0000","last_peered":"2024-09-10T20:36:38.773202+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:36:38.773202+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19505152,"num_objects":4762,"num_object_clones":0,"num_object_copies":9524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12573,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1201}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'15680","reported_seq":15966,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:38.819212+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:36:38.819212+0000","last_peered":"2024-09-10T20:36:38.819212+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:36:38.819212+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20185088,"num_objects":4928,"num_object_clones":0,"num_object_copies":9856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12275,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1442}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'15861","reported_seq":15994,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:38.813875+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:36:38.813875+0000","last_peered":"2024-09-10T20:36:38.813875+0000","last_clean":"2024-09-10T20:36:38.813875+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:36:38.813875+0000","last_undegraded":"2024-09-10T20:36:38.813875+0000","last_fullsized":"2024-09-10T20:36:38.813875+0000","mapping_epoch":34,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19968000,"num_objects":4875,"num_object_clones":0,"num_object_copies":9750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12932,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'15492","reported_seq":15692,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:39.173945+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:36:39.173945+0000","last_peered":"2024-09-10T20:36:39.173945+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:36:39.173945+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20262912,"num_objects":4947,"num_object_clones":0,"num_object_copies":9894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13485,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1450}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318026+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318026+0000","last_peered":"2024-09-10T20:36:30.318026+0000","last_clean":"2024-09-10T20:36:30.318026+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318026+0000","last_undegraded":"2024-09-10T20:36:30.318026+0000","last_fullsized":"2024-09-10T20:36:30.318026+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'15849","reported_seq":16065,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:38.852697+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:36:38.852697+0000","last_peered":"2024-09-10T20:36:38.852697+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:36:38.852697+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19993542,"num_objects":4883,"num_object_clones":0,"num_object_copies":9766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15847,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1317}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315256+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315256+0000","last_peered":"2024-09-10T20:36:30.315256+0000","last_clean":"2024-09-10T20:36:30.315256+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315256+0000","last_undegraded":"2024-09-10T20:36:30.315256+0000","last_fullsized":"2024-09-10T20:36:30.315256+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'15678","reported_seq":15870,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:39.031287+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:36:39.031287+0000","last_peered":"2024-09-10T20:36:39.031287+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:36:39.031287+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20017152,"num_objects":4887,"num_object_clones":0,"num_object_copies":9774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13506,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1281}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'15809","reported_seq":23179,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:39.190200+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:36:39.190200+0000","last_peered":"2024-09-10T20:36:39.190200+0000","last_clean":"2024-09-10T20:36:39.190200+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:36:39.190200+0000","last_undegraded":"2024-09-10T20:36:39.190200+0000","last_fullsized":"2024-09-10T20:36:39.190200+0000","mapping_epoch":48,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19886080,"num_objects":4855,"num_object_clones":0,"num_object_copies":9710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13595,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'15906","reported_seq":16162,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:38.870298+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:36:38.870298+0000","last_peered":"2024-09-10T20:36:38.870298+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:36:38.870298+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20168704,"num_objects":4924,"num_object_clones":0,"num_object_copies":9848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12629,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1332}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'15731","reported_seq":16514,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:38.822275+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:36:38.822275+0000","last_peered":"2024-09-10T20:36:38.822275+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:36:38.822275+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19877888,"num_objects":4853,"num_object_clones":0,"num_object_copies":9706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3218,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12425,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":317,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1635}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319607+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319607+0000","last_peered":"2024-09-10T20:36:30.319607+0000","last_clean":"2024-09-10T20:36:30.319607+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319607+0000","last_undegraded":"2024-09-10T20:36:30.319607+0000","last_fullsized":"2024-09-10T20:36:30.319607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'16078","reported_seq":16335,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:38.809864+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:36:38.809864+0000","last_peered":"2024-09-10T20:36:38.809864+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:36:38.809864+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20103168,"num_objects":4908,"num_object_clones":0,"num_object_copies":9816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12905,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1253}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'15715","reported_seq":22842,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:40.604603+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:36:40.604603+0000","last_peered":"2024-09-10T20:36:40.604603+0000","last_clean":"2024-09-10T20:36:40.604603+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:36:40.604603+0000","last_undegraded":"2024-09-10T20:36:40.604603+0000","last_fullsized":"2024-09-10T20:36:40.604603+0000","mapping_epoch":46,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19665379,"num_objects":4802,"num_object_clones":0,"num_object_copies":9604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12322,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":320337345,"num_objects":78211,"num_object_clones":0,"num_object_copies":156422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31735,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78211,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":217206,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11414,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":815505408,"data_stored":815439456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160810,"ondisk_log_size":160810,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182497,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":693808,"kb_used_data":292892,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93678032,"statfs":{"total":96636764160,"available":95926304768,"internally_reserved":0,"allocated":299921408,"data_stored":299659460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":410513066},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,10,0,0,2,2,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986269,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":593604,"kb_used_data":273140,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93778236,"statfs":{"total":96636764160,"available":96028913664,"internally_reserved":0,"allocated":279695360,"data_stored":279433417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328134314},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659216,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594300,"kb_used_data":233572,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93777540,"statfs":{"total":96636764160,"available":96028200960,"internally_reserved":0,"allocated":239177728,"data_stored":238864739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":369356462},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,2,5,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":299458560,"data_stored":299454947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":237342720,"data_stored":237288066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":278704128,"data_stored":278696443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:36:42.701 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:36:43.243 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:43.245 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:36:43.269 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":300,"stamp":"2024-09-10T20:36:42.990947+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":320337364,"num_objects":78213,"num_object_clones":0,"num_object_copies":156426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31735,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78213,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":217208,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11416,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160813,"ondisk_log_size":160813,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1881712,"kb_used_data":799604,"kb_used_omap":12,"kb_used_meta":1082035,"kb_avail":281233808,"statfs":{"total":289910292480,"available":287983419392,"internally_reserved":0,"allocated":818794496,"data_stored":817957616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1108003842},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,10,2,3,8,7,9,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18661376,"num_objects":-4556,"num_object_clones":0,"num_object_copies":-9112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-266,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4556,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007343"},"pg_stats":[{"pgid":"2.f","version":"49'15768","reported_seq":15990,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:38.810240+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:36:38.810240+0000","last_peered":"2024-09-10T20:36:38.810240+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:36:38.810240+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20029440,"num_objects":4890,"num_object_clones":0,"num_object_copies":9780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15768,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1349}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'16029","reported_seq":16149,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:39.161357+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:36:39.161357+0000","last_peered":"2024-09-10T20:36:39.161357+0000","last_clean":"2024-09-10T20:36:39.161357+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:36:39.161357+0000","last_undegraded":"2024-09-10T20:36:39.161357+0000","last_fullsized":"2024-09-10T20:36:39.161357+0000","mapping_epoch":35,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20123648,"num_objects":4913,"num_object_clones":0,"num_object_copies":9826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14215,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'15873","reported_seq":23230,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:40.587732+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:36:40.587732+0000","last_peered":"2024-09-10T20:36:40.587732+0000","last_clean":"2024-09-10T20:36:40.587732+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:36:40.587732+0000","last_undegraded":"2024-09-10T20:36:40.587732+0000","last_fullsized":"2024-09-10T20:36:40.587732+0000","mapping_epoch":44,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19832856,"num_objects":4843,"num_object_clones":0,"num_object_copies":9686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4843,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14001,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'15664","reported_seq":15754,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:39.078967+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:36:39.078967+0000","last_peered":"2024-09-10T20:36:39.078967+0000","last_clean":"2024-09-10T20:36:39.078967+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:36:39.078967+0000","last_undegraded":"2024-09-10T20:36:39.078967+0000","last_fullsized":"2024-09-10T20:36:39.078967+0000","mapping_epoch":35,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20234240,"num_objects":4940,"num_object_clones":0,"num_object_copies":9880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15664,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'16068","reported_seq":16204,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:38.809408+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:36:38.809408+0000","last_peered":"2024-09-10T20:36:38.809408+0000","last_clean":"2024-09-10T20:36:38.809408+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:36:38.809408+0000","last_undegraded":"2024-09-10T20:36:38.809408+0000","last_fullsized":"2024-09-10T20:36:38.809408+0000","mapping_epoch":34,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20484096,"num_objects":5001,"num_object_clones":0,"num_object_copies":10002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13064,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'15709","reported_seq":15961,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:38.773202+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:36:38.773202+0000","last_peered":"2024-09-10T20:36:38.773202+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:36:38.773202+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19505152,"num_objects":4762,"num_object_clones":0,"num_object_copies":9524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12573,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1201}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'15680","reported_seq":15966,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:38.819212+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:36:38.819212+0000","last_peered":"2024-09-10T20:36:38.819212+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:36:38.819212+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20185088,"num_objects":4928,"num_object_clones":0,"num_object_copies":9856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12275,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1442}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'15861","reported_seq":15994,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:38.813875+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:36:38.813875+0000","last_peered":"2024-09-10T20:36:38.813875+0000","last_clean":"2024-09-10T20:36:38.813875+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:36:38.813875+0000","last_undegraded":"2024-09-10T20:36:38.813875+0000","last_fullsized":"2024-09-10T20:36:38.813875+0000","mapping_epoch":34,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19968000,"num_objects":4875,"num_object_clones":0,"num_object_copies":9750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12932,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'15492","reported_seq":15692,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:39.173945+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:36:39.173945+0000","last_peered":"2024-09-10T20:36:39.173945+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:36:39.173945+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20262912,"num_objects":4947,"num_object_clones":0,"num_object_copies":9894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13485,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1450}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318026+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318026+0000","last_peered":"2024-09-10T20:36:30.318026+0000","last_clean":"2024-09-10T20:36:30.318026+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318026+0000","last_undegraded":"2024-09-10T20:36:30.318026+0000","last_fullsized":"2024-09-10T20:36:30.318026+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'15849","reported_seq":16065,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:38.852697+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:36:38.852697+0000","last_peered":"2024-09-10T20:36:38.852697+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:36:38.852697+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19993542,"num_objects":4883,"num_object_clones":0,"num_object_copies":9766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15847,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1317}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315256+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315256+0000","last_peered":"2024-09-10T20:36:30.315256+0000","last_clean":"2024-09-10T20:36:30.315256+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315256+0000","last_undegraded":"2024-09-10T20:36:30.315256+0000","last_fullsized":"2024-09-10T20:36:30.315256+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'15678","reported_seq":15870,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:39.031287+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:36:39.031287+0000","last_peered":"2024-09-10T20:36:39.031287+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:36:39.031287+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20017152,"num_objects":4887,"num_object_clones":0,"num_object_copies":9774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13506,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1281}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'15809","reported_seq":23179,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:39.190200+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:36:39.190200+0000","last_peered":"2024-09-10T20:36:39.190200+0000","last_clean":"2024-09-10T20:36:39.190200+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:36:39.190200+0000","last_undegraded":"2024-09-10T20:36:39.190200+0000","last_fullsized":"2024-09-10T20:36:39.190200+0000","mapping_epoch":48,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19886080,"num_objects":4855,"num_object_clones":0,"num_object_copies":9710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13595,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'15906","reported_seq":16162,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:38.870298+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:36:38.870298+0000","last_peered":"2024-09-10T20:36:38.870298+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:36:38.870298+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20168704,"num_objects":4924,"num_object_clones":0,"num_object_copies":9848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12629,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1332}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'15731","reported_seq":16514,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:38.822275+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:36:38.822275+0000","last_peered":"2024-09-10T20:36:38.822275+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:36:38.822275+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19877888,"num_objects":4853,"num_object_clones":0,"num_object_copies":9706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3218,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12425,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":317,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1635}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319607+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319607+0000","last_peered":"2024-09-10T20:36:30.319607+0000","last_clean":"2024-09-10T20:36:30.319607+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319607+0000","last_undegraded":"2024-09-10T20:36:30.319607+0000","last_fullsized":"2024-09-10T20:36:30.319607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'16078","reported_seq":16335,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:38.809864+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:36:38.809864+0000","last_peered":"2024-09-10T20:36:38.809864+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:36:38.809864+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20103168,"num_objects":4908,"num_object_clones":0,"num_object_copies":9816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12905,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1253}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'15715","reported_seq":22842,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:40.604603+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:36:40.604603+0000","last_peered":"2024-09-10T20:36:40.604603+0000","last_clean":"2024-09-10T20:36:40.604603+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:36:40.604603+0000","last_undegraded":"2024-09-10T20:36:40.604603+0000","last_fullsized":"2024-09-10T20:36:40.604603+0000","mapping_epoch":46,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19665379,"num_objects":4802,"num_object_clones":0,"num_object_copies":9604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12322,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":320337345,"num_objects":78211,"num_object_clones":0,"num_object_copies":156422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31735,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78211,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":217206,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11414,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":815505408,"data_stored":815439456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160810,"ondisk_log_size":160810,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182497,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":693808,"kb_used_data":292892,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93678032,"statfs":{"total":96636764160,"available":95926304768,"internally_reserved":0,"allocated":299921408,"data_stored":299659460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":410513066},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,10,0,0,2,2,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986269,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":593604,"kb_used_data":273140,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93778236,"statfs":{"total":96636764160,"available":96028913664,"internally_reserved":0,"allocated":279695360,"data_stored":279433417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328134314},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659216,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594300,"kb_used_data":233572,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93777540,"statfs":{"total":96636764160,"available":96028200960,"internally_reserved":0,"allocated":239177728,"data_stored":238864739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":369356462},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,2,5,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":299458560,"data_stored":299454947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":237342720,"data_stored":237288066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":278704128,"data_stored":278696443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:36:43.271 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:36:43.271 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'15768", 'reported_seq': 15990, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:38.810240+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:36:38.810240+0000', 'last_peered': '2024-09-10T20:36:38.810240+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:36:38.810240+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "43'5700", 'ondisk_log_start': "43'5700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20029440, 'num_objects': 4890, 'num_object_clones': 0, 'num_object_copies': 9780, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4890, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15768, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1349}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:43.271 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:36:43.271 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'15709", 'reported_seq': 15961, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:38.773202+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:36:38.773202+0000', 'last_peered': '2024-09-10T20:36:38.773202+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:36:38.773202+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "43'5700", 'ondisk_log_start': "43'5700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19505152, 'num_objects': 4762, 'num_object_clones': 0, 'num_object_copies': 9524, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4762, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12573, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1201}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:43.271 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:36:43.271 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'15680", 'reported_seq': 15966, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:38.819212+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:36:38.819212+0000', 'last_peered': '2024-09-10T20:36:38.819212+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:36:38.819212+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "43'5600", 'ondisk_log_start': "43'5600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20185088, 'num_objects': 4928, 'num_object_clones': 0, 'num_object_copies': 9856, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4928, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12275, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1442}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:43.271 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:36:43.271 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'15492", 'reported_seq': 15692, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:39.173945+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:36:39.173945+0000', 'last_peered': '2024-09-10T20:36:39.173945+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:36:39.173945+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "43'5400", 'ondisk_log_start': "43'5400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20262912, 'num_objects': 4947, 'num_object_clones': 0, 'num_object_copies': 9894, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4947, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13485, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1450}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:36:43.272 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:36:43.272 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'15849", 'reported_seq': 16065, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:38.852697+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:36:38.852697+0000', 'last_peered': '2024-09-10T20:36:38.852697+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:36:38.852697+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "43'5800", 'ondisk_log_start': "43'5800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19993542, 'num_objects': 4883, 'num_object_clones': 0, 'num_object_copies': 9766, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4883, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15847, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1317}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:43.272 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:36:43.272 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'15678", 'reported_seq': 15870, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:39.031287+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:36:39.031287+0000', 'last_peered': '2024-09-10T20:36:39.031287+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:36:39.031287+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "43'5600", 'ondisk_log_start': "43'5600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20017152, 'num_objects': 4887, 'num_object_clones': 0, 'num_object_copies': 9774, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4887, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13506, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1281}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:36:43.272 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:36:43.272 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'15906", 'reported_seq': 16162, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:38.870298+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:36:38.870298+0000', 'last_peered': '2024-09-10T20:36:38.870298+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:36:38.870298+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "43'5900", 'ondisk_log_start': "43'5900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20168704, 'num_objects': 4924, 'num_object_clones': 0, 'num_object_copies': 9848, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4924, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12629, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1332}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:43.272 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:36:43.272 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'15731", 'reported_seq': 16514, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:38.822275+0000', 'last_change': '2024-09-10T20:36:28.778006+0000', 'last_active': '2024-09-10T20:36:38.822275+0000', 'last_peered': '2024-09-10T20:36:38.822275+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:36:38.822275+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "43'5700", 'ondisk_log_start': "43'5700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19877888, 'num_objects': 4853, 'num_object_clones': 0, 'num_object_copies': 9706, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3218, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4853, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12425, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 317, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1635}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:43.272 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:36:43.272 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'16078", 'reported_seq': 16335, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:38.809864+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:36:38.809864+0000', 'last_peered': '2024-09-10T20:36:38.809864+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:36:38.809864+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "43'6000", 'ondisk_log_start': "43'6000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20103168, 'num_objects': 4908, 'num_object_clones': 0, 'num_object_copies': 9816, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4908, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12905, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1253}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:43.273 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:36:43.992 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:43.992 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31735/156426 objects degraded (20.288%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":515,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":78213,"data_bytes":320337364,"bytes_used":1926873088,"bytes_avail":287983419392,"bytes_total":289910292480,"degraded_objects":31735,"degraded_total":156426,"degraded_ratio":0.2028754810581361,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":268,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":379},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:36:44.024 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:36:44.025 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:36:44.596 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:44.596 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:36:44.623 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":300,"stamp":"2024-09-10T20:36:42.990947+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":320337364,"num_objects":78213,"num_object_clones":0,"num_object_copies":156426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31735,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78213,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":217208,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11416,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160813,"ondisk_log_size":160813,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1881712,"kb_used_data":799604,"kb_used_omap":12,"kb_used_meta":1082035,"kb_avail":281233808,"statfs":{"total":289910292480,"available":287983419392,"internally_reserved":0,"allocated":818794496,"data_stored":817957616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1108003842},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,10,2,3,8,7,9,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18661376,"num_objects":-4556,"num_object_clones":0,"num_object_copies":-9112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-266,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4556,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007343"},"pg_stats":[{"pgid":"2.f","version":"49'15768","reported_seq":15990,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:38.810240+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:36:38.810240+0000","last_peered":"2024-09-10T20:36:38.810240+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:36:38.810240+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20029440,"num_objects":4890,"num_object_clones":0,"num_object_copies":9780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15768,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1349}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'16029","reported_seq":16149,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:39.161357+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:36:39.161357+0000","last_peered":"2024-09-10T20:36:39.161357+0000","last_clean":"2024-09-10T20:36:39.161357+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:36:39.161357+0000","last_undegraded":"2024-09-10T20:36:39.161357+0000","last_fullsized":"2024-09-10T20:36:39.161357+0000","mapping_epoch":35,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20123648,"num_objects":4913,"num_object_clones":0,"num_object_copies":9826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14215,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'15873","reported_seq":23230,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:40.587732+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:36:40.587732+0000","last_peered":"2024-09-10T20:36:40.587732+0000","last_clean":"2024-09-10T20:36:40.587732+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:36:40.587732+0000","last_undegraded":"2024-09-10T20:36:40.587732+0000","last_fullsized":"2024-09-10T20:36:40.587732+0000","mapping_epoch":44,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19832856,"num_objects":4843,"num_object_clones":0,"num_object_copies":9686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4843,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14001,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'15664","reported_seq":15754,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:39.078967+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:36:39.078967+0000","last_peered":"2024-09-10T20:36:39.078967+0000","last_clean":"2024-09-10T20:36:39.078967+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:36:39.078967+0000","last_undegraded":"2024-09-10T20:36:39.078967+0000","last_fullsized":"2024-09-10T20:36:39.078967+0000","mapping_epoch":35,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20234240,"num_objects":4940,"num_object_clones":0,"num_object_copies":9880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15664,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'16068","reported_seq":16204,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:38.809408+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:36:38.809408+0000","last_peered":"2024-09-10T20:36:38.809408+0000","last_clean":"2024-09-10T20:36:38.809408+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:36:38.809408+0000","last_undegraded":"2024-09-10T20:36:38.809408+0000","last_fullsized":"2024-09-10T20:36:38.809408+0000","mapping_epoch":34,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20484096,"num_objects":5001,"num_object_clones":0,"num_object_copies":10002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13064,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'15709","reported_seq":15961,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:38.773202+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:36:38.773202+0000","last_peered":"2024-09-10T20:36:38.773202+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:36:38.773202+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19505152,"num_objects":4762,"num_object_clones":0,"num_object_copies":9524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12573,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1201}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'15680","reported_seq":15966,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:38.819212+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:36:38.819212+0000","last_peered":"2024-09-10T20:36:38.819212+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:36:38.819212+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20185088,"num_objects":4928,"num_object_clones":0,"num_object_copies":9856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12275,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1442}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'15861","reported_seq":15994,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:38.813875+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:36:38.813875+0000","last_peered":"2024-09-10T20:36:38.813875+0000","last_clean":"2024-09-10T20:36:38.813875+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:36:38.813875+0000","last_undegraded":"2024-09-10T20:36:38.813875+0000","last_fullsized":"2024-09-10T20:36:38.813875+0000","mapping_epoch":34,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19968000,"num_objects":4875,"num_object_clones":0,"num_object_copies":9750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12932,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'15492","reported_seq":15692,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:39.173945+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:36:39.173945+0000","last_peered":"2024-09-10T20:36:39.173945+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:36:39.173945+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20262912,"num_objects":4947,"num_object_clones":0,"num_object_copies":9894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13485,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1450}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318026+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318026+0000","last_peered":"2024-09-10T20:36:30.318026+0000","last_clean":"2024-09-10T20:36:30.318026+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318026+0000","last_undegraded":"2024-09-10T20:36:30.318026+0000","last_fullsized":"2024-09-10T20:36:30.318026+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'15849","reported_seq":16065,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:38.852697+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:36:38.852697+0000","last_peered":"2024-09-10T20:36:38.852697+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:36:38.852697+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19993542,"num_objects":4883,"num_object_clones":0,"num_object_copies":9766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15847,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1317}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315256+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315256+0000","last_peered":"2024-09-10T20:36:30.315256+0000","last_clean":"2024-09-10T20:36:30.315256+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315256+0000","last_undegraded":"2024-09-10T20:36:30.315256+0000","last_fullsized":"2024-09-10T20:36:30.315256+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'15678","reported_seq":15870,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:39.031287+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:36:39.031287+0000","last_peered":"2024-09-10T20:36:39.031287+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:36:39.031287+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20017152,"num_objects":4887,"num_object_clones":0,"num_object_copies":9774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13506,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1281}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'15809","reported_seq":23179,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:39.190200+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:36:39.190200+0000","last_peered":"2024-09-10T20:36:39.190200+0000","last_clean":"2024-09-10T20:36:39.190200+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:36:39.190200+0000","last_undegraded":"2024-09-10T20:36:39.190200+0000","last_fullsized":"2024-09-10T20:36:39.190200+0000","mapping_epoch":48,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19886080,"num_objects":4855,"num_object_clones":0,"num_object_copies":9710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13595,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'15906","reported_seq":16162,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:38.870298+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:36:38.870298+0000","last_peered":"2024-09-10T20:36:38.870298+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:36:38.870298+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20168704,"num_objects":4924,"num_object_clones":0,"num_object_copies":9848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12629,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1332}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'15731","reported_seq":16514,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:38.822275+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:36:38.822275+0000","last_peered":"2024-09-10T20:36:38.822275+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:36:38.822275+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19877888,"num_objects":4853,"num_object_clones":0,"num_object_copies":9706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3218,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12425,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":317,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1635}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319607+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319607+0000","last_peered":"2024-09-10T20:36:30.319607+0000","last_clean":"2024-09-10T20:36:30.319607+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319607+0000","last_undegraded":"2024-09-10T20:36:30.319607+0000","last_fullsized":"2024-09-10T20:36:30.319607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'16078","reported_seq":16335,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:38.809864+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:36:38.809864+0000","last_peered":"2024-09-10T20:36:38.809864+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:36:38.809864+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20103168,"num_objects":4908,"num_object_clones":0,"num_object_copies":9816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12905,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1253}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'15715","reported_seq":22842,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:40.604603+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:36:40.604603+0000","last_peered":"2024-09-10T20:36:40.604603+0000","last_clean":"2024-09-10T20:36:40.604603+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:36:40.604603+0000","last_undegraded":"2024-09-10T20:36:40.604603+0000","last_fullsized":"2024-09-10T20:36:40.604603+0000","mapping_epoch":46,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19665379,"num_objects":4802,"num_object_clones":0,"num_object_copies":9604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12322,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":320337345,"num_objects":78211,"num_object_clones":0,"num_object_copies":156422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31735,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78211,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":217206,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11414,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":815505408,"data_stored":815439456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160810,"ondisk_log_size":160810,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182497,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":693808,"kb_used_data":292892,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93678032,"statfs":{"total":96636764160,"available":95926304768,"internally_reserved":0,"allocated":299921408,"data_stored":299659460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":410513066},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,10,0,0,2,2,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986269,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":593604,"kb_used_data":273140,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93778236,"statfs":{"total":96636764160,"available":96028913664,"internally_reserved":0,"allocated":279695360,"data_stored":279433417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328134314},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659216,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594300,"kb_used_data":233572,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93777540,"statfs":{"total":96636764160,"available":96028200960,"internally_reserved":0,"allocated":239177728,"data_stored":238864739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":369356462},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,2,5,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":299458560,"data_stored":299454947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":237342720,"data_stored":237288066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":278704128,"data_stored":278696443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:36:47.625 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:36:48.204 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:48.205 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:36:48.233 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-09-10T20:36:46.992433+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":310236628,"num_objects":75747,"num_object_clones":0,"num_object_copies":151494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31638,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75747,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":219674,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11513,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160679,"ondisk_log_size":160679,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1880480,"kb_used_data":780304,"kb_used_omap":12,"kb_used_meta":1100083,"kb_avail":281235040,"statfs":{"total":289910292480,"available":287984680960,"internally_reserved":0,"allocated":799031296,"data_stored":798194416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1126484994},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,4,12,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20975616,"num_objects":-5121,"num_object_clones":0,"num_object_copies":-10242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-211,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5121,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007449"},"pg_stats":[{"pgid":"2.f","version":"49'15917","reported_seq":16138,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:43.956109+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:36:43.956109+0000","last_peered":"2024-09-10T20:36:43.956109+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:36:43.956109+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19419136,"num_objects":4741,"num_object_clones":0,"num_object_copies":9482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15917,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1200}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'16175","reported_seq":16295,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:44.112749+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:36:44.112749+0000","last_peered":"2024-09-10T20:36:44.112749+0000","last_clean":"2024-09-10T20:36:44.112749+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:36:44.112749+0000","last_undegraded":"2024-09-10T20:36:44.112749+0000","last_fullsized":"2024-09-10T20:36:44.112749+0000","mapping_epoch":35,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19525632,"num_objects":4767,"num_object_clones":0,"num_object_copies":9534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14361,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'16038","reported_seq":23395,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:45.685237+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:36:45.685237+0000","last_peered":"2024-09-10T20:36:45.685237+0000","last_clean":"2024-09-10T20:36:45.685237+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:36:45.685237+0000","last_undegraded":"2024-09-10T20:36:45.685237+0000","last_fullsized":"2024-09-10T20:36:45.685237+0000","mapping_epoch":44,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19157016,"num_objects":4678,"num_object_clones":0,"num_object_copies":9356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4678,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14166,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'15806","reported_seq":15895,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:44.085767+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:36:44.085767+0000","last_peered":"2024-09-10T20:36:44.085767+0000","last_clean":"2024-09-10T20:36:44.085767+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:36:44.085767+0000","last_undegraded":"2024-09-10T20:36:44.085767+0000","last_fullsized":"2024-09-10T20:36:44.085767+0000","mapping_epoch":35,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19652608,"num_objects":4798,"num_object_clones":0,"num_object_copies":9596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'16235","reported_seq":16371,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:43.918581+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:36:43.918581+0000","last_peered":"2024-09-10T20:36:43.918581+0000","last_clean":"2024-09-10T20:36:43.918581+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:36:43.918581+0000","last_undegraded":"2024-09-10T20:36:43.918581+0000","last_fullsized":"2024-09-10T20:36:43.918581+0000","mapping_epoch":34,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19800064,"num_objects":4834,"num_object_clones":0,"num_object_copies":9668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4834,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13231,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'15858","reported_seq":16110,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:43.878378+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:36:43.878378+0000","last_peered":"2024-09-10T20:36:43.878378+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:36:43.878378+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18894848,"num_objects":4613,"num_object_clones":0,"num_object_copies":9226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12722,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1052}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'15822","reported_seq":16108,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:43.887040+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:36:43.887040+0000","last_peered":"2024-09-10T20:36:43.887040+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:36:43.887040+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19603456,"num_objects":4786,"num_object_clones":0,"num_object_copies":9572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12417,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1300}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'16021","reported_seq":16155,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:43.829106+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:36:43.829106+0000","last_peered":"2024-09-10T20:36:43.829106+0000","last_clean":"2024-09-10T20:36:43.829106+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:36:43.829106+0000","last_undegraded":"2024-09-10T20:36:43.829106+0000","last_fullsized":"2024-09-10T20:36:43.829106+0000","mapping_epoch":34,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19312640,"num_objects":4715,"num_object_clones":0,"num_object_copies":9430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13092,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'15642","reported_seq":15842,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:44.135141+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:36:44.135141+0000","last_peered":"2024-09-10T20:36:44.135141+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:36:44.135141+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19648512,"num_objects":4797,"num_object_clones":0,"num_object_copies":9594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13635,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1300}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318026+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318026+0000","last_peered":"2024-09-10T20:36:30.318026+0000","last_clean":"2024-09-10T20:36:30.318026+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318026+0000","last_undegraded":"2024-09-10T20:36:30.318026+0000","last_fullsized":"2024-09-10T20:36:30.318026+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'16014","reported_seq":16229,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:43.940385+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:36:43.940385+0000","last_peered":"2024-09-10T20:36:43.940385+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:36:43.940385+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19317702,"num_objects":4718,"num_object_clones":0,"num_object_copies":9436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16012,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1152}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315256+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315256+0000","last_peered":"2024-09-10T20:36:30.315256+0000","last_clean":"2024-09-10T20:36:30.315256+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315256+0000","last_undegraded":"2024-09-10T20:36:30.315256+0000","last_fullsized":"2024-09-10T20:36:30.315256+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'15829","reported_seq":16021,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:44.142534+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:36:44.142534+0000","last_peered":"2024-09-10T20:36:44.142534+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:36:44.142534+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19398656,"num_objects":4736,"num_object_clones":0,"num_object_copies":9472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13657,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1130}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'15964","reported_seq":23334,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:44.142777+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:36:44.142777+0000","last_peered":"2024-09-10T20:36:44.142777+0000","last_clean":"2024-09-10T20:36:44.142777+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:36:44.142777+0000","last_undegraded":"2024-09-10T20:36:44.142777+0000","last_fullsized":"2024-09-10T20:36:44.142777+0000","mapping_epoch":48,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19251200,"num_objects":4700,"num_object_clones":0,"num_object_copies":9400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13750,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'16058","reported_seq":16315,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:43.925508+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:36:43.925508+0000","last_peered":"2024-09-10T20:36:43.925508+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:36:43.925508+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19546112,"num_objects":4772,"num_object_clones":0,"num_object_copies":9544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12781,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1180}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'15894","reported_seq":16871,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:43.919286+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:36:43.919286+0000","last_peered":"2024-09-10T20:36:43.919286+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:36:43.919286+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19210240,"num_objects":4690,"num_object_clones":0,"num_object_copies":9380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3121,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12588,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":414,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1569}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319607+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319607+0000","last_peered":"2024-09-10T20:36:30.319607+0000","last_clean":"2024-09-10T20:36:30.319607+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319607+0000","last_undegraded":"2024-09-10T20:36:30.319607+0000","last_fullsized":"2024-09-10T20:36:30.319607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'16237","reported_seq":16493,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:43.953512+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:36:43.953512+0000","last_peered":"2024-09-10T20:36:43.953512+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:36:43.953512+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19451904,"num_objects":4749,"num_object_clones":0,"num_object_copies":9498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13064,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'15866","reported_seq":22993,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:45.716129+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:36:45.716129+0000","last_peered":"2024-09-10T20:36:45.716129+0000","last_clean":"2024-09-10T20:36:45.716129+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:36:45.716129+0000","last_undegraded":"2024-09-10T20:36:45.716129+0000","last_fullsized":"2024-09-10T20:36:45.716129+0000","mapping_epoch":46,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19046883,"num_objects":4651,"num_object_clones":0,"num_object_copies":9302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12473,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":310236609,"num_objects":75745,"num_object_clones":0,"num_object_copies":151490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31638,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75745,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":219672,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11511,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":795717632,"data_stored":795651680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160676,"ondisk_log_size":160676,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182498,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":678244,"kb_used_data":285628,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93693596,"statfs":{"total":96636764160,"available":95942242304,"internally_reserved":0,"allocated":292483072,"data_stored":292221124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":401993386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,1,7,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986270,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592788,"kb_used_data":268156,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93779052,"statfs":{"total":96636764160,"available":96029749248,"internally_reserved":0,"allocated":274591744,"data_stored":274329801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332394154},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659217,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609448,"kb_used_data":226520,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93762392,"statfs":{"total":96636764160,"available":96012689408,"internally_reserved":0,"allocated":231956480,"data_stored":231643491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":392097454},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":291905536,"data_stored":291901923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":230117376,"data_stored":230062722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":273694720,"data_stored":273687035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:36:48.235 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:36:48.822 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:48.825 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:36:48.858 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-09-10T20:36:46.992433+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":310236628,"num_objects":75747,"num_object_clones":0,"num_object_copies":151494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31638,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75747,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":219674,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11513,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160679,"ondisk_log_size":160679,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1880480,"kb_used_data":780304,"kb_used_omap":12,"kb_used_meta":1100083,"kb_avail":281235040,"statfs":{"total":289910292480,"available":287984680960,"internally_reserved":0,"allocated":799031296,"data_stored":798194416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1126484994},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,4,12,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20975616,"num_objects":-5121,"num_object_clones":0,"num_object_copies":-10242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-211,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5121,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007449"},"pg_stats":[{"pgid":"2.f","version":"49'15917","reported_seq":16138,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:43.956109+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:36:43.956109+0000","last_peered":"2024-09-10T20:36:43.956109+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:36:43.956109+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19419136,"num_objects":4741,"num_object_clones":0,"num_object_copies":9482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15917,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1200}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'16175","reported_seq":16295,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:44.112749+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:36:44.112749+0000","last_peered":"2024-09-10T20:36:44.112749+0000","last_clean":"2024-09-10T20:36:44.112749+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:36:44.112749+0000","last_undegraded":"2024-09-10T20:36:44.112749+0000","last_fullsized":"2024-09-10T20:36:44.112749+0000","mapping_epoch":35,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19525632,"num_objects":4767,"num_object_clones":0,"num_object_copies":9534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14361,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'16038","reported_seq":23395,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:45.685237+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:36:45.685237+0000","last_peered":"2024-09-10T20:36:45.685237+0000","last_clean":"2024-09-10T20:36:45.685237+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:36:45.685237+0000","last_undegraded":"2024-09-10T20:36:45.685237+0000","last_fullsized":"2024-09-10T20:36:45.685237+0000","mapping_epoch":44,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19157016,"num_objects":4678,"num_object_clones":0,"num_object_copies":9356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4678,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14166,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'15806","reported_seq":15895,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:44.085767+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:36:44.085767+0000","last_peered":"2024-09-10T20:36:44.085767+0000","last_clean":"2024-09-10T20:36:44.085767+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:36:44.085767+0000","last_undegraded":"2024-09-10T20:36:44.085767+0000","last_fullsized":"2024-09-10T20:36:44.085767+0000","mapping_epoch":35,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19652608,"num_objects":4798,"num_object_clones":0,"num_object_copies":9596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'16235","reported_seq":16371,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:43.918581+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:36:43.918581+0000","last_peered":"2024-09-10T20:36:43.918581+0000","last_clean":"2024-09-10T20:36:43.918581+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:36:43.918581+0000","last_undegraded":"2024-09-10T20:36:43.918581+0000","last_fullsized":"2024-09-10T20:36:43.918581+0000","mapping_epoch":34,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19800064,"num_objects":4834,"num_object_clones":0,"num_object_copies":9668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4834,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13231,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'15858","reported_seq":16110,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:43.878378+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:36:43.878378+0000","last_peered":"2024-09-10T20:36:43.878378+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:36:43.878378+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18894848,"num_objects":4613,"num_object_clones":0,"num_object_copies":9226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12722,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1052}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'15822","reported_seq":16108,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:43.887040+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:36:43.887040+0000","last_peered":"2024-09-10T20:36:43.887040+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:36:43.887040+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19603456,"num_objects":4786,"num_object_clones":0,"num_object_copies":9572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12417,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1300}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'16021","reported_seq":16155,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:43.829106+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:36:43.829106+0000","last_peered":"2024-09-10T20:36:43.829106+0000","last_clean":"2024-09-10T20:36:43.829106+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:36:43.829106+0000","last_undegraded":"2024-09-10T20:36:43.829106+0000","last_fullsized":"2024-09-10T20:36:43.829106+0000","mapping_epoch":34,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19312640,"num_objects":4715,"num_object_clones":0,"num_object_copies":9430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13092,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'15642","reported_seq":15842,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:44.135141+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:36:44.135141+0000","last_peered":"2024-09-10T20:36:44.135141+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:36:44.135141+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19648512,"num_objects":4797,"num_object_clones":0,"num_object_copies":9594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13635,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1300}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318026+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318026+0000","last_peered":"2024-09-10T20:36:30.318026+0000","last_clean":"2024-09-10T20:36:30.318026+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318026+0000","last_undegraded":"2024-09-10T20:36:30.318026+0000","last_fullsized":"2024-09-10T20:36:30.318026+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'16014","reported_seq":16229,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:43.940385+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:36:43.940385+0000","last_peered":"2024-09-10T20:36:43.940385+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:36:43.940385+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19317702,"num_objects":4718,"num_object_clones":0,"num_object_copies":9436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16012,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1152}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315256+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315256+0000","last_peered":"2024-09-10T20:36:30.315256+0000","last_clean":"2024-09-10T20:36:30.315256+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315256+0000","last_undegraded":"2024-09-10T20:36:30.315256+0000","last_fullsized":"2024-09-10T20:36:30.315256+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'15829","reported_seq":16021,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:44.142534+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:36:44.142534+0000","last_peered":"2024-09-10T20:36:44.142534+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:36:44.142534+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19398656,"num_objects":4736,"num_object_clones":0,"num_object_copies":9472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13657,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1130}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'15964","reported_seq":23334,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:44.142777+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:36:44.142777+0000","last_peered":"2024-09-10T20:36:44.142777+0000","last_clean":"2024-09-10T20:36:44.142777+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:36:44.142777+0000","last_undegraded":"2024-09-10T20:36:44.142777+0000","last_fullsized":"2024-09-10T20:36:44.142777+0000","mapping_epoch":48,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19251200,"num_objects":4700,"num_object_clones":0,"num_object_copies":9400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13750,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'16058","reported_seq":16315,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:43.925508+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:36:43.925508+0000","last_peered":"2024-09-10T20:36:43.925508+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:36:43.925508+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19546112,"num_objects":4772,"num_object_clones":0,"num_object_copies":9544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12781,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1180}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'15894","reported_seq":16871,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:43.919286+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:36:43.919286+0000","last_peered":"2024-09-10T20:36:43.919286+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:36:43.919286+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19210240,"num_objects":4690,"num_object_clones":0,"num_object_copies":9380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3121,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12588,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":414,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1569}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319607+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319607+0000","last_peered":"2024-09-10T20:36:30.319607+0000","last_clean":"2024-09-10T20:36:30.319607+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319607+0000","last_undegraded":"2024-09-10T20:36:30.319607+0000","last_fullsized":"2024-09-10T20:36:30.319607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'16237","reported_seq":16493,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:43.953512+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:36:43.953512+0000","last_peered":"2024-09-10T20:36:43.953512+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:36:43.953512+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19451904,"num_objects":4749,"num_object_clones":0,"num_object_copies":9498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13064,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'15866","reported_seq":22993,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:45.716129+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:36:45.716129+0000","last_peered":"2024-09-10T20:36:45.716129+0000","last_clean":"2024-09-10T20:36:45.716129+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:36:45.716129+0000","last_undegraded":"2024-09-10T20:36:45.716129+0000","last_fullsized":"2024-09-10T20:36:45.716129+0000","mapping_epoch":46,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19046883,"num_objects":4651,"num_object_clones":0,"num_object_copies":9302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12473,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":310236609,"num_objects":75745,"num_object_clones":0,"num_object_copies":151490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31638,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75745,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":219672,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11511,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":795717632,"data_stored":795651680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160676,"ondisk_log_size":160676,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182498,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":678244,"kb_used_data":285628,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93693596,"statfs":{"total":96636764160,"available":95942242304,"internally_reserved":0,"allocated":292483072,"data_stored":292221124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":401993386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,1,7,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986270,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592788,"kb_used_data":268156,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93779052,"statfs":{"total":96636764160,"available":96029749248,"internally_reserved":0,"allocated":274591744,"data_stored":274329801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332394154},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659217,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609448,"kb_used_data":226520,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93762392,"statfs":{"total":96636764160,"available":96012689408,"internally_reserved":0,"allocated":231956480,"data_stored":231643491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":392097454},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":291905536,"data_stored":291901923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":230117376,"data_stored":230062722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":273694720,"data_stored":273687035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:36:48.860 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:36:48.860 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'15917", 'reported_seq': 16138, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:43.956109+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:36:43.956109+0000', 'last_peered': '2024-09-10T20:36:43.956109+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:36:43.956109+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "43'5900", 'ondisk_log_start': "43'5900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19419136, 'num_objects': 4741, 'num_object_clones': 0, 'num_object_copies': 9482, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4741, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15917, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1200}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:48.860 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:36:48.860 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'15858", 'reported_seq': 16110, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:43.878378+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:36:43.878378+0000', 'last_peered': '2024-09-10T20:36:43.878378+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:36:43.878378+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "43'5800", 'ondisk_log_start': "43'5800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18894848, 'num_objects': 4613, 'num_object_clones': 0, 'num_object_copies': 9226, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4613, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12722, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1052}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:48.860 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:36:48.860 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'15822", 'reported_seq': 16108, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:43.887040+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:36:43.887040+0000', 'last_peered': '2024-09-10T20:36:43.887040+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:36:43.887040+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "43'5800", 'ondisk_log_start': "43'5800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19603456, 'num_objects': 4786, 'num_object_clones': 0, 'num_object_copies': 9572, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4786, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12417, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1300}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:48.860 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:36:48.860 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'15642", 'reported_seq': 15842, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:44.135141+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:36:44.135141+0000', 'last_peered': '2024-09-10T20:36:44.135141+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:36:44.135141+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "43'5600", 'ondisk_log_start': "43'5600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19648512, 'num_objects': 4797, 'num_object_clones': 0, 'num_object_copies': 9594, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4797, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13635, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1300}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:36:48.861 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:36:48.861 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'16014", 'reported_seq': 16229, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:43.940385+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:36:43.940385+0000', 'last_peered': '2024-09-10T20:36:43.940385+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:36:43.940385+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "43'6000", 'ondisk_log_start': "43'6000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19317702, 'num_objects': 4718, 'num_object_clones': 0, 'num_object_copies': 9436, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4718, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16012, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1152}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:48.861 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:36:48.861 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'15829", 'reported_seq': 16021, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:44.142534+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:36:44.142534+0000', 'last_peered': '2024-09-10T20:36:44.142534+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:36:44.142534+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "43'5800", 'ondisk_log_start': "43'5800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19398656, 'num_objects': 4736, 'num_object_clones': 0, 'num_object_copies': 9472, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4736, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13657, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1130}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:36:48.861 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:36:48.861 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'16058", 'reported_seq': 16315, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:43.925508+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:36:43.925508+0000', 'last_peered': '2024-09-10T20:36:43.925508+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:36:43.925508+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "43'6000", 'ondisk_log_start': "43'6000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19546112, 'num_objects': 4772, 'num_object_clones': 0, 'num_object_copies': 9544, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4772, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12781, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1180}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:48.861 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:36:48.861 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'15894", 'reported_seq': 16871, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:43.919286+0000', 'last_change': '2024-09-10T20:36:28.778006+0000', 'last_active': '2024-09-10T20:36:43.919286+0000', 'last_peered': '2024-09-10T20:36:43.919286+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:36:43.919286+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "43'5800", 'ondisk_log_start': "43'5800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19210240, 'num_objects': 4690, 'num_object_clones': 0, 'num_object_copies': 9380, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3121, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4690, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12588, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 414, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1569}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:48.861 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:36:48.861 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'16237", 'reported_seq': 16493, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:43.953512+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:36:43.953512+0000', 'last_peered': '2024-09-10T20:36:43.953512+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:36:43.953512+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "43'6200", 'ondisk_log_start': "43'6200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19451904, 'num_objects': 4749, 'num_object_clones': 0, 'num_object_copies': 9498, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4749, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13064, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:48.862 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:36:49.562 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:49.562 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31525/148908 objects degraded (21.171%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":521,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":74454,"data_bytes":304940500,"bytes_used":1922666496,"bytes_avail":287987625984,"bytes_total":289910292480,"degraded_objects":31525,"degraded_total":148908,"degraded_ratio":0.2117079001799769,"recovering_objects_per_sec":26,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":324,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":508},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:36:49.587 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:36:49.587 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:36:50.153 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:50.156 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:36:50.176 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":303,"stamp":"2024-09-10T20:36:48.993172+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":304940500,"num_objects":74454,"num_object_clones":0,"num_object_copies":148908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31525,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74454,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":220967,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11626,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160872,"ondisk_log_size":160872,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1877604,"kb_used_data":773288,"kb_used_omap":12,"kb_used_meta":1104243,"kb_avail":281237916,"statfs":{"total":289910292480,"available":287987625984,"internally_reserved":0,"allocated":791846912,"data_stored":791010032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1130744834},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,6,6,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25026560,"num_objects":-6110,"num_object_clones":0,"num_object_copies":-12220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-324,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6110,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007710"},"pg_stats":[{"pgid":"2.f","version":"49'16074","reported_seq":16296,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:48.973987+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:36:48.973987+0000","last_peered":"2024-09-10T20:36:48.973987+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:36:48.973987+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18776064,"num_objects":4584,"num_object_clones":0,"num_object_copies":9168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16074,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1043}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'16175","reported_seq":16295,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:44.112749+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:36:44.112749+0000","last_peered":"2024-09-10T20:36:44.112749+0000","last_clean":"2024-09-10T20:36:44.112749+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:36:44.112749+0000","last_undegraded":"2024-09-10T20:36:44.112749+0000","last_fullsized":"2024-09-10T20:36:44.112749+0000","mapping_epoch":35,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19525632,"num_objects":4767,"num_object_clones":0,"num_object_copies":9534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14361,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'16038","reported_seq":23395,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:45.685237+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:36:45.685237+0000","last_peered":"2024-09-10T20:36:45.685237+0000","last_clean":"2024-09-10T20:36:45.685237+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:36:45.685237+0000","last_undegraded":"2024-09-10T20:36:45.685237+0000","last_fullsized":"2024-09-10T20:36:45.685237+0000","mapping_epoch":44,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19157016,"num_objects":4678,"num_object_clones":0,"num_object_copies":9356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4678,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14166,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'15806","reported_seq":15895,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:44.085767+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:36:44.085767+0000","last_peered":"2024-09-10T20:36:44.085767+0000","last_clean":"2024-09-10T20:36:44.085767+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:36:44.085767+0000","last_undegraded":"2024-09-10T20:36:44.085767+0000","last_fullsized":"2024-09-10T20:36:44.085767+0000","mapping_epoch":35,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19652608,"num_objects":4798,"num_object_clones":0,"num_object_copies":9596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'16398","reported_seq":16534,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:48.837264+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:36:48.837264+0000","last_peered":"2024-09-10T20:36:48.837264+0000","last_clean":"2024-09-10T20:36:48.837264+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:36:48.837264+0000","last_undegraded":"2024-09-10T20:36:48.837264+0000","last_fullsized":"2024-09-10T20:36:48.837264+0000","mapping_epoch":34,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19132416,"num_objects":4671,"num_object_clones":0,"num_object_copies":9342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13394,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'15985","reported_seq":16237,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:48.962436+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:36:48.962436+0000","last_peered":"2024-09-10T20:36:48.962436+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:36:48.962436+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18374656,"num_objects":4486,"num_object_clones":0,"num_object_copies":8972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12849,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":925}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'15956","reported_seq":16242,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:48.875262+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:36:48.875262+0000","last_peered":"2024-09-10T20:36:48.875262+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:36:48.875262+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19054592,"num_objects":4652,"num_object_clones":0,"num_object_copies":9304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12551,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1166}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'16164","reported_seq":16298,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:48.940166+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:36:48.940166+0000","last_peered":"2024-09-10T20:36:48.940166+0000","last_clean":"2024-09-10T20:36:48.940166+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:36:48.940166+0000","last_undegraded":"2024-09-10T20:36:48.940166+0000","last_fullsized":"2024-09-10T20:36:48.940166+0000","mapping_epoch":34,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18726912,"num_objects":4572,"num_object_clones":0,"num_object_copies":9144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13235,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'15642","reported_seq":15842,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:44.135141+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:36:44.135141+0000","last_peered":"2024-09-10T20:36:44.135141+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:36:44.135141+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19648512,"num_objects":4797,"num_object_clones":0,"num_object_copies":9594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13635,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1300}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318026+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318026+0000","last_peered":"2024-09-10T20:36:30.318026+0000","last_clean":"2024-09-10T20:36:30.318026+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318026+0000","last_undegraded":"2024-09-10T20:36:30.318026+0000","last_fullsized":"2024-09-10T20:36:30.318026+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'16161","reported_seq":16377,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:48.749378+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:36:48.749378+0000","last_peered":"2024-09-10T20:36:48.749378+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:36:48.749378+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18715590,"num_objects":4571,"num_object_clones":0,"num_object_copies":9142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16159,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1005}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315256+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315256+0000","last_peered":"2024-09-10T20:36:30.315256+0000","last_clean":"2024-09-10T20:36:30.315256+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315256+0000","last_undegraded":"2024-09-10T20:36:30.315256+0000","last_fullsized":"2024-09-10T20:36:30.315256+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'15829","reported_seq":16021,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:44.142534+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:36:44.142534+0000","last_peered":"2024-09-10T20:36:44.142534+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:36:44.142534+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19398656,"num_objects":4736,"num_object_clones":0,"num_object_copies":9472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13657,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1130}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'15964","reported_seq":23334,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:44.142777+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:36:44.142777+0000","last_peered":"2024-09-10T20:36:44.142777+0000","last_clean":"2024-09-10T20:36:44.142777+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:36:44.142777+0000","last_undegraded":"2024-09-10T20:36:44.142777+0000","last_fullsized":"2024-09-10T20:36:44.142777+0000","mapping_epoch":48,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19251200,"num_objects":4700,"num_object_clones":0,"num_object_copies":9400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13750,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'16202","reported_seq":16459,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:48.929035+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:36:48.929035+0000","last_peered":"2024-09-10T20:36:48.929035+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:36:48.929035+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18956288,"num_objects":4628,"num_object_clones":0,"num_object_copies":9256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12925,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1036}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'16027","reported_seq":17228,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:48.924306+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:36:48.924306+0000","last_peered":"2024-09-10T20:36:48.924306+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:36:48.924306+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18665472,"num_objects":4557,"num_object_clones":0,"num_object_copies":9114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3008,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4557,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12721,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1549}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319607+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319607+0000","last_peered":"2024-09-10T20:36:30.319607+0000","last_clean":"2024-09-10T20:36:30.319607+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319607+0000","last_undegraded":"2024-09-10T20:36:30.319607+0000","last_fullsized":"2024-09-10T20:36:30.319607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'16382","reported_seq":16639,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:48.858472+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:36:48.858472+0000","last_peered":"2024-09-10T20:36:48.858472+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:36:48.858472+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18857984,"num_objects":4604,"num_object_clones":0,"num_object_copies":9208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13209,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":949}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'15866","reported_seq":22993,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:45.716129+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:36:45.716129+0000","last_peered":"2024-09-10T20:36:45.716129+0000","last_clean":"2024-09-10T20:36:45.716129+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:36:45.716129+0000","last_undegraded":"2024-09-10T20:36:45.716129+0000","last_fullsized":"2024-09-10T20:36:45.716129+0000","mapping_epoch":46,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19046883,"num_objects":4651,"num_object_clones":0,"num_object_copies":9302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12473,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":304940481,"num_objects":74452,"num_object_clones":0,"num_object_copies":148904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31525,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74452,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":220965,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11624,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":788557824,"data_stored":788491872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160869,"ondisk_log_size":160869,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182499,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":675368,"kb_used_data":278612,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93696472,"statfs":{"total":96636764160,"available":95945187328,"internally_reserved":0,"allocated":285298688,"data_stored":285036740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,1,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986270,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592788,"kb_used_data":268156,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93779052,"statfs":{"total":96636764160,"available":96029749248,"internally_reserved":0,"allocated":274591744,"data_stored":274329801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332394154},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659217,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609448,"kb_used_data":226520,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93762392,"statfs":{"total":96636764160,"available":96012689408,"internally_reserved":0,"allocated":231956480,"data_stored":231643491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":392097454},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":284745728,"data_stored":284742115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":230117376,"data_stored":230062722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":273694720,"data_stored":273687035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:36:53.179 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:36:53.764 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:53.766 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:36:53.789 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2024-09-10T20:36:52.995726+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":300660180,"num_objects":73409,"num_object_clones":0,"num_object_copies":146818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31525,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73409,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":222012,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11626,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160817,"ondisk_log_size":160817,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1897392,"kb_used_data":761136,"kb_used_omap":12,"kb_used_meta":1136179,"kb_avail":281218128,"statfs":{"total":289910292480,"available":287967363072,"internally_reserved":0,"allocated":779403264,"data_stored":778566384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1163447298},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,10,8,12,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19677184,"num_objects":-4804,"num_object_clones":0,"num_object_copies":-9608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-210,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4804,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008809"},"pg_stats":[{"pgid":"2.f","version":"49'16074","reported_seq":16296,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:48.973987+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:36:48.973987+0000","last_peered":"2024-09-10T20:36:48.973987+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:36:48.973987+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18776064,"num_objects":4584,"num_object_clones":0,"num_object_copies":9168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16074,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1043}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'16318","reported_seq":16438,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:49.171224+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:36:49.171224+0000","last_peered":"2024-09-10T20:36:49.171224+0000","last_clean":"2024-09-10T20:36:49.171224+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:36:49.171224+0000","last_undegraded":"2024-09-10T20:36:49.171224+0000","last_fullsized":"2024-09-10T20:36:49.171224+0000","mapping_epoch":35,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18939904,"num_objects":4624,"num_object_clones":0,"num_object_copies":9248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14504,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'16200","reported_seq":23557,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:50.794087+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:36:50.794087+0000","last_peered":"2024-09-10T20:36:50.794087+0000","last_clean":"2024-09-10T20:36:50.794087+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:36:50.794087+0000","last_undegraded":"2024-09-10T20:36:50.794087+0000","last_fullsized":"2024-09-10T20:36:50.794087+0000","mapping_epoch":44,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18493464,"num_objects":4516,"num_object_clones":0,"num_object_copies":9032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4516,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14328,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'15951","reported_seq":16041,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:49.246220+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:36:49.246220+0000","last_peered":"2024-09-10T20:36:49.246220+0000","last_clean":"2024-09-10T20:36:49.246220+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:36:49.246220+0000","last_undegraded":"2024-09-10T20:36:49.246220+0000","last_fullsized":"2024-09-10T20:36:49.246220+0000","mapping_epoch":35,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19058688,"num_objects":4653,"num_object_clones":0,"num_object_copies":9306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4653,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15951,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'16398","reported_seq":16534,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:48.837264+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:36:48.837264+0000","last_peered":"2024-09-10T20:36:48.837264+0000","last_clean":"2024-09-10T20:36:48.837264+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:36:48.837264+0000","last_undegraded":"2024-09-10T20:36:48.837264+0000","last_fullsized":"2024-09-10T20:36:48.837264+0000","mapping_epoch":34,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19132416,"num_objects":4671,"num_object_clones":0,"num_object_copies":9342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13394,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'15985","reported_seq":16237,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:48.962436+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:36:48.962436+0000","last_peered":"2024-09-10T20:36:48.962436+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:36:48.962436+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18374656,"num_objects":4486,"num_object_clones":0,"num_object_copies":8972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12849,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":925}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'15956","reported_seq":16242,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:48.875262+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:36:48.875262+0000","last_peered":"2024-09-10T20:36:48.875262+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:36:48.875262+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19054592,"num_objects":4652,"num_object_clones":0,"num_object_copies":9304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12551,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1166}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'16164","reported_seq":16298,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:48.940166+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:36:48.940166+0000","last_peered":"2024-09-10T20:36:48.940166+0000","last_clean":"2024-09-10T20:36:48.940166+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:36:48.940166+0000","last_undegraded":"2024-09-10T20:36:48.940166+0000","last_fullsized":"2024-09-10T20:36:48.940166+0000","mapping_epoch":34,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18726912,"num_objects":4572,"num_object_clones":0,"num_object_copies":9144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13235,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'15806","reported_seq":16006,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:49.182481+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:36:49.182481+0000","last_peered":"2024-09-10T20:36:49.182481+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:36:49.182481+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18976768,"num_objects":4633,"num_object_clones":0,"num_object_copies":9266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13799,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1136}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318026+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318026+0000","last_peered":"2024-09-10T20:36:30.318026+0000","last_clean":"2024-09-10T20:36:30.318026+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318026+0000","last_undegraded":"2024-09-10T20:36:30.318026+0000","last_fullsized":"2024-09-10T20:36:30.318026+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'16161","reported_seq":16377,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:48.749378+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:36:48.749378+0000","last_peered":"2024-09-10T20:36:48.749378+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:36:48.749378+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18715590,"num_objects":4571,"num_object_clones":0,"num_object_copies":9142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16159,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1005}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315256+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315256+0000","last_peered":"2024-09-10T20:36:30.315256+0000","last_clean":"2024-09-10T20:36:30.315256+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315256+0000","last_undegraded":"2024-09-10T20:36:30.315256+0000","last_fullsized":"2024-09-10T20:36:30.315256+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'15964","reported_seq":16156,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:49.159134+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:36:49.159134+0000","last_peered":"2024-09-10T20:36:49.159134+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:36:49.159134+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18845696,"num_objects":4601,"num_object_clones":0,"num_object_copies":9202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13792,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":995}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'16109","reported_seq":23479,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:49.152144+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:36:49.152144+0000","last_peered":"2024-09-10T20:36:49.152144+0000","last_clean":"2024-09-10T20:36:49.152144+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:36:49.152144+0000","last_undegraded":"2024-09-10T20:36:49.152144+0000","last_fullsized":"2024-09-10T20:36:49.152144+0000","mapping_epoch":48,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18657280,"num_objects":4555,"num_object_clones":0,"num_object_copies":9110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13895,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'16202","reported_seq":16459,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:48.929035+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:36:48.929035+0000","last_peered":"2024-09-10T20:36:48.929035+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:36:48.929035+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18956288,"num_objects":4628,"num_object_clones":0,"num_object_copies":9256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12925,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1036}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'16027","reported_seq":17228,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:48.924306+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:36:48.924306+0000","last_peered":"2024-09-10T20:36:48.924306+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:36:48.924306+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18665472,"num_objects":4557,"num_object_clones":0,"num_object_copies":9114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3008,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4557,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12721,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1549}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319607+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319607+0000","last_peered":"2024-09-10T20:36:30.319607+0000","last_clean":"2024-09-10T20:36:30.319607+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319607+0000","last_undegraded":"2024-09-10T20:36:30.319607+0000","last_fullsized":"2024-09-10T20:36:30.319607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'16382","reported_seq":16639,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:48.858472+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:36:48.858472+0000","last_peered":"2024-09-10T20:36:48.858472+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:36:48.858472+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18857984,"num_objects":4604,"num_object_clones":0,"num_object_copies":9208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13209,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":949}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'16017","reported_seq":23144,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:50.697766+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:36:50.697766+0000","last_peered":"2024-09-10T20:36:50.697766+0000","last_clean":"2024-09-10T20:36:50.697766+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:36:50.697766+0000","last_undegraded":"2024-09-10T20:36:50.697766+0000","last_fullsized":"2024-09-10T20:36:50.697766+0000","mapping_epoch":46,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18428387,"num_objects":4500,"num_object_clones":0,"num_object_copies":9000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12624,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":300660161,"num_objects":73407,"num_object_clones":0,"num_object_copies":146814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31525,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73407,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":222010,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11624,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":776044544,"data_stored":775978592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160814,"ondisk_log_size":160814,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182499,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":675368,"kb_used_data":278612,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93696472,"statfs":{"total":96636764160,"available":95945187328,"internally_reserved":0,"allocated":285298688,"data_stored":285036740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,1,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986271,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592100,"kb_used_data":263316,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93779740,"statfs":{"total":96636764160,"available":96030453760,"internally_reserved":0,"allocated":269635584,"data_stored":269373641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336653994},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,2,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659218,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":629924,"kb_used_data":219208,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93741916,"statfs":{"total":96636764160,"available":95991721984,"internally_reserved":0,"allocated":224468992,"data_stored":224156003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":420540078},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,5,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":284745728,"data_stored":284742115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":222617600,"data_stored":222562946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":268681216,"data_stored":268673531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:36:53.790 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:36:54.344 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:54.346 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:36:54.371 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2024-09-10T20:36:52.995726+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":300660180,"num_objects":73409,"num_object_clones":0,"num_object_copies":146818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31525,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73409,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":222012,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11626,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160817,"ondisk_log_size":160817,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1897392,"kb_used_data":761136,"kb_used_omap":12,"kb_used_meta":1136179,"kb_avail":281218128,"statfs":{"total":289910292480,"available":287967363072,"internally_reserved":0,"allocated":779403264,"data_stored":778566384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1163447298},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,10,8,12,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19677184,"num_objects":-4804,"num_object_clones":0,"num_object_copies":-9608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-210,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4804,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008809"},"pg_stats":[{"pgid":"2.f","version":"49'16074","reported_seq":16296,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:48.973987+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:36:48.973987+0000","last_peered":"2024-09-10T20:36:48.973987+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:36:48.973987+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18776064,"num_objects":4584,"num_object_clones":0,"num_object_copies":9168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16074,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1043}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'16318","reported_seq":16438,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:49.171224+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:36:49.171224+0000","last_peered":"2024-09-10T20:36:49.171224+0000","last_clean":"2024-09-10T20:36:49.171224+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:36:49.171224+0000","last_undegraded":"2024-09-10T20:36:49.171224+0000","last_fullsized":"2024-09-10T20:36:49.171224+0000","mapping_epoch":35,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18939904,"num_objects":4624,"num_object_clones":0,"num_object_copies":9248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14504,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'16200","reported_seq":23557,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:50.794087+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:36:50.794087+0000","last_peered":"2024-09-10T20:36:50.794087+0000","last_clean":"2024-09-10T20:36:50.794087+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:36:50.794087+0000","last_undegraded":"2024-09-10T20:36:50.794087+0000","last_fullsized":"2024-09-10T20:36:50.794087+0000","mapping_epoch":44,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18493464,"num_objects":4516,"num_object_clones":0,"num_object_copies":9032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4516,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14328,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'15951","reported_seq":16041,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:49.246220+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:36:49.246220+0000","last_peered":"2024-09-10T20:36:49.246220+0000","last_clean":"2024-09-10T20:36:49.246220+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:36:49.246220+0000","last_undegraded":"2024-09-10T20:36:49.246220+0000","last_fullsized":"2024-09-10T20:36:49.246220+0000","mapping_epoch":35,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19058688,"num_objects":4653,"num_object_clones":0,"num_object_copies":9306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4653,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15951,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'16398","reported_seq":16534,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:48.837264+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:36:48.837264+0000","last_peered":"2024-09-10T20:36:48.837264+0000","last_clean":"2024-09-10T20:36:48.837264+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:36:48.837264+0000","last_undegraded":"2024-09-10T20:36:48.837264+0000","last_fullsized":"2024-09-10T20:36:48.837264+0000","mapping_epoch":34,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19132416,"num_objects":4671,"num_object_clones":0,"num_object_copies":9342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13394,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'15985","reported_seq":16237,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:48.962436+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:36:48.962436+0000","last_peered":"2024-09-10T20:36:48.962436+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:36:48.962436+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18374656,"num_objects":4486,"num_object_clones":0,"num_object_copies":8972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12849,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":925}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'15956","reported_seq":16242,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:48.875262+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:36:48.875262+0000","last_peered":"2024-09-10T20:36:48.875262+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:36:48.875262+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19054592,"num_objects":4652,"num_object_clones":0,"num_object_copies":9304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12551,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1166}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'16164","reported_seq":16298,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:48.940166+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:36:48.940166+0000","last_peered":"2024-09-10T20:36:48.940166+0000","last_clean":"2024-09-10T20:36:48.940166+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:36:48.940166+0000","last_undegraded":"2024-09-10T20:36:48.940166+0000","last_fullsized":"2024-09-10T20:36:48.940166+0000","mapping_epoch":34,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18726912,"num_objects":4572,"num_object_clones":0,"num_object_copies":9144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13235,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'15806","reported_seq":16006,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:49.182481+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:36:49.182481+0000","last_peered":"2024-09-10T20:36:49.182481+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:36:49.182481+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18976768,"num_objects":4633,"num_object_clones":0,"num_object_copies":9266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13799,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1136}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318026+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318026+0000","last_peered":"2024-09-10T20:36:30.318026+0000","last_clean":"2024-09-10T20:36:30.318026+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318026+0000","last_undegraded":"2024-09-10T20:36:30.318026+0000","last_fullsized":"2024-09-10T20:36:30.318026+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'16161","reported_seq":16377,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:48.749378+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:36:48.749378+0000","last_peered":"2024-09-10T20:36:48.749378+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:36:48.749378+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18715590,"num_objects":4571,"num_object_clones":0,"num_object_copies":9142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16159,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1005}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315256+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315256+0000","last_peered":"2024-09-10T20:36:30.315256+0000","last_clean":"2024-09-10T20:36:30.315256+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315256+0000","last_undegraded":"2024-09-10T20:36:30.315256+0000","last_fullsized":"2024-09-10T20:36:30.315256+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'15964","reported_seq":16156,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:49.159134+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:36:49.159134+0000","last_peered":"2024-09-10T20:36:49.159134+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:36:49.159134+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18845696,"num_objects":4601,"num_object_clones":0,"num_object_copies":9202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13792,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":995}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'16109","reported_seq":23479,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:49.152144+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:36:49.152144+0000","last_peered":"2024-09-10T20:36:49.152144+0000","last_clean":"2024-09-10T20:36:49.152144+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:36:49.152144+0000","last_undegraded":"2024-09-10T20:36:49.152144+0000","last_fullsized":"2024-09-10T20:36:49.152144+0000","mapping_epoch":48,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18657280,"num_objects":4555,"num_object_clones":0,"num_object_copies":9110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13895,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'16202","reported_seq":16459,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:48.929035+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:36:48.929035+0000","last_peered":"2024-09-10T20:36:48.929035+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:36:48.929035+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18956288,"num_objects":4628,"num_object_clones":0,"num_object_copies":9256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12925,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1036}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'16027","reported_seq":17228,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:48.924306+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:36:48.924306+0000","last_peered":"2024-09-10T20:36:48.924306+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:36:48.924306+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18665472,"num_objects":4557,"num_object_clones":0,"num_object_copies":9114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3008,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4557,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12721,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1549}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319607+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319607+0000","last_peered":"2024-09-10T20:36:30.319607+0000","last_clean":"2024-09-10T20:36:30.319607+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319607+0000","last_undegraded":"2024-09-10T20:36:30.319607+0000","last_fullsized":"2024-09-10T20:36:30.319607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'16382","reported_seq":16639,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:48.858472+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:36:48.858472+0000","last_peered":"2024-09-10T20:36:48.858472+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:36:48.858472+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18857984,"num_objects":4604,"num_object_clones":0,"num_object_copies":9208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13209,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":949}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'16017","reported_seq":23144,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:50.697766+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:36:50.697766+0000","last_peered":"2024-09-10T20:36:50.697766+0000","last_clean":"2024-09-10T20:36:50.697766+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:36:50.697766+0000","last_undegraded":"2024-09-10T20:36:50.697766+0000","last_fullsized":"2024-09-10T20:36:50.697766+0000","mapping_epoch":46,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18428387,"num_objects":4500,"num_object_clones":0,"num_object_copies":9000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12624,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":300660161,"num_objects":73407,"num_object_clones":0,"num_object_copies":146814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31525,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73407,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":222010,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11624,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":776044544,"data_stored":775978592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160814,"ondisk_log_size":160814,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182499,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":675368,"kb_used_data":278612,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93696472,"statfs":{"total":96636764160,"available":95945187328,"internally_reserved":0,"allocated":285298688,"data_stored":285036740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,1,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986271,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592100,"kb_used_data":263316,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93779740,"statfs":{"total":96636764160,"available":96030453760,"internally_reserved":0,"allocated":269635584,"data_stored":269373641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336653994},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,2,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659218,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":629924,"kb_used_data":219208,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93741916,"statfs":{"total":96636764160,"available":95991721984,"internally_reserved":0,"allocated":224468992,"data_stored":224156003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":420540078},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,5,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":284745728,"data_stored":284742115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":222617600,"data_stored":222562946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":268681216,"data_stored":268673531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:36:54.373 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:36:54.373 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'16074", 'reported_seq': 16296, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:48.973987+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:36:48.973987+0000', 'last_peered': '2024-09-10T20:36:48.973987+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:36:48.973987+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "43'6000", 'ondisk_log_start': "43'6000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18776064, 'num_objects': 4584, 'num_object_clones': 0, 'num_object_copies': 9168, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4584, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16074, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1043}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:54.373 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:36:54.373 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'15985", 'reported_seq': 16237, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:48.962436+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:36:48.962436+0000', 'last_peered': '2024-09-10T20:36:48.962436+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:36:48.962436+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "43'5900", 'ondisk_log_start': "43'5900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18374656, 'num_objects': 4486, 'num_object_clones': 0, 'num_object_copies': 8972, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4486, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12849, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 925}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:54.373 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:36:54.373 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'15956", 'reported_seq': 16242, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:48.875262+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:36:48.875262+0000', 'last_peered': '2024-09-10T20:36:48.875262+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:36:48.875262+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "43'5900", 'ondisk_log_start': "43'5900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19054592, 'num_objects': 4652, 'num_object_clones': 0, 'num_object_copies': 9304, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4652, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12551, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1166}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:54.373 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:36:54.374 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'15806", 'reported_seq': 16006, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:49.182481+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:36:49.182481+0000', 'last_peered': '2024-09-10T20:36:49.182481+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:36:49.182481+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "43'5800", 'ondisk_log_start': "43'5800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18976768, 'num_objects': 4633, 'num_object_clones': 0, 'num_object_copies': 9266, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4633, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13799, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1136}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:36:54.374 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:36:54.374 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'16161", 'reported_seq': 16377, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:48.749378+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:36:48.749378+0000', 'last_peered': '2024-09-10T20:36:48.749378+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:36:48.749378+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "43'6100", 'ondisk_log_start': "43'6100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18715590, 'num_objects': 4571, 'num_object_clones': 0, 'num_object_copies': 9142, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4571, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16159, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1005}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:54.374 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:36:54.374 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'15964", 'reported_seq': 16156, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:49.159134+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:36:49.159134+0000', 'last_peered': '2024-09-10T20:36:49.159134+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:36:49.159134+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "43'5900", 'ondisk_log_start': "43'5900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18845696, 'num_objects': 4601, 'num_object_clones': 0, 'num_object_copies': 9202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4601, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13792, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 995}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:36:54.374 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:36:54.374 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'16202", 'reported_seq': 16459, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:48.929035+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:36:48.929035+0000', 'last_peered': '2024-09-10T20:36:48.929035+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:36:48.929035+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "43'6200", 'ondisk_log_start': "43'6200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18956288, 'num_objects': 4628, 'num_object_clones': 0, 'num_object_copies': 9256, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4628, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12925, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1036}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:54.374 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:36:54.374 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'16027", 'reported_seq': 17228, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:48.924306+0000', 'last_change': '2024-09-10T20:36:28.778006+0000', 'last_active': '2024-09-10T20:36:48.924306+0000', 'last_peered': '2024-09-10T20:36:48.924306+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:36:48.924306+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "43'6000", 'ondisk_log_start': "43'6000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18665472, 'num_objects': 4557, 'num_object_clones': 0, 'num_object_copies': 9114, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3008, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4557, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12721, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 527, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1549}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:54.375 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:36:54.375 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'16382", 'reported_seq': 16639, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:48.858472+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:36:48.858472+0000', 'last_peered': '2024-09-10T20:36:48.858472+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:36:48.858472+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "43'6300", 'ondisk_log_start': "43'6300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18857984, 'num_objects': 4604, 'num_object_clones': 0, 'num_object_copies': 9208, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4604, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13209, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 949}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:54.375 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:36:55.062 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:55.062 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31525/146818 objects degraded (21.472%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":526,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":73409,"data_bytes":300660180,"bytes_used":1942929408,"bytes_avail":287967363072,"bytes_total":289910292480,"degraded_objects":31525,"degraded_total":146818,"degraded_ratio":0.21472162813823917,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":210,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":400},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:36:55.091 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:36:55.091 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:36:55.661 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:55.661 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:36:55.686 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2024-09-10T20:36:54.996489+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":291661268,"num_objects":71212,"num_object_clones":0,"num_object_copies":142424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31429,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71212,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":224209,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11722,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160814,"ondisk_log_size":160814,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1894816,"kb_used_data":748816,"kb_used_omap":12,"kb_used_meta":1145907,"kb_avail":281220704,"statfs":{"total":289910292480,"available":287970000896,"internally_reserved":0,"allocated":766787584,"data_stored":765950704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1173408770},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,6,10,8,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28676096,"num_objects":-7001,"num_object_clones":0,"num_object_copies":-14002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7001,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005542"},"pg_stats":[{"pgid":"2.f","version":"49'16247","reported_seq":16467,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:53.984717+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:36:53.984717+0000","last_peered":"2024-09-10T20:36:53.984717+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:36:53.984717+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18067456,"num_objects":4411,"num_object_clones":0,"num_object_copies":8822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16247,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":870}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'16458","reported_seq":16578,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:54.083648+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:36:54.083648+0000","last_peered":"2024-09-10T20:36:54.083648+0000","last_clean":"2024-09-10T20:36:54.083648+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:36:54.083648+0000","last_undegraded":"2024-09-10T20:36:54.083648+0000","last_fullsized":"2024-09-10T20:36:54.083648+0000","mapping_epoch":35,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18366464,"num_objects":4484,"num_object_clones":0,"num_object_copies":8968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14644,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'16200","reported_seq":23557,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:50.794087+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:36:50.794087+0000","last_peered":"2024-09-10T20:36:50.794087+0000","last_clean":"2024-09-10T20:36:50.794087+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:36:50.794087+0000","last_undegraded":"2024-09-10T20:36:50.794087+0000","last_fullsized":"2024-09-10T20:36:50.794087+0000","mapping_epoch":44,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18493464,"num_objects":4516,"num_object_clones":0,"num_object_copies":9032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4516,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14328,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'16096","reported_seq":16186,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:54.233011+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:36:54.233011+0000","last_peered":"2024-09-10T20:36:54.233011+0000","last_clean":"2024-09-10T20:36:54.233011+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:36:54.233011+0000","last_undegraded":"2024-09-10T20:36:54.233011+0000","last_fullsized":"2024-09-10T20:36:54.233011+0000","mapping_epoch":35,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18464768,"num_objects":4508,"num_object_clones":0,"num_object_copies":9016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16096,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'16560","reported_seq":16696,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:53.916673+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:36:53.916673+0000","last_peered":"2024-09-10T20:36:53.916673+0000","last_clean":"2024-09-10T20:36:53.916673+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:36:53.916673+0000","last_undegraded":"2024-09-10T20:36:53.916673+0000","last_fullsized":"2024-09-10T20:36:53.916673+0000","mapping_epoch":34,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18468864,"num_objects":4509,"num_object_clones":0,"num_object_copies":9018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13556,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'16125","reported_seq":16377,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:53.934217+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:36:53.934217+0000","last_peered":"2024-09-10T20:36:53.934217+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:36:53.934217+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17801216,"num_objects":4346,"num_object_clones":0,"num_object_copies":8692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12989,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":785}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'16115","reported_seq":16400,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:53.944424+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:36:53.944424+0000","last_peered":"2024-09-10T20:36:53.944424+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:36:53.944424+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18403328,"num_objects":4493,"num_object_clones":0,"num_object_copies":8986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12710,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1007}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'16347","reported_seq":16481,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:53.954981+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:36:53.954981+0000","last_peered":"2024-09-10T20:36:53.954981+0000","last_clean":"2024-09-10T20:36:53.954981+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:36:53.954981+0000","last_undegraded":"2024-09-10T20:36:53.954981+0000","last_fullsized":"2024-09-10T20:36:53.954981+0000","mapping_epoch":34,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17977344,"num_objects":4389,"num_object_clones":0,"num_object_copies":8778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13418,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'15961","reported_seq":16161,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:54.203913+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:36:54.203913+0000","last_peered":"2024-09-10T20:36:54.203913+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:36:54.203913+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18341888,"num_objects":4478,"num_object_clones":0,"num_object_copies":8956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13954,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":981}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318026+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318026+0000","last_peered":"2024-09-10T20:36:30.318026+0000","last_clean":"2024-09-10T20:36:30.318026+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318026+0000","last_undegraded":"2024-09-10T20:36:30.318026+0000","last_fullsized":"2024-09-10T20:36:30.318026+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'16313","reported_seq":16529,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:53.946834+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:36:53.946834+0000","last_peered":"2024-09-10T20:36:53.946834+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:36:53.946834+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18092998,"num_objects":4419,"num_object_clones":0,"num_object_copies":8838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16311,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":853}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315256+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315256+0000","last_peered":"2024-09-10T20:36:30.315256+0000","last_clean":"2024-09-10T20:36:30.315256+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315256+0000","last_undegraded":"2024-09-10T20:36:30.315256+0000","last_fullsized":"2024-09-10T20:36:30.315256+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'16138","reported_seq":16330,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:54.177580+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:36:54.177580+0000","last_peered":"2024-09-10T20:36:54.177580+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:36:54.177580+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18132992,"num_objects":4427,"num_object_clones":0,"num_object_copies":8854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13966,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":821}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'16267","reported_seq":23637,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:54.165692+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:36:54.165692+0000","last_peered":"2024-09-10T20:36:54.165692+0000","last_clean":"2024-09-10T20:36:54.165692+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:36:54.165692+0000","last_undegraded":"2024-09-10T20:36:54.165692+0000","last_fullsized":"2024-09-10T20:36:54.165692+0000","mapping_epoch":48,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18010112,"num_objects":4397,"num_object_clones":0,"num_object_copies":8794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14053,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'16329","reported_seq":16586,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:53.936786+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:36:53.936786+0000","last_peered":"2024-09-10T20:36:53.936786+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:36:53.936786+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18436096,"num_objects":4501,"num_object_clones":0,"num_object_copies":9002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13052,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":909}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'16177","reported_seq":17572,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:53.969849+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:36:53.969849+0000","last_peered":"2024-09-10T20:36:53.969849+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:36:53.969849+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18051072,"num_objects":4407,"num_object_clones":0,"num_object_copies":8814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2912,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12871,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":623,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319607+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319607+0000","last_peered":"2024-09-10T20:36:30.319607+0000","last_clean":"2024-09-10T20:36:30.319607+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319607+0000","last_undegraded":"2024-09-10T20:36:30.319607+0000","last_fullsized":"2024-09-10T20:36:30.319607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'16561","reported_seq":16815,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:53.974031+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:36:53.974031+0000","last_peered":"2024-09-10T20:36:53.974031+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:36:53.974031+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18124800,"num_objects":4425,"num_object_clones":0,"num_object_copies":8850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13388,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":770}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'16017","reported_seq":23144,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:50.697766+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:36:50.697766+0000","last_peered":"2024-09-10T20:36:50.697766+0000","last_clean":"2024-09-10T20:36:50.697766+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:36:50.697766+0000","last_undegraded":"2024-09-10T20:36:50.697766+0000","last_fullsized":"2024-09-10T20:36:50.697766+0000","mapping_epoch":46,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18428387,"num_objects":4500,"num_object_clones":0,"num_object_copies":9000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12624,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":291661249,"num_objects":71210,"num_object_clones":0,"num_object_copies":142420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31429,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71210,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":224207,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11720,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":763359232,"data_stored":763293280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160811,"ondisk_log_size":160811,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182500,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":690256,"kb_used_data":271284,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93681584,"statfs":{"total":96636764160,"available":95929942016,"internally_reserved":0,"allocated":277794816,"data_stored":277532868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":428994218},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,5,2,1,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986272,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574636,"kb_used_data":258324,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93797204,"statfs":{"total":96636764160,"available":96048336896,"internally_reserved":0,"allocated":264523776,"data_stored":264261833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323874474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,3,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659218,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":629924,"kb_used_data":219208,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93741916,"statfs":{"total":96636764160,"available":95991721984,"internally_reserved":0,"allocated":224468992,"data_stored":224156003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":420540078},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,5,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":277147648,"data_stored":277144035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":222617600,"data_stored":222562946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":263593984,"data_stored":263586299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:36:58.688 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:36:59.241 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:59.242 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:36:59.270 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2024-09-10T20:36:58.997552+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":290485716,"num_objects":70925,"num_object_clones":0,"num_object_copies":141850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31429,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70925,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":224496,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11722,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160801,"ondisk_log_size":160801,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1891572,"kb_used_data":741420,"kb_used_omap":12,"kb_used_meta":1150067,"kb_avail":281223948,"statfs":{"total":289910292480,"available":287973322752,"internally_reserved":0,"allocated":759214080,"data_stored":758377200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1177668610},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,8,8,5,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19750912,"num_objects":-4822,"num_object_clones":0,"num_object_copies":-9644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-209,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4822,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005119"},"pg_stats":[{"pgid":"2.f","version":"49'16247","reported_seq":16467,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:53.984717+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:36:53.984717+0000","last_peered":"2024-09-10T20:36:53.984717+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:36:53.984717+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18067456,"num_objects":4411,"num_object_clones":0,"num_object_copies":8822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16247,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":870}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'16458","reported_seq":16578,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:54.083648+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:36:54.083648+0000","last_peered":"2024-09-10T20:36:54.083648+0000","last_clean":"2024-09-10T20:36:54.083648+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:36:54.083648+0000","last_undegraded":"2024-09-10T20:36:54.083648+0000","last_fullsized":"2024-09-10T20:36:54.083648+0000","mapping_epoch":35,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18366464,"num_objects":4484,"num_object_clones":0,"num_object_copies":8968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14644,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'16339","reported_seq":23697,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:55.826127+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:36:55.826127+0000","last_peered":"2024-09-10T20:36:55.826127+0000","last_clean":"2024-09-10T20:36:55.826127+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:36:55.826127+0000","last_undegraded":"2024-09-10T20:36:55.826127+0000","last_fullsized":"2024-09-10T20:36:55.826127+0000","mapping_epoch":44,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17924120,"num_objects":4377,"num_object_clones":0,"num_object_copies":8754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4377,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14467,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'16096","reported_seq":16186,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:54.233011+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:36:54.233011+0000","last_peered":"2024-09-10T20:36:54.233011+0000","last_clean":"2024-09-10T20:36:54.233011+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:36:54.233011+0000","last_undegraded":"2024-09-10T20:36:54.233011+0000","last_fullsized":"2024-09-10T20:36:54.233011+0000","mapping_epoch":35,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18464768,"num_objects":4508,"num_object_clones":0,"num_object_copies":9016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16096,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'16560","reported_seq":16696,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:53.916673+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:36:53.916673+0000","last_peered":"2024-09-10T20:36:53.916673+0000","last_clean":"2024-09-10T20:36:53.916673+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:36:53.916673+0000","last_undegraded":"2024-09-10T20:36:53.916673+0000","last_fullsized":"2024-09-10T20:36:53.916673+0000","mapping_epoch":34,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18468864,"num_objects":4509,"num_object_clones":0,"num_object_copies":9018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13556,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'16125","reported_seq":16377,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:53.934217+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:36:53.934217+0000","last_peered":"2024-09-10T20:36:53.934217+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:36:53.934217+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17801216,"num_objects":4346,"num_object_clones":0,"num_object_copies":8692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12989,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":785}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'16115","reported_seq":16400,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:53.944424+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:36:53.944424+0000","last_peered":"2024-09-10T20:36:53.944424+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:36:53.944424+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18403328,"num_objects":4493,"num_object_clones":0,"num_object_copies":8986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12710,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1007}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'16347","reported_seq":16481,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:53.954981+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:36:53.954981+0000","last_peered":"2024-09-10T20:36:53.954981+0000","last_clean":"2024-09-10T20:36:53.954981+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:36:53.954981+0000","last_undegraded":"2024-09-10T20:36:53.954981+0000","last_fullsized":"2024-09-10T20:36:53.954981+0000","mapping_epoch":34,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17977344,"num_objects":4389,"num_object_clones":0,"num_object_copies":8778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13418,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'15961","reported_seq":16161,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:54.203913+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:36:54.203913+0000","last_peered":"2024-09-10T20:36:54.203913+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:36:54.203913+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18341888,"num_objects":4478,"num_object_clones":0,"num_object_copies":8956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13954,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":981}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318026+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318026+0000","last_peered":"2024-09-10T20:36:30.318026+0000","last_clean":"2024-09-10T20:36:30.318026+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318026+0000","last_undegraded":"2024-09-10T20:36:30.318026+0000","last_fullsized":"2024-09-10T20:36:30.318026+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'16313","reported_seq":16529,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:53.946834+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:36:53.946834+0000","last_peered":"2024-09-10T20:36:53.946834+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:36:53.946834+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18092998,"num_objects":4419,"num_object_clones":0,"num_object_copies":8838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16311,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":853}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315256+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315256+0000","last_peered":"2024-09-10T20:36:30.315256+0000","last_clean":"2024-09-10T20:36:30.315256+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315256+0000","last_undegraded":"2024-09-10T20:36:30.315256+0000","last_fullsized":"2024-09-10T20:36:30.315256+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'16138","reported_seq":16330,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:54.177580+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:36:54.177580+0000","last_peered":"2024-09-10T20:36:54.177580+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:36:54.177580+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18132992,"num_objects":4427,"num_object_clones":0,"num_object_copies":8854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13966,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":821}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'16267","reported_seq":23637,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:54.165692+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:36:54.165692+0000","last_peered":"2024-09-10T20:36:54.165692+0000","last_clean":"2024-09-10T20:36:54.165692+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:36:54.165692+0000","last_undegraded":"2024-09-10T20:36:54.165692+0000","last_fullsized":"2024-09-10T20:36:54.165692+0000","mapping_epoch":48,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18010112,"num_objects":4397,"num_object_clones":0,"num_object_copies":8794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14053,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'16329","reported_seq":16586,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:53.936786+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:36:53.936786+0000","last_peered":"2024-09-10T20:36:53.936786+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:36:53.936786+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18436096,"num_objects":4501,"num_object_clones":0,"num_object_copies":9002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13052,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":909}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'16177","reported_seq":17572,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:53.969849+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:36:53.969849+0000","last_peered":"2024-09-10T20:36:53.969849+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:36:53.969849+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18051072,"num_objects":4407,"num_object_clones":0,"num_object_copies":8814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2912,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12871,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":623,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319607+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319607+0000","last_peered":"2024-09-10T20:36:30.319607+0000","last_clean":"2024-09-10T20:36:30.319607+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319607+0000","last_undegraded":"2024-09-10T20:36:30.319607+0000","last_fullsized":"2024-09-10T20:36:30.319607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'16561","reported_seq":16815,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:53.974031+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:36:53.974031+0000","last_peered":"2024-09-10T20:36:53.974031+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:36:53.974031+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18124800,"num_objects":4425,"num_object_clones":0,"num_object_copies":8850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13388,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":770}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'16165","reported_seq":23292,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:55.801640+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:36:55.801640+0000","last_peered":"2024-09-10T20:36:55.801640+0000","last_clean":"2024-09-10T20:36:55.801640+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:36:55.801640+0000","last_undegraded":"2024-09-10T20:36:55.801640+0000","last_fullsized":"2024-09-10T20:36:55.801640+0000","mapping_epoch":46,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17822179,"num_objects":4352,"num_object_clones":0,"num_object_copies":8704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12772,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":290485697,"num_objects":70923,"num_object_clones":0,"num_object_copies":141846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31429,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70923,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":224494,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11720,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":755945472,"data_stored":755879520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160798,"ondisk_log_size":160798,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182500,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":690256,"kb_used_data":271284,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93681584,"statfs":{"total":96636764160,"available":95929942016,"internally_reserved":0,"allocated":277794816,"data_stored":277532868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":428994218},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,5,2,1,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986272,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574636,"kb_used_data":258324,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93797204,"statfs":{"total":96636764160,"available":96048336896,"internally_reserved":0,"allocated":264523776,"data_stored":264261833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323874474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,3,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659219,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":626680,"kb_used_data":211812,"kb_used_omap":4,"kb_used_meta":414843,"kb_avail":93745160,"statfs":{"total":96636764160,"available":95995043840,"internally_reserved":0,"allocated":216895488,"data_stored":216582499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":424799918},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":277147648,"data_stored":277144035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":215203840,"data_stored":215149186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":263593984,"data_stored":263586299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:36:59.272 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:36:59.834 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:36:59.837 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:36:59.860 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2024-09-10T20:36:58.997552+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":290485716,"num_objects":70925,"num_object_clones":0,"num_object_copies":141850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31429,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70925,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":224496,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11722,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160801,"ondisk_log_size":160801,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1891572,"kb_used_data":741420,"kb_used_omap":12,"kb_used_meta":1150067,"kb_avail":281223948,"statfs":{"total":289910292480,"available":287973322752,"internally_reserved":0,"allocated":759214080,"data_stored":758377200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1177668610},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,8,8,5,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19750912,"num_objects":-4822,"num_object_clones":0,"num_object_copies":-9644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-209,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4822,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005119"},"pg_stats":[{"pgid":"2.f","version":"49'16247","reported_seq":16467,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:53.984717+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:36:53.984717+0000","last_peered":"2024-09-10T20:36:53.984717+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:36:53.984717+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18067456,"num_objects":4411,"num_object_clones":0,"num_object_copies":8822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16247,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":870}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'16458","reported_seq":16578,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:54.083648+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:36:54.083648+0000","last_peered":"2024-09-10T20:36:54.083648+0000","last_clean":"2024-09-10T20:36:54.083648+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:36:54.083648+0000","last_undegraded":"2024-09-10T20:36:54.083648+0000","last_fullsized":"2024-09-10T20:36:54.083648+0000","mapping_epoch":35,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18366464,"num_objects":4484,"num_object_clones":0,"num_object_copies":8968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14644,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'16339","reported_seq":23697,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:55.826127+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:36:55.826127+0000","last_peered":"2024-09-10T20:36:55.826127+0000","last_clean":"2024-09-10T20:36:55.826127+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:36:55.826127+0000","last_undegraded":"2024-09-10T20:36:55.826127+0000","last_fullsized":"2024-09-10T20:36:55.826127+0000","mapping_epoch":44,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17924120,"num_objects":4377,"num_object_clones":0,"num_object_copies":8754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4377,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14467,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'16096","reported_seq":16186,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:54.233011+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:36:54.233011+0000","last_peered":"2024-09-10T20:36:54.233011+0000","last_clean":"2024-09-10T20:36:54.233011+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:36:54.233011+0000","last_undegraded":"2024-09-10T20:36:54.233011+0000","last_fullsized":"2024-09-10T20:36:54.233011+0000","mapping_epoch":35,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18464768,"num_objects":4508,"num_object_clones":0,"num_object_copies":9016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16096,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'16560","reported_seq":16696,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:53.916673+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:36:53.916673+0000","last_peered":"2024-09-10T20:36:53.916673+0000","last_clean":"2024-09-10T20:36:53.916673+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:36:53.916673+0000","last_undegraded":"2024-09-10T20:36:53.916673+0000","last_fullsized":"2024-09-10T20:36:53.916673+0000","mapping_epoch":34,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18468864,"num_objects":4509,"num_object_clones":0,"num_object_copies":9018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13556,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'16125","reported_seq":16377,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:53.934217+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:36:53.934217+0000","last_peered":"2024-09-10T20:36:53.934217+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:36:53.934217+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17801216,"num_objects":4346,"num_object_clones":0,"num_object_copies":8692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12989,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":785}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'16115","reported_seq":16400,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:53.944424+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:36:53.944424+0000","last_peered":"2024-09-10T20:36:53.944424+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:36:53.944424+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18403328,"num_objects":4493,"num_object_clones":0,"num_object_copies":8986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12710,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1007}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'16347","reported_seq":16481,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:53.954981+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:36:53.954981+0000","last_peered":"2024-09-10T20:36:53.954981+0000","last_clean":"2024-09-10T20:36:53.954981+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:36:53.954981+0000","last_undegraded":"2024-09-10T20:36:53.954981+0000","last_fullsized":"2024-09-10T20:36:53.954981+0000","mapping_epoch":34,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17977344,"num_objects":4389,"num_object_clones":0,"num_object_copies":8778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13418,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'15961","reported_seq":16161,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:54.203913+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:36:54.203913+0000","last_peered":"2024-09-10T20:36:54.203913+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:36:54.203913+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18341888,"num_objects":4478,"num_object_clones":0,"num_object_copies":8956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13954,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":981}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318026+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318026+0000","last_peered":"2024-09-10T20:36:30.318026+0000","last_clean":"2024-09-10T20:36:30.318026+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318026+0000","last_undegraded":"2024-09-10T20:36:30.318026+0000","last_fullsized":"2024-09-10T20:36:30.318026+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'16313","reported_seq":16529,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:53.946834+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:36:53.946834+0000","last_peered":"2024-09-10T20:36:53.946834+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:36:53.946834+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18092998,"num_objects":4419,"num_object_clones":0,"num_object_copies":8838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16311,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":853}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315256+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315256+0000","last_peered":"2024-09-10T20:36:30.315256+0000","last_clean":"2024-09-10T20:36:30.315256+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315256+0000","last_undegraded":"2024-09-10T20:36:30.315256+0000","last_fullsized":"2024-09-10T20:36:30.315256+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'16138","reported_seq":16330,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:54.177580+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:36:54.177580+0000","last_peered":"2024-09-10T20:36:54.177580+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:36:54.177580+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18132992,"num_objects":4427,"num_object_clones":0,"num_object_copies":8854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13966,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":821}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'16267","reported_seq":23637,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:54.165692+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:36:54.165692+0000","last_peered":"2024-09-10T20:36:54.165692+0000","last_clean":"2024-09-10T20:36:54.165692+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:36:54.165692+0000","last_undegraded":"2024-09-10T20:36:54.165692+0000","last_fullsized":"2024-09-10T20:36:54.165692+0000","mapping_epoch":48,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18010112,"num_objects":4397,"num_object_clones":0,"num_object_copies":8794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14053,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'16329","reported_seq":16586,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:53.936786+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:36:53.936786+0000","last_peered":"2024-09-10T20:36:53.936786+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:36:53.936786+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18436096,"num_objects":4501,"num_object_clones":0,"num_object_copies":9002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13052,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":909}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'16177","reported_seq":17572,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:53.969849+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:36:53.969849+0000","last_peered":"2024-09-10T20:36:53.969849+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:36:53.969849+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18051072,"num_objects":4407,"num_object_clones":0,"num_object_copies":8814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2912,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12871,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":623,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319607+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319607+0000","last_peered":"2024-09-10T20:36:30.319607+0000","last_clean":"2024-09-10T20:36:30.319607+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319607+0000","last_undegraded":"2024-09-10T20:36:30.319607+0000","last_fullsized":"2024-09-10T20:36:30.319607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'16561","reported_seq":16815,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:53.974031+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:36:53.974031+0000","last_peered":"2024-09-10T20:36:53.974031+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:36:53.974031+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18124800,"num_objects":4425,"num_object_clones":0,"num_object_copies":8850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13388,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":770}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'16165","reported_seq":23292,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:55.801640+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:36:55.801640+0000","last_peered":"2024-09-10T20:36:55.801640+0000","last_clean":"2024-09-10T20:36:55.801640+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:36:55.801640+0000","last_undegraded":"2024-09-10T20:36:55.801640+0000","last_fullsized":"2024-09-10T20:36:55.801640+0000","mapping_epoch":46,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17822179,"num_objects":4352,"num_object_clones":0,"num_object_copies":8704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12772,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":290485697,"num_objects":70923,"num_object_clones":0,"num_object_copies":141846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31429,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70923,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":224494,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11720,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":755945472,"data_stored":755879520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160798,"ondisk_log_size":160798,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182500,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":690256,"kb_used_data":271284,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93681584,"statfs":{"total":96636764160,"available":95929942016,"internally_reserved":0,"allocated":277794816,"data_stored":277532868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":428994218},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,5,2,1,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986272,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574636,"kb_used_data":258324,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93797204,"statfs":{"total":96636764160,"available":96048336896,"internally_reserved":0,"allocated":264523776,"data_stored":264261833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323874474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,3,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659219,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":626680,"kb_used_data":211812,"kb_used_omap":4,"kb_used_meta":414843,"kb_avail":93745160,"statfs":{"total":96636764160,"available":95995043840,"internally_reserved":0,"allocated":216895488,"data_stored":216582499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":424799918},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":277147648,"data_stored":277144035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":215203840,"data_stored":215149186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":263593984,"data_stored":263586299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:36:59.862 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:36:59.862 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'16247", 'reported_seq': 16467, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:53.984717+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:36:53.984717+0000', 'last_peered': '2024-09-10T20:36:53.984717+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:36:53.984717+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "43'6200", 'ondisk_log_start': "43'6200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18067456, 'num_objects': 4411, 'num_object_clones': 0, 'num_object_copies': 8822, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4411, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16247, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 870}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:59.862 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:36:59.862 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'16125", 'reported_seq': 16377, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:53.934217+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:36:53.934217+0000', 'last_peered': '2024-09-10T20:36:53.934217+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:36:53.934217+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "43'6100", 'ondisk_log_start': "43'6100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17801216, 'num_objects': 4346, 'num_object_clones': 0, 'num_object_copies': 8692, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4346, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12989, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 785}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:59.862 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:36:59.862 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'16115", 'reported_seq': 16400, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:53.944424+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:36:53.944424+0000', 'last_peered': '2024-09-10T20:36:53.944424+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:36:53.944424+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "43'6100", 'ondisk_log_start': "43'6100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18403328, 'num_objects': 4493, 'num_object_clones': 0, 'num_object_copies': 8986, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4493, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12710, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1007}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:59.862 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:36:59.862 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'15961", 'reported_seq': 16161, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:54.203913+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:36:54.203913+0000', 'last_peered': '2024-09-10T20:36:54.203913+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:36:54.203913+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "43'5900", 'ondisk_log_start': "43'5900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18341888, 'num_objects': 4478, 'num_object_clones': 0, 'num_object_copies': 8956, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4478, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13954, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 981}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:36:59.862 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:36:59.863 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'16313", 'reported_seq': 16529, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:53.946834+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:36:53.946834+0000', 'last_peered': '2024-09-10T20:36:53.946834+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:36:53.946834+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "43'6300", 'ondisk_log_start': "43'6300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18092998, 'num_objects': 4419, 'num_object_clones': 0, 'num_object_copies': 8838, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4419, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16311, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 853}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:59.863 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:36:59.863 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'16138", 'reported_seq': 16330, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:54.177580+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:36:54.177580+0000', 'last_peered': '2024-09-10T20:36:54.177580+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:36:54.177580+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "43'6100", 'ondisk_log_start': "43'6100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18132992, 'num_objects': 4427, 'num_object_clones': 0, 'num_object_copies': 8854, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4427, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13966, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 821}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:36:59.863 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:36:59.863 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'16329", 'reported_seq': 16586, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:53.936786+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:36:53.936786+0000', 'last_peered': '2024-09-10T20:36:53.936786+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:36:53.936786+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "43'6300", 'ondisk_log_start': "43'6300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18436096, 'num_objects': 4501, 'num_object_clones': 0, 'num_object_copies': 9002, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4501, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13052, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 909}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:59.863 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:36:59.863 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'16177", 'reported_seq': 17572, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:53.969849+0000', 'last_change': '2024-09-10T20:36:28.778006+0000', 'last_active': '2024-09-10T20:36:53.969849+0000', 'last_peered': '2024-09-10T20:36:53.969849+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:36:53.969849+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "43'6100", 'ondisk_log_start': "43'6100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18051072, 'num_objects': 4407, 'num_object_clones': 0, 'num_object_copies': 8814, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2912, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4407, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12871, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 623, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:59.863 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:36:59.863 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'16561", 'reported_seq': 16815, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:36:53.974031+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:36:53.974031+0000', 'last_peered': '2024-09-10T20:36:53.974031+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:36:53.974031+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "43'6500", 'ondisk_log_start': "43'6500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18124800, 'num_objects': 4425, 'num_object_clones': 0, 'num_object_copies': 8850, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4425, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13388, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 770}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:36:59.864 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:37:00.539 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:00.539 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31429/141850 objects degraded (22.157%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":532,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":70925,"data_bytes":290485716,"bytes_used":1936969728,"bytes_avail":287973322752,"bytes_total":289910292480,"degraded_objects":31429,"degraded_total":141850,"degraded_ratio":0.22156503348607687,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":209,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":401},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:37:00.565 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:37:00.566 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:37:01.137 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:01.139 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:37:01.167 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":309,"stamp":"2024-09-10T20:37:00.998187+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":280331732,"num_objects":68446,"num_object_clones":0,"num_object_copies":136892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31317,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68446,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":226975,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11834,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160780,"ondisk_log_size":160780,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1875964,"kb_used_data":721672,"kb_used_omap":12,"kb_used_meta":1154227,"kb_avail":281239556,"statfs":{"total":289910292480,"available":287989305344,"internally_reserved":0,"allocated":738992128,"data_stored":738155248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1181928450},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,6,18,7,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24608768,"num_objects":-6008,"num_object_clones":0,"num_object_copies":-12016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6008,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005015"},"pg_stats":[{"pgid":"2.f","version":"49'16387","reported_seq":16603,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:58.991333+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:36:58.991333+0000","last_peered":"2024-09-10T20:36:58.991333+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:36:58.991333+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17494016,"num_objects":4271,"num_object_clones":0,"num_object_copies":8542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16387,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":730}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'16606","reported_seq":16726,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:59.252625+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:36:59.252625+0000","last_peered":"2024-09-10T20:36:59.252625+0000","last_clean":"2024-09-10T20:36:59.252625+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:36:59.252625+0000","last_undegraded":"2024-09-10T20:36:59.252625+0000","last_fullsized":"2024-09-10T20:36:59.252625+0000","mapping_epoch":35,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17760256,"num_objects":4336,"num_object_clones":0,"num_object_copies":8672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14792,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'16490","reported_seq":23848,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:00.853306+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:37:00.853306+0000","last_peered":"2024-09-10T20:37:00.853306+0000","last_clean":"2024-09-10T20:37:00.853306+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:37:00.853306+0000","last_undegraded":"2024-09-10T20:37:00.853306+0000","last_fullsized":"2024-09-10T20:37:00.853306+0000","mapping_epoch":44,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17305624,"num_objects":4226,"num_object_clones":0,"num_object_copies":8452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4226,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14618,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'16242","reported_seq":16332,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:59.260281+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:36:59.260281+0000","last_peered":"2024-09-10T20:36:59.260281+0000","last_clean":"2024-09-10T20:36:59.260281+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:36:59.260281+0000","last_undegraded":"2024-09-10T20:36:59.260281+0000","last_fullsized":"2024-09-10T20:36:59.260281+0000","mapping_epoch":35,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17866752,"num_objects":4362,"num_object_clones":0,"num_object_copies":8724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16242,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'16708","reported_seq":16844,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:58.918217+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:36:58.918217+0000","last_peered":"2024-09-10T20:36:58.918217+0000","last_clean":"2024-09-10T20:36:58.918217+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:36:58.918217+0000","last_undegraded":"2024-09-10T20:36:58.918217+0000","last_fullsized":"2024-09-10T20:36:58.918217+0000","mapping_epoch":34,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17862656,"num_objects":4361,"num_object_clones":0,"num_object_copies":8722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13704,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'16278","reported_seq":16530,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:58.964742+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:36:58.964742+0000","last_peered":"2024-09-10T20:36:58.964742+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:36:58.964742+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17174528,"num_objects":4193,"num_object_clones":0,"num_object_copies":8386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13142,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":632}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'16292","reported_seq":16578,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:58.960000+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:36:58.960000+0000","last_peered":"2024-09-10T20:36:58.960000+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:36:58.960000+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17678336,"num_objects":4316,"num_object_clones":0,"num_object_copies":8632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12887,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":830}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'16500","reported_seq":16634,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:58.903157+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:36:58.903157+0000","last_peered":"2024-09-10T20:36:58.903157+0000","last_clean":"2024-09-10T20:36:58.903157+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:36:58.903157+0000","last_undegraded":"2024-09-10T20:36:58.903157+0000","last_fullsized":"2024-09-10T20:36:58.903157+0000","mapping_epoch":34,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17350656,"num_objects":4236,"num_object_clones":0,"num_object_copies":8472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13571,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'16135","reported_seq":16335,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:59.264177+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:36:59.264177+0000","last_peered":"2024-09-10T20:36:59.264177+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:36:59.264177+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17629184,"num_objects":4304,"num_object_clones":0,"num_object_copies":8608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14128,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":807}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318026+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318026+0000","last_peered":"2024-09-10T20:36:30.318026+0000","last_clean":"2024-09-10T20:36:30.318026+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318026+0000","last_undegraded":"2024-09-10T20:36:30.318026+0000","last_fullsized":"2024-09-10T20:36:30.318026+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'16468","reported_seq":16684,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:58.976827+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:36:58.976827+0000","last_peered":"2024-09-10T20:36:58.976827+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:36:58.976827+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17458118,"num_objects":4264,"num_object_clones":0,"num_object_copies":8528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16466,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":698}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315256+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315256+0000","last_peered":"2024-09-10T20:36:30.315256+0000","last_clean":"2024-09-10T20:36:30.315256+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315256+0000","last_undegraded":"2024-09-10T20:36:30.315256+0000","last_fullsized":"2024-09-10T20:36:30.315256+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'16314","reported_seq":16506,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:59.211855+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:36:59.211855+0000","last_peered":"2024-09-10T20:36:59.211855+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:36:59.211855+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17412096,"num_objects":4251,"num_object_clones":0,"num_object_copies":8502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14142,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":645}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'16425","reported_seq":23795,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:59.231048+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:36:59.231048+0000","last_peered":"2024-09-10T20:36:59.231048+0000","last_clean":"2024-09-10T20:36:59.231048+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:36:59.231048+0000","last_undegraded":"2024-09-10T20:36:59.231048+0000","last_fullsized":"2024-09-10T20:36:59.231048+0000","mapping_epoch":48,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17362944,"num_objects":4239,"num_object_clones":0,"num_object_copies":8478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14211,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'16491","reported_seq":16746,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:58.976410+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:36:58.976410+0000","last_peered":"2024-09-10T20:36:58.976410+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:36:58.976410+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17772544,"num_objects":4339,"num_object_clones":0,"num_object_copies":8678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13214,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":747}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'16314","reported_seq":17933,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:59.005052+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:36:59.005052+0000","last_peered":"2024-09-10T20:36:59.005052+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:36:59.005052+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17489920,"num_objects":4270,"num_object_clones":0,"num_object_copies":8540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2800,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13008,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":735,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319607+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319607+0000","last_peered":"2024-09-10T20:36:30.319607+0000","last_clean":"2024-09-10T20:36:30.319607+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319607+0000","last_undegraded":"2024-09-10T20:36:30.319607+0000","last_fullsized":"2024-09-10T20:36:30.319607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'16707","reported_seq":16964,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:58.947430+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:36:58.947430+0000","last_peered":"2024-09-10T20:36:58.947430+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:36:58.947430+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17526784,"num_objects":4279,"num_object_clones":0,"num_object_copies":8558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13534,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":624}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'16320","reported_seq":23447,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:00.868115+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:37:00.868115+0000","last_peered":"2024-09-10T20:37:00.868115+0000","last_clean":"2024-09-10T20:37:00.868115+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:37:00.868115+0000","last_undegraded":"2024-09-10T20:37:00.868115+0000","last_fullsized":"2024-09-10T20:37:00.868115+0000","mapping_epoch":46,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17187299,"num_objects":4197,"num_object_clones":0,"num_object_copies":8394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4197,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12927,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":280331713,"num_objects":68444,"num_object_clones":0,"num_object_copies":136888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31317,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68444,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":226973,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11832,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":735686656,"data_stored":735620704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160777,"ondisk_log_size":160777,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182501,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":674808,"kb_used_data":264164,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93697032,"statfs":{"total":96636764160,"available":95945760768,"internally_reserved":0,"allocated":270503936,"data_stored":270241988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":420474538},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,11,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986273,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":573720,"kb_used_data":253256,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93798120,"statfs":{"total":96636764160,"available":96049274880,"internally_reserved":0,"allocated":259334144,"data_stored":259072201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328134314},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659220,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627436,"kb_used_data":204252,"kb_used_omap":4,"kb_used_meta":423163,"kb_avail":93744404,"statfs":{"total":96636764160,"available":95994269696,"internally_reserved":0,"allocated":209154048,"data_stored":208841059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":433319598},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":269795328,"data_stored":269791715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":207441920,"data_stored":207387266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":258449408,"data_stored":258441723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:37:04.170 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:37:04.726 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:04.728 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:37:04.752 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-09-10T20:37:02.999923+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":280331732,"num_objects":68446,"num_object_clones":0,"num_object_copies":136892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31317,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68446,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":226975,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11834,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160780,"ondisk_log_size":160780,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1875964,"kb_used_data":721672,"kb_used_omap":12,"kb_used_meta":1154227,"kb_avail":281239556,"statfs":{"total":289910292480,"available":287989305344,"internally_reserved":0,"allocated":738992128,"data_stored":738155248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1181928450},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,6,18,7,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20328448,"num_objects":-4963,"num_object_clones":0,"num_object_copies":-9926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4963,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006001"},"pg_stats":[{"pgid":"2.f","version":"49'16387","reported_seq":16603,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:58.991333+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:36:58.991333+0000","last_peered":"2024-09-10T20:36:58.991333+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:36:58.991333+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17494016,"num_objects":4271,"num_object_clones":0,"num_object_copies":8542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16387,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":730}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'16606","reported_seq":16726,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:59.252625+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:36:59.252625+0000","last_peered":"2024-09-10T20:36:59.252625+0000","last_clean":"2024-09-10T20:36:59.252625+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:36:59.252625+0000","last_undegraded":"2024-09-10T20:36:59.252625+0000","last_fullsized":"2024-09-10T20:36:59.252625+0000","mapping_epoch":35,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17760256,"num_objects":4336,"num_object_clones":0,"num_object_copies":8672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14792,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'16490","reported_seq":23848,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:00.853306+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:37:00.853306+0000","last_peered":"2024-09-10T20:37:00.853306+0000","last_clean":"2024-09-10T20:37:00.853306+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:37:00.853306+0000","last_undegraded":"2024-09-10T20:37:00.853306+0000","last_fullsized":"2024-09-10T20:37:00.853306+0000","mapping_epoch":44,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17305624,"num_objects":4226,"num_object_clones":0,"num_object_copies":8452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4226,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14618,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'16242","reported_seq":16332,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:59.260281+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:36:59.260281+0000","last_peered":"2024-09-10T20:36:59.260281+0000","last_clean":"2024-09-10T20:36:59.260281+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:36:59.260281+0000","last_undegraded":"2024-09-10T20:36:59.260281+0000","last_fullsized":"2024-09-10T20:36:59.260281+0000","mapping_epoch":35,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17866752,"num_objects":4362,"num_object_clones":0,"num_object_copies":8724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16242,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'16708","reported_seq":16844,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:58.918217+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:36:58.918217+0000","last_peered":"2024-09-10T20:36:58.918217+0000","last_clean":"2024-09-10T20:36:58.918217+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:36:58.918217+0000","last_undegraded":"2024-09-10T20:36:58.918217+0000","last_fullsized":"2024-09-10T20:36:58.918217+0000","mapping_epoch":34,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17862656,"num_objects":4361,"num_object_clones":0,"num_object_copies":8722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13704,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'16278","reported_seq":16530,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:58.964742+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:36:58.964742+0000","last_peered":"2024-09-10T20:36:58.964742+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:36:58.964742+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17174528,"num_objects":4193,"num_object_clones":0,"num_object_copies":8386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13142,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":632}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'16292","reported_seq":16578,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:58.960000+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:36:58.960000+0000","last_peered":"2024-09-10T20:36:58.960000+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:36:58.960000+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17678336,"num_objects":4316,"num_object_clones":0,"num_object_copies":8632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12887,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":830}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'16500","reported_seq":16634,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:58.903157+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:36:58.903157+0000","last_peered":"2024-09-10T20:36:58.903157+0000","last_clean":"2024-09-10T20:36:58.903157+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:36:58.903157+0000","last_undegraded":"2024-09-10T20:36:58.903157+0000","last_fullsized":"2024-09-10T20:36:58.903157+0000","mapping_epoch":34,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17350656,"num_objects":4236,"num_object_clones":0,"num_object_copies":8472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13571,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'16135","reported_seq":16335,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:59.264177+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:36:59.264177+0000","last_peered":"2024-09-10T20:36:59.264177+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:36:59.264177+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17629184,"num_objects":4304,"num_object_clones":0,"num_object_copies":8608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14128,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":807}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318026+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318026+0000","last_peered":"2024-09-10T20:36:30.318026+0000","last_clean":"2024-09-10T20:36:30.318026+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318026+0000","last_undegraded":"2024-09-10T20:36:30.318026+0000","last_fullsized":"2024-09-10T20:36:30.318026+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'16468","reported_seq":16684,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:58.976827+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:36:58.976827+0000","last_peered":"2024-09-10T20:36:58.976827+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:36:58.976827+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17458118,"num_objects":4264,"num_object_clones":0,"num_object_copies":8528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16466,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":698}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315256+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315256+0000","last_peered":"2024-09-10T20:36:30.315256+0000","last_clean":"2024-09-10T20:36:30.315256+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315256+0000","last_undegraded":"2024-09-10T20:36:30.315256+0000","last_fullsized":"2024-09-10T20:36:30.315256+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'16314","reported_seq":16506,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:59.211855+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:36:59.211855+0000","last_peered":"2024-09-10T20:36:59.211855+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:36:59.211855+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17412096,"num_objects":4251,"num_object_clones":0,"num_object_copies":8502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14142,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":645}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'16425","reported_seq":23795,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:59.231048+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:36:59.231048+0000","last_peered":"2024-09-10T20:36:59.231048+0000","last_clean":"2024-09-10T20:36:59.231048+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:36:59.231048+0000","last_undegraded":"2024-09-10T20:36:59.231048+0000","last_fullsized":"2024-09-10T20:36:59.231048+0000","mapping_epoch":48,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17362944,"num_objects":4239,"num_object_clones":0,"num_object_copies":8478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14211,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'16491","reported_seq":16746,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:58.976410+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:36:58.976410+0000","last_peered":"2024-09-10T20:36:58.976410+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:36:58.976410+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17772544,"num_objects":4339,"num_object_clones":0,"num_object_copies":8678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13214,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":747}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'16314","reported_seq":17933,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:59.005052+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:36:59.005052+0000","last_peered":"2024-09-10T20:36:59.005052+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:36:59.005052+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17489920,"num_objects":4270,"num_object_clones":0,"num_object_copies":8540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2800,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13008,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":735,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319607+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319607+0000","last_peered":"2024-09-10T20:36:30.319607+0000","last_clean":"2024-09-10T20:36:30.319607+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319607+0000","last_undegraded":"2024-09-10T20:36:30.319607+0000","last_fullsized":"2024-09-10T20:36:30.319607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'16707","reported_seq":16964,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:36:58.947430+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:36:58.947430+0000","last_peered":"2024-09-10T20:36:58.947430+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:36:58.947430+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17526784,"num_objects":4279,"num_object_clones":0,"num_object_copies":8558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13534,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":624}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'16320","reported_seq":23447,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:00.868115+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:37:00.868115+0000","last_peered":"2024-09-10T20:37:00.868115+0000","last_clean":"2024-09-10T20:37:00.868115+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:37:00.868115+0000","last_undegraded":"2024-09-10T20:37:00.868115+0000","last_fullsized":"2024-09-10T20:37:00.868115+0000","mapping_epoch":46,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17187299,"num_objects":4197,"num_object_clones":0,"num_object_copies":8394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4197,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12927,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":280331713,"num_objects":68444,"num_object_clones":0,"num_object_copies":136888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31317,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68444,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":226973,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11832,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":735686656,"data_stored":735620704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160777,"ondisk_log_size":160777,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182501,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":674808,"kb_used_data":264164,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93697032,"statfs":{"total":96636764160,"available":95945760768,"internally_reserved":0,"allocated":270503936,"data_stored":270241988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":420474538},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,11,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986273,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":573720,"kb_used_data":253256,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93798120,"statfs":{"total":96636764160,"available":96049274880,"internally_reserved":0,"allocated":259334144,"data_stored":259072201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328134314},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659220,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627436,"kb_used_data":204252,"kb_used_omap":4,"kb_used_meta":423163,"kb_avail":93744404,"statfs":{"total":96636764160,"available":95994269696,"internally_reserved":0,"allocated":209154048,"data_stored":208841059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":433319598},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":269795328,"data_stored":269791715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":207441920,"data_stored":207387266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":258449408,"data_stored":258441723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:37:04.753 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:37:05.305 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:05.306 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:37:05.337 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2024-09-10T20:37:05.000642+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":271500756,"num_objects":66290,"num_object_clones":0,"num_object_copies":132580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31210,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66290,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":229131,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11941,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1872136,"kb_used_data":709512,"kb_used_omap":12,"kb_used_meta":1162547,"kb_avail":281243384,"statfs":{"total":289910292480,"available":287993225216,"internally_reserved":0,"allocated":726540288,"data_stored":725703408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1190448130},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,5,2,6,10,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29159424,"num_objects":-7119,"num_object_clones":0,"num_object_copies":-14238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-315,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7119,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004915"},"pg_stats":[{"pgid":"2.f","version":"49'16536","reported_seq":16758,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:04.010087+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:37:04.010087+0000","last_peered":"2024-09-10T20:37:04.010087+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:37:04.010087+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16883712,"num_objects":4122,"num_object_clones":0,"num_object_copies":8244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16536,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":581}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'16756","reported_seq":16875,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:04.261578+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:37:04.261578+0000","last_peered":"2024-09-10T20:37:04.261578+0000","last_clean":"2024-09-10T20:37:04.261578+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:37:04.261578+0000","last_undegraded":"2024-09-10T20:37:04.261578+0000","last_fullsized":"2024-09-10T20:37:04.261578+0000","mapping_epoch":35,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17145856,"num_objects":4186,"num_object_clones":0,"num_object_copies":8372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14942,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'16490","reported_seq":23848,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:00.853306+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:37:00.853306+0000","last_peered":"2024-09-10T20:37:00.853306+0000","last_clean":"2024-09-10T20:37:00.853306+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:37:00.853306+0000","last_undegraded":"2024-09-10T20:37:00.853306+0000","last_fullsized":"2024-09-10T20:37:00.853306+0000","mapping_epoch":44,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17305624,"num_objects":4226,"num_object_clones":0,"num_object_copies":8452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4226,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14618,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'16413","reported_seq":16500,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:04.301780+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:37:04.301780+0000","last_peered":"2024-09-10T20:37:04.301780+0000","last_clean":"2024-09-10T20:37:04.301780+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:37:04.301780+0000","last_undegraded":"2024-09-10T20:37:04.301780+0000","last_fullsized":"2024-09-10T20:37:04.301780+0000","mapping_epoch":35,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17166336,"num_objects":4191,"num_object_clones":0,"num_object_copies":8382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16413,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'16851","reported_seq":16987,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:04.056433+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:37:04.056433+0000","last_peered":"2024-09-10T20:37:04.056433+0000","last_clean":"2024-09-10T20:37:04.056433+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:37:04.056433+0000","last_undegraded":"2024-09-10T20:37:04.056433+0000","last_fullsized":"2024-09-10T20:37:04.056433+0000","mapping_epoch":34,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17276928,"num_objects":4218,"num_object_clones":0,"num_object_copies":8436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13847,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'16447","reported_seq":16699,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:03.990920+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:37:03.990920+0000","last_peered":"2024-09-10T20:37:03.990920+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:37:03.990920+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16482304,"num_objects":4024,"num_object_clones":0,"num_object_copies":8048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13311,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":463}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'16445","reported_seq":16731,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:04.056879+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:37:04.056879+0000","last_peered":"2024-09-10T20:37:04.056879+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:37:04.056879+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17051648,"num_objects":4163,"num_object_clones":0,"num_object_copies":8326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13040,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":677}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'16645","reported_seq":16779,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:04.036703+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:37:04.036703+0000","last_peered":"2024-09-10T20:37:04.036703+0000","last_clean":"2024-09-10T20:37:04.036703+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:37:04.036703+0000","last_undegraded":"2024-09-10T20:37:04.036703+0000","last_fullsized":"2024-09-10T20:37:04.036703+0000","mapping_epoch":34,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16756736,"num_objects":4091,"num_object_clones":0,"num_object_copies":8182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13716,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'16284","reported_seq":16484,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:04.274248+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:37:04.274248+0000","last_peered":"2024-09-10T20:37:04.274248+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:37:04.274248+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17018880,"num_objects":4155,"num_object_clones":0,"num_object_copies":8310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14277,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":658}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318026+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318026+0000","last_peered":"2024-09-10T20:36:30.318026+0000","last_clean":"2024-09-10T20:36:30.318026+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318026+0000","last_undegraded":"2024-09-10T20:36:30.318026+0000","last_fullsized":"2024-09-10T20:36:30.318026+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'16602","reported_seq":16818,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:03.968134+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:37:03.968134+0000","last_peered":"2024-09-10T20:37:03.968134+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:37:03.968134+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16909254,"num_objects":4130,"num_object_clones":0,"num_object_copies":8260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16600,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":564}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315256+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315256+0000","last_peered":"2024-09-10T20:36:30.315256+0000","last_clean":"2024-09-10T20:36:30.315256+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315256+0000","last_undegraded":"2024-09-10T20:36:30.315256+0000","last_fullsized":"2024-09-10T20:36:30.315256+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'16464","reported_seq":16656,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:04.242044+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:37:04.242044+0000","last_peered":"2024-09-10T20:37:04.242044+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:37:04.242044+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16797696,"num_objects":4101,"num_object_clones":0,"num_object_copies":8202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14292,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":495}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'16578","reported_seq":23947,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:04.302434+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:37:04.302434+0000","last_peered":"2024-09-10T20:37:04.302434+0000","last_clean":"2024-09-10T20:37:04.302434+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:37:04.302434+0000","last_undegraded":"2024-09-10T20:37:04.302434+0000","last_fullsized":"2024-09-10T20:37:04.302434+0000","mapping_epoch":48,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16736256,"num_objects":4086,"num_object_clones":0,"num_object_copies":8172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14364,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'16650","reported_seq":16903,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:04.076990+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:37:04.076990+0000","last_peered":"2024-09-10T20:37:04.076990+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:37:04.076990+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17121280,"num_objects":4180,"num_object_clones":0,"num_object_copies":8360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13373,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":588}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'16486","reported_seq":18319,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:04.072149+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:37:04.072149+0000","last_peered":"2024-09-10T20:37:04.072149+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:37:04.072149+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16785408,"num_objects":4098,"num_object_clones":0,"num_object_copies":8196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2693,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13180,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":842,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1405}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319607+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319607+0000","last_peered":"2024-09-10T20:36:30.319607+0000","last_clean":"2024-09-10T20:36:30.319607+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319607+0000","last_undegraded":"2024-09-10T20:36:30.319607+0000","last_fullsized":"2024-09-10T20:36:30.319607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'16866","reported_seq":17123,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:03.969197+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:37:03.969197+0000","last_peered":"2024-09-10T20:37:03.969197+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:37:03.969197+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16875520,"num_objects":4120,"num_object_clones":0,"num_object_copies":8240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13693,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":465}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'16320","reported_seq":23447,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:00.868115+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:37:00.868115+0000","last_peered":"2024-09-10T20:37:00.868115+0000","last_clean":"2024-09-10T20:37:00.868115+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:37:00.868115+0000","last_undegraded":"2024-09-10T20:37:00.868115+0000","last_fullsized":"2024-09-10T20:37:00.868115+0000","mapping_epoch":46,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17187299,"num_objects":4197,"num_object_clones":0,"num_object_copies":8394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4197,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12927,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":271500737,"num_objects":66288,"num_object_clones":0,"num_object_copies":132576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31210,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66288,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":229129,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11939,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":723156992,"data_stored":723091040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160833,"ondisk_log_size":160833,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182502,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":671552,"kb_used_data":256736,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93700288,"statfs":{"total":96636764160,"available":95949094912,"internally_reserved":0,"allocated":262897664,"data_stored":262635716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":424734378},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,3,0,2,5,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986274,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":573148,"kb_used_data":248524,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93798692,"statfs":{"total":96636764160,"available":96049860608,"internally_reserved":0,"allocated":254488576,"data_stored":254226633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332394154},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659220,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627436,"kb_used_data":204252,"kb_used_omap":4,"kb_used_meta":423163,"kb_avail":93744404,"statfs":{"total":96636764160,"available":95994269696,"internally_reserved":0,"allocated":209154048,"data_stored":208841059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":433319598},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":262193152,"data_stored":262189539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":207441920,"data_stored":207387266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":253521920,"data_stored":253514235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:37:05.341 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:37:05.341 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'16536", 'reported_seq': 16758, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:04.010087+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:37:04.010087+0000', 'last_peered': '2024-09-10T20:37:04.010087+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:37:04.010087+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "43'6500", 'ondisk_log_start': "43'6500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16883712, 'num_objects': 4122, 'num_object_clones': 0, 'num_object_copies': 8244, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4122, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16536, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 581}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:05.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:37:05.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'16447", 'reported_seq': 16699, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:03.990920+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:37:03.990920+0000', 'last_peered': '2024-09-10T20:37:03.990920+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:37:03.990920+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "43'6400", 'ondisk_log_start': "43'6400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16482304, 'num_objects': 4024, 'num_object_clones': 0, 'num_object_copies': 8048, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4024, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13311, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 463}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:05.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:37:05.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'16445", 'reported_seq': 16731, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:04.056879+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:37:04.056879+0000', 'last_peered': '2024-09-10T20:37:04.056879+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:37:04.056879+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "43'6400", 'ondisk_log_start': "43'6400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17051648, 'num_objects': 4163, 'num_object_clones': 0, 'num_object_copies': 8326, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4163, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13040, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 677}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:05.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:37:05.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'16284", 'reported_seq': 16484, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:04.274248+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:37:04.274248+0000', 'last_peered': '2024-09-10T20:37:04.274248+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:37:04.274248+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "43'6200", 'ondisk_log_start': "43'6200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17018880, 'num_objects': 4155, 'num_object_clones': 0, 'num_object_copies': 8310, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4155, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14277, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 658}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:37:05.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:37:05.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'16602", 'reported_seq': 16818, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:03.968134+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:37:03.968134+0000', 'last_peered': '2024-09-10T20:37:03.968134+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:37:03.968134+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "43'6600", 'ondisk_log_start': "43'6600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16909254, 'num_objects': 4130, 'num_object_clones': 0, 'num_object_copies': 8260, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4130, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16600, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 564}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:05.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:37:05.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'16464", 'reported_seq': 16656, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:04.242044+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:37:04.242044+0000', 'last_peered': '2024-09-10T20:37:04.242044+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:37:04.242044+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "43'6400", 'ondisk_log_start': "43'6400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16797696, 'num_objects': 4101, 'num_object_clones': 0, 'num_object_copies': 8202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4101, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14292, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 495}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:37:05.343 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:37:05.343 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'16650", 'reported_seq': 16903, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:04.076990+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:37:04.076990+0000', 'last_peered': '2024-09-10T20:37:04.076990+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:37:04.076990+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "43'6600", 'ondisk_log_start': "43'6600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17121280, 'num_objects': 4180, 'num_object_clones': 0, 'num_object_copies': 8360, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4180, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13373, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 588}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:05.343 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:37:05.343 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'16486", 'reported_seq': 18319, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:04.072149+0000', 'last_change': '2024-09-10T20:36:28.778006+0000', 'last_active': '2024-09-10T20:37:04.072149+0000', 'last_peered': '2024-09-10T20:37:04.072149+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:37:04.072149+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "43'6400", 'ondisk_log_start': "43'6400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16785408, 'num_objects': 4098, 'num_object_clones': 0, 'num_object_copies': 8196, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2693, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4098, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13180, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 842, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1405}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:05.343 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:37:05.343 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'16866", 'reported_seq': 17123, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:03.969197+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:37:03.969197+0000', 'last_peered': '2024-09-10T20:37:03.969197+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:37:03.969197+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16875520, 'num_objects': 4120, 'num_object_clones': 0, 'num_object_copies': 8240, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4120, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13693, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 465}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:05.343 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:37:06.035 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:06.035 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31210/132580 objects degraded (23.541%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":537,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":66290,"data_bytes":271500756,"bytes_used":1917067264,"bytes_avail":287993225216,"bytes_total":289910292480,"degraded_objects":31210,"degraded_total":132580,"degraded_ratio":0.23540503846734048,"recovering_objects_per_sec":26,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":315,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":593},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:37:06.058 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:37:06.058 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:37:06.613 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:06.614 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:37:06.642 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2024-09-10T20:37:05.000642+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":271500756,"num_objects":66290,"num_object_clones":0,"num_object_copies":132580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31210,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66290,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":229131,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11941,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1872136,"kb_used_data":709512,"kb_used_omap":12,"kb_used_meta":1162547,"kb_avail":281243384,"statfs":{"total":289910292480,"available":287993225216,"internally_reserved":0,"allocated":726540288,"data_stored":725703408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1190448130},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,5,2,6,10,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29159424,"num_objects":-7119,"num_object_clones":0,"num_object_copies":-14238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-315,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7119,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004915"},"pg_stats":[{"pgid":"2.f","version":"49'16536","reported_seq":16758,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:04.010087+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:37:04.010087+0000","last_peered":"2024-09-10T20:37:04.010087+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:37:04.010087+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16883712,"num_objects":4122,"num_object_clones":0,"num_object_copies":8244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16536,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":581}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'16756","reported_seq":16875,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:04.261578+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:37:04.261578+0000","last_peered":"2024-09-10T20:37:04.261578+0000","last_clean":"2024-09-10T20:37:04.261578+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:37:04.261578+0000","last_undegraded":"2024-09-10T20:37:04.261578+0000","last_fullsized":"2024-09-10T20:37:04.261578+0000","mapping_epoch":35,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17145856,"num_objects":4186,"num_object_clones":0,"num_object_copies":8372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14942,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'16490","reported_seq":23848,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:00.853306+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:37:00.853306+0000","last_peered":"2024-09-10T20:37:00.853306+0000","last_clean":"2024-09-10T20:37:00.853306+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:37:00.853306+0000","last_undegraded":"2024-09-10T20:37:00.853306+0000","last_fullsized":"2024-09-10T20:37:00.853306+0000","mapping_epoch":44,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17305624,"num_objects":4226,"num_object_clones":0,"num_object_copies":8452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4226,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14618,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'16413","reported_seq":16500,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:04.301780+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:37:04.301780+0000","last_peered":"2024-09-10T20:37:04.301780+0000","last_clean":"2024-09-10T20:37:04.301780+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:37:04.301780+0000","last_undegraded":"2024-09-10T20:37:04.301780+0000","last_fullsized":"2024-09-10T20:37:04.301780+0000","mapping_epoch":35,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17166336,"num_objects":4191,"num_object_clones":0,"num_object_copies":8382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16413,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'16851","reported_seq":16987,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:04.056433+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:37:04.056433+0000","last_peered":"2024-09-10T20:37:04.056433+0000","last_clean":"2024-09-10T20:37:04.056433+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:37:04.056433+0000","last_undegraded":"2024-09-10T20:37:04.056433+0000","last_fullsized":"2024-09-10T20:37:04.056433+0000","mapping_epoch":34,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17276928,"num_objects":4218,"num_object_clones":0,"num_object_copies":8436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13847,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'16447","reported_seq":16699,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:03.990920+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:37:03.990920+0000","last_peered":"2024-09-10T20:37:03.990920+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:37:03.990920+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16482304,"num_objects":4024,"num_object_clones":0,"num_object_copies":8048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13311,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":463}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'16445","reported_seq":16731,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:04.056879+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:37:04.056879+0000","last_peered":"2024-09-10T20:37:04.056879+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:37:04.056879+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17051648,"num_objects":4163,"num_object_clones":0,"num_object_copies":8326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13040,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":677}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'16645","reported_seq":16779,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:04.036703+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:37:04.036703+0000","last_peered":"2024-09-10T20:37:04.036703+0000","last_clean":"2024-09-10T20:37:04.036703+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:37:04.036703+0000","last_undegraded":"2024-09-10T20:37:04.036703+0000","last_fullsized":"2024-09-10T20:37:04.036703+0000","mapping_epoch":34,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16756736,"num_objects":4091,"num_object_clones":0,"num_object_copies":8182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13716,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'16284","reported_seq":16484,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:04.274248+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:37:04.274248+0000","last_peered":"2024-09-10T20:37:04.274248+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:37:04.274248+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17018880,"num_objects":4155,"num_object_clones":0,"num_object_copies":8310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14277,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":658}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318026+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318026+0000","last_peered":"2024-09-10T20:36:30.318026+0000","last_clean":"2024-09-10T20:36:30.318026+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318026+0000","last_undegraded":"2024-09-10T20:36:30.318026+0000","last_fullsized":"2024-09-10T20:36:30.318026+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'16602","reported_seq":16818,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:03.968134+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:37:03.968134+0000","last_peered":"2024-09-10T20:37:03.968134+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:37:03.968134+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16909254,"num_objects":4130,"num_object_clones":0,"num_object_copies":8260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16600,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":564}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315256+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315256+0000","last_peered":"2024-09-10T20:36:30.315256+0000","last_clean":"2024-09-10T20:36:30.315256+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315256+0000","last_undegraded":"2024-09-10T20:36:30.315256+0000","last_fullsized":"2024-09-10T20:36:30.315256+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'16464","reported_seq":16656,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:04.242044+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:37:04.242044+0000","last_peered":"2024-09-10T20:37:04.242044+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:37:04.242044+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16797696,"num_objects":4101,"num_object_clones":0,"num_object_copies":8202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14292,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":495}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'16578","reported_seq":23947,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:04.302434+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:37:04.302434+0000","last_peered":"2024-09-10T20:37:04.302434+0000","last_clean":"2024-09-10T20:37:04.302434+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:37:04.302434+0000","last_undegraded":"2024-09-10T20:37:04.302434+0000","last_fullsized":"2024-09-10T20:37:04.302434+0000","mapping_epoch":48,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16736256,"num_objects":4086,"num_object_clones":0,"num_object_copies":8172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14364,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'16650","reported_seq":16903,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:04.076990+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:37:04.076990+0000","last_peered":"2024-09-10T20:37:04.076990+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:37:04.076990+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17121280,"num_objects":4180,"num_object_clones":0,"num_object_copies":8360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13373,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":588}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'16486","reported_seq":18319,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:04.072149+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:37:04.072149+0000","last_peered":"2024-09-10T20:37:04.072149+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:37:04.072149+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16785408,"num_objects":4098,"num_object_clones":0,"num_object_copies":8196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2693,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13180,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":842,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1405}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319607+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319607+0000","last_peered":"2024-09-10T20:36:30.319607+0000","last_clean":"2024-09-10T20:36:30.319607+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319607+0000","last_undegraded":"2024-09-10T20:36:30.319607+0000","last_fullsized":"2024-09-10T20:36:30.319607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'16866","reported_seq":17123,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:03.969197+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:37:03.969197+0000","last_peered":"2024-09-10T20:37:03.969197+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:37:03.969197+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16875520,"num_objects":4120,"num_object_clones":0,"num_object_copies":8240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13693,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":465}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'16320","reported_seq":23447,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:00.868115+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:37:00.868115+0000","last_peered":"2024-09-10T20:37:00.868115+0000","last_clean":"2024-09-10T20:37:00.868115+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:37:00.868115+0000","last_undegraded":"2024-09-10T20:37:00.868115+0000","last_fullsized":"2024-09-10T20:37:00.868115+0000","mapping_epoch":46,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17187299,"num_objects":4197,"num_object_clones":0,"num_object_copies":8394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4197,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12927,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":271500737,"num_objects":66288,"num_object_clones":0,"num_object_copies":132576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31210,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66288,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":229129,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11939,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":723156992,"data_stored":723091040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160833,"ondisk_log_size":160833,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182502,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":671552,"kb_used_data":256736,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93700288,"statfs":{"total":96636764160,"available":95949094912,"internally_reserved":0,"allocated":262897664,"data_stored":262635716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":424734378},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,3,0,2,5,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986274,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":573148,"kb_used_data":248524,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93798692,"statfs":{"total":96636764160,"available":96049860608,"internally_reserved":0,"allocated":254488576,"data_stored":254226633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332394154},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659220,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627436,"kb_used_data":204252,"kb_used_omap":4,"kb_used_meta":423163,"kb_avail":93744404,"statfs":{"total":96636764160,"available":95994269696,"internally_reserved":0,"allocated":209154048,"data_stored":208841059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":433319598},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":262193152,"data_stored":262189539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":207441920,"data_stored":207387266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":253521920,"data_stored":253514235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:37:09.645 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:37:10.199 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:10.200 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:37:10.225 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":313,"stamp":"2024-09-10T20:37:09.001912+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":270288340,"num_objects":65994,"num_object_clones":0,"num_object_copies":131988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31210,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65994,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":229427,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11941,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160832,"ondisk_log_size":160832,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1851156,"kb_used_data":701012,"kb_used_omap":12,"kb_used_meta":1150067,"kb_avail":281264364,"statfs":{"total":289910292480,"available":288014708736,"internally_reserved":0,"allocated":717836288,"data_stored":716999408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1177668610},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,3,4,2,4,10,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20197376,"num_objects":-4931,"num_object_clones":0,"num_object_copies":-9862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-219,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4931,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":219,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004820"},"pg_stats":[{"pgid":"2.f","version":"49'16536","reported_seq":16758,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:04.010087+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:37:04.010087+0000","last_peered":"2024-09-10T20:37:04.010087+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:37:04.010087+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16883712,"num_objects":4122,"num_object_clones":0,"num_object_copies":8244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16536,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":581}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'16756","reported_seq":16875,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:04.261578+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:37:04.261578+0000","last_peered":"2024-09-10T20:37:04.261578+0000","last_clean":"2024-09-10T20:37:04.261578+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:37:04.261578+0000","last_undegraded":"2024-09-10T20:37:04.261578+0000","last_fullsized":"2024-09-10T20:37:04.261578+0000","mapping_epoch":35,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17145856,"num_objects":4186,"num_object_clones":0,"num_object_copies":8372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14942,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'16639","reported_seq":23997,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:05.897219+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:37:05.897219+0000","last_peered":"2024-09-10T20:37:05.897219+0000","last_clean":"2024-09-10T20:37:05.897219+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:37:05.897219+0000","last_undegraded":"2024-09-10T20:37:05.897219+0000","last_fullsized":"2024-09-10T20:37:05.897219+0000","mapping_epoch":44,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16695320,"num_objects":4077,"num_object_clones":0,"num_object_copies":8154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4077,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14767,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'16413","reported_seq":16500,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:04.301780+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:37:04.301780+0000","last_peered":"2024-09-10T20:37:04.301780+0000","last_clean":"2024-09-10T20:37:04.301780+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:37:04.301780+0000","last_undegraded":"2024-09-10T20:37:04.301780+0000","last_fullsized":"2024-09-10T20:37:04.301780+0000","mapping_epoch":35,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17166336,"num_objects":4191,"num_object_clones":0,"num_object_copies":8382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16413,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'16851","reported_seq":16987,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:04.056433+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:37:04.056433+0000","last_peered":"2024-09-10T20:37:04.056433+0000","last_clean":"2024-09-10T20:37:04.056433+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:37:04.056433+0000","last_undegraded":"2024-09-10T20:37:04.056433+0000","last_fullsized":"2024-09-10T20:37:04.056433+0000","mapping_epoch":34,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17276928,"num_objects":4218,"num_object_clones":0,"num_object_copies":8436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13847,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'16447","reported_seq":16699,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:03.990920+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:37:03.990920+0000","last_peered":"2024-09-10T20:37:03.990920+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:37:03.990920+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16482304,"num_objects":4024,"num_object_clones":0,"num_object_copies":8048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13311,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":463}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'16445","reported_seq":16731,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:04.056879+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:37:04.056879+0000","last_peered":"2024-09-10T20:37:04.056879+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:37:04.056879+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17051648,"num_objects":4163,"num_object_clones":0,"num_object_copies":8326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13040,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":677}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'16645","reported_seq":16779,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:04.036703+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:37:04.036703+0000","last_peered":"2024-09-10T20:37:04.036703+0000","last_clean":"2024-09-10T20:37:04.036703+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:37:04.036703+0000","last_undegraded":"2024-09-10T20:37:04.036703+0000","last_fullsized":"2024-09-10T20:37:04.036703+0000","mapping_epoch":34,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16756736,"num_objects":4091,"num_object_clones":0,"num_object_copies":8182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13716,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'16284","reported_seq":16484,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:04.274248+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:37:04.274248+0000","last_peered":"2024-09-10T20:37:04.274248+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:37:04.274248+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17018880,"num_objects":4155,"num_object_clones":0,"num_object_copies":8310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14277,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":658}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318026+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318026+0000","last_peered":"2024-09-10T20:36:30.318026+0000","last_clean":"2024-09-10T20:36:30.318026+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318026+0000","last_undegraded":"2024-09-10T20:36:30.318026+0000","last_fullsized":"2024-09-10T20:36:30.318026+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'16602","reported_seq":16818,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:03.968134+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:37:03.968134+0000","last_peered":"2024-09-10T20:37:03.968134+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:37:03.968134+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16909254,"num_objects":4130,"num_object_clones":0,"num_object_copies":8260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16600,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":564}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315256+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315256+0000","last_peered":"2024-09-10T20:36:30.315256+0000","last_clean":"2024-09-10T20:36:30.315256+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315256+0000","last_undegraded":"2024-09-10T20:36:30.315256+0000","last_fullsized":"2024-09-10T20:36:30.315256+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'16464","reported_seq":16656,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:04.242044+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:37:04.242044+0000","last_peered":"2024-09-10T20:37:04.242044+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:37:04.242044+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16797696,"num_objects":4101,"num_object_clones":0,"num_object_copies":8202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14292,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":495}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'16578","reported_seq":23947,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:04.302434+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:37:04.302434+0000","last_peered":"2024-09-10T20:37:04.302434+0000","last_clean":"2024-09-10T20:37:04.302434+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:37:04.302434+0000","last_undegraded":"2024-09-10T20:37:04.302434+0000","last_fullsized":"2024-09-10T20:37:04.302434+0000","mapping_epoch":48,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16736256,"num_objects":4086,"num_object_clones":0,"num_object_copies":8172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14364,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'16650","reported_seq":16903,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:04.076990+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:37:04.076990+0000","last_peered":"2024-09-10T20:37:04.076990+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:37:04.076990+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17121280,"num_objects":4180,"num_object_clones":0,"num_object_copies":8360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13373,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":588}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'16486","reported_seq":18319,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:04.072149+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:37:04.072149+0000","last_peered":"2024-09-10T20:37:04.072149+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:37:04.072149+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16785408,"num_objects":4098,"num_object_clones":0,"num_object_copies":8196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2693,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13180,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":842,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1405}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319607+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319607+0000","last_peered":"2024-09-10T20:36:30.319607+0000","last_clean":"2024-09-10T20:36:30.319607+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319607+0000","last_undegraded":"2024-09-10T20:36:30.319607+0000","last_fullsized":"2024-09-10T20:36:30.319607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'16866","reported_seq":17123,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:03.969197+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:37:03.969197+0000","last_peered":"2024-09-10T20:37:03.969197+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:37:03.969197+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16875520,"num_objects":4120,"num_object_clones":0,"num_object_copies":8240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13693,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":465}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'16467","reported_seq":23594,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:05.875592+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:37:05.875592+0000","last_peered":"2024-09-10T20:37:05.875592+0000","last_clean":"2024-09-10T20:37:05.875592+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:37:05.875592+0000","last_undegraded":"2024-09-10T20:37:05.875592+0000","last_fullsized":"2024-09-10T20:37:05.875592+0000","mapping_epoch":46,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16585187,"num_objects":4050,"num_object_clones":0,"num_object_copies":8100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13074,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":270288321,"num_objects":65992,"num_object_clones":0,"num_object_copies":131984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31210,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65992,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":229425,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11939,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":715673600,"data_stored":715607648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160829,"ondisk_log_size":160829,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182502,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":671552,"kb_used_data":256736,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93700288,"statfs":{"total":96636764160,"available":95949094912,"internally_reserved":0,"allocated":262897664,"data_stored":262635716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":424734378},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,3,0,2,5,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986274,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":573148,"kb_used_data":248524,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93798692,"statfs":{"total":96636764160,"available":96049860608,"internally_reserved":0,"allocated":254488576,"data_stored":254226633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332394154},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659221,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":606456,"kb_used_data":195752,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93765384,"statfs":{"total":96636764160,"available":96015753216,"internally_reserved":0,"allocated":200450048,"data_stored":200137059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":420540078},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,1,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":262193152,"data_stored":262189539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":199958528,"data_stored":199903874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":253521920,"data_stored":253514235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:37:10.226 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:37:10.773 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:10.774 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:37:10.801 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":313,"stamp":"2024-09-10T20:37:09.001912+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":270288340,"num_objects":65994,"num_object_clones":0,"num_object_copies":131988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31210,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65994,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":229427,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11941,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160832,"ondisk_log_size":160832,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1851156,"kb_used_data":701012,"kb_used_omap":12,"kb_used_meta":1150067,"kb_avail":281264364,"statfs":{"total":289910292480,"available":288014708736,"internally_reserved":0,"allocated":717836288,"data_stored":716999408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1177668610},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,3,4,2,4,10,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20197376,"num_objects":-4931,"num_object_clones":0,"num_object_copies":-9862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-219,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4931,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":219,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004820"},"pg_stats":[{"pgid":"2.f","version":"49'16536","reported_seq":16758,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:04.010087+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:37:04.010087+0000","last_peered":"2024-09-10T20:37:04.010087+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:37:04.010087+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16883712,"num_objects":4122,"num_object_clones":0,"num_object_copies":8244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16536,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":581}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'16756","reported_seq":16875,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:04.261578+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:37:04.261578+0000","last_peered":"2024-09-10T20:37:04.261578+0000","last_clean":"2024-09-10T20:37:04.261578+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:37:04.261578+0000","last_undegraded":"2024-09-10T20:37:04.261578+0000","last_fullsized":"2024-09-10T20:37:04.261578+0000","mapping_epoch":35,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17145856,"num_objects":4186,"num_object_clones":0,"num_object_copies":8372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14942,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'16639","reported_seq":23997,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:05.897219+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:37:05.897219+0000","last_peered":"2024-09-10T20:37:05.897219+0000","last_clean":"2024-09-10T20:37:05.897219+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:37:05.897219+0000","last_undegraded":"2024-09-10T20:37:05.897219+0000","last_fullsized":"2024-09-10T20:37:05.897219+0000","mapping_epoch":44,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16695320,"num_objects":4077,"num_object_clones":0,"num_object_copies":8154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4077,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14767,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'16413","reported_seq":16500,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:04.301780+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:37:04.301780+0000","last_peered":"2024-09-10T20:37:04.301780+0000","last_clean":"2024-09-10T20:37:04.301780+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:37:04.301780+0000","last_undegraded":"2024-09-10T20:37:04.301780+0000","last_fullsized":"2024-09-10T20:37:04.301780+0000","mapping_epoch":35,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17166336,"num_objects":4191,"num_object_clones":0,"num_object_copies":8382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16413,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'16851","reported_seq":16987,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:04.056433+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:37:04.056433+0000","last_peered":"2024-09-10T20:37:04.056433+0000","last_clean":"2024-09-10T20:37:04.056433+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:37:04.056433+0000","last_undegraded":"2024-09-10T20:37:04.056433+0000","last_fullsized":"2024-09-10T20:37:04.056433+0000","mapping_epoch":34,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17276928,"num_objects":4218,"num_object_clones":0,"num_object_copies":8436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13847,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'16447","reported_seq":16699,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:03.990920+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:37:03.990920+0000","last_peered":"2024-09-10T20:37:03.990920+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:37:03.990920+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16482304,"num_objects":4024,"num_object_clones":0,"num_object_copies":8048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13311,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":463}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'16445","reported_seq":16731,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:04.056879+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:37:04.056879+0000","last_peered":"2024-09-10T20:37:04.056879+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:37:04.056879+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17051648,"num_objects":4163,"num_object_clones":0,"num_object_copies":8326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13040,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":677}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'16645","reported_seq":16779,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:04.036703+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:37:04.036703+0000","last_peered":"2024-09-10T20:37:04.036703+0000","last_clean":"2024-09-10T20:37:04.036703+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:37:04.036703+0000","last_undegraded":"2024-09-10T20:37:04.036703+0000","last_fullsized":"2024-09-10T20:37:04.036703+0000","mapping_epoch":34,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16756736,"num_objects":4091,"num_object_clones":0,"num_object_copies":8182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13716,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'16284","reported_seq":16484,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:04.274248+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:37:04.274248+0000","last_peered":"2024-09-10T20:37:04.274248+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:37:04.274248+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17018880,"num_objects":4155,"num_object_clones":0,"num_object_copies":8310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14277,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":658}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318026+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318026+0000","last_peered":"2024-09-10T20:36:30.318026+0000","last_clean":"2024-09-10T20:36:30.318026+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318026+0000","last_undegraded":"2024-09-10T20:36:30.318026+0000","last_fullsized":"2024-09-10T20:36:30.318026+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'16602","reported_seq":16818,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:03.968134+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:37:03.968134+0000","last_peered":"2024-09-10T20:37:03.968134+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:37:03.968134+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16909254,"num_objects":4130,"num_object_clones":0,"num_object_copies":8260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16600,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":564}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315256+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315256+0000","last_peered":"2024-09-10T20:36:30.315256+0000","last_clean":"2024-09-10T20:36:30.315256+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315256+0000","last_undegraded":"2024-09-10T20:36:30.315256+0000","last_fullsized":"2024-09-10T20:36:30.315256+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'16464","reported_seq":16656,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:04.242044+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:37:04.242044+0000","last_peered":"2024-09-10T20:37:04.242044+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:37:04.242044+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16797696,"num_objects":4101,"num_object_clones":0,"num_object_copies":8202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14292,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":495}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'16578","reported_seq":23947,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:04.302434+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:37:04.302434+0000","last_peered":"2024-09-10T20:37:04.302434+0000","last_clean":"2024-09-10T20:37:04.302434+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:37:04.302434+0000","last_undegraded":"2024-09-10T20:37:04.302434+0000","last_fullsized":"2024-09-10T20:37:04.302434+0000","mapping_epoch":48,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16736256,"num_objects":4086,"num_object_clones":0,"num_object_copies":8172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14364,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'16650","reported_seq":16903,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:04.076990+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:37:04.076990+0000","last_peered":"2024-09-10T20:37:04.076990+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:37:04.076990+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17121280,"num_objects":4180,"num_object_clones":0,"num_object_copies":8360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13373,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":588}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'16486","reported_seq":18319,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:04.072149+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:37:04.072149+0000","last_peered":"2024-09-10T20:37:04.072149+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:37:04.072149+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16785408,"num_objects":4098,"num_object_clones":0,"num_object_copies":8196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2693,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13180,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":842,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1405}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319607+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319607+0000","last_peered":"2024-09-10T20:36:30.319607+0000","last_clean":"2024-09-10T20:36:30.319607+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319607+0000","last_undegraded":"2024-09-10T20:36:30.319607+0000","last_fullsized":"2024-09-10T20:36:30.319607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'16866","reported_seq":17123,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:03.969197+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:37:03.969197+0000","last_peered":"2024-09-10T20:37:03.969197+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:37:03.969197+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16875520,"num_objects":4120,"num_object_clones":0,"num_object_copies":8240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13693,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":465}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'16467","reported_seq":23594,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:05.875592+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:37:05.875592+0000","last_peered":"2024-09-10T20:37:05.875592+0000","last_clean":"2024-09-10T20:37:05.875592+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:37:05.875592+0000","last_undegraded":"2024-09-10T20:37:05.875592+0000","last_fullsized":"2024-09-10T20:37:05.875592+0000","mapping_epoch":46,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16585187,"num_objects":4050,"num_object_clones":0,"num_object_copies":8100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13074,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":270288321,"num_objects":65992,"num_object_clones":0,"num_object_copies":131984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31210,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65992,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":229425,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11939,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":715673600,"data_stored":715607648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160829,"ondisk_log_size":160829,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182502,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":671552,"kb_used_data":256736,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93700288,"statfs":{"total":96636764160,"available":95949094912,"internally_reserved":0,"allocated":262897664,"data_stored":262635716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":424734378},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,3,0,2,5,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986274,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":573148,"kb_used_data":248524,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93798692,"statfs":{"total":96636764160,"available":96049860608,"internally_reserved":0,"allocated":254488576,"data_stored":254226633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332394154},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659221,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":606456,"kb_used_data":195752,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93765384,"statfs":{"total":96636764160,"available":96015753216,"internally_reserved":0,"allocated":200450048,"data_stored":200137059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":420540078},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,1,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":262193152,"data_stored":262189539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":199958528,"data_stored":199903874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":253521920,"data_stored":253514235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:37:10.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:37:10.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'16536", 'reported_seq': 16758, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:04.010087+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:37:04.010087+0000', 'last_peered': '2024-09-10T20:37:04.010087+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:37:04.010087+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "43'6500", 'ondisk_log_start': "43'6500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16883712, 'num_objects': 4122, 'num_object_clones': 0, 'num_object_copies': 8244, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4122, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16536, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 581}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:10.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:37:10.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'16447", 'reported_seq': 16699, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:03.990920+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:37:03.990920+0000', 'last_peered': '2024-09-10T20:37:03.990920+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:37:03.990920+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "43'6400", 'ondisk_log_start': "43'6400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16482304, 'num_objects': 4024, 'num_object_clones': 0, 'num_object_copies': 8048, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4024, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13311, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 463}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:10.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:37:10.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'16445", 'reported_seq': 16731, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:04.056879+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:37:04.056879+0000', 'last_peered': '2024-09-10T20:37:04.056879+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:37:04.056879+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "43'6400", 'ondisk_log_start': "43'6400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17051648, 'num_objects': 4163, 'num_object_clones': 0, 'num_object_copies': 8326, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4163, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13040, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 677}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:10.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:37:10.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'16284", 'reported_seq': 16484, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:04.274248+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:37:04.274248+0000', 'last_peered': '2024-09-10T20:37:04.274248+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:37:04.274248+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "43'6200", 'ondisk_log_start': "43'6200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17018880, 'num_objects': 4155, 'num_object_clones': 0, 'num_object_copies': 8310, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4155, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14277, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 658}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:37:10.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:37:10.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'16602", 'reported_seq': 16818, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:03.968134+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:37:03.968134+0000', 'last_peered': '2024-09-10T20:37:03.968134+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:37:03.968134+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "43'6600", 'ondisk_log_start': "43'6600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16909254, 'num_objects': 4130, 'num_object_clones': 0, 'num_object_copies': 8260, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4130, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16600, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 564}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:10.804 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:37:10.804 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'16464", 'reported_seq': 16656, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:04.242044+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:37:04.242044+0000', 'last_peered': '2024-09-10T20:37:04.242044+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:37:04.242044+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "43'6400", 'ondisk_log_start': "43'6400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16797696, 'num_objects': 4101, 'num_object_clones': 0, 'num_object_copies': 8202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4101, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14292, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 495}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:37:10.804 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:37:10.804 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'16650", 'reported_seq': 16903, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:04.076990+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:37:04.076990+0000', 'last_peered': '2024-09-10T20:37:04.076990+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:37:04.076990+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "43'6600", 'ondisk_log_start': "43'6600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17121280, 'num_objects': 4180, 'num_object_clones': 0, 'num_object_copies': 8360, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4180, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13373, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 588}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:10.804 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:37:10.804 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'16486", 'reported_seq': 18319, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:04.072149+0000', 'last_change': '2024-09-10T20:36:28.778006+0000', 'last_active': '2024-09-10T20:37:04.072149+0000', 'last_peered': '2024-09-10T20:37:04.072149+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:37:04.072149+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "43'6400", 'ondisk_log_start': "43'6400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16785408, 'num_objects': 4098, 'num_object_clones': 0, 'num_object_copies': 8196, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2693, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4098, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13180, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 842, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1405}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:10.804 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:37:10.804 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'16866", 'reported_seq': 17123, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:03.969197+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:37:03.969197+0000', 'last_peered': '2024-09-10T20:37:03.969197+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:37:03.969197+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16875520, 'num_objects': 4120, 'num_object_clones': 0, 'num_object_copies': 8240, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4120, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13693, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 465}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:10.805 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:37:11.473 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:11.473 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31210/131988 objects degraded (23.646%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":543,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":65994,"data_bytes":270288340,"bytes_used":1895583744,"bytes_avail":288014708736,"bytes_total":289910292480,"degraded_objects":31210,"degraded_total":131988,"degraded_ratio":0.2364608903839743,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":219,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":410},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:37:11.500 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:37:11.501 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:37:12.090 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:12.090 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:37:12.112 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-09-10T20:37:11.002684+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":261498324,"num_objects":63848,"num_object_clones":0,"num_object_copies":127696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31119,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63848,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":231573,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12032,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160678,"ondisk_log_size":160678,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1847208,"kb_used_data":688748,"kb_used_omap":12,"kb_used_meta":1158387,"kb_avail":281268312,"statfs":{"total":289910292480,"available":288018751488,"internally_reserved":0,"allocated":705277952,"data_stored":704441072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1186188290},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,1,6,6,8,10,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28987392,"num_objects":-7077,"num_object_clones":0,"num_object_copies":-14154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-310,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7077,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005131"},"pg_stats":[{"pgid":"2.f","version":"49'16715","reported_seq":16937,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:09.071554+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:37:09.071554+0000","last_peered":"2024-09-10T20:37:09.071554+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:37:09.071554+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16150528,"num_objects":3943,"num_object_clones":0,"num_object_copies":7886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3943,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16715,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":402}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'16925","reported_seq":17045,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:09.181987+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:37:09.181987+0000","last_peered":"2024-09-10T20:37:09.181987+0000","last_clean":"2024-09-10T20:37:09.181987+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:37:09.181987+0000","last_undegraded":"2024-09-10T20:37:09.181987+0000","last_fullsized":"2024-09-10T20:37:09.181987+0000","mapping_epoch":35,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16453632,"num_objects":4017,"num_object_clones":0,"num_object_copies":8034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15111,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'16639","reported_seq":23997,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:05.897219+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:37:05.897219+0000","last_peered":"2024-09-10T20:37:05.897219+0000","last_clean":"2024-09-10T20:37:05.897219+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:37:05.897219+0000","last_undegraded":"2024-09-10T20:37:05.897219+0000","last_fullsized":"2024-09-10T20:37:05.897219+0000","mapping_epoch":44,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16695320,"num_objects":4077,"num_object_clones":0,"num_object_copies":8154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4077,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14767,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'16555","reported_seq":16645,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:09.250487+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:37:09.250487+0000","last_peered":"2024-09-10T20:37:09.250487+0000","last_clean":"2024-09-10T20:37:09.250487+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:37:09.250487+0000","last_undegraded":"2024-09-10T20:37:09.250487+0000","last_fullsized":"2024-09-10T20:37:09.250487+0000","mapping_epoch":35,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16584704,"num_objects":4049,"num_object_clones":0,"num_object_copies":8098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16555,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'17005","reported_seq":17141,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:08.976285+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:37:08.976285+0000","last_peered":"2024-09-10T20:37:08.976285+0000","last_clean":"2024-09-10T20:37:08.976285+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:37:08.976285+0000","last_undegraded":"2024-09-10T20:37:08.976285+0000","last_fullsized":"2024-09-10T20:37:08.976285+0000","mapping_epoch":34,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16646144,"num_objects":4064,"num_object_clones":0,"num_object_copies":8128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14001,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'16602","reported_seq":16853,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:09.066077+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:37:09.066077+0000","last_peered":"2024-09-10T20:37:09.066077+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:37:09.066077+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15847424,"num_objects":3869,"num_object_clones":0,"num_object_copies":7738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13466,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":308}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'16588","reported_seq":16874,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:09.095825+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:37:09.095825+0000","last_peered":"2024-09-10T20:37:09.095825+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:37:09.095825+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16465920,"num_objects":4020,"num_object_clones":0,"num_object_copies":8040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13183,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":534}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'16804","reported_seq":16938,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:09.058021+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:37:09.058021+0000","last_peered":"2024-09-10T20:37:09.058021+0000","last_clean":"2024-09-10T20:37:09.058021+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:37:09.058021+0000","last_undegraded":"2024-09-10T20:37:09.058021+0000","last_fullsized":"2024-09-10T20:37:09.058021+0000","mapping_epoch":34,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16105472,"num_objects":3932,"num_object_clones":0,"num_object_copies":7864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3932,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13875,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'16445","reported_seq":16644,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:09.255155+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:37:09.255155+0000","last_peered":"2024-09-10T20:37:09.255155+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:37:09.255155+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16359424,"num_objects":3994,"num_object_clones":0,"num_object_copies":7988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14438,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":497}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318026+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318026+0000","last_peered":"2024-09-10T20:36:30.318026+0000","last_clean":"2024-09-10T20:36:30.318026+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318026+0000","last_undegraded":"2024-09-10T20:36:30.318026+0000","last_fullsized":"2024-09-10T20:36:30.318026+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'16760","reported_seq":16974,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:09.083868+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:37:09.083868+0000","last_peered":"2024-09-10T20:37:09.083868+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:37:09.083868+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16262086,"num_objects":3972,"num_object_clones":0,"num_object_copies":7944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16758,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":406}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315256+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315256+0000","last_peered":"2024-09-10T20:36:30.315256+0000","last_clean":"2024-09-10T20:36:30.315256+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315256+0000","last_undegraded":"2024-09-10T20:36:30.315256+0000","last_fullsized":"2024-09-10T20:36:30.315256+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'16610","reported_seq":16801,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:09.205024+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:37:09.205024+0000","last_peered":"2024-09-10T20:37:09.205024+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:37:09.205024+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16199680,"num_objects":3955,"num_object_clones":0,"num_object_copies":7910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14438,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":349}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'16731","reported_seq":24101,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:09.271906+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:37:09.271906+0000","last_peered":"2024-09-10T20:37:09.271906+0000","last_clean":"2024-09-10T20:37:09.271906+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:37:09.271906+0000","last_undegraded":"2024-09-10T20:37:09.271906+0000","last_fullsized":"2024-09-10T20:37:09.271906+0000","mapping_epoch":48,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16109568,"num_objects":3933,"num_object_clones":0,"num_object_copies":7866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3933,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14517,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'16789","reported_seq":17046,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:09.050029+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:37:09.050029+0000","last_peered":"2024-09-10T20:37:09.050029+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:37:09.050029+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16551936,"num_objects":4041,"num_object_clones":0,"num_object_copies":8082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13512,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":449}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'16664","reported_seq":18679,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:09.069402+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:37:09.069402+0000","last_peered":"2024-09-10T20:37:09.069402+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:37:09.069402+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16056320,"num_objects":3920,"num_object_clones":0,"num_object_copies":7840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13358,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":933,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1318}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319607+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319607+0000","last_peered":"2024-09-10T20:36:30.319607+0000","last_clean":"2024-09-10T20:36:30.319607+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319607+0000","last_undegraded":"2024-09-10T20:36:30.319607+0000","last_fullsized":"2024-09-10T20:36:30.319607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'16976","reported_seq":17233,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:09.034316+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:37:09.034316+0000","last_peered":"2024-09-10T20:37:09.034316+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:37:09.034316+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16424960,"num_objects":4010,"num_object_clones":0,"num_object_copies":8020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13803,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":355}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'16467","reported_seq":23594,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:05.875592+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:37:05.875592+0000","last_peered":"2024-09-10T20:37:05.875592+0000","last_clean":"2024-09-10T20:37:05.875592+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:37:05.875592+0000","last_undegraded":"2024-09-10T20:37:05.875592+0000","last_fullsized":"2024-09-10T20:37:05.875592+0000","mapping_epoch":46,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16585187,"num_objects":4050,"num_object_clones":0,"num_object_copies":8100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13074,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":261498305,"num_objects":63846,"num_object_clones":0,"num_object_copies":127692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31119,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63846,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":231571,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12030,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":703041536,"data_stored":702975584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160675,"ondisk_log_size":160675,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182503,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":668488,"kb_used_data":249520,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93703352,"statfs":{"total":96636764160,"available":95952232448,"internally_reserved":0,"allocated":255508480,"data_stored":255246532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":428994218},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,4,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986275,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572264,"kb_used_data":243476,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93799576,"statfs":{"total":96636764160,"available":96050765824,"internally_reserved":0,"allocated":249319424,"data_stored":249057481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336653994},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,3,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659221,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":606456,"kb_used_data":195752,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93765384,"statfs":{"total":96636764160,"available":96015753216,"internally_reserved":0,"allocated":200450048,"data_stored":200137059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":420540078},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,1,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":254644224,"data_stored":254640611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":199958528,"data_stored":199903874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":248438784,"data_stored":248431099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:37:15.114 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:37:15.662 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:15.663 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:37:15.685 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2024-09-10T20:37:15.004185+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":251368916,"num_objects":61375,"num_object_clones":0,"num_object_copies":122750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61375,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":234046,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12129,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160951,"ondisk_log_size":160951,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1815292,"kb_used_data":669296,"kb_used_omap":12,"kb_used_meta":1145907,"kb_avail":281300228,"statfs":{"total":289910292480,"available":288051433472,"internally_reserved":0,"allocated":685359104,"data_stored":684522224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1173408770},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,2,4,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28962816,"num_objects":-7071,"num_object_clones":0,"num_object_copies":-14142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-295,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7071,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":295,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004261"},"pg_stats":[{"pgid":"2.f","version":"49'16860","reported_seq":17082,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:14.089886+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:37:14.089886+0000","last_peered":"2024-09-10T20:37:14.089886+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:37:14.089886+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15556608,"num_objects":3798,"num_object_clones":0,"num_object_copies":7596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16860,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":257}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'17093","reported_seq":17213,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:14.340010+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:37:14.340010+0000","last_peered":"2024-09-10T20:37:14.340010+0000","last_clean":"2024-09-10T20:37:14.340010+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:37:14.340010+0000","last_undegraded":"2024-09-10T20:37:14.340010+0000","last_fullsized":"2024-09-10T20:37:14.340010+0000","mapping_epoch":35,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15765504,"num_objects":3849,"num_object_clones":0,"num_object_copies":7698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15279,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'16789","reported_seq":24147,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:10.929627+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:37:10.929627+0000","last_peered":"2024-09-10T20:37:10.929627+0000","last_clean":"2024-09-10T20:37:10.929627+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:37:10.929627+0000","last_undegraded":"2024-09-10T20:37:10.929627+0000","last_fullsized":"2024-09-10T20:37:10.929627+0000","mapping_epoch":44,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16080920,"num_objects":3927,"num_object_clones":0,"num_object_copies":7854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3927,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14917,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'16695","reported_seq":16785,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:14.286120+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:37:14.286120+0000","last_peered":"2024-09-10T20:37:14.286120+0000","last_clean":"2024-09-10T20:37:14.286120+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:37:14.286120+0000","last_undegraded":"2024-09-10T20:37:14.286120+0000","last_fullsized":"2024-09-10T20:37:14.286120+0000","mapping_epoch":35,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16011264,"num_objects":3909,"num_object_clones":0,"num_object_copies":7818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16695,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'17165","reported_seq":17301,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:14.111999+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:37:14.111999+0000","last_peered":"2024-09-10T20:37:14.111999+0000","last_clean":"2024-09-10T20:37:14.111999+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:37:14.111999+0000","last_undegraded":"2024-09-10T20:37:14.111999+0000","last_fullsized":"2024-09-10T20:37:14.111999+0000","mapping_epoch":34,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15990784,"num_objects":3904,"num_object_clones":0,"num_object_copies":7808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14161,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'16742","reported_seq":16994,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:14.113242+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:37:14.113242+0000","last_peered":"2024-09-10T20:37:14.113242+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:37:14.113242+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15273984,"num_objects":3729,"num_object_clones":0,"num_object_copies":7458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13606,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":168}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'16751","reported_seq":17036,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:14.034580+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:37:14.034580+0000","last_peered":"2024-09-10T20:37:14.034580+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:37:14.034580+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15798272,"num_objects":3857,"num_object_clones":0,"num_object_copies":7714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13346,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'16941","reported_seq":17075,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:14.119479+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:37:14.119479+0000","last_peered":"2024-09-10T20:37:14.119479+0000","last_clean":"2024-09-10T20:37:14.119479+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:37:14.119479+0000","last_undegraded":"2024-09-10T20:37:14.119479+0000","last_fullsized":"2024-09-10T20:37:14.119479+0000","mapping_epoch":34,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15544320,"num_objects":3795,"num_object_clones":0,"num_object_copies":7590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14012,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'16600","reported_seq":16800,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:14.344183+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:37:14.344183+0000","last_peered":"2024-09-10T20:37:14.344183+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:37:14.344183+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15724544,"num_objects":3839,"num_object_clones":0,"num_object_copies":7678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14593,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":342}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318026+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318026+0000","last_peered":"2024-09-10T20:36:30.318026+0000","last_clean":"2024-09-10T20:36:30.318026+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318026+0000","last_undegraded":"2024-09-10T20:36:30.318026+0000","last_fullsized":"2024-09-10T20:36:30.318026+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'16909","reported_seq":17125,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:14.127790+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:37:14.127790+0000","last_peered":"2024-09-10T20:37:14.127790+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:37:14.127790+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15651782,"num_objects":3823,"num_object_clones":0,"num_object_copies":7646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16907,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":257}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315256+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315256+0000","last_peered":"2024-09-10T20:36:30.315256+0000","last_clean":"2024-09-10T20:36:30.315256+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315256+0000","last_undegraded":"2024-09-10T20:36:30.315256+0000","last_fullsized":"2024-09-10T20:36:30.315256+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'16753","reported_seq":16945,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:14.332610+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:37:14.332610+0000","last_peered":"2024-09-10T20:37:14.332610+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:37:14.332610+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15613952,"num_objects":3812,"num_object_clones":0,"num_object_copies":7624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3812,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14581,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":206}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'16892","reported_seq":24261,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:14.308354+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:37:14.308354+0000","last_peered":"2024-09-10T20:37:14.308354+0000","last_clean":"2024-09-10T20:37:14.308354+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:37:14.308354+0000","last_undegraded":"2024-09-10T20:37:14.308354+0000","last_fullsized":"2024-09-10T20:37:14.308354+0000","mapping_epoch":48,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15450112,"num_objects":3772,"num_object_clones":0,"num_object_copies":7544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14678,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'16962","reported_seq":17219,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:14.093202+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:37:14.093202+0000","last_peered":"2024-09-10T20:37:14.093202+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:37:14.093202+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15843328,"num_objects":3868,"num_object_clones":0,"num_object_copies":7736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13685,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":276}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'16828","reported_seq":19035,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:14.113421+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:37:14.113421+0000","last_peered":"2024-09-10T20:37:14.113421+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:37:14.113421+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15384576,"num_objects":3756,"num_object_clones":0,"num_object_copies":7512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2505,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13522,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1030,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1251}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319607+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319607+0000","last_peered":"2024-09-10T20:36:30.319607+0000","last_clean":"2024-09-10T20:36:30.319607+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319607+0000","last_undegraded":"2024-09-10T20:36:30.319607+0000","last_fullsized":"2024-09-10T20:36:30.319607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'17155","reported_seq":17412,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:14.062456+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:37:14.062456+0000","last_peered":"2024-09-10T20:37:14.062456+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:37:14.062456+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15691776,"num_objects":3831,"num_object_clones":0,"num_object_copies":7662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13982,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":176}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'16613","reported_seq":23740,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:10.955204+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:37:10.955204+0000","last_peered":"2024-09-10T20:37:10.955204+0000","last_clean":"2024-09-10T20:37:10.955204+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:37:10.955204+0000","last_undegraded":"2024-09-10T20:37:10.955204+0000","last_fullsized":"2024-09-10T20:37:10.955204+0000","mapping_epoch":46,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15987171,"num_objects":3904,"num_object_clones":0,"num_object_copies":7808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13220,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":251368897,"num_objects":61373,"num_object_clones":0,"num_object_copies":122746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61373,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":234044,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12127,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":682631168,"data_stored":682565216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160948,"ondisk_log_size":160948,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182504,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":652888,"kb_used_data":242240,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93718952,"statfs":{"total":96636764160,"available":95968206848,"internally_reserved":0,"allocated":248053760,"data_stored":247791812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":420474538},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986276,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":555088,"kb_used_data":238776,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93816752,"statfs":{"total":96636764160,"available":96068354048,"internally_reserved":0,"allocated":244506624,"data_stored":244244681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323874474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659222,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":607316,"kb_used_data":188280,"kb_used_omap":4,"kb_used_meta":419003,"kb_avail":93764524,"statfs":{"total":96636764160,"available":96014872576,"internally_reserved":0,"allocated":192798720,"data_stored":192485731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":429059758},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":246980608,"data_stored":246976995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":192212992,"data_stored":192158338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":243437568,"data_stored":243429883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:37:15.687 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:37:16.233 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:16.234 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:37:16.260 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2024-09-10T20:37:15.004185+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":251368916,"num_objects":61375,"num_object_clones":0,"num_object_copies":122750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61375,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":234046,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12129,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160951,"ondisk_log_size":160951,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1815292,"kb_used_data":669296,"kb_used_omap":12,"kb_used_meta":1145907,"kb_avail":281300228,"statfs":{"total":289910292480,"available":288051433472,"internally_reserved":0,"allocated":685359104,"data_stored":684522224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1173408770},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,2,4,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28962816,"num_objects":-7071,"num_object_clones":0,"num_object_copies":-14142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-295,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7071,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":295,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004261"},"pg_stats":[{"pgid":"2.f","version":"49'16860","reported_seq":17082,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:14.089886+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:37:14.089886+0000","last_peered":"2024-09-10T20:37:14.089886+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:37:14.089886+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15556608,"num_objects":3798,"num_object_clones":0,"num_object_copies":7596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16860,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":257}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'17093","reported_seq":17213,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:14.340010+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:37:14.340010+0000","last_peered":"2024-09-10T20:37:14.340010+0000","last_clean":"2024-09-10T20:37:14.340010+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:37:14.340010+0000","last_undegraded":"2024-09-10T20:37:14.340010+0000","last_fullsized":"2024-09-10T20:37:14.340010+0000","mapping_epoch":35,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15765504,"num_objects":3849,"num_object_clones":0,"num_object_copies":7698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15279,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'16789","reported_seq":24147,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:10.929627+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:37:10.929627+0000","last_peered":"2024-09-10T20:37:10.929627+0000","last_clean":"2024-09-10T20:37:10.929627+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:37:10.929627+0000","last_undegraded":"2024-09-10T20:37:10.929627+0000","last_fullsized":"2024-09-10T20:37:10.929627+0000","mapping_epoch":44,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16080920,"num_objects":3927,"num_object_clones":0,"num_object_copies":7854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3927,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14917,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'16695","reported_seq":16785,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:14.286120+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:37:14.286120+0000","last_peered":"2024-09-10T20:37:14.286120+0000","last_clean":"2024-09-10T20:37:14.286120+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:37:14.286120+0000","last_undegraded":"2024-09-10T20:37:14.286120+0000","last_fullsized":"2024-09-10T20:37:14.286120+0000","mapping_epoch":35,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16011264,"num_objects":3909,"num_object_clones":0,"num_object_copies":7818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16695,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'17165","reported_seq":17301,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:14.111999+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:37:14.111999+0000","last_peered":"2024-09-10T20:37:14.111999+0000","last_clean":"2024-09-10T20:37:14.111999+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:37:14.111999+0000","last_undegraded":"2024-09-10T20:37:14.111999+0000","last_fullsized":"2024-09-10T20:37:14.111999+0000","mapping_epoch":34,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15990784,"num_objects":3904,"num_object_clones":0,"num_object_copies":7808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14161,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'16742","reported_seq":16994,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:14.113242+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:37:14.113242+0000","last_peered":"2024-09-10T20:37:14.113242+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:37:14.113242+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15273984,"num_objects":3729,"num_object_clones":0,"num_object_copies":7458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13606,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":168}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'16751","reported_seq":17036,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:14.034580+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:37:14.034580+0000","last_peered":"2024-09-10T20:37:14.034580+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:37:14.034580+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15798272,"num_objects":3857,"num_object_clones":0,"num_object_copies":7714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13346,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'16941","reported_seq":17075,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:14.119479+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:37:14.119479+0000","last_peered":"2024-09-10T20:37:14.119479+0000","last_clean":"2024-09-10T20:37:14.119479+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:37:14.119479+0000","last_undegraded":"2024-09-10T20:37:14.119479+0000","last_fullsized":"2024-09-10T20:37:14.119479+0000","mapping_epoch":34,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15544320,"num_objects":3795,"num_object_clones":0,"num_object_copies":7590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14012,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'16600","reported_seq":16800,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:14.344183+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:37:14.344183+0000","last_peered":"2024-09-10T20:37:14.344183+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:37:14.344183+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15724544,"num_objects":3839,"num_object_clones":0,"num_object_copies":7678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14593,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":342}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318026+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318026+0000","last_peered":"2024-09-10T20:36:30.318026+0000","last_clean":"2024-09-10T20:36:30.318026+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318026+0000","last_undegraded":"2024-09-10T20:36:30.318026+0000","last_fullsized":"2024-09-10T20:36:30.318026+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'16909","reported_seq":17125,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:14.127790+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:37:14.127790+0000","last_peered":"2024-09-10T20:37:14.127790+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:37:14.127790+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15651782,"num_objects":3823,"num_object_clones":0,"num_object_copies":7646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16907,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":257}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315256+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315256+0000","last_peered":"2024-09-10T20:36:30.315256+0000","last_clean":"2024-09-10T20:36:30.315256+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315256+0000","last_undegraded":"2024-09-10T20:36:30.315256+0000","last_fullsized":"2024-09-10T20:36:30.315256+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'16753","reported_seq":16945,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:14.332610+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:37:14.332610+0000","last_peered":"2024-09-10T20:37:14.332610+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:37:14.332610+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15613952,"num_objects":3812,"num_object_clones":0,"num_object_copies":7624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3812,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14581,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":206}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'16892","reported_seq":24261,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:14.308354+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:37:14.308354+0000","last_peered":"2024-09-10T20:37:14.308354+0000","last_clean":"2024-09-10T20:37:14.308354+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:37:14.308354+0000","last_undegraded":"2024-09-10T20:37:14.308354+0000","last_fullsized":"2024-09-10T20:37:14.308354+0000","mapping_epoch":48,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15450112,"num_objects":3772,"num_object_clones":0,"num_object_copies":7544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14678,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'16962","reported_seq":17219,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:14.093202+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:37:14.093202+0000","last_peered":"2024-09-10T20:37:14.093202+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:37:14.093202+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15843328,"num_objects":3868,"num_object_clones":0,"num_object_copies":7736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13685,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":276}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'16828","reported_seq":19035,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:14.113421+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:37:14.113421+0000","last_peered":"2024-09-10T20:37:14.113421+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:37:14.113421+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15384576,"num_objects":3756,"num_object_clones":0,"num_object_copies":7512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2505,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13522,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1030,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1251}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319607+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319607+0000","last_peered":"2024-09-10T20:36:30.319607+0000","last_clean":"2024-09-10T20:36:30.319607+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319607+0000","last_undegraded":"2024-09-10T20:36:30.319607+0000","last_fullsized":"2024-09-10T20:36:30.319607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'17155","reported_seq":17412,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:14.062456+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:37:14.062456+0000","last_peered":"2024-09-10T20:37:14.062456+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:37:14.062456+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15691776,"num_objects":3831,"num_object_clones":0,"num_object_copies":7662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13982,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":176}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'16613","reported_seq":23740,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:10.955204+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:37:10.955204+0000","last_peered":"2024-09-10T20:37:10.955204+0000","last_clean":"2024-09-10T20:37:10.955204+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:37:10.955204+0000","last_undegraded":"2024-09-10T20:37:10.955204+0000","last_fullsized":"2024-09-10T20:37:10.955204+0000","mapping_epoch":46,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15987171,"num_objects":3904,"num_object_clones":0,"num_object_copies":7808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13220,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":251368897,"num_objects":61373,"num_object_clones":0,"num_object_copies":122746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61373,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":234044,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12127,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":682631168,"data_stored":682565216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160948,"ondisk_log_size":160948,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182504,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":652888,"kb_used_data":242240,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93718952,"statfs":{"total":96636764160,"available":95968206848,"internally_reserved":0,"allocated":248053760,"data_stored":247791812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":420474538},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986276,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":555088,"kb_used_data":238776,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93816752,"statfs":{"total":96636764160,"available":96068354048,"internally_reserved":0,"allocated":244506624,"data_stored":244244681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323874474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659222,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":607316,"kb_used_data":188280,"kb_used_omap":4,"kb_used_meta":419003,"kb_avail":93764524,"statfs":{"total":96636764160,"available":96014872576,"internally_reserved":0,"allocated":192798720,"data_stored":192485731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":429059758},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":246980608,"data_stored":246976995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":192212992,"data_stored":192158338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":243437568,"data_stored":243429883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:37:16.262 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:37:16.262 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'16860", 'reported_seq': 17082, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:14.089886+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:37:14.089886+0000', 'last_peered': '2024-09-10T20:37:14.089886+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:37:14.089886+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15556608, 'num_objects': 3798, 'num_object_clones': 0, 'num_object_copies': 7596, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3798, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16860, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 257}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:16.262 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:37:16.262 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'16742", 'reported_seq': 16994, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:14.113242+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:37:14.113242+0000', 'last_peered': '2024-09-10T20:37:14.113242+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:37:14.113242+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "43'6700", 'ondisk_log_start': "43'6700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15273984, 'num_objects': 3729, 'num_object_clones': 0, 'num_object_copies': 7458, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3729, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13606, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 168}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:16.262 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:37:16.262 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'16751", 'reported_seq': 17036, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:14.034580+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:37:14.034580+0000', 'last_peered': '2024-09-10T20:37:14.034580+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:37:14.034580+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "43'6700", 'ondisk_log_start': "43'6700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15798272, 'num_objects': 3857, 'num_object_clones': 0, 'num_object_copies': 7714, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3857, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13346, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:16.262 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:37:16.262 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'16600", 'reported_seq': 16800, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:14.344183+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:37:14.344183+0000', 'last_peered': '2024-09-10T20:37:14.344183+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:37:14.344183+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "43'6500", 'ondisk_log_start': "43'6500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15724544, 'num_objects': 3839, 'num_object_clones': 0, 'num_object_copies': 7678, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3839, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14593, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 342}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:37:16.263 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:37:16.263 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'16909", 'reported_seq': 17125, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:14.127790+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:37:14.127790+0000', 'last_peered': '2024-09-10T20:37:14.127790+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:37:14.127790+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "43'6900", 'ondisk_log_start': "43'6900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15651782, 'num_objects': 3823, 'num_object_clones': 0, 'num_object_copies': 7646, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3823, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16907, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 257}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:16.263 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:37:16.263 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'16753", 'reported_seq': 16945, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:14.332610+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:37:14.332610+0000', 'last_peered': '2024-09-10T20:37:14.332610+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:37:14.332610+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "43'6700", 'ondisk_log_start': "43'6700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15613952, 'num_objects': 3812, 'num_object_clones': 0, 'num_object_copies': 7624, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3812, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14581, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 206}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:37:16.263 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:37:16.263 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'16962", 'reported_seq': 17219, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:14.093202+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:37:14.093202+0000', 'last_peered': '2024-09-10T20:37:14.093202+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:37:14.093202+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "43'6900", 'ondisk_log_start': "43'6900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15843328, 'num_objects': 3868, 'num_object_clones': 0, 'num_object_copies': 7736, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3868, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13685, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 276}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:16.263 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:37:16.263 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'16828", 'reported_seq': 19035, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:14.113421+0000', 'last_change': '2024-09-10T20:36:28.778006+0000', 'last_active': '2024-09-10T20:37:14.113421+0000', 'last_peered': '2024-09-10T20:37:14.113421+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:37:14.113421+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15384576, 'num_objects': 3756, 'num_object_clones': 0, 'num_object_copies': 7512, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2505, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3756, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13522, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1030, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1251}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:16.264 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:37:16.264 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'17155", 'reported_seq': 17412, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:14.062456+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:37:14.062456+0000', 'last_peered': '2024-09-10T20:37:14.062456+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:37:14.062456+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "43'7100", 'ondisk_log_start': "43'7100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15691776, 'num_objects': 3831, 'num_object_clones': 0, 'num_object_copies': 7662, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3831, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13982, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 176}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:16.264 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:37:16.976 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:16.976 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31022/122750 objects degraded (25.273%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":548,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":61375,"data_bytes":251368916,"bytes_used":1858859008,"bytes_avail":288051433472,"bytes_total":289910292480,"degraded_objects":31022,"degraded_total":122750,"degraded_ratio":0.25272505091649694,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":295,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":589},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:37:17.002 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:37:17.002 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:37:17.579 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:17.584 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:37:17.611 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-09-10T20:37:17.004986+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":250029524,"num_objects":61048,"num_object_clones":0,"num_object_copies":122096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61048,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":234373,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12129,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160978,"ondisk_log_size":160978,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1795364,"kb_used_data":661860,"kb_used_omap":12,"kb_used_meta":1133427,"kb_avail":281320156,"statfs":{"total":289910292480,"available":288071839744,"internally_reserved":0,"allocated":677744640,"data_stored":676907760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1160629250},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,6,4,7,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21471232,"num_objects":-5242,"num_object_clones":0,"num_object_copies":-10484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-188,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5242,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5242,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004344"},"pg_stats":[{"pgid":"2.f","version":"49'16860","reported_seq":17082,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:14.089886+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:37:14.089886+0000","last_peered":"2024-09-10T20:37:14.089886+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:37:14.089886+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15556608,"num_objects":3798,"num_object_clones":0,"num_object_copies":7596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16860,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":257}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'17093","reported_seq":17213,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:14.340010+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:37:14.340010+0000","last_peered":"2024-09-10T20:37:14.340010+0000","last_clean":"2024-09-10T20:37:14.340010+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:37:14.340010+0000","last_undegraded":"2024-09-10T20:37:14.340010+0000","last_fullsized":"2024-09-10T20:37:14.340010+0000","mapping_epoch":35,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15765504,"num_objects":3849,"num_object_clones":0,"num_object_copies":7698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15279,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'16947","reported_seq":24305,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:16.026360+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:37:16.026360+0000","last_peered":"2024-09-10T20:37:16.026360+0000","last_clean":"2024-09-10T20:37:16.026360+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:37:16.026360+0000","last_undegraded":"2024-09-10T20:37:16.026360+0000","last_fullsized":"2024-09-10T20:37:16.026360+0000","mapping_epoch":44,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15433752,"num_objects":3769,"num_object_clones":0,"num_object_copies":7538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3769,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15075,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'16695","reported_seq":16785,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:14.286120+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:37:14.286120+0000","last_peered":"2024-09-10T20:37:14.286120+0000","last_clean":"2024-09-10T20:37:14.286120+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:37:14.286120+0000","last_undegraded":"2024-09-10T20:37:14.286120+0000","last_fullsized":"2024-09-10T20:37:14.286120+0000","mapping_epoch":35,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16011264,"num_objects":3909,"num_object_clones":0,"num_object_copies":7818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16695,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'17165","reported_seq":17301,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:14.111999+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:37:14.111999+0000","last_peered":"2024-09-10T20:37:14.111999+0000","last_clean":"2024-09-10T20:37:14.111999+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:37:14.111999+0000","last_undegraded":"2024-09-10T20:37:14.111999+0000","last_fullsized":"2024-09-10T20:37:14.111999+0000","mapping_epoch":34,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15990784,"num_objects":3904,"num_object_clones":0,"num_object_copies":7808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14161,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'16742","reported_seq":16994,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:14.113242+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:37:14.113242+0000","last_peered":"2024-09-10T20:37:14.113242+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:37:14.113242+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15273984,"num_objects":3729,"num_object_clones":0,"num_object_copies":7458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13606,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":168}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'16751","reported_seq":17036,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:14.034580+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:37:14.034580+0000","last_peered":"2024-09-10T20:37:14.034580+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:37:14.034580+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15798272,"num_objects":3857,"num_object_clones":0,"num_object_copies":7714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13346,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'16941","reported_seq":17075,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:14.119479+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:37:14.119479+0000","last_peered":"2024-09-10T20:37:14.119479+0000","last_clean":"2024-09-10T20:37:14.119479+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:37:14.119479+0000","last_undegraded":"2024-09-10T20:37:14.119479+0000","last_fullsized":"2024-09-10T20:37:14.119479+0000","mapping_epoch":34,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15544320,"num_objects":3795,"num_object_clones":0,"num_object_copies":7590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14012,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'16600","reported_seq":16800,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:14.344183+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:37:14.344183+0000","last_peered":"2024-09-10T20:37:14.344183+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:37:14.344183+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15724544,"num_objects":3839,"num_object_clones":0,"num_object_copies":7678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14593,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":342}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318026+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318026+0000","last_peered":"2024-09-10T20:36:30.318026+0000","last_clean":"2024-09-10T20:36:30.318026+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318026+0000","last_undegraded":"2024-09-10T20:36:30.318026+0000","last_fullsized":"2024-09-10T20:36:30.318026+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'16909","reported_seq":17125,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:14.127790+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:37:14.127790+0000","last_peered":"2024-09-10T20:37:14.127790+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:37:14.127790+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15651782,"num_objects":3823,"num_object_clones":0,"num_object_copies":7646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16907,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":257}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315256+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315256+0000","last_peered":"2024-09-10T20:36:30.315256+0000","last_clean":"2024-09-10T20:36:30.315256+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315256+0000","last_undegraded":"2024-09-10T20:36:30.315256+0000","last_fullsized":"2024-09-10T20:36:30.315256+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'16753","reported_seq":16945,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:14.332610+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:37:14.332610+0000","last_peered":"2024-09-10T20:37:14.332610+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:37:14.332610+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15613952,"num_objects":3812,"num_object_clones":0,"num_object_copies":7624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3812,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14581,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":206}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'16892","reported_seq":24261,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:14.308354+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:37:14.308354+0000","last_peered":"2024-09-10T20:37:14.308354+0000","last_clean":"2024-09-10T20:37:14.308354+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:37:14.308354+0000","last_undegraded":"2024-09-10T20:37:14.308354+0000","last_fullsized":"2024-09-10T20:37:14.308354+0000","mapping_epoch":48,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15450112,"num_objects":3772,"num_object_clones":0,"num_object_copies":7544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14678,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'16962","reported_seq":17219,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:14.093202+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:37:14.093202+0000","last_peered":"2024-09-10T20:37:14.093202+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:37:14.093202+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15843328,"num_objects":3868,"num_object_clones":0,"num_object_copies":7736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13685,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":276}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'16828","reported_seq":19035,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:14.113421+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:37:14.113421+0000","last_peered":"2024-09-10T20:37:14.113421+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:37:14.113421+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15384576,"num_objects":3756,"num_object_clones":0,"num_object_copies":7512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2505,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13522,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1030,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1251}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319607+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319607+0000","last_peered":"2024-09-10T20:36:30.319607+0000","last_clean":"2024-09-10T20:36:30.319607+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319607+0000","last_undegraded":"2024-09-10T20:36:30.319607+0000","last_fullsized":"2024-09-10T20:36:30.319607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'17155","reported_seq":17412,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:14.062456+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:37:14.062456+0000","last_peered":"2024-09-10T20:37:14.062456+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:37:14.062456+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15691776,"num_objects":3831,"num_object_clones":0,"num_object_copies":7662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13982,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":176}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'16782","reported_seq":23909,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:16.010057+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:37:16.010057+0000","last_peered":"2024-09-10T20:37:16.010057+0000","last_clean":"2024-09-10T20:37:16.010057+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:37:16.010057+0000","last_undegraded":"2024-09-10T20:37:16.010057+0000","last_fullsized":"2024-09-10T20:37:16.010057+0000","mapping_epoch":46,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15294947,"num_objects":3735,"num_object_clones":0,"num_object_copies":7470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13389,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":250029505,"num_objects":61046,"num_object_clones":0,"num_object_copies":122092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61046,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":234371,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12127,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":675037184,"data_stored":674971232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160975,"ondisk_log_size":160975,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182504,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":652888,"kb_used_data":242240,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93718952,"statfs":{"total":96636764160,"available":95968206848,"internally_reserved":0,"allocated":248053760,"data_stored":247791812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":420474538},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986276,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":555088,"kb_used_data":238776,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93816752,"statfs":{"total":96636764160,"available":96068354048,"internally_reserved":0,"allocated":244506624,"data_stored":244244681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323874474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659223,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":587388,"kb_used_data":180844,"kb_used_omap":4,"kb_used_meta":406523,"kb_avail":93784452,"statfs":{"total":96636764160,"available":96035278848,"internally_reserved":0,"allocated":185184256,"data_stored":184871267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":416280238},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,2,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":246980608,"data_stored":246976995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":184619008,"data_stored":184564354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":243437568,"data_stored":243429883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:37:20.613 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:37:21.231 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:21.237 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:37:21.258 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":319,"stamp":"2024-09-10T20:37:21.006307+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":241219028,"num_objects":58897,"num_object_clones":0,"num_object_copies":117794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30914,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58897,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":236524,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12237,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160729,"ondisk_log_size":160729,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1809528,"kb_used_data":649660,"kb_used_omap":12,"kb_used_meta":1159795,"kb_avail":281305992,"statfs":{"total":289910292480,"available":288057335808,"internally_reserved":0,"allocated":665251840,"data_stored":664414960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1187630082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,2,8,8,11,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29069312,"num_objects":-7097,"num_object_clones":0,"num_object_copies":-14194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-296,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7097,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7097,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004394"},"pg_stats":[{"pgid":"2.f","version":"49'17005","reported_seq":17225,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:19.229745+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:37:19.229745+0000","last_peered":"2024-09-10T20:37:19.229745+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:37:19.229745+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14962688,"num_objects":3653,"num_object_clones":0,"num_object_copies":7306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3653,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17005,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":112}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'17258","reported_seq":17377,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:19.332797+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:37:19.332797+0000","last_peered":"2024-09-10T20:37:19.332797+0000","last_clean":"2024-09-10T20:37:19.332797+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:37:19.332797+0000","last_undegraded":"2024-09-10T20:37:19.332797+0000","last_fullsized":"2024-09-10T20:37:19.332797+0000","mapping_epoch":35,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15089664,"num_objects":3684,"num_object_clones":0,"num_object_copies":7368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3684,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15444,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'16947","reported_seq":24305,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:16.026360+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:37:16.026360+0000","last_peered":"2024-09-10T20:37:16.026360+0000","last_clean":"2024-09-10T20:37:16.026360+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:37:16.026360+0000","last_undegraded":"2024-09-10T20:37:16.026360+0000","last_fullsized":"2024-09-10T20:37:16.026360+0000","mapping_epoch":44,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15433752,"num_objects":3769,"num_object_clones":0,"num_object_copies":7538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3769,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15075,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'16856","reported_seq":16946,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:19.365652+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:37:19.365652+0000","last_peered":"2024-09-10T20:37:19.365652+0000","last_clean":"2024-09-10T20:37:19.365652+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:37:19.365652+0000","last_undegraded":"2024-09-10T20:37:19.365652+0000","last_fullsized":"2024-09-10T20:37:19.365652+0000","mapping_epoch":35,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15351808,"num_objects":3748,"num_object_clones":0,"num_object_copies":7496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16856,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'17321","reported_seq":17457,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:19.185792+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:37:19.185792+0000","last_peered":"2024-09-10T20:37:19.185792+0000","last_clean":"2024-09-10T20:37:19.185792+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:37:19.185792+0000","last_undegraded":"2024-09-10T20:37:19.185792+0000","last_fullsized":"2024-09-10T20:37:19.185792+0000","mapping_epoch":34,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15351808,"num_objects":3748,"num_object_clones":0,"num_object_copies":7496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14317,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'16902","reported_seq":17154,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:19.137446+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:37:19.137446+0000","last_peered":"2024-09-10T20:37:19.137446+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:37:19.137446+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14618624,"num_objects":3569,"num_object_clones":0,"num_object_copies":7138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13766,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'16888","reported_seq":17174,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:19.128217+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:37:19.128217+0000","last_peered":"2024-09-10T20:37:19.128217+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:37:19.128217+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15237120,"num_objects":3720,"num_object_clones":0,"num_object_copies":7440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13483,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":234}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'17098","reported_seq":17232,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:19.166065+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:37:19.166065+0000","last_peered":"2024-09-10T20:37:19.166065+0000","last_clean":"2024-09-10T20:37:19.166065+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:37:19.166065+0000","last_undegraded":"2024-09-10T20:37:19.166065+0000","last_fullsized":"2024-09-10T20:37:19.166065+0000","mapping_epoch":34,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14901248,"num_objects":3638,"num_object_clones":0,"num_object_copies":7276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14169,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'16752","reported_seq":16952,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:19.364604+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:37:19.364604+0000","last_peered":"2024-09-10T20:37:19.364604+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:37:19.364604+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15101952,"num_objects":3687,"num_object_clones":0,"num_object_copies":7374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3687,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14745,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":190}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318026+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318026+0000","last_peered":"2024-09-10T20:36:30.318026+0000","last_clean":"2024-09-10T20:36:30.318026+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318026+0000","last_undegraded":"2024-09-10T20:36:30.318026+0000","last_fullsized":"2024-09-10T20:36:30.318026+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'17067","reported_seq":17283,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:19.217595+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:37:19.217595+0000","last_peered":"2024-09-10T20:37:19.217595+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:37:19.217595+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15004614,"num_objects":3665,"num_object_clones":0,"num_object_copies":7330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17065,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":99}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'16904","reported_seq":17096,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:19.312814+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:37:19.312814+0000","last_peered":"2024-09-10T20:37:19.312814+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:37:19.312814+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14995456,"num_objects":3661,"num_object_clones":0,"num_object_copies":7322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14732,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":55}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'17054","reported_seq":24425,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:19.321837+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:37:19.321837+0000","last_peered":"2024-09-10T20:37:19.321837+0000","last_clean":"2024-09-10T20:37:19.321837+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:37:19.321837+0000","last_undegraded":"2024-09-10T20:37:19.321837+0000","last_fullsized":"2024-09-10T20:37:19.321837+0000","mapping_epoch":48,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14786560,"num_objects":3610,"num_object_clones":0,"num_object_copies":7220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3610,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14840,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'17117","reported_seq":17373,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:19.176173+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:37:19.176173+0000","last_peered":"2024-09-10T20:37:19.176173+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:37:19.176173+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15208448,"num_objects":3713,"num_object_clones":0,"num_object_copies":7426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13840,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":121}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'16967","reported_seq":19392,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:19.215281+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:37:19.215281+0000","last_peered":"2024-09-10T20:37:19.215281+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:37:19.215281+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14815232,"num_objects":3617,"num_object_clones":0,"num_object_copies":7234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2397,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13661,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1138,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1220}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319607+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319607+0000","last_peered":"2024-09-10T20:36:30.319607+0000","last_clean":"2024-09-10T20:36:30.319607+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319607+0000","last_undegraded":"2024-09-10T20:36:30.319607+0000","last_fullsized":"2024-09-10T20:36:30.319607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'17308","reported_seq":17565,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:19.152493+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:37:19.152493+0000","last_peered":"2024-09-10T20:37:19.152493+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:37:19.152493+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15065088,"num_objects":3678,"num_object_clones":0,"num_object_copies":7356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3678,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14135,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":23}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'16782","reported_seq":23909,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:16.010057+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:37:16.010057+0000","last_peered":"2024-09-10T20:37:16.010057+0000","last_clean":"2024-09-10T20:37:16.010057+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:37:16.010057+0000","last_undegraded":"2024-09-10T20:37:16.010057+0000","last_fullsized":"2024-09-10T20:37:16.010057+0000","mapping_epoch":46,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15294947,"num_objects":3735,"num_object_clones":0,"num_object_copies":7470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13389,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":241219009,"num_objects":58895,"num_object_clones":0,"num_object_copies":117790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30914,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58895,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":236522,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12235,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":662368256,"data_stored":662302304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160726,"ondisk_log_size":160726,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182505,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":649868,"kb_used_data":235060,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93721972,"statfs":{"total":96636764160,"available":95971299328,"internally_reserved":0,"allocated":240701440,"data_stored":240439492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":424734378},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,1,0,4,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986277,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572272,"kb_used_data":233756,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93799568,"statfs":{"total":96636764160,"available":96050757632,"internally_reserved":0,"allocated":239366144,"data_stored":239104201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346615466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659223,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":587388,"kb_used_data":180844,"kb_used_omap":4,"kb_used_meta":406523,"kb_avail":93784452,"statfs":{"total":96636764160,"available":96035278848,"internally_reserved":0,"allocated":185184256,"data_stored":184871267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":416280238},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,2,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":239452160,"data_stored":239448547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":184619008,"data_stored":184564354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":238297088,"data_stored":238289403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:37:21.259 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:37:21.828 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:21.829 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:37:21.856 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":319,"stamp":"2024-09-10T20:37:21.006307+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":241219028,"num_objects":58897,"num_object_clones":0,"num_object_copies":117794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30914,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58897,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":236524,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12237,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160729,"ondisk_log_size":160729,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1809528,"kb_used_data":649660,"kb_used_omap":12,"kb_used_meta":1159795,"kb_avail":281305992,"statfs":{"total":289910292480,"available":288057335808,"internally_reserved":0,"allocated":665251840,"data_stored":664414960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1187630082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,2,8,8,11,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29069312,"num_objects":-7097,"num_object_clones":0,"num_object_copies":-14194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-296,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7097,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7097,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004394"},"pg_stats":[{"pgid":"2.f","version":"49'17005","reported_seq":17225,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:19.229745+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:37:19.229745+0000","last_peered":"2024-09-10T20:37:19.229745+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:37:19.229745+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14962688,"num_objects":3653,"num_object_clones":0,"num_object_copies":7306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3653,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17005,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":112}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'17258","reported_seq":17377,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:19.332797+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:37:19.332797+0000","last_peered":"2024-09-10T20:37:19.332797+0000","last_clean":"2024-09-10T20:37:19.332797+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:37:19.332797+0000","last_undegraded":"2024-09-10T20:37:19.332797+0000","last_fullsized":"2024-09-10T20:37:19.332797+0000","mapping_epoch":35,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15089664,"num_objects":3684,"num_object_clones":0,"num_object_copies":7368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3684,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15444,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'16947","reported_seq":24305,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:16.026360+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:37:16.026360+0000","last_peered":"2024-09-10T20:37:16.026360+0000","last_clean":"2024-09-10T20:37:16.026360+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:37:16.026360+0000","last_undegraded":"2024-09-10T20:37:16.026360+0000","last_fullsized":"2024-09-10T20:37:16.026360+0000","mapping_epoch":44,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15433752,"num_objects":3769,"num_object_clones":0,"num_object_copies":7538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3769,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15075,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'16856","reported_seq":16946,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:19.365652+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:37:19.365652+0000","last_peered":"2024-09-10T20:37:19.365652+0000","last_clean":"2024-09-10T20:37:19.365652+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:37:19.365652+0000","last_undegraded":"2024-09-10T20:37:19.365652+0000","last_fullsized":"2024-09-10T20:37:19.365652+0000","mapping_epoch":35,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15351808,"num_objects":3748,"num_object_clones":0,"num_object_copies":7496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16856,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'17321","reported_seq":17457,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:19.185792+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:37:19.185792+0000","last_peered":"2024-09-10T20:37:19.185792+0000","last_clean":"2024-09-10T20:37:19.185792+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:37:19.185792+0000","last_undegraded":"2024-09-10T20:37:19.185792+0000","last_fullsized":"2024-09-10T20:37:19.185792+0000","mapping_epoch":34,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15351808,"num_objects":3748,"num_object_clones":0,"num_object_copies":7496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14317,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'16902","reported_seq":17154,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:19.137446+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:37:19.137446+0000","last_peered":"2024-09-10T20:37:19.137446+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:37:19.137446+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14618624,"num_objects":3569,"num_object_clones":0,"num_object_copies":7138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13766,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'16888","reported_seq":17174,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:19.128217+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:37:19.128217+0000","last_peered":"2024-09-10T20:37:19.128217+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:37:19.128217+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15237120,"num_objects":3720,"num_object_clones":0,"num_object_copies":7440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13483,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":234}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'17098","reported_seq":17232,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:19.166065+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:37:19.166065+0000","last_peered":"2024-09-10T20:37:19.166065+0000","last_clean":"2024-09-10T20:37:19.166065+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:37:19.166065+0000","last_undegraded":"2024-09-10T20:37:19.166065+0000","last_fullsized":"2024-09-10T20:37:19.166065+0000","mapping_epoch":34,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14901248,"num_objects":3638,"num_object_clones":0,"num_object_copies":7276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14169,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'16752","reported_seq":16952,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:19.364604+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:37:19.364604+0000","last_peered":"2024-09-10T20:37:19.364604+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:37:19.364604+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15101952,"num_objects":3687,"num_object_clones":0,"num_object_copies":7374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3687,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14745,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":190}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318026+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318026+0000","last_peered":"2024-09-10T20:36:30.318026+0000","last_clean":"2024-09-10T20:36:30.318026+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318026+0000","last_undegraded":"2024-09-10T20:36:30.318026+0000","last_fullsized":"2024-09-10T20:36:30.318026+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'17067","reported_seq":17283,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:19.217595+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:37:19.217595+0000","last_peered":"2024-09-10T20:37:19.217595+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:37:19.217595+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15004614,"num_objects":3665,"num_object_clones":0,"num_object_copies":7330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17065,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":99}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'16904","reported_seq":17096,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:19.312814+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:37:19.312814+0000","last_peered":"2024-09-10T20:37:19.312814+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:37:19.312814+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14995456,"num_objects":3661,"num_object_clones":0,"num_object_copies":7322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14732,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":55}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'17054","reported_seq":24425,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:19.321837+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:37:19.321837+0000","last_peered":"2024-09-10T20:37:19.321837+0000","last_clean":"2024-09-10T20:37:19.321837+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:37:19.321837+0000","last_undegraded":"2024-09-10T20:37:19.321837+0000","last_fullsized":"2024-09-10T20:37:19.321837+0000","mapping_epoch":48,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14786560,"num_objects":3610,"num_object_clones":0,"num_object_copies":7220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3610,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14840,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'17117","reported_seq":17373,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:19.176173+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:37:19.176173+0000","last_peered":"2024-09-10T20:37:19.176173+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:37:19.176173+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15208448,"num_objects":3713,"num_object_clones":0,"num_object_copies":7426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13840,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":121}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'16967","reported_seq":19392,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:19.215281+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:37:19.215281+0000","last_peered":"2024-09-10T20:37:19.215281+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:37:19.215281+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14815232,"num_objects":3617,"num_object_clones":0,"num_object_copies":7234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2397,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13661,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1138,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1220}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319607+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319607+0000","last_peered":"2024-09-10T20:36:30.319607+0000","last_clean":"2024-09-10T20:36:30.319607+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319607+0000","last_undegraded":"2024-09-10T20:36:30.319607+0000","last_fullsized":"2024-09-10T20:36:30.319607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'17308","reported_seq":17565,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:19.152493+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:37:19.152493+0000","last_peered":"2024-09-10T20:37:19.152493+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:37:19.152493+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15065088,"num_objects":3678,"num_object_clones":0,"num_object_copies":7356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3678,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14135,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":23}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'16782","reported_seq":23909,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:16.010057+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:37:16.010057+0000","last_peered":"2024-09-10T20:37:16.010057+0000","last_clean":"2024-09-10T20:37:16.010057+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:37:16.010057+0000","last_undegraded":"2024-09-10T20:37:16.010057+0000","last_fullsized":"2024-09-10T20:37:16.010057+0000","mapping_epoch":46,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15294947,"num_objects":3735,"num_object_clones":0,"num_object_copies":7470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13389,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":241219009,"num_objects":58895,"num_object_clones":0,"num_object_copies":117790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30914,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58895,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":236522,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12235,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":662368256,"data_stored":662302304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160726,"ondisk_log_size":160726,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182505,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":649868,"kb_used_data":235060,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93721972,"statfs":{"total":96636764160,"available":95971299328,"internally_reserved":0,"allocated":240701440,"data_stored":240439492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":424734378},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,1,0,4,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986277,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572272,"kb_used_data":233756,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93799568,"statfs":{"total":96636764160,"available":96050757632,"internally_reserved":0,"allocated":239366144,"data_stored":239104201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346615466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659223,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":587388,"kb_used_data":180844,"kb_used_omap":4,"kb_used_meta":406523,"kb_avail":93784452,"statfs":{"total":96636764160,"available":96035278848,"internally_reserved":0,"allocated":185184256,"data_stored":184871267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":416280238},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,2,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":239452160,"data_stored":239448547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":184619008,"data_stored":184564354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":238297088,"data_stored":238289403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:37:21.858 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:37:21.858 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'17005", 'reported_seq': 17225, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:19.229745+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:37:19.229745+0000', 'last_peered': '2024-09-10T20:37:19.229745+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:37:19.229745+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "43'7000", 'ondisk_log_start': "43'7000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14962688, 'num_objects': 3653, 'num_object_clones': 0, 'num_object_copies': 7306, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3653, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17005, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 112}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:21.858 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:37:21.858 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'16902", 'reported_seq': 17154, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:19.137446+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:37:19.137446+0000', 'last_peered': '2024-09-10T20:37:19.137446+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:37:19.137446+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "43'6900", 'ondisk_log_start': "43'6900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14618624, 'num_objects': 3569, 'num_object_clones': 0, 'num_object_copies': 7138, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3569, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13766, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 8}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:21.858 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:37:21.858 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'16888", 'reported_seq': 17174, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:19.128217+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:37:19.128217+0000', 'last_peered': '2024-09-10T20:37:19.128217+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:37:19.128217+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15237120, 'num_objects': 3720, 'num_object_clones': 0, 'num_object_copies': 7440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3720, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13483, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 234}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:21.858 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:37:21.859 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'16752", 'reported_seq': 16952, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:19.364604+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:37:19.364604+0000', 'last_peered': '2024-09-10T20:37:19.364604+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:37:19.364604+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "43'6700", 'ondisk_log_start': "43'6700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15101952, 'num_objects': 3687, 'num_object_clones': 0, 'num_object_copies': 7374, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3687, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14745, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 190}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:37:21.859 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:37:21.859 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'17067", 'reported_seq': 17283, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:19.217595+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:37:19.217595+0000', 'last_peered': '2024-09-10T20:37:19.217595+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:37:19.217595+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "43'7000", 'ondisk_log_start': "43'7000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15004614, 'num_objects': 3665, 'num_object_clones': 0, 'num_object_copies': 7330, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3665, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17065, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 99}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:21.859 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:37:21.859 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'16904", 'reported_seq': 17096, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:19.312814+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:37:19.312814+0000', 'last_peered': '2024-09-10T20:37:19.312814+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:37:19.312814+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "43'6900", 'ondisk_log_start': "43'6900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14995456, 'num_objects': 3661, 'num_object_clones': 0, 'num_object_copies': 7322, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3661, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14732, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 55}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:37:21.860 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:37:21.860 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'17117", 'reported_seq': 17373, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:19.176173+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:37:19.176173+0000', 'last_peered': '2024-09-10T20:37:19.176173+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:37:19.176173+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "43'7100", 'ondisk_log_start': "43'7100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15208448, 'num_objects': 3713, 'num_object_clones': 0, 'num_object_copies': 7426, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3713, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13840, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 121}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:21.860 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:37:21.860 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'16967", 'reported_seq': 19392, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:19.215281+0000', 'last_change': '2024-09-10T20:36:28.778006+0000', 'last_active': '2024-09-10T20:37:19.215281+0000', 'last_peered': '2024-09-10T20:37:19.215281+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:37:19.215281+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "43'6900", 'ondisk_log_start': "43'6900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14815232, 'num_objects': 3617, 'num_object_clones': 0, 'num_object_copies': 7234, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2397, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3617, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13661, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1138, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1220}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:21.860 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:37:21.861 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'17308", 'reported_seq': 17565, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:19.152493+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:37:19.152493+0000', 'last_peered': '2024-09-10T20:37:19.152493+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:37:19.152493+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "43'7300", 'ondisk_log_start': "43'7300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15065088, 'num_objects': 3678, 'num_object_clones': 0, 'num_object_copies': 7356, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3678, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14135, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 23}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:21.861 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:37:22.557 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:22.557 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 30914/117794 objects degraded (26.244%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":554,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":58897,"data_bytes":241219028,"bytes_used":1852956672,"bytes_avail":288057335808,"bytes_total":289910292480,"degraded_objects":30914,"degraded_total":117794,"degraded_ratio":0.26244121092755152,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":296,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":591},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:37:22.575 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:37:22.575 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:37:23.150 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:23.153 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:37:23.172 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":320,"stamp":"2024-09-10T20:37:23.007152+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":240117204,"num_objects":58628,"num_object_clones":0,"num_object_copies":117256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30914,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58628,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":236793,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12237,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160698,"ondisk_log_size":160698,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1810692,"kb_used_data":642496,"kb_used_omap":12,"kb_used_meta":1168115,"kb_avail":281304828,"statfs":{"total":289910292480,"available":288056143872,"internally_reserved":0,"allocated":657915904,"data_stored":657079024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1196149762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,2,4,12,10,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21381120,"num_objects":-5220,"num_object_clones":0,"num_object_copies":-10440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-205,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5220,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":205,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004467"},"pg_stats":[{"pgid":"2.f","version":"49'17005","reported_seq":17225,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:19.229745+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:37:19.229745+0000","last_peered":"2024-09-10T20:37:19.229745+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:37:19.229745+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14962688,"num_objects":3653,"num_object_clones":0,"num_object_copies":7306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3653,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17005,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":112}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'17258","reported_seq":17377,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:19.332797+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:37:19.332797+0000","last_peered":"2024-09-10T20:37:19.332797+0000","last_clean":"2024-09-10T20:37:19.332797+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:37:19.332797+0000","last_undegraded":"2024-09-10T20:37:19.332797+0000","last_fullsized":"2024-09-10T20:37:19.332797+0000","mapping_epoch":35,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15089664,"num_objects":3684,"num_object_clones":0,"num_object_copies":7368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3684,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15444,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'17096","reported_seq":24454,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:21.009054+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:37:21.009054+0000","last_peered":"2024-09-10T20:37:21.009054+0000","last_clean":"2024-09-10T20:37:21.009054+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:37:21.009054+0000","last_undegraded":"2024-09-10T20:37:21.009054+0000","last_fullsized":"2024-09-10T20:37:21.009054+0000","mapping_epoch":44,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14823448,"num_objects":3620,"num_object_clones":0,"num_object_copies":7240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3620,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15224,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'16856","reported_seq":16946,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:19.365652+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:37:19.365652+0000","last_peered":"2024-09-10T20:37:19.365652+0000","last_clean":"2024-09-10T20:37:19.365652+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:37:19.365652+0000","last_undegraded":"2024-09-10T20:37:19.365652+0000","last_fullsized":"2024-09-10T20:37:19.365652+0000","mapping_epoch":35,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15351808,"num_objects":3748,"num_object_clones":0,"num_object_copies":7496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16856,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'17321","reported_seq":17457,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:19.185792+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:37:19.185792+0000","last_peered":"2024-09-10T20:37:19.185792+0000","last_clean":"2024-09-10T20:37:19.185792+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:37:19.185792+0000","last_undegraded":"2024-09-10T20:37:19.185792+0000","last_fullsized":"2024-09-10T20:37:19.185792+0000","mapping_epoch":34,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15351808,"num_objects":3748,"num_object_clones":0,"num_object_copies":7496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14317,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'16902","reported_seq":17154,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:19.137446+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:37:19.137446+0000","last_peered":"2024-09-10T20:37:19.137446+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:37:19.137446+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14618624,"num_objects":3569,"num_object_clones":0,"num_object_copies":7138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13766,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'16888","reported_seq":17174,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:19.128217+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:37:19.128217+0000","last_peered":"2024-09-10T20:37:19.128217+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:37:19.128217+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15237120,"num_objects":3720,"num_object_clones":0,"num_object_copies":7440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13483,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":234}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'17098","reported_seq":17232,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:19.166065+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:37:19.166065+0000","last_peered":"2024-09-10T20:37:19.166065+0000","last_clean":"2024-09-10T20:37:19.166065+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:37:19.166065+0000","last_undegraded":"2024-09-10T20:37:19.166065+0000","last_fullsized":"2024-09-10T20:37:19.166065+0000","mapping_epoch":34,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14901248,"num_objects":3638,"num_object_clones":0,"num_object_copies":7276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14169,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'16752","reported_seq":16952,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:19.364604+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:37:19.364604+0000","last_peered":"2024-09-10T20:37:19.364604+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:37:19.364604+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15101952,"num_objects":3687,"num_object_clones":0,"num_object_copies":7374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3687,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14745,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":190}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318026+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318026+0000","last_peered":"2024-09-10T20:36:30.318026+0000","last_clean":"2024-09-10T20:36:30.318026+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318026+0000","last_undegraded":"2024-09-10T20:36:30.318026+0000","last_fullsized":"2024-09-10T20:36:30.318026+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'17067","reported_seq":17283,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:19.217595+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:37:19.217595+0000","last_peered":"2024-09-10T20:37:19.217595+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:37:19.217595+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15004614,"num_objects":3665,"num_object_clones":0,"num_object_copies":7330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17065,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":99}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'16904","reported_seq":17096,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:19.312814+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:37:19.312814+0000","last_peered":"2024-09-10T20:37:19.312814+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:37:19.312814+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14995456,"num_objects":3661,"num_object_clones":0,"num_object_copies":7322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14732,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":55}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'17054","reported_seq":24425,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:19.321837+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:37:19.321837+0000","last_peered":"2024-09-10T20:37:19.321837+0000","last_clean":"2024-09-10T20:37:19.321837+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:37:19.321837+0000","last_undegraded":"2024-09-10T20:37:19.321837+0000","last_fullsized":"2024-09-10T20:37:19.321837+0000","mapping_epoch":48,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14786560,"num_objects":3610,"num_object_clones":0,"num_object_copies":7220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3610,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14840,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'17117","reported_seq":17373,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:19.176173+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:37:19.176173+0000","last_peered":"2024-09-10T20:37:19.176173+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:37:19.176173+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15208448,"num_objects":3713,"num_object_clones":0,"num_object_copies":7426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13840,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":121}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'16967","reported_seq":19392,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:19.215281+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:37:19.215281+0000","last_peered":"2024-09-10T20:37:19.215281+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:37:19.215281+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14815232,"num_objects":3617,"num_object_clones":0,"num_object_copies":7234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2397,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13661,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1138,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1220}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319607+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319607+0000","last_peered":"2024-09-10T20:36:30.319607+0000","last_clean":"2024-09-10T20:36:30.319607+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319607+0000","last_undegraded":"2024-09-10T20:36:30.319607+0000","last_fullsized":"2024-09-10T20:36:30.319607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'17308","reported_seq":17565,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:19.152493+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:37:19.152493+0000","last_peered":"2024-09-10T20:37:19.152493+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:37:19.152493+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15065088,"num_objects":3678,"num_object_clones":0,"num_object_copies":7356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3678,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14135,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":23}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'16902","reported_seq":24029,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:21.062222+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:37:21.062222+0000","last_peered":"2024-09-10T20:37:21.062222+0000","last_clean":"2024-09-10T20:37:21.062222+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:37:21.062222+0000","last_undegraded":"2024-09-10T20:37:21.062222+0000","last_fullsized":"2024-09-10T20:37:21.062222+0000","mapping_epoch":46,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14803427,"num_objects":3615,"num_object_clones":0,"num_object_copies":7230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13509,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":240117185,"num_objects":58626,"num_object_clones":0,"num_object_copies":117252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30914,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58626,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":236791,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12235,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":654876672,"data_stored":654810720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160695,"ondisk_log_size":160695,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182505,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":649868,"kb_used_data":235060,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93721972,"statfs":{"total":96636764160,"available":95971299328,"internally_reserved":0,"allocated":240701440,"data_stored":240439492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":424734378},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,1,0,4,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986277,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572272,"kb_used_data":233756,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93799568,"statfs":{"total":96636764160,"available":96050757632,"internally_reserved":0,"allocated":239366144,"data_stored":239104201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346615466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659224,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":588552,"kb_used_data":173680,"kb_used_omap":4,"kb_used_meta":414843,"kb_avail":93783288,"statfs":{"total":96636764160,"available":96034086912,"internally_reserved":0,"allocated":177848320,"data_stored":177535331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":424799918},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,6,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":239452160,"data_stored":239448547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":177127424,"data_stored":177072770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":238297088,"data_stored":238289403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:37:26.174 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:37:26.761 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:26.762 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:37:26.789 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-09-10T20:37:25.007809+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":231470548,"num_objects":56517,"num_object_clones":0,"num_object_copies":113034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30806,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56517,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":238904,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12345,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160809,"ondisk_log_size":160809,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1807248,"kb_used_data":630732,"kb_used_omap":12,"kb_used_meta":1176435,"kb_avail":281308272,"statfs":{"total":289910292480,"available":288059670528,"internally_reserved":0,"allocated":645869568,"data_stored":645032688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1204669442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,6,10,8,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28815360,"num_objects":-7035,"num_object_clones":0,"num_object_copies":-14070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-313,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7035,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004306"},"pg_stats":[{"pgid":"2.f","version":"49'17165","reported_seq":17387,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:24.246831+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:37:24.246831+0000","last_peered":"2024-09-10T20:37:24.246831+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:37:24.246831+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14307328,"num_objects":3493,"num_object_clones":0,"num_object_copies":6986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17165,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-48}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'17415","reported_seq":17533,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:24.352492+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:37:24.352492+0000","last_peered":"2024-09-10T20:37:24.352492+0000","last_clean":"2024-09-10T20:37:24.352492+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:37:24.352492+0000","last_undegraded":"2024-09-10T20:37:24.352492+0000","last_fullsized":"2024-09-10T20:37:24.352492+0000","mapping_epoch":35,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14446592,"num_objects":3527,"num_object_clones":0,"num_object_copies":7054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15601,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'17096","reported_seq":24454,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:21.009054+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:37:21.009054+0000","last_peered":"2024-09-10T20:37:21.009054+0000","last_clean":"2024-09-10T20:37:21.009054+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:37:21.009054+0000","last_undegraded":"2024-09-10T20:37:21.009054+0000","last_fullsized":"2024-09-10T20:37:21.009054+0000","mapping_epoch":44,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14823448,"num_objects":3620,"num_object_clones":0,"num_object_copies":7240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3620,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15224,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'16998","reported_seq":17088,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:24.359840+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:37:24.359840+0000","last_peered":"2024-09-10T20:37:24.359840+0000","last_clean":"2024-09-10T20:37:24.359840+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:37:24.359840+0000","last_undegraded":"2024-09-10T20:37:24.359840+0000","last_fullsized":"2024-09-10T20:37:24.359840+0000","mapping_epoch":35,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14770176,"num_objects":3606,"num_object_clones":0,"num_object_copies":7212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16998,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'17468","reported_seq":17601,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:24.200519+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:37:24.200519+0000","last_peered":"2024-09-10T20:37:24.200519+0000","last_clean":"2024-09-10T20:37:24.200519+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:37:24.200519+0000","last_undegraded":"2024-09-10T20:37:24.200519+0000","last_fullsized":"2024-09-10T20:37:24.200519+0000","mapping_epoch":34,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14749696,"num_objects":3601,"num_object_clones":0,"num_object_copies":7202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14464,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'17051","reported_seq":17303,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:24.232810+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:37:24.232810+0000","last_peered":"2024-09-10T20:37:24.232810+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:37:24.232810+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14008320,"num_objects":3420,"num_object_clones":0,"num_object_copies":6840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13915,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-141}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'17041","reported_seq":17327,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:24.267936+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:37:24.267936+0000","last_peered":"2024-09-10T20:37:24.267936+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:37:24.267936+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14610432,"num_objects":3567,"num_object_clones":0,"num_object_copies":7134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13636,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":81}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'17244","reported_seq":17378,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:24.155435+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:37:24.155435+0000","last_peered":"2024-09-10T20:37:24.155435+0000","last_clean":"2024-09-10T20:37:24.155435+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:37:24.155435+0000","last_undegraded":"2024-09-10T20:37:24.155435+0000","last_fullsized":"2024-09-10T20:37:24.155435+0000","mapping_epoch":34,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14303232,"num_objects":3492,"num_object_clones":0,"num_object_copies":6984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14315,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'16922","reported_seq":17122,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:24.390917+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:37:24.390917+0000","last_peered":"2024-09-10T20:37:24.390917+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:37:24.390917+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14405632,"num_objects":3517,"num_object_clones":0,"num_object_copies":7034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14915,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":20}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318026+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318026+0000","last_peered":"2024-09-10T20:36:30.318026+0000","last_clean":"2024-09-10T20:36:30.318026+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318026+0000","last_undegraded":"2024-09-10T20:36:30.318026+0000","last_fullsized":"2024-09-10T20:36:30.318026+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'17211","reported_seq":17428,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:24.264132+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:37:24.264132+0000","last_peered":"2024-09-10T20:37:24.264132+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:37:24.264132+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14414790,"num_objects":3521,"num_object_clones":0,"num_object_copies":7042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17209,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-45}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'17057","reported_seq":17249,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:24.175012+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:37:24.175012+0000","last_peered":"2024-09-10T20:37:24.175012+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:37:24.175012+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14368768,"num_objects":3508,"num_object_clones":0,"num_object_copies":7016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14885,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-98}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'17196","reported_seq":24567,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:24.379766+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:37:24.379766+0000","last_peered":"2024-09-10T20:37:24.379766+0000","last_clean":"2024-09-10T20:37:24.379766+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:37:24.379766+0000","last_undegraded":"2024-09-10T20:37:24.379766+0000","last_fullsized":"2024-09-10T20:37:24.379766+0000","mapping_epoch":48,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14204928,"num_objects":3468,"num_object_clones":0,"num_object_copies":6936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3468,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14982,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'17279","reported_seq":17536,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:24.220950+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:37:24.220950+0000","last_peered":"2024-09-10T20:37:24.220950+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:37:24.220950+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14544896,"num_objects":3551,"num_object_clones":0,"num_object_copies":7102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14002,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-41}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'17106","reported_seq":19747,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:24.274031+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:37:24.274031+0000","last_peered":"2024-09-10T20:37:24.274031+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:37:24.274031+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14245888,"num_objects":3478,"num_object_clones":0,"num_object_copies":6956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2289,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13800,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1246,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1189}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319607+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319607+0000","last_peered":"2024-09-10T20:36:30.319607+0000","last_clean":"2024-09-10T20:36:30.319607+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319607+0000","last_undegraded":"2024-09-10T20:36:30.319607+0000","last_fullsized":"2024-09-10T20:36:30.319607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'17455","reported_seq":17712,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:24.258559+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:37:24.258559+0000","last_peered":"2024-09-10T20:37:24.258559+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:37:24.258559+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14462976,"num_objects":3531,"num_object_clones":0,"num_object_copies":7062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3531,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14282,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-124}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'16902","reported_seq":24029,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:21.062222+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:37:21.062222+0000","last_peered":"2024-09-10T20:37:21.062222+0000","last_clean":"2024-09-10T20:37:21.062222+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:37:21.062222+0000","last_undegraded":"2024-09-10T20:37:21.062222+0000","last_fullsized":"2024-09-10T20:37:21.062222+0000","mapping_epoch":46,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14803427,"num_objects":3615,"num_object_clones":0,"num_object_copies":7230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13509,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":231470529,"num_objects":56515,"num_object_clones":0,"num_object_copies":113030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30806,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56515,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":238902,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12343,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":642666496,"data_stored":642600544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160806,"ondisk_log_size":160806,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182506,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":647040,"kb_used_data":228080,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93724800,"statfs":{"total":96636764160,"available":95974195200,"internally_reserved":0,"allocated":233553920,"data_stored":233291972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":428994218},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,4,4,1,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986278,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":571656,"kb_used_data":228972,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93800184,"statfs":{"total":96636764160,"available":96051388416,"internally_reserved":0,"allocated":234467328,"data_stored":234205385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350875306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659224,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":588552,"kb_used_data":173680,"kb_used_omap":4,"kb_used_meta":414843,"kb_avail":93783288,"statfs":{"total":96636764160,"available":96034086912,"internally_reserved":0,"allocated":177848320,"data_stored":177535331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":424799918},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,6,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":232173568,"data_stored":232169955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":177127424,"data_stored":177072770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":233365504,"data_stored":233357819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:37:26.790 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:37:27.375 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:27.377 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:37:27.410 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-09-10T20:37:27.008441+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":230073812,"num_objects":56176,"num_object_clones":0,"num_object_copies":112352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30806,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56176,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":239245,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12345,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160850,"ondisk_log_size":160850,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1804032,"kb_used_data":623356,"kb_used_omap":12,"kb_used_meta":1180595,"kb_avail":281311488,"statfs":{"total":289910292480,"available":288062963712,"internally_reserved":0,"allocated":638316544,"data_stored":637479664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1208929282},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,9,5,12,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21295104,"num_objects":-5199,"num_object_clones":0,"num_object_copies":-10398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-216,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5199,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004255"},"pg_stats":[{"pgid":"2.f","version":"49'17165","reported_seq":17387,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:24.246831+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:37:24.246831+0000","last_peered":"2024-09-10T20:37:24.246831+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:37:24.246831+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14307328,"num_objects":3493,"num_object_clones":0,"num_object_copies":6986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17165,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-48}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'17415","reported_seq":17533,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:24.352492+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:37:24.352492+0000","last_peered":"2024-09-10T20:37:24.352492+0000","last_clean":"2024-09-10T20:37:24.352492+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:37:24.352492+0000","last_undegraded":"2024-09-10T20:37:24.352492+0000","last_fullsized":"2024-09-10T20:37:24.352492+0000","mapping_epoch":35,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14446592,"num_objects":3527,"num_object_clones":0,"num_object_copies":7054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15601,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'17276","reported_seq":24632,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:26.101222+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:37:26.101222+0000","last_peered":"2024-09-10T20:37:26.101222+0000","last_clean":"2024-09-10T20:37:26.101222+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:37:26.101222+0000","last_undegraded":"2024-09-10T20:37:26.101222+0000","last_fullsized":"2024-09-10T20:37:26.101222+0000","mapping_epoch":44,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14086168,"num_objects":3440,"num_object_clones":0,"num_object_copies":6880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3440,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15404,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'16998","reported_seq":17088,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:24.359840+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:37:24.359840+0000","last_peered":"2024-09-10T20:37:24.359840+0000","last_clean":"2024-09-10T20:37:24.359840+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:37:24.359840+0000","last_undegraded":"2024-09-10T20:37:24.359840+0000","last_fullsized":"2024-09-10T20:37:24.359840+0000","mapping_epoch":35,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14770176,"num_objects":3606,"num_object_clones":0,"num_object_copies":7212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16998,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'17468","reported_seq":17601,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:24.200519+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:37:24.200519+0000","last_peered":"2024-09-10T20:37:24.200519+0000","last_clean":"2024-09-10T20:37:24.200519+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:37:24.200519+0000","last_undegraded":"2024-09-10T20:37:24.200519+0000","last_fullsized":"2024-09-10T20:37:24.200519+0000","mapping_epoch":34,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14749696,"num_objects":3601,"num_object_clones":0,"num_object_copies":7202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14464,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'17051","reported_seq":17303,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:24.232810+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:37:24.232810+0000","last_peered":"2024-09-10T20:37:24.232810+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:37:24.232810+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14008320,"num_objects":3420,"num_object_clones":0,"num_object_copies":6840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13915,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-141}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'17041","reported_seq":17327,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:24.267936+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:37:24.267936+0000","last_peered":"2024-09-10T20:37:24.267936+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:37:24.267936+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14610432,"num_objects":3567,"num_object_clones":0,"num_object_copies":7134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13636,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":81}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'17244","reported_seq":17378,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:24.155435+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:37:24.155435+0000","last_peered":"2024-09-10T20:37:24.155435+0000","last_clean":"2024-09-10T20:37:24.155435+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:37:24.155435+0000","last_undegraded":"2024-09-10T20:37:24.155435+0000","last_fullsized":"2024-09-10T20:37:24.155435+0000","mapping_epoch":34,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14303232,"num_objects":3492,"num_object_clones":0,"num_object_copies":6984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14315,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'16922","reported_seq":17122,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:24.390917+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:37:24.390917+0000","last_peered":"2024-09-10T20:37:24.390917+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:37:24.390917+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14405632,"num_objects":3517,"num_object_clones":0,"num_object_copies":7034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14915,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":20}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318026+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318026+0000","last_peered":"2024-09-10T20:36:30.318026+0000","last_clean":"2024-09-10T20:36:30.318026+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318026+0000","last_undegraded":"2024-09-10T20:36:30.318026+0000","last_fullsized":"2024-09-10T20:36:30.318026+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'17211","reported_seq":17428,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:24.264132+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:37:24.264132+0000","last_peered":"2024-09-10T20:37:24.264132+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:37:24.264132+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14414790,"num_objects":3521,"num_object_clones":0,"num_object_copies":7042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17209,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-45}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'17057","reported_seq":17249,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:24.175012+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:37:24.175012+0000","last_peered":"2024-09-10T20:37:24.175012+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:37:24.175012+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14368768,"num_objects":3508,"num_object_clones":0,"num_object_copies":7016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14885,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-98}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'17196","reported_seq":24567,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:24.379766+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:37:24.379766+0000","last_peered":"2024-09-10T20:37:24.379766+0000","last_clean":"2024-09-10T20:37:24.379766+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:37:24.379766+0000","last_undegraded":"2024-09-10T20:37:24.379766+0000","last_fullsized":"2024-09-10T20:37:24.379766+0000","mapping_epoch":48,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14204928,"num_objects":3468,"num_object_clones":0,"num_object_copies":6936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3468,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14982,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'17279","reported_seq":17536,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:24.220950+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:37:24.220950+0000","last_peered":"2024-09-10T20:37:24.220950+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:37:24.220950+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14544896,"num_objects":3551,"num_object_clones":0,"num_object_copies":7102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14002,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-41}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'17106","reported_seq":19747,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:24.274031+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:37:24.274031+0000","last_peered":"2024-09-10T20:37:24.274031+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:37:24.274031+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14245888,"num_objects":3478,"num_object_clones":0,"num_object_copies":6956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2289,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13800,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1246,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1189}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319607+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319607+0000","last_peered":"2024-09-10T20:36:30.319607+0000","last_clean":"2024-09-10T20:36:30.319607+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319607+0000","last_undegraded":"2024-09-10T20:36:30.319607+0000","last_fullsized":"2024-09-10T20:36:30.319607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'17455","reported_seq":17712,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:24.258559+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:37:24.258559+0000","last_peered":"2024-09-10T20:37:24.258559+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:37:24.258559+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14462976,"num_objects":3531,"num_object_clones":0,"num_object_copies":7062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3531,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14282,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-124}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'17063","reported_seq":24190,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:26.145718+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:37:26.145718+0000","last_peered":"2024-09-10T20:37:26.145718+0000","last_clean":"2024-09-10T20:37:26.145718+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:37:26.145718+0000","last_undegraded":"2024-09-10T20:37:26.145718+0000","last_fullsized":"2024-09-10T20:37:26.145718+0000","mapping_epoch":46,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14143971,"num_objects":3454,"num_object_clones":0,"num_object_copies":6908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13670,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":230073793,"num_objects":56174,"num_object_clones":0,"num_object_copies":112348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30806,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56174,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":239243,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12343,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":635015168,"data_stored":634949216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160847,"ondisk_log_size":160847,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182506,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":647040,"kb_used_data":228080,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93724800,"statfs":{"total":96636764160,"available":95974195200,"internally_reserved":0,"allocated":233553920,"data_stored":233291972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":428994218},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,4,4,1,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986278,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":571656,"kb_used_data":228972,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93800184,"statfs":{"total":96636764160,"available":96051388416,"internally_reserved":0,"allocated":234467328,"data_stored":234205385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350875306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659225,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":585336,"kb_used_data":166304,"kb_used_omap":4,"kb_used_meta":419003,"kb_avail":93786504,"statfs":{"total":96636764160,"available":96037380096,"internally_reserved":0,"allocated":170295296,"data_stored":169982307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":429059758},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,8,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":232173568,"data_stored":232169955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":169476096,"data_stored":169421442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":233365504,"data_stored":233357819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:37:27.411 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:37:27.411 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'17165", 'reported_seq': 17387, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:24.246831+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:37:24.246831+0000', 'last_peered': '2024-09-10T20:37:24.246831+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:37:24.246831+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "43'7100", 'ondisk_log_start': "43'7100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14307328, 'num_objects': 3493, 'num_object_clones': 0, 'num_object_copies': 6986, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3493, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17165, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -48}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:27.411 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:37:27.411 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'17051", 'reported_seq': 17303, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:24.232810+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:37:24.232810+0000', 'last_peered': '2024-09-10T20:37:24.232810+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:37:24.232810+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "43'7000", 'ondisk_log_start': "43'7000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14008320, 'num_objects': 3420, 'num_object_clones': 0, 'num_object_copies': 6840, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3420, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13915, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -141}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:27.412 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:37:27.412 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'17041", 'reported_seq': 17327, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:24.267936+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:37:24.267936+0000', 'last_peered': '2024-09-10T20:37:24.267936+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:37:24.267936+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "43'7000", 'ondisk_log_start': "43'7000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14610432, 'num_objects': 3567, 'num_object_clones': 0, 'num_object_copies': 7134, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3567, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13636, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 81}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:27.412 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:37:27.412 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'16922", 'reported_seq': 17122, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:24.390917+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:37:24.390917+0000', 'last_peered': '2024-09-10T20:37:24.390917+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:37:24.390917+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "43'6900", 'ondisk_log_start': "43'6900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14405632, 'num_objects': 3517, 'num_object_clones': 0, 'num_object_copies': 7034, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3517, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14915, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 20}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:37:27.412 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:37:27.412 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'17211", 'reported_seq': 17428, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:24.264132+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:37:24.264132+0000', 'last_peered': '2024-09-10T20:37:24.264132+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:37:24.264132+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "43'7200", 'ondisk_log_start': "43'7200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14414790, 'num_objects': 3521, 'num_object_clones': 0, 'num_object_copies': 7042, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3521, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17209, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -45}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:27.412 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:37:27.412 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'17057", 'reported_seq': 17249, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:24.175012+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:37:24.175012+0000', 'last_peered': '2024-09-10T20:37:24.175012+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:37:24.175012+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "43'7000", 'ondisk_log_start': "43'7000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14368768, 'num_objects': 3508, 'num_object_clones': 0, 'num_object_copies': 7016, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3508, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14885, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -98}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:37:27.412 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:37:27.412 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'17279", 'reported_seq': 17536, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:24.220950+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:37:24.220950+0000', 'last_peered': '2024-09-10T20:37:24.220950+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:37:24.220950+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "43'7200", 'ondisk_log_start': "43'7200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14544896, 'num_objects': 3551, 'num_object_clones': 0, 'num_object_copies': 7102, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3551, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14002, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -41}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:27.413 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:37:27.413 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'17106", 'reported_seq': 19747, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:24.274031+0000', 'last_change': '2024-09-10T20:36:28.778006+0000', 'last_active': '2024-09-10T20:37:24.274031+0000', 'last_peered': '2024-09-10T20:37:24.274031+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:37:24.274031+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "43'7100", 'ondisk_log_start': "43'7100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14245888, 'num_objects': 3478, 'num_object_clones': 0, 'num_object_copies': 6956, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2289, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3478, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13800, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1246, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1189}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:27.413 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:37:27.413 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'17455", 'reported_seq': 17712, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:24.258559+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:37:24.258559+0000', 'last_peered': '2024-09-10T20:37:24.258559+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:37:24.258559+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "43'7400", 'ondisk_log_start': "43'7400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14462976, 'num_objects': 3531, 'num_object_clones': 0, 'num_object_copies': 7062, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3531, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14282, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -124}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:27.414 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:37:28.109 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:28.110 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 30806/112352 objects degraded (27.419%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":559,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":56176,"data_bytes":230073812,"bytes_used":1847328768,"bytes_avail":288062963712,"bytes_total":289910292480,"degraded_objects":30806,"degraded_total":112352,"degraded_ratio":0.2741918256906864,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":216,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":433},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:37:28.139 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:37:28.139 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:37:28.688 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:28.690 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:37:28.726 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-09-10T20:37:27.008441+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":230073812,"num_objects":56176,"num_object_clones":0,"num_object_copies":112352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30806,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56176,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":239245,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12345,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160850,"ondisk_log_size":160850,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1804032,"kb_used_data":623356,"kb_used_omap":12,"kb_used_meta":1180595,"kb_avail":281311488,"statfs":{"total":289910292480,"available":288062963712,"internally_reserved":0,"allocated":638316544,"data_stored":637479664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1208929282},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,9,5,12,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21295104,"num_objects":-5199,"num_object_clones":0,"num_object_copies":-10398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-216,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5199,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004255"},"pg_stats":[{"pgid":"2.f","version":"49'17165","reported_seq":17387,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:24.246831+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:37:24.246831+0000","last_peered":"2024-09-10T20:37:24.246831+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:37:24.246831+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14307328,"num_objects":3493,"num_object_clones":0,"num_object_copies":6986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17165,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-48}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'17415","reported_seq":17533,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:24.352492+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:37:24.352492+0000","last_peered":"2024-09-10T20:37:24.352492+0000","last_clean":"2024-09-10T20:37:24.352492+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:37:24.352492+0000","last_undegraded":"2024-09-10T20:37:24.352492+0000","last_fullsized":"2024-09-10T20:37:24.352492+0000","mapping_epoch":35,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14446592,"num_objects":3527,"num_object_clones":0,"num_object_copies":7054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15601,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'17276","reported_seq":24632,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:26.101222+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:37:26.101222+0000","last_peered":"2024-09-10T20:37:26.101222+0000","last_clean":"2024-09-10T20:37:26.101222+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:37:26.101222+0000","last_undegraded":"2024-09-10T20:37:26.101222+0000","last_fullsized":"2024-09-10T20:37:26.101222+0000","mapping_epoch":44,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14086168,"num_objects":3440,"num_object_clones":0,"num_object_copies":6880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3440,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15404,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'16998","reported_seq":17088,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:24.359840+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:37:24.359840+0000","last_peered":"2024-09-10T20:37:24.359840+0000","last_clean":"2024-09-10T20:37:24.359840+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:37:24.359840+0000","last_undegraded":"2024-09-10T20:37:24.359840+0000","last_fullsized":"2024-09-10T20:37:24.359840+0000","mapping_epoch":35,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14770176,"num_objects":3606,"num_object_clones":0,"num_object_copies":7212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16998,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'17468","reported_seq":17601,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:24.200519+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:37:24.200519+0000","last_peered":"2024-09-10T20:37:24.200519+0000","last_clean":"2024-09-10T20:37:24.200519+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:37:24.200519+0000","last_undegraded":"2024-09-10T20:37:24.200519+0000","last_fullsized":"2024-09-10T20:37:24.200519+0000","mapping_epoch":34,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14749696,"num_objects":3601,"num_object_clones":0,"num_object_copies":7202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14464,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'17051","reported_seq":17303,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:24.232810+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:37:24.232810+0000","last_peered":"2024-09-10T20:37:24.232810+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:37:24.232810+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14008320,"num_objects":3420,"num_object_clones":0,"num_object_copies":6840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13915,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-141}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'17041","reported_seq":17327,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:24.267936+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:37:24.267936+0000","last_peered":"2024-09-10T20:37:24.267936+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:37:24.267936+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14610432,"num_objects":3567,"num_object_clones":0,"num_object_copies":7134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13636,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":81}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'17244","reported_seq":17378,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:24.155435+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:37:24.155435+0000","last_peered":"2024-09-10T20:37:24.155435+0000","last_clean":"2024-09-10T20:37:24.155435+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:37:24.155435+0000","last_undegraded":"2024-09-10T20:37:24.155435+0000","last_fullsized":"2024-09-10T20:37:24.155435+0000","mapping_epoch":34,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14303232,"num_objects":3492,"num_object_clones":0,"num_object_copies":6984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14315,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'16922","reported_seq":17122,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:24.390917+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:37:24.390917+0000","last_peered":"2024-09-10T20:37:24.390917+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:37:24.390917+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14405632,"num_objects":3517,"num_object_clones":0,"num_object_copies":7034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14915,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":20}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318026+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318026+0000","last_peered":"2024-09-10T20:36:30.318026+0000","last_clean":"2024-09-10T20:36:30.318026+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318026+0000","last_undegraded":"2024-09-10T20:36:30.318026+0000","last_fullsized":"2024-09-10T20:36:30.318026+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'17211","reported_seq":17428,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:24.264132+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:37:24.264132+0000","last_peered":"2024-09-10T20:37:24.264132+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:37:24.264132+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14414790,"num_objects":3521,"num_object_clones":0,"num_object_copies":7042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17209,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-45}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'17057","reported_seq":17249,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:24.175012+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:37:24.175012+0000","last_peered":"2024-09-10T20:37:24.175012+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:37:24.175012+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14368768,"num_objects":3508,"num_object_clones":0,"num_object_copies":7016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14885,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-98}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'17196","reported_seq":24567,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:24.379766+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:37:24.379766+0000","last_peered":"2024-09-10T20:37:24.379766+0000","last_clean":"2024-09-10T20:37:24.379766+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:37:24.379766+0000","last_undegraded":"2024-09-10T20:37:24.379766+0000","last_fullsized":"2024-09-10T20:37:24.379766+0000","mapping_epoch":48,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14204928,"num_objects":3468,"num_object_clones":0,"num_object_copies":6936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3468,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14982,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'17279","reported_seq":17536,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:24.220950+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:37:24.220950+0000","last_peered":"2024-09-10T20:37:24.220950+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:37:24.220950+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14544896,"num_objects":3551,"num_object_clones":0,"num_object_copies":7102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14002,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-41}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'17106","reported_seq":19747,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:24.274031+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:37:24.274031+0000","last_peered":"2024-09-10T20:37:24.274031+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:37:24.274031+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14245888,"num_objects":3478,"num_object_clones":0,"num_object_copies":6956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2289,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13800,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1246,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1189}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319607+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319607+0000","last_peered":"2024-09-10T20:36:30.319607+0000","last_clean":"2024-09-10T20:36:30.319607+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319607+0000","last_undegraded":"2024-09-10T20:36:30.319607+0000","last_fullsized":"2024-09-10T20:36:30.319607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'17455","reported_seq":17712,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:24.258559+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:37:24.258559+0000","last_peered":"2024-09-10T20:37:24.258559+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:37:24.258559+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14462976,"num_objects":3531,"num_object_clones":0,"num_object_copies":7062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3531,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14282,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-124}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'17063","reported_seq":24190,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:26.145718+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:37:26.145718+0000","last_peered":"2024-09-10T20:37:26.145718+0000","last_clean":"2024-09-10T20:37:26.145718+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:37:26.145718+0000","last_undegraded":"2024-09-10T20:37:26.145718+0000","last_fullsized":"2024-09-10T20:37:26.145718+0000","mapping_epoch":46,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14143971,"num_objects":3454,"num_object_clones":0,"num_object_copies":6908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13670,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":230073793,"num_objects":56174,"num_object_clones":0,"num_object_copies":112348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30806,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56174,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":239243,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12343,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":635015168,"data_stored":634949216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160847,"ondisk_log_size":160847,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182506,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":647040,"kb_used_data":228080,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93724800,"statfs":{"total":96636764160,"available":95974195200,"internally_reserved":0,"allocated":233553920,"data_stored":233291972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":428994218},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,4,4,1,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986278,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":571656,"kb_used_data":228972,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93800184,"statfs":{"total":96636764160,"available":96051388416,"internally_reserved":0,"allocated":234467328,"data_stored":234205385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350875306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659225,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":585336,"kb_used_data":166304,"kb_used_omap":4,"kb_used_meta":419003,"kb_avail":93786504,"statfs":{"total":96636764160,"available":96037380096,"internally_reserved":0,"allocated":170295296,"data_stored":169982307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":429059758},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,8,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":232173568,"data_stored":232169955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":169476096,"data_stored":169421442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":233365504,"data_stored":233357819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:37:31.728 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:37:32.267 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:32.269 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:37:32.289 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2024-09-10T20:37:31.009758+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":221410772,"num_objects":54061,"num_object_clones":0,"num_object_copies":108122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30701,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54061,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":241360,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12450,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160765,"ondisk_log_size":160765,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1787504,"kb_used_data":610980,"kb_used_omap":12,"kb_used_meta":1176435,"kb_avail":281328016,"statfs":{"total":289910292480,"available":288079888384,"internally_reserved":0,"allocated":625643520,"data_stored":624806640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1204669442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,8,1,10,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28618752,"num_objects":-6987,"num_object_clones":0,"num_object_copies":-13974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-321,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6987,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6987,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":321,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004114"},"pg_stats":[{"pgid":"2.f","version":"49'17305","reported_seq":17527,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:29.286965+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:37:29.286965+0000","last_peered":"2024-09-10T20:37:29.286965+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:37:29.286965+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13733888,"num_objects":3353,"num_object_clones":0,"num_object_copies":6706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17305,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-188}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'17568","reported_seq":17688,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:29.365269+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:37:29.365269+0000","last_peered":"2024-09-10T20:37:29.365269+0000","last_clean":"2024-09-10T20:37:29.365269+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:37:29.365269+0000","last_undegraded":"2024-09-10T20:37:29.365269+0000","last_fullsized":"2024-09-10T20:37:29.365269+0000","mapping_epoch":35,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13819904,"num_objects":3374,"num_object_clones":0,"num_object_copies":6748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3374,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15754,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'17276","reported_seq":24632,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:26.101222+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:37:26.101222+0000","last_peered":"2024-09-10T20:37:26.101222+0000","last_clean":"2024-09-10T20:37:26.101222+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:37:26.101222+0000","last_undegraded":"2024-09-10T20:37:26.101222+0000","last_fullsized":"2024-09-10T20:37:26.101222+0000","mapping_epoch":44,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14086168,"num_objects":3440,"num_object_clones":0,"num_object_copies":6880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3440,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15404,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'17170","reported_seq":17260,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:29.349030+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:37:29.349030+0000","last_peered":"2024-09-10T20:37:29.349030+0000","last_clean":"2024-09-10T20:37:29.349030+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:37:29.349030+0000","last_undegraded":"2024-09-10T20:37:29.349030+0000","last_fullsized":"2024-09-10T20:37:29.349030+0000","mapping_epoch":35,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14065664,"num_objects":3434,"num_object_clones":0,"num_object_copies":6868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17170,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'17614","reported_seq":17750,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:29.176743+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:37:29.176743+0000","last_peered":"2024-09-10T20:37:29.176743+0000","last_clean":"2024-09-10T20:37:29.176743+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:37:29.176743+0000","last_undegraded":"2024-09-10T20:37:29.176743+0000","last_fullsized":"2024-09-10T20:37:29.176743+0000","mapping_epoch":34,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14151680,"num_objects":3455,"num_object_clones":0,"num_object_copies":6910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14610,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'17175","reported_seq":17427,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:29.110155+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:37:29.110155+0000","last_peered":"2024-09-10T20:37:29.110155+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:37:29.110155+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13500416,"num_objects":3296,"num_object_clones":0,"num_object_copies":6592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14039,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-265}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'17192","reported_seq":17478,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:29.276851+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:37:29.276851+0000","last_peered":"2024-09-10T20:37:29.276851+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:37:29.276851+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13991936,"num_objects":3416,"num_object_clones":0,"num_object_copies":6832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13787,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-70}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'17413","reported_seq":17547,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:29.290981+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:37:29.290981+0000","last_peered":"2024-09-10T20:37:29.290981+0000","last_clean":"2024-09-10T20:37:29.290981+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:37:29.290981+0000","last_undegraded":"2024-09-10T20:37:29.290981+0000","last_fullsized":"2024-09-10T20:37:29.290981+0000","mapping_epoch":34,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13611008,"num_objects":3323,"num_object_clones":0,"num_object_copies":6646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14484,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'17071","reported_seq":17271,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:29.373886+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:37:29.373886+0000","last_peered":"2024-09-10T20:37:29.373886+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:37:29.373886+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13795328,"num_objects":3368,"num_object_clones":0,"num_object_copies":6736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3368,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15064,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-129}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318026+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318026+0000","last_peered":"2024-09-10T20:36:30.318026+0000","last_clean":"2024-09-10T20:36:30.318026+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318026+0000","last_undegraded":"2024-09-10T20:36:30.318026+0000","last_fullsized":"2024-09-10T20:36:30.318026+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'17384","reported_seq":17601,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:29.210183+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:37:29.210183+0000","last_peered":"2024-09-10T20:37:29.210183+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:37:29.210183+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13706182,"num_objects":3348,"num_object_clones":0,"num_object_copies":6696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17382,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-218}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'17201","reported_seq":17393,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:29.338115+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:37:29.338115+0000","last_peered":"2024-09-10T20:37:29.338115+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:37:29.338115+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13778944,"num_objects":3364,"num_object_clones":0,"num_object_copies":6728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15029,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-242}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'17346","reported_seq":24717,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:29.417445+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:37:29.417445+0000","last_peered":"2024-09-10T20:37:29.417445+0000","last_clean":"2024-09-10T20:37:29.417445+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:37:29.417445+0000","last_undegraded":"2024-09-10T20:37:29.417445+0000","last_fullsized":"2024-09-10T20:37:29.417445+0000","mapping_epoch":48,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13590528,"num_objects":3318,"num_object_clones":0,"num_object_copies":6636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15132,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'17420","reported_seq":17677,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:29.284987+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:37:29.284987+0000","last_peered":"2024-09-10T20:37:29.284987+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:37:29.284987+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13967360,"num_objects":3410,"num_object_clones":0,"num_object_copies":6820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14143,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-182}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'17258","reported_seq":20109,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:29.296843+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:37:29.296843+0000","last_peered":"2024-09-10T20:37:29.296843+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:37:29.296843+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13623296,"num_objects":3326,"num_object_clones":0,"num_object_copies":6652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13952,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1351,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1142}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319607+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319607+0000","last_peered":"2024-09-10T20:36:30.319607+0000","last_clean":"2024-09-10T20:36:30.319607+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319607+0000","last_undegraded":"2024-09-10T20:36:30.319607+0000","last_fullsized":"2024-09-10T20:36:30.319607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'17606","reported_seq":17864,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:29.306887+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:37:29.306887+0000","last_peered":"2024-09-10T20:37:29.306887+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:37:29.306887+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13844480,"num_objects":3380,"num_object_clones":0,"num_object_copies":6760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14433,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-275}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'17063","reported_seq":24190,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:26.145718+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:37:26.145718+0000","last_peered":"2024-09-10T20:37:26.145718+0000","last_clean":"2024-09-10T20:37:26.145718+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:37:26.145718+0000","last_undegraded":"2024-09-10T20:37:26.145718+0000","last_fullsized":"2024-09-10T20:37:26.145718+0000","mapping_epoch":46,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14143971,"num_objects":3454,"num_object_clones":0,"num_object_copies":6908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13670,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":221410753,"num_objects":54059,"num_object_clones":0,"num_object_copies":108118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30701,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54059,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":241358,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12448,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":622276608,"data_stored":622210656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160762,"ondisk_log_size":160762,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182507,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":631336,"kb_used_data":220680,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93740504,"statfs":{"total":96636764160,"available":95990276096,"internally_reserved":0,"allocated":225976320,"data_stored":225714372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":420474538},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,0,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986279,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":570832,"kb_used_data":223996,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93801008,"statfs":{"total":96636764160,"available":96052232192,"internally_reserved":0,"allocated":229371904,"data_stored":229109961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355135146},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,0,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659225,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":585336,"kb_used_data":166304,"kb_used_omap":4,"kb_used_meta":419003,"kb_avail":93786504,"statfs":{"total":96636764160,"available":96037380096,"internally_reserved":0,"allocated":170295296,"data_stored":169982307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":429059758},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,8,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":224636928,"data_stored":224633315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":169476096,"data_stored":169421442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":228163584,"data_stored":228155899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:37:32.291 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:37:32.847 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:32.851 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:37:32.877 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2024-09-10T20:37:31.009758+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":221410772,"num_objects":54061,"num_object_clones":0,"num_object_copies":108122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30701,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54061,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":241360,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12450,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160765,"ondisk_log_size":160765,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1787504,"kb_used_data":610980,"kb_used_omap":12,"kb_used_meta":1176435,"kb_avail":281328016,"statfs":{"total":289910292480,"available":288079888384,"internally_reserved":0,"allocated":625643520,"data_stored":624806640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1204669442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,8,1,10,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28618752,"num_objects":-6987,"num_object_clones":0,"num_object_copies":-13974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-321,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6987,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6987,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":321,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004114"},"pg_stats":[{"pgid":"2.f","version":"49'17305","reported_seq":17527,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:29.286965+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:37:29.286965+0000","last_peered":"2024-09-10T20:37:29.286965+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:37:29.286965+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13733888,"num_objects":3353,"num_object_clones":0,"num_object_copies":6706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17305,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-188}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'17568","reported_seq":17688,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:29.365269+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:37:29.365269+0000","last_peered":"2024-09-10T20:37:29.365269+0000","last_clean":"2024-09-10T20:37:29.365269+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:37:29.365269+0000","last_undegraded":"2024-09-10T20:37:29.365269+0000","last_fullsized":"2024-09-10T20:37:29.365269+0000","mapping_epoch":35,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13819904,"num_objects":3374,"num_object_clones":0,"num_object_copies":6748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3374,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15754,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'17276","reported_seq":24632,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:26.101222+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:37:26.101222+0000","last_peered":"2024-09-10T20:37:26.101222+0000","last_clean":"2024-09-10T20:37:26.101222+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:37:26.101222+0000","last_undegraded":"2024-09-10T20:37:26.101222+0000","last_fullsized":"2024-09-10T20:37:26.101222+0000","mapping_epoch":44,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14086168,"num_objects":3440,"num_object_clones":0,"num_object_copies":6880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3440,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15404,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'17170","reported_seq":17260,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:29.349030+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:37:29.349030+0000","last_peered":"2024-09-10T20:37:29.349030+0000","last_clean":"2024-09-10T20:37:29.349030+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:37:29.349030+0000","last_undegraded":"2024-09-10T20:37:29.349030+0000","last_fullsized":"2024-09-10T20:37:29.349030+0000","mapping_epoch":35,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14065664,"num_objects":3434,"num_object_clones":0,"num_object_copies":6868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17170,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'17614","reported_seq":17750,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:29.176743+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:37:29.176743+0000","last_peered":"2024-09-10T20:37:29.176743+0000","last_clean":"2024-09-10T20:37:29.176743+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:37:29.176743+0000","last_undegraded":"2024-09-10T20:37:29.176743+0000","last_fullsized":"2024-09-10T20:37:29.176743+0000","mapping_epoch":34,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14151680,"num_objects":3455,"num_object_clones":0,"num_object_copies":6910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14610,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'17175","reported_seq":17427,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:29.110155+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:37:29.110155+0000","last_peered":"2024-09-10T20:37:29.110155+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:37:29.110155+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13500416,"num_objects":3296,"num_object_clones":0,"num_object_copies":6592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14039,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-265}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'17192","reported_seq":17478,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:29.276851+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:37:29.276851+0000","last_peered":"2024-09-10T20:37:29.276851+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:37:29.276851+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13991936,"num_objects":3416,"num_object_clones":0,"num_object_copies":6832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13787,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-70}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'17413","reported_seq":17547,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:29.290981+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:37:29.290981+0000","last_peered":"2024-09-10T20:37:29.290981+0000","last_clean":"2024-09-10T20:37:29.290981+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:37:29.290981+0000","last_undegraded":"2024-09-10T20:37:29.290981+0000","last_fullsized":"2024-09-10T20:37:29.290981+0000","mapping_epoch":34,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13611008,"num_objects":3323,"num_object_clones":0,"num_object_copies":6646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14484,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'17071","reported_seq":17271,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:29.373886+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:37:29.373886+0000","last_peered":"2024-09-10T20:37:29.373886+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:37:29.373886+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13795328,"num_objects":3368,"num_object_clones":0,"num_object_copies":6736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3368,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15064,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-129}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318026+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318026+0000","last_peered":"2024-09-10T20:36:30.318026+0000","last_clean":"2024-09-10T20:36:30.318026+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318026+0000","last_undegraded":"2024-09-10T20:36:30.318026+0000","last_fullsized":"2024-09-10T20:36:30.318026+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'17384","reported_seq":17601,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:29.210183+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:37:29.210183+0000","last_peered":"2024-09-10T20:37:29.210183+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:37:29.210183+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13706182,"num_objects":3348,"num_object_clones":0,"num_object_copies":6696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17382,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-218}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'17201","reported_seq":17393,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:29.338115+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:37:29.338115+0000","last_peered":"2024-09-10T20:37:29.338115+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:37:29.338115+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13778944,"num_objects":3364,"num_object_clones":0,"num_object_copies":6728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15029,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-242}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'17346","reported_seq":24717,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:29.417445+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:37:29.417445+0000","last_peered":"2024-09-10T20:37:29.417445+0000","last_clean":"2024-09-10T20:37:29.417445+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:37:29.417445+0000","last_undegraded":"2024-09-10T20:37:29.417445+0000","last_fullsized":"2024-09-10T20:37:29.417445+0000","mapping_epoch":48,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13590528,"num_objects":3318,"num_object_clones":0,"num_object_copies":6636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15132,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'17420","reported_seq":17677,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:29.284987+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:37:29.284987+0000","last_peered":"2024-09-10T20:37:29.284987+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:37:29.284987+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13967360,"num_objects":3410,"num_object_clones":0,"num_object_copies":6820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14143,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-182}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'17258","reported_seq":20109,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:29.296843+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:37:29.296843+0000","last_peered":"2024-09-10T20:37:29.296843+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:37:29.296843+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13623296,"num_objects":3326,"num_object_clones":0,"num_object_copies":6652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13952,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1351,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1142}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319607+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319607+0000","last_peered":"2024-09-10T20:36:30.319607+0000","last_clean":"2024-09-10T20:36:30.319607+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319607+0000","last_undegraded":"2024-09-10T20:36:30.319607+0000","last_fullsized":"2024-09-10T20:36:30.319607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'17606","reported_seq":17864,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:29.306887+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:37:29.306887+0000","last_peered":"2024-09-10T20:37:29.306887+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:37:29.306887+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13844480,"num_objects":3380,"num_object_clones":0,"num_object_copies":6760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14433,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-275}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'17063","reported_seq":24190,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:26.145718+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:37:26.145718+0000","last_peered":"2024-09-10T20:37:26.145718+0000","last_clean":"2024-09-10T20:37:26.145718+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:37:26.145718+0000","last_undegraded":"2024-09-10T20:37:26.145718+0000","last_fullsized":"2024-09-10T20:37:26.145718+0000","mapping_epoch":46,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14143971,"num_objects":3454,"num_object_clones":0,"num_object_copies":6908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13670,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":221410753,"num_objects":54059,"num_object_clones":0,"num_object_copies":108118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30701,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54059,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":241358,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12448,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":622276608,"data_stored":622210656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160762,"ondisk_log_size":160762,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182507,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":631336,"kb_used_data":220680,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93740504,"statfs":{"total":96636764160,"available":95990276096,"internally_reserved":0,"allocated":225976320,"data_stored":225714372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":420474538},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,0,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986279,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":570832,"kb_used_data":223996,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93801008,"statfs":{"total":96636764160,"available":96052232192,"internally_reserved":0,"allocated":229371904,"data_stored":229109961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355135146},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,0,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659225,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":585336,"kb_used_data":166304,"kb_used_omap":4,"kb_used_meta":419003,"kb_avail":93786504,"statfs":{"total":96636764160,"available":96037380096,"internally_reserved":0,"allocated":170295296,"data_stored":169982307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":429059758},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,8,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":224636928,"data_stored":224633315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":169476096,"data_stored":169421442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":228163584,"data_stored":228155899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:37:32.878 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:37:32.878 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'17305", 'reported_seq': 17527, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:29.286965+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:37:29.286965+0000', 'last_peered': '2024-09-10T20:37:29.286965+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:37:29.286965+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "43'7300", 'ondisk_log_start': "43'7300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13733888, 'num_objects': 3353, 'num_object_clones': 0, 'num_object_copies': 6706, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3353, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17305, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -188}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:32.878 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:37:32.878 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'17175", 'reported_seq': 17427, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:29.110155+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:37:29.110155+0000', 'last_peered': '2024-09-10T20:37:29.110155+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:37:29.110155+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "43'7100", 'ondisk_log_start': "43'7100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13500416, 'num_objects': 3296, 'num_object_clones': 0, 'num_object_copies': 6592, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3296, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14039, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -265}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:32.879 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:37:32.879 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'17192", 'reported_seq': 17478, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:29.276851+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:37:29.276851+0000', 'last_peered': '2024-09-10T20:37:29.276851+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:37:29.276851+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "43'7100", 'ondisk_log_start': "43'7100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13991936, 'num_objects': 3416, 'num_object_clones': 0, 'num_object_copies': 6832, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3416, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13787, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -70}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:32.879 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:37:32.879 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'17071", 'reported_seq': 17271, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:29.373886+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:37:29.373886+0000', 'last_peered': '2024-09-10T20:37:29.373886+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:37:29.373886+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "43'7000", 'ondisk_log_start': "43'7000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13795328, 'num_objects': 3368, 'num_object_clones': 0, 'num_object_copies': 6736, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3368, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15064, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -129}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:37:32.879 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:37:32.879 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'17384", 'reported_seq': 17601, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:29.210183+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:37:29.210183+0000', 'last_peered': '2024-09-10T20:37:29.210183+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:37:29.210183+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "43'7300", 'ondisk_log_start': "43'7300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13706182, 'num_objects': 3348, 'num_object_clones': 0, 'num_object_copies': 6696, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3348, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17382, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -218}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:32.879 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:37:32.879 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'17201", 'reported_seq': 17393, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:29.338115+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:37:29.338115+0000', 'last_peered': '2024-09-10T20:37:29.338115+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:37:29.338115+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "43'7200", 'ondisk_log_start': "43'7200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13778944, 'num_objects': 3364, 'num_object_clones': 0, 'num_object_copies': 6728, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3364, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15029, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -242}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:37:32.879 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:37:32.879 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'17420", 'reported_seq': 17677, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:29.284987+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:37:29.284987+0000', 'last_peered': '2024-09-10T20:37:29.284987+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:37:29.284987+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "43'7400", 'ondisk_log_start': "43'7400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13967360, 'num_objects': 3410, 'num_object_clones': 0, 'num_object_copies': 6820, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3410, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14143, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -182}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:32.880 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:37:32.880 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'17258", 'reported_seq': 20109, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:29.296843+0000', 'last_change': '2024-09-10T20:36:28.778006+0000', 'last_active': '2024-09-10T20:37:29.296843+0000', 'last_peered': '2024-09-10T20:37:29.296843+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:37:29.296843+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "43'7200", 'ondisk_log_start': "43'7200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13623296, 'num_objects': 3326, 'num_object_clones': 0, 'num_object_copies': 6652, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2184, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3326, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13952, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1351, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1142}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:32.880 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:37:32.880 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'17606", 'reported_seq': 17864, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:29.306887+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:37:29.306887+0000', 'last_peered': '2024-09-10T20:37:29.306887+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:37:29.306887+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "43'7600", 'ondisk_log_start': "43'7600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13844480, 'num_objects': 3380, 'num_object_clones': 0, 'num_object_copies': 6760, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3380, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14433, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -275}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:32.880 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:37:33.601 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:33.601 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 30701/107494 objects degraded (28.561%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":565,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":53747,"data_bytes":220124628,"bytes_used":1832939520,"bytes_avail":288077352960,"bytes_total":289910292480,"degraded_objects":30701,"degraded_total":107494,"degraded_ratio":0.28560663851005635,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":213,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":429},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:37:33.627 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:37:33.627 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:37:34.190 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:34.192 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:37:34.214 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":325,"stamp":"2024-09-10T20:37:33.010473+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":220124628,"num_objects":53747,"num_object_clones":0,"num_object_copies":107494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30701,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53747,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":241674,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12450,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160679,"ondisk_log_size":160679,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1789980,"kb_used_data":603736,"kb_used_omap":12,"kb_used_meta":1186163,"kb_avail":281325540,"statfs":{"total":289910292480,"available":288077352960,"internally_reserved":0,"allocated":618225664,"data_stored":617388784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1214630914},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,6,4,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21094400,"num_objects":-5150,"num_object_clones":0,"num_object_copies":-10300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-213,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5150,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004165"},"pg_stats":[{"pgid":"2.f","version":"49'17305","reported_seq":17527,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:29.286965+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:37:29.286965+0000","last_peered":"2024-09-10T20:37:29.286965+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:37:29.286965+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13733888,"num_objects":3353,"num_object_clones":0,"num_object_copies":6706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17305,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-188}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'17568","reported_seq":17688,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:29.365269+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:37:29.365269+0000","last_peered":"2024-09-10T20:37:29.365269+0000","last_clean":"2024-09-10T20:37:29.365269+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:37:29.365269+0000","last_undegraded":"2024-09-10T20:37:29.365269+0000","last_fullsized":"2024-09-10T20:37:29.365269+0000","mapping_epoch":35,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13819904,"num_objects":3374,"num_object_clones":0,"num_object_copies":6748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3374,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15754,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'17432","reported_seq":24790,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:31.133926+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:37:31.133926+0000","last_peered":"2024-09-10T20:37:31.133926+0000","last_clean":"2024-09-10T20:37:31.133926+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:37:31.133926+0000","last_undegraded":"2024-09-10T20:37:31.133926+0000","last_fullsized":"2024-09-10T20:37:31.133926+0000","mapping_epoch":44,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13447192,"num_objects":3284,"num_object_clones":0,"num_object_copies":6568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3284,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15560,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'17170","reported_seq":17260,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:29.349030+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:37:29.349030+0000","last_peered":"2024-09-10T20:37:29.349030+0000","last_clean":"2024-09-10T20:37:29.349030+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:37:29.349030+0000","last_undegraded":"2024-09-10T20:37:29.349030+0000","last_fullsized":"2024-09-10T20:37:29.349030+0000","mapping_epoch":35,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14065664,"num_objects":3434,"num_object_clones":0,"num_object_copies":6868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17170,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'17614","reported_seq":17750,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:29.176743+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:37:29.176743+0000","last_peered":"2024-09-10T20:37:29.176743+0000","last_clean":"2024-09-10T20:37:29.176743+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:37:29.176743+0000","last_undegraded":"2024-09-10T20:37:29.176743+0000","last_fullsized":"2024-09-10T20:37:29.176743+0000","mapping_epoch":34,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14151680,"num_objects":3455,"num_object_clones":0,"num_object_copies":6910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14610,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'17175","reported_seq":17427,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:29.110155+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:37:29.110155+0000","last_peered":"2024-09-10T20:37:29.110155+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:37:29.110155+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13500416,"num_objects":3296,"num_object_clones":0,"num_object_copies":6592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14039,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-265}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'17192","reported_seq":17478,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:29.276851+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:37:29.276851+0000","last_peered":"2024-09-10T20:37:29.276851+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:37:29.276851+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13991936,"num_objects":3416,"num_object_clones":0,"num_object_copies":6832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13787,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-70}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'17413","reported_seq":17547,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:29.290981+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:37:29.290981+0000","last_peered":"2024-09-10T20:37:29.290981+0000","last_clean":"2024-09-10T20:37:29.290981+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:37:29.290981+0000","last_undegraded":"2024-09-10T20:37:29.290981+0000","last_fullsized":"2024-09-10T20:37:29.290981+0000","mapping_epoch":34,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13611008,"num_objects":3323,"num_object_clones":0,"num_object_copies":6646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14484,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'17071","reported_seq":17271,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:29.373886+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:37:29.373886+0000","last_peered":"2024-09-10T20:37:29.373886+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:37:29.373886+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13795328,"num_objects":3368,"num_object_clones":0,"num_object_copies":6736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3368,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15064,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-129}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318026+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318026+0000","last_peered":"2024-09-10T20:36:30.318026+0000","last_clean":"2024-09-10T20:36:30.318026+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318026+0000","last_undegraded":"2024-09-10T20:36:30.318026+0000","last_fullsized":"2024-09-10T20:36:30.318026+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'17384","reported_seq":17601,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:29.210183+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:37:29.210183+0000","last_peered":"2024-09-10T20:37:29.210183+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:37:29.210183+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13706182,"num_objects":3348,"num_object_clones":0,"num_object_copies":6696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17382,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-218}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'17201","reported_seq":17393,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:29.338115+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:37:29.338115+0000","last_peered":"2024-09-10T20:37:29.338115+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:37:29.338115+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13778944,"num_objects":3364,"num_object_clones":0,"num_object_copies":6728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15029,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-242}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'17346","reported_seq":24717,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:29.417445+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:37:29.417445+0000","last_peered":"2024-09-10T20:37:29.417445+0000","last_clean":"2024-09-10T20:37:29.417445+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:37:29.417445+0000","last_undegraded":"2024-09-10T20:37:29.417445+0000","last_fullsized":"2024-09-10T20:37:29.417445+0000","mapping_epoch":48,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13590528,"num_objects":3318,"num_object_clones":0,"num_object_copies":6636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15132,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'17420","reported_seq":17677,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:29.284987+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:37:29.284987+0000","last_peered":"2024-09-10T20:37:29.284987+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:37:29.284987+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13967360,"num_objects":3410,"num_object_clones":0,"num_object_copies":6820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14143,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-182}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'17258","reported_seq":20109,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:29.296843+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:37:29.296843+0000","last_peered":"2024-09-10T20:37:29.296843+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:37:29.296843+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13623296,"num_objects":3326,"num_object_clones":0,"num_object_copies":6652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13952,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1351,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1142}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319607+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319607+0000","last_peered":"2024-09-10T20:36:30.319607+0000","last_clean":"2024-09-10T20:36:30.319607+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319607+0000","last_undegraded":"2024-09-10T20:36:30.319607+0000","last_fullsized":"2024-09-10T20:36:30.319607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'17606","reported_seq":17864,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:29.306887+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:37:29.306887+0000","last_peered":"2024-09-10T20:37:29.306887+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:37:29.306887+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13844480,"num_objects":3380,"num_object_clones":0,"num_object_copies":6760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14433,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-275}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'17221","reported_seq":24348,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:31.116162+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:37:31.116162+0000","last_peered":"2024-09-10T20:37:31.116162+0000","last_clean":"2024-09-10T20:37:31.116162+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:37:31.116162+0000","last_undegraded":"2024-09-10T20:37:31.116162+0000","last_fullsized":"2024-09-10T20:37:31.116162+0000","mapping_epoch":46,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13496803,"num_objects":3296,"num_object_clones":0,"num_object_copies":6592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13828,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":220124609,"num_objects":53745,"num_object_clones":0,"num_object_copies":107490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30701,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53745,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":241672,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12448,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":614817792,"data_stored":614751840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160676,"ondisk_log_size":160676,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182507,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":631336,"kb_used_data":220680,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93740504,"statfs":{"total":96636764160,"available":95990276096,"internally_reserved":0,"allocated":225976320,"data_stored":225714372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":420474538},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,0,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986279,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":570832,"kb_used_data":223996,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93801008,"statfs":{"total":96636764160,"available":96052232192,"internally_reserved":0,"allocated":229371904,"data_stored":229109961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355135146},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,0,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659226,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":587812,"kb_used_data":159060,"kb_used_omap":4,"kb_used_meta":428731,"kb_avail":93784028,"statfs":{"total":96636764160,"available":96034844672,"internally_reserved":0,"allocated":162877440,"data_stored":162564451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":439021230},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,4,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":224636928,"data_stored":224633315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":162017280,"data_stored":161962626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":228163584,"data_stored":228155899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:37:37.216 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:37:37.775 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:37.778 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:37:37.797 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-09-10T20:37:37.011617+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":210154964,"num_objects":51313,"num_object_clones":0,"num_object_copies":102626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30587,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51313,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":244108,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12564,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160713,"ondisk_log_size":160713,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1770432,"kb_used_data":584196,"kb_used_omap":12,"kb_used_meta":1186163,"kb_avail":281345088,"statfs":{"total":289910292480,"available":288097370112,"internally_reserved":0,"allocated":598216704,"data_stored":597379824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1214630914},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,5,5,8,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21315584,"num_objects":-5204,"num_object_clones":0,"num_object_copies":-10408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-219,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5204,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":219,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003807"},"pg_stats":[{"pgid":"2.f","version":"49'17466","reported_seq":17688,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:34.326024+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:37:34.326024+0000","last_peered":"2024-09-10T20:37:34.326024+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:37:34.326024+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13074432,"num_objects":3192,"num_object_clones":0,"num_object_copies":6384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17466,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-349}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'17711","reported_seq":17831,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:34.364987+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:37:34.364987+0000","last_peered":"2024-09-10T20:37:34.364987+0000","last_clean":"2024-09-10T20:37:34.364987+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:37:34.364987+0000","last_undegraded":"2024-09-10T20:37:34.364987+0000","last_fullsized":"2024-09-10T20:37:34.364987+0000","mapping_epoch":35,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13234176,"num_objects":3231,"num_object_clones":0,"num_object_copies":6462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15897,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'17579","reported_seq":24937,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:36.165196+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:37:36.165196+0000","last_peered":"2024-09-10T20:37:36.165196+0000","last_clean":"2024-09-10T20:37:36.165196+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:37:36.165196+0000","last_undegraded":"2024-09-10T20:37:36.165196+0000","last_fullsized":"2024-09-10T20:37:36.165196+0000","mapping_epoch":44,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12845080,"num_objects":3137,"num_object_clones":0,"num_object_copies":6274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3137,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15707,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'17307","reported_seq":17397,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:34.387958+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:37:34.387958+0000","last_peered":"2024-09-10T20:37:34.387958+0000","last_clean":"2024-09-10T20:37:34.387958+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:37:34.387958+0000","last_undegraded":"2024-09-10T20:37:34.387958+0000","last_fullsized":"2024-09-10T20:37:34.387958+0000","mapping_epoch":35,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13504512,"num_objects":3297,"num_object_clones":0,"num_object_copies":6594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17307,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'17767","reported_seq":17903,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:34.327919+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:37:34.327919+0000","last_peered":"2024-09-10T20:37:34.327919+0000","last_clean":"2024-09-10T20:37:34.327919+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:37:34.327919+0000","last_undegraded":"2024-09-10T20:37:34.327919+0000","last_fullsized":"2024-09-10T20:37:34.327919+0000","mapping_epoch":34,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13524992,"num_objects":3302,"num_object_clones":0,"num_object_copies":6604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14763,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'17312","reported_seq":17564,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:34.248592+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:37:34.248592+0000","last_peered":"2024-09-10T20:37:34.248592+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:37:34.248592+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12939264,"num_objects":3159,"num_object_clones":0,"num_object_copies":6318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14176,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-402}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'17339","reported_seq":17624,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:34.330241+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:37:34.330241+0000","last_peered":"2024-09-10T20:37:34.330241+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:37:34.330241+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13389824,"num_objects":3269,"num_object_clones":0,"num_object_copies":6538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13934,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-217}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'17570","reported_seq":17704,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:34.216561+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:37:34.216561+0000","last_peered":"2024-09-10T20:37:34.216561+0000","last_clean":"2024-09-10T20:37:34.216561+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:37:34.216561+0000","last_undegraded":"2024-09-10T20:37:34.216561+0000","last_fullsized":"2024-09-10T20:37:34.216561+0000","mapping_epoch":34,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12967936,"num_objects":3166,"num_object_clones":0,"num_object_copies":6332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14641,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'17203","reported_seq":17403,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:34.271718+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:37:34.271718+0000","last_peered":"2024-09-10T20:37:34.271718+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:37:34.271718+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13254656,"num_objects":3236,"num_object_clones":0,"num_object_copies":6472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15196,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-261}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318059+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318059+0000","last_peered":"2024-09-10T20:36:30.318059+0000","last_clean":"2024-09-10T20:36:30.318059+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318059+0000","last_undegraded":"2024-09-10T20:36:30.318059+0000","last_fullsized":"2024-09-10T20:36:30.318059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'17541","reported_seq":17756,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:34.282332+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:37:34.282332+0000","last_peered":"2024-09-10T20:37:34.282332+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:37:34.282332+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13063110,"num_objects":3191,"num_object_clones":0,"num_object_copies":6382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17539,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-375}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'17366","reported_seq":17558,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:34.386594+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:37:34.386594+0000","last_peered":"2024-09-10T20:37:34.386594+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:37:34.386594+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13103104,"num_objects":3199,"num_object_clones":0,"num_object_copies":6398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15194,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-407}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'17502","reported_seq":24873,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:34.407144+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:37:34.407144+0000","last_peered":"2024-09-10T20:37:34.407144+0000","last_clean":"2024-09-10T20:37:34.407144+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:37:34.407144+0000","last_undegraded":"2024-09-10T20:37:34.407144+0000","last_fullsized":"2024-09-10T20:37:34.407144+0000","mapping_epoch":48,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12951552,"num_objects":3162,"num_object_clones":0,"num_object_copies":6324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15288,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'17586","reported_seq":17843,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:34.323200+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:37:34.323200+0000","last_peered":"2024-09-10T20:37:34.323200+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:37:34.323200+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13287424,"num_objects":3244,"num_object_clones":0,"num_object_copies":6488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14309,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-348}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'17426","reported_seq":20504,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:34.338779+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:37:34.338779+0000","last_peered":"2024-09-10T20:37:34.338779+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:37:34.338779+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12935168,"num_objects":3158,"num_object_clones":0,"num_object_copies":6316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14120,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1465,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1088}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319644+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319644+0000","last_peered":"2024-09-10T20:36:30.319644+0000","last_clean":"2024-09-10T20:36:30.319644+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319644+0000","last_undegraded":"2024-09-10T20:36:30.319644+0000","last_fullsized":"2024-09-10T20:36:30.319644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'17764","reported_seq":18022,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:34.234941+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:37:34.234941+0000","last_peered":"2024-09-10T20:37:34.234941+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:37:34.234941+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13197312,"num_objects":3222,"num_object_clones":0,"num_object_copies":6444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14591,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'17371","reported_seq":24497,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:36.216649+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:37:36.216649+0000","last_peered":"2024-09-10T20:37:36.216649+0000","last_clean":"2024-09-10T20:37:36.216649+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:37:36.216649+0000","last_undegraded":"2024-09-10T20:37:36.216649+0000","last_fullsized":"2024-09-10T20:37:36.216649+0000","mapping_epoch":46,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12882403,"num_objects":3146,"num_object_clones":0,"num_object_copies":6292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13978,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":210154945,"num_objects":51311,"num_object_clones":0,"num_object_copies":102622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30587,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51311,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":244106,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12562,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":594870272,"data_stored":594804320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160710,"ondisk_log_size":160710,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182508,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":628032,"kb_used_data":213232,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93743808,"statfs":{"total":96636764160,"available":95993659392,"internally_reserved":0,"allocated":218349568,"data_stored":218087620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":424734378},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986280,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":557736,"kb_used_data":219212,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93814104,"statfs":{"total":96636764160,"available":96065642496,"internally_reserved":0,"allocated":224473088,"data_stored":224211145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346615466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659227,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":584664,"kb_used_data":151752,"kb_used_omap":4,"kb_used_meta":432891,"kb_avail":93787176,"statfs":{"total":96636764160,"available":96038068224,"internally_reserved":0,"allocated":155394048,"data_stored":155081059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":443281070},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,2,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":217063424,"data_stored":217059811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":154484736,"data_stored":154430082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":223322112,"data_stored":223314427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:37:37.799 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:37:38.369 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:38.372 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:37:38.393 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-09-10T20:37:37.011617+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":210154964,"num_objects":51313,"num_object_clones":0,"num_object_copies":102626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30587,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51313,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":244108,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12564,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160713,"ondisk_log_size":160713,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1770432,"kb_used_data":584196,"kb_used_omap":12,"kb_used_meta":1186163,"kb_avail":281345088,"statfs":{"total":289910292480,"available":288097370112,"internally_reserved":0,"allocated":598216704,"data_stored":597379824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1214630914},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,5,5,8,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21315584,"num_objects":-5204,"num_object_clones":0,"num_object_copies":-10408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-219,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5204,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":219,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003807"},"pg_stats":[{"pgid":"2.f","version":"49'17466","reported_seq":17688,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:34.326024+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:37:34.326024+0000","last_peered":"2024-09-10T20:37:34.326024+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:37:34.326024+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13074432,"num_objects":3192,"num_object_clones":0,"num_object_copies":6384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17466,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-349}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'17711","reported_seq":17831,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:34.364987+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:37:34.364987+0000","last_peered":"2024-09-10T20:37:34.364987+0000","last_clean":"2024-09-10T20:37:34.364987+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:37:34.364987+0000","last_undegraded":"2024-09-10T20:37:34.364987+0000","last_fullsized":"2024-09-10T20:37:34.364987+0000","mapping_epoch":35,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13234176,"num_objects":3231,"num_object_clones":0,"num_object_copies":6462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15897,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'17579","reported_seq":24937,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:36.165196+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:37:36.165196+0000","last_peered":"2024-09-10T20:37:36.165196+0000","last_clean":"2024-09-10T20:37:36.165196+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:37:36.165196+0000","last_undegraded":"2024-09-10T20:37:36.165196+0000","last_fullsized":"2024-09-10T20:37:36.165196+0000","mapping_epoch":44,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12845080,"num_objects":3137,"num_object_clones":0,"num_object_copies":6274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3137,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15707,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'17307","reported_seq":17397,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:34.387958+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:37:34.387958+0000","last_peered":"2024-09-10T20:37:34.387958+0000","last_clean":"2024-09-10T20:37:34.387958+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:37:34.387958+0000","last_undegraded":"2024-09-10T20:37:34.387958+0000","last_fullsized":"2024-09-10T20:37:34.387958+0000","mapping_epoch":35,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13504512,"num_objects":3297,"num_object_clones":0,"num_object_copies":6594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17307,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'17767","reported_seq":17903,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:34.327919+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:37:34.327919+0000","last_peered":"2024-09-10T20:37:34.327919+0000","last_clean":"2024-09-10T20:37:34.327919+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:37:34.327919+0000","last_undegraded":"2024-09-10T20:37:34.327919+0000","last_fullsized":"2024-09-10T20:37:34.327919+0000","mapping_epoch":34,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13524992,"num_objects":3302,"num_object_clones":0,"num_object_copies":6604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14763,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'17312","reported_seq":17564,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:34.248592+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:37:34.248592+0000","last_peered":"2024-09-10T20:37:34.248592+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:37:34.248592+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12939264,"num_objects":3159,"num_object_clones":0,"num_object_copies":6318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14176,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-402}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'17339","reported_seq":17624,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:34.330241+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:37:34.330241+0000","last_peered":"2024-09-10T20:37:34.330241+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:37:34.330241+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13389824,"num_objects":3269,"num_object_clones":0,"num_object_copies":6538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13934,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-217}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'17570","reported_seq":17704,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:34.216561+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:37:34.216561+0000","last_peered":"2024-09-10T20:37:34.216561+0000","last_clean":"2024-09-10T20:37:34.216561+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:37:34.216561+0000","last_undegraded":"2024-09-10T20:37:34.216561+0000","last_fullsized":"2024-09-10T20:37:34.216561+0000","mapping_epoch":34,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12967936,"num_objects":3166,"num_object_clones":0,"num_object_copies":6332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14641,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'17203","reported_seq":17403,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:34.271718+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:37:34.271718+0000","last_peered":"2024-09-10T20:37:34.271718+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:37:34.271718+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13254656,"num_objects":3236,"num_object_clones":0,"num_object_copies":6472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15196,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-261}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318059+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318059+0000","last_peered":"2024-09-10T20:36:30.318059+0000","last_clean":"2024-09-10T20:36:30.318059+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318059+0000","last_undegraded":"2024-09-10T20:36:30.318059+0000","last_fullsized":"2024-09-10T20:36:30.318059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'17541","reported_seq":17756,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:34.282332+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:37:34.282332+0000","last_peered":"2024-09-10T20:37:34.282332+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:37:34.282332+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13063110,"num_objects":3191,"num_object_clones":0,"num_object_copies":6382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17539,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-375}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'17366","reported_seq":17558,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:34.386594+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:37:34.386594+0000","last_peered":"2024-09-10T20:37:34.386594+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:37:34.386594+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13103104,"num_objects":3199,"num_object_clones":0,"num_object_copies":6398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15194,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-407}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'17502","reported_seq":24873,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:34.407144+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:37:34.407144+0000","last_peered":"2024-09-10T20:37:34.407144+0000","last_clean":"2024-09-10T20:37:34.407144+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:37:34.407144+0000","last_undegraded":"2024-09-10T20:37:34.407144+0000","last_fullsized":"2024-09-10T20:37:34.407144+0000","mapping_epoch":48,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12951552,"num_objects":3162,"num_object_clones":0,"num_object_copies":6324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15288,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'17586","reported_seq":17843,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:34.323200+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:37:34.323200+0000","last_peered":"2024-09-10T20:37:34.323200+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:37:34.323200+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13287424,"num_objects":3244,"num_object_clones":0,"num_object_copies":6488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14309,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-348}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'17426","reported_seq":20504,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:34.338779+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:37:34.338779+0000","last_peered":"2024-09-10T20:37:34.338779+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:37:34.338779+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12935168,"num_objects":3158,"num_object_clones":0,"num_object_copies":6316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14120,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1465,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1088}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319644+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319644+0000","last_peered":"2024-09-10T20:36:30.319644+0000","last_clean":"2024-09-10T20:36:30.319644+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319644+0000","last_undegraded":"2024-09-10T20:36:30.319644+0000","last_fullsized":"2024-09-10T20:36:30.319644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'17764","reported_seq":18022,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:34.234941+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:37:34.234941+0000","last_peered":"2024-09-10T20:37:34.234941+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:37:34.234941+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13197312,"num_objects":3222,"num_object_clones":0,"num_object_copies":6444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14591,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'17371","reported_seq":24497,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:36.216649+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:37:36.216649+0000","last_peered":"2024-09-10T20:37:36.216649+0000","last_clean":"2024-09-10T20:37:36.216649+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:37:36.216649+0000","last_undegraded":"2024-09-10T20:37:36.216649+0000","last_fullsized":"2024-09-10T20:37:36.216649+0000","mapping_epoch":46,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12882403,"num_objects":3146,"num_object_clones":0,"num_object_copies":6292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13978,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":210154945,"num_objects":51311,"num_object_clones":0,"num_object_copies":102622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30587,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51311,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":244106,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12562,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":594870272,"data_stored":594804320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160710,"ondisk_log_size":160710,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182508,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":628032,"kb_used_data":213232,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93743808,"statfs":{"total":96636764160,"available":95993659392,"internally_reserved":0,"allocated":218349568,"data_stored":218087620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":424734378},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986280,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":557736,"kb_used_data":219212,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93814104,"statfs":{"total":96636764160,"available":96065642496,"internally_reserved":0,"allocated":224473088,"data_stored":224211145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346615466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659227,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":584664,"kb_used_data":151752,"kb_used_omap":4,"kb_used_meta":432891,"kb_avail":93787176,"statfs":{"total":96636764160,"available":96038068224,"internally_reserved":0,"allocated":155394048,"data_stored":155081059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":443281070},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,2,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":217063424,"data_stored":217059811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":154484736,"data_stored":154430082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":223322112,"data_stored":223314427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:37:38.394 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:37:38.394 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'17466", 'reported_seq': 17688, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:34.326024+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:37:34.326024+0000', 'last_peered': '2024-09-10T20:37:34.326024+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:37:34.326024+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "43'7400", 'ondisk_log_start': "43'7400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13074432, 'num_objects': 3192, 'num_object_clones': 0, 'num_object_copies': 6384, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3192, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17466, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -349}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:38.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:37:38.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'17312", 'reported_seq': 17564, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:34.248592+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:37:34.248592+0000', 'last_peered': '2024-09-10T20:37:34.248592+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:37:34.248592+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "43'7300", 'ondisk_log_start': "43'7300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12939264, 'num_objects': 3159, 'num_object_clones': 0, 'num_object_copies': 6318, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3159, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14176, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -402}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:38.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:37:38.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'17339", 'reported_seq': 17624, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:34.330241+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:37:34.330241+0000', 'last_peered': '2024-09-10T20:37:34.330241+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:37:34.330241+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "43'7300", 'ondisk_log_start': "43'7300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13389824, 'num_objects': 3269, 'num_object_clones': 0, 'num_object_copies': 6538, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3269, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13934, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -217}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:38.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:37:38.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'17203", 'reported_seq': 17403, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:34.271718+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:37:34.271718+0000', 'last_peered': '2024-09-10T20:37:34.271718+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:37:34.271718+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "43'7200", 'ondisk_log_start': "43'7200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13254656, 'num_objects': 3236, 'num_object_clones': 0, 'num_object_copies': 6472, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3236, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15196, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -261}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:37:38.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:37:38.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'17541", 'reported_seq': 17756, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:34.282332+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:37:34.282332+0000', 'last_peered': '2024-09-10T20:37:34.282332+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:37:34.282332+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "43'7500", 'ondisk_log_start': "43'7500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13063110, 'num_objects': 3191, 'num_object_clones': 0, 'num_object_copies': 6382, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3191, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17539, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -375}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:38.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:37:38.396 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'17366", 'reported_seq': 17558, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:34.386594+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:37:34.386594+0000', 'last_peered': '2024-09-10T20:37:34.386594+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:37:34.386594+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "43'7300", 'ondisk_log_start': "43'7300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13103104, 'num_objects': 3199, 'num_object_clones': 0, 'num_object_copies': 6398, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3199, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15194, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -407}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:37:38.396 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:37:38.396 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'17586", 'reported_seq': 17843, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:34.323200+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:37:34.323200+0000', 'last_peered': '2024-09-10T20:37:34.323200+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:37:34.323200+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "43'7500", 'ondisk_log_start': "43'7500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13287424, 'num_objects': 3244, 'num_object_clones': 0, 'num_object_copies': 6488, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3244, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14309, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -348}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:38.396 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:37:38.396 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'17426", 'reported_seq': 20504, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:34.338779+0000', 'last_change': '2024-09-10T20:36:28.778006+0000', 'last_active': '2024-09-10T20:37:34.338779+0000', 'last_peered': '2024-09-10T20:37:34.338779+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:37:34.338779+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "43'7400", 'ondisk_log_start': "43'7400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12935168, 'num_objects': 3158, 'num_object_clones': 0, 'num_object_copies': 6316, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3158, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14120, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1465, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1088}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:38.396 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:37:38.396 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'17764", 'reported_seq': 18022, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:34.234941+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:37:34.234941+0000', 'last_peered': '2024-09-10T20:37:34.234941+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:37:34.234941+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "43'7700", 'ondisk_log_start': "43'7700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13197312, 'num_objects': 3222, 'num_object_clones': 0, 'num_object_copies': 6444, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3222, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14591, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -433}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:38.396 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:37:39.098 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:39.098 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 30587/102626 objects degraded (29.804%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":570,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":51313,"data_bytes":210154964,"bytes_used":1812922368,"bytes_avail":288097370112,"bytes_total":289910292480,"degraded_objects":30587,"degraded_total":102626,"degraded_ratio":0.29804338081967535,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":219,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":433},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:37:39.124 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:37:39.124 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:37:39.703 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:39.704 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:37:39.727 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2024-09-10T20:37:39.012285+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":210154964,"num_objects":51313,"num_object_clones":0,"num_object_copies":102626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30587,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51313,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":244108,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12564,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160713,"ondisk_log_size":160713,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1770432,"kb_used_data":584196,"kb_used_omap":12,"kb_used_meta":1186163,"kb_avail":281345088,"statfs":{"total":289910292480,"available":288097370112,"internally_reserved":0,"allocated":598216704,"data_stored":597379824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1214630914},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,5,5,8,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19918848,"num_objects":-4863,"num_object_clones":0,"num_object_copies":-9726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-219,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4863,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":219,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003844"},"pg_stats":[{"pgid":"2.f","version":"49'17466","reported_seq":17688,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:34.326024+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:37:34.326024+0000","last_peered":"2024-09-10T20:37:34.326024+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:37:34.326024+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13074432,"num_objects":3192,"num_object_clones":0,"num_object_copies":6384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17466,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-349}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'17711","reported_seq":17831,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:34.364987+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:37:34.364987+0000","last_peered":"2024-09-10T20:37:34.364987+0000","last_clean":"2024-09-10T20:37:34.364987+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:37:34.364987+0000","last_undegraded":"2024-09-10T20:37:34.364987+0000","last_fullsized":"2024-09-10T20:37:34.364987+0000","mapping_epoch":35,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13234176,"num_objects":3231,"num_object_clones":0,"num_object_copies":6462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15897,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'17579","reported_seq":24937,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:36.165196+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:37:36.165196+0000","last_peered":"2024-09-10T20:37:36.165196+0000","last_clean":"2024-09-10T20:37:36.165196+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:37:36.165196+0000","last_undegraded":"2024-09-10T20:37:36.165196+0000","last_fullsized":"2024-09-10T20:37:36.165196+0000","mapping_epoch":44,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12845080,"num_objects":3137,"num_object_clones":0,"num_object_copies":6274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3137,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15707,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'17307","reported_seq":17397,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:34.387958+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:37:34.387958+0000","last_peered":"2024-09-10T20:37:34.387958+0000","last_clean":"2024-09-10T20:37:34.387958+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:37:34.387958+0000","last_undegraded":"2024-09-10T20:37:34.387958+0000","last_fullsized":"2024-09-10T20:37:34.387958+0000","mapping_epoch":35,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13504512,"num_objects":3297,"num_object_clones":0,"num_object_copies":6594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17307,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'17767","reported_seq":17903,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:34.327919+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:37:34.327919+0000","last_peered":"2024-09-10T20:37:34.327919+0000","last_clean":"2024-09-10T20:37:34.327919+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:37:34.327919+0000","last_undegraded":"2024-09-10T20:37:34.327919+0000","last_fullsized":"2024-09-10T20:37:34.327919+0000","mapping_epoch":34,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13524992,"num_objects":3302,"num_object_clones":0,"num_object_copies":6604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14763,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'17312","reported_seq":17564,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:34.248592+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:37:34.248592+0000","last_peered":"2024-09-10T20:37:34.248592+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:37:34.248592+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12939264,"num_objects":3159,"num_object_clones":0,"num_object_copies":6318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14176,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-402}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'17339","reported_seq":17624,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:34.330241+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:37:34.330241+0000","last_peered":"2024-09-10T20:37:34.330241+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:37:34.330241+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13389824,"num_objects":3269,"num_object_clones":0,"num_object_copies":6538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13934,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-217}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'17570","reported_seq":17704,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:34.216561+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:37:34.216561+0000","last_peered":"2024-09-10T20:37:34.216561+0000","last_clean":"2024-09-10T20:37:34.216561+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:37:34.216561+0000","last_undegraded":"2024-09-10T20:37:34.216561+0000","last_fullsized":"2024-09-10T20:37:34.216561+0000","mapping_epoch":34,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12967936,"num_objects":3166,"num_object_clones":0,"num_object_copies":6332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14641,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'17203","reported_seq":17403,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:34.271718+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:37:34.271718+0000","last_peered":"2024-09-10T20:37:34.271718+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:37:34.271718+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13254656,"num_objects":3236,"num_object_clones":0,"num_object_copies":6472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15196,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-261}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318059+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318059+0000","last_peered":"2024-09-10T20:36:30.318059+0000","last_clean":"2024-09-10T20:36:30.318059+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318059+0000","last_undegraded":"2024-09-10T20:36:30.318059+0000","last_fullsized":"2024-09-10T20:36:30.318059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'17541","reported_seq":17756,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:34.282332+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:37:34.282332+0000","last_peered":"2024-09-10T20:37:34.282332+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:37:34.282332+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13063110,"num_objects":3191,"num_object_clones":0,"num_object_copies":6382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17539,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-375}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'17366","reported_seq":17558,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:34.386594+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:37:34.386594+0000","last_peered":"2024-09-10T20:37:34.386594+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:37:34.386594+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13103104,"num_objects":3199,"num_object_clones":0,"num_object_copies":6398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15194,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-407}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'17502","reported_seq":24873,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:34.407144+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:37:34.407144+0000","last_peered":"2024-09-10T20:37:34.407144+0000","last_clean":"2024-09-10T20:37:34.407144+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:37:34.407144+0000","last_undegraded":"2024-09-10T20:37:34.407144+0000","last_fullsized":"2024-09-10T20:37:34.407144+0000","mapping_epoch":48,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12951552,"num_objects":3162,"num_object_clones":0,"num_object_copies":6324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15288,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'17586","reported_seq":17843,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:34.323200+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:37:34.323200+0000","last_peered":"2024-09-10T20:37:34.323200+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:37:34.323200+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13287424,"num_objects":3244,"num_object_clones":0,"num_object_copies":6488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14309,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-348}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'17426","reported_seq":20504,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:34.338779+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:37:34.338779+0000","last_peered":"2024-09-10T20:37:34.338779+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:37:34.338779+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12935168,"num_objects":3158,"num_object_clones":0,"num_object_copies":6316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14120,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1465,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1088}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319644+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319644+0000","last_peered":"2024-09-10T20:36:30.319644+0000","last_clean":"2024-09-10T20:36:30.319644+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319644+0000","last_undegraded":"2024-09-10T20:36:30.319644+0000","last_fullsized":"2024-09-10T20:36:30.319644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'17764","reported_seq":18022,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:34.234941+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:37:34.234941+0000","last_peered":"2024-09-10T20:37:34.234941+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:37:34.234941+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13197312,"num_objects":3222,"num_object_clones":0,"num_object_copies":6444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14591,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'17371","reported_seq":24497,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:36.216649+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:37:36.216649+0000","last_peered":"2024-09-10T20:37:36.216649+0000","last_clean":"2024-09-10T20:37:36.216649+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:37:36.216649+0000","last_undegraded":"2024-09-10T20:37:36.216649+0000","last_fullsized":"2024-09-10T20:37:36.216649+0000","mapping_epoch":46,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12882403,"num_objects":3146,"num_object_clones":0,"num_object_copies":6292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13978,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":210154945,"num_objects":51311,"num_object_clones":0,"num_object_copies":102622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30587,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51311,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":244106,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12562,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":594870272,"data_stored":594804320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160710,"ondisk_log_size":160710,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182508,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":628032,"kb_used_data":213232,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93743808,"statfs":{"total":96636764160,"available":95993659392,"internally_reserved":0,"allocated":218349568,"data_stored":218087620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":424734378},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986280,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":557736,"kb_used_data":219212,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93814104,"statfs":{"total":96636764160,"available":96065642496,"internally_reserved":0,"allocated":224473088,"data_stored":224211145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346615466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659227,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":584664,"kb_used_data":151752,"kb_used_omap":4,"kb_used_meta":432891,"kb_avail":93787176,"statfs":{"total":96636764160,"available":96038068224,"internally_reserved":0,"allocated":155394048,"data_stored":155081059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":443281070},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,2,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":217063424,"data_stored":217059811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":154484736,"data_stored":154430082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":223322112,"data_stored":223314427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:37:42.729 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:37:43.288 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:43.289 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:37:43.309 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":330,"stamp":"2024-09-10T20:37:43.013196+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":200009172,"num_objects":48836,"num_object_clones":0,"num_object_copies":97672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30464,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48836,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":246585,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12687,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160890,"ondisk_log_size":160890,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1755020,"kb_used_data":564616,"kb_used_omap":12,"kb_used_meta":1190323,"kb_avail":281360500,"statfs":{"total":289910292480,"available":288113152000,"internally_reserved":0,"allocated":578166784,"data_stored":577329904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1218890754},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,9,7,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21401600,"num_objects":-5225,"num_object_clones":0,"num_object_copies":-10450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5225,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003438"},"pg_stats":[{"pgid":"2.f","version":"49'17591","reported_seq":17813,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:39.276707+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:37:39.276707+0000","last_peered":"2024-09-10T20:37:39.276707+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:37:39.276707+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12562432,"num_objects":3067,"num_object_clones":0,"num_object_copies":6134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17591,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-474}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'17883","reported_seq":18002,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:39.379908+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:37:39.379908+0000","last_peered":"2024-09-10T20:37:39.379908+0000","last_clean":"2024-09-10T20:37:39.379908+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:37:39.379908+0000","last_undegraded":"2024-09-10T20:37:39.379908+0000","last_fullsized":"2024-09-10T20:37:39.379908+0000","mapping_epoch":35,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12529664,"num_objects":3059,"num_object_clones":0,"num_object_copies":6118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16069,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'17735","reported_seq":25093,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:41.246946+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:37:41.246946+0000","last_peered":"2024-09-10T20:37:41.246946+0000","last_clean":"2024-09-10T20:37:41.246946+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:37:41.246946+0000","last_undegraded":"2024-09-10T20:37:41.246946+0000","last_fullsized":"2024-09-10T20:37:41.246946+0000","mapping_epoch":44,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12206104,"num_objects":2981,"num_object_clones":0,"num_object_copies":5962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2981,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15863,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'17448","reported_seq":17538,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:39.356111+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:37:39.356111+0000","last_peered":"2024-09-10T20:37:39.356111+0000","last_clean":"2024-09-10T20:37:39.356111+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:37:39.356111+0000","last_undegraded":"2024-09-10T20:37:39.356111+0000","last_fullsized":"2024-09-10T20:37:39.356111+0000","mapping_epoch":35,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12926976,"num_objects":3156,"num_object_clones":0,"num_object_copies":6312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17448,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'17929","reported_seq":18065,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:39.331232+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:37:39.331232+0000","last_peered":"2024-09-10T20:37:39.331232+0000","last_clean":"2024-09-10T20:37:39.331232+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:37:39.331232+0000","last_undegraded":"2024-09-10T20:37:39.331232+0000","last_fullsized":"2024-09-10T20:37:39.331232+0000","mapping_epoch":34,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12861440,"num_objects":3140,"num_object_clones":0,"num_object_copies":6280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14925,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'17461","reported_seq":17713,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:39.303011+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:37:39.303011+0000","last_peered":"2024-09-10T20:37:39.303011+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:37:39.303011+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12328960,"num_objects":3010,"num_object_clones":0,"num_object_copies":6020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14325,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-551}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'17495","reported_seq":17778,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:39.311440+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:37:39.311440+0000","last_peered":"2024-09-10T20:37:39.311440+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:37:39.311440+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12750848,"num_objects":3113,"num_object_clones":0,"num_object_copies":6226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14090,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-373}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'17710","reported_seq":17845,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:39.359175+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:37:39.359175+0000","last_peered":"2024-09-10T20:37:39.359175+0000","last_clean":"2024-09-10T20:37:39.359175+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:37:39.359175+0000","last_undegraded":"2024-09-10T20:37:39.359175+0000","last_fullsized":"2024-09-10T20:37:39.359175+0000","mapping_epoch":34,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12394496,"num_objects":3026,"num_object_clones":0,"num_object_copies":6052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14781,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'17378","reported_seq":17578,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:39.380686+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:37:39.380686+0000","last_peered":"2024-09-10T20:37:39.380686+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:37:39.380686+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12537856,"num_objects":3061,"num_object_clones":0,"num_object_copies":6122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3061,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15371,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-436}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318059+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318059+0000","last_peered":"2024-09-10T20:36:30.318059+0000","last_clean":"2024-09-10T20:36:30.318059+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318059+0000","last_undegraded":"2024-09-10T20:36:30.318059+0000","last_fullsized":"2024-09-10T20:36:30.318059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'17700","reported_seq":17917,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:39.341231+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:37:39.341231+0000","last_peered":"2024-09-10T20:37:39.341231+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:37:39.341231+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12411846,"num_objects":3032,"num_object_clones":0,"num_object_copies":6064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17698,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-534}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'17532","reported_seq":17724,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:39.433417+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:37:39.433417+0000","last_peered":"2024-09-10T20:37:39.433417+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:37:39.433417+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12423168,"num_objects":3033,"num_object_clones":0,"num_object_copies":6066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15360,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-573}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'17661","reported_seq":25032,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:39.357715+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:37:39.357715+0000","last_peered":"2024-09-10T20:37:39.357715+0000","last_clean":"2024-09-10T20:37:39.357715+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:37:39.357715+0000","last_undegraded":"2024-09-10T20:37:39.357715+0000","last_fullsized":"2024-09-10T20:37:39.357715+0000","mapping_epoch":48,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12300288,"num_objects":3003,"num_object_clones":0,"num_object_copies":6006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15447,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'17743","reported_seq":18000,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:39.304894+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:37:39.304894+0000","last_peered":"2024-09-10T20:37:39.304894+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:37:39.304894+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12644352,"num_objects":3087,"num_object_clones":0,"num_object_copies":6174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3087,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14466,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-505}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'17573","reported_seq":20898,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:39.329322+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:37:39.329322+0000","last_peered":"2024-09-10T20:37:39.329322+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:37:39.329322+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12333056,"num_objects":3011,"num_object_clones":0,"num_object_copies":6022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1947,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14267,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1588,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1064}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319644+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319644+0000","last_peered":"2024-09-10T20:36:30.319644+0000","last_clean":"2024-09-10T20:36:30.319644+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319644+0000","last_undegraded":"2024-09-10T20:36:30.319644+0000","last_fullsized":"2024-09-10T20:36:30.319644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'17902","reported_seq":18160,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:39.277864+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:37:39.277864+0000","last_peered":"2024-09-10T20:37:39.277864+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:37:39.277864+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12632064,"num_objects":3084,"num_object_clones":0,"num_object_copies":6168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14729,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-571}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'17546","reported_seq":24673,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:41.229072+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:37:41.229072+0000","last_peered":"2024-09-10T20:37:41.229072+0000","last_clean":"2024-09-10T20:37:41.229072+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:37:41.229072+0000","last_undegraded":"2024-09-10T20:37:41.229072+0000","last_fullsized":"2024-09-10T20:37:41.229072+0000","mapping_epoch":46,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12165603,"num_objects":2971,"num_object_clones":0,"num_object_copies":5942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14153,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":200009153,"num_objects":48834,"num_object_clones":0,"num_object_copies":97668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30464,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48834,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":246583,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12685,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":574750720,"data_stored":574684768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160887,"ondisk_log_size":160887,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182509,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612492,"kb_used_data":206004,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93759348,"statfs":{"total":96636764160,"available":96009572352,"internally_reserved":0,"allocated":210948096,"data_stored":210686148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986281,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":556892,"kb_used_data":214208,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93814948,"statfs":{"total":96636764160,"available":96066506752,"internally_reserved":0,"allocated":219348992,"data_stored":219087049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350875306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659228,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":585636,"kb_used_data":144404,"kb_used_omap":4,"kb_used_meta":441211,"kb_avail":93786204,"statfs":{"total":96636764160,"available":96037072896,"internally_reserved":0,"allocated":147869696,"data_stored":147556707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":451800750},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":209682432,"data_stored":209678819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":146911232,"data_stored":146856578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":218157056,"data_stored":218149371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:37:43.311 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:37:43.853 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:43.856 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:37:43.883 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":330,"stamp":"2024-09-10T20:37:43.013196+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":200009172,"num_objects":48836,"num_object_clones":0,"num_object_copies":97672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30464,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48836,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":246585,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12687,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160890,"ondisk_log_size":160890,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1755020,"kb_used_data":564616,"kb_used_omap":12,"kb_used_meta":1190323,"kb_avail":281360500,"statfs":{"total":289910292480,"available":288113152000,"internally_reserved":0,"allocated":578166784,"data_stored":577329904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1218890754},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,9,7,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21401600,"num_objects":-5225,"num_object_clones":0,"num_object_copies":-10450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5225,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003438"},"pg_stats":[{"pgid":"2.f","version":"49'17591","reported_seq":17813,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:39.276707+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:37:39.276707+0000","last_peered":"2024-09-10T20:37:39.276707+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:37:39.276707+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12562432,"num_objects":3067,"num_object_clones":0,"num_object_copies":6134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17591,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-474}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'17883","reported_seq":18002,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:39.379908+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:37:39.379908+0000","last_peered":"2024-09-10T20:37:39.379908+0000","last_clean":"2024-09-10T20:37:39.379908+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:37:39.379908+0000","last_undegraded":"2024-09-10T20:37:39.379908+0000","last_fullsized":"2024-09-10T20:37:39.379908+0000","mapping_epoch":35,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12529664,"num_objects":3059,"num_object_clones":0,"num_object_copies":6118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16069,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'17735","reported_seq":25093,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:41.246946+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:37:41.246946+0000","last_peered":"2024-09-10T20:37:41.246946+0000","last_clean":"2024-09-10T20:37:41.246946+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:37:41.246946+0000","last_undegraded":"2024-09-10T20:37:41.246946+0000","last_fullsized":"2024-09-10T20:37:41.246946+0000","mapping_epoch":44,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12206104,"num_objects":2981,"num_object_clones":0,"num_object_copies":5962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2981,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15863,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'17448","reported_seq":17538,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:39.356111+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:37:39.356111+0000","last_peered":"2024-09-10T20:37:39.356111+0000","last_clean":"2024-09-10T20:37:39.356111+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:37:39.356111+0000","last_undegraded":"2024-09-10T20:37:39.356111+0000","last_fullsized":"2024-09-10T20:37:39.356111+0000","mapping_epoch":35,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12926976,"num_objects":3156,"num_object_clones":0,"num_object_copies":6312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17448,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'17929","reported_seq":18065,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:39.331232+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:37:39.331232+0000","last_peered":"2024-09-10T20:37:39.331232+0000","last_clean":"2024-09-10T20:37:39.331232+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:37:39.331232+0000","last_undegraded":"2024-09-10T20:37:39.331232+0000","last_fullsized":"2024-09-10T20:37:39.331232+0000","mapping_epoch":34,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12861440,"num_objects":3140,"num_object_clones":0,"num_object_copies":6280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14925,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'17461","reported_seq":17713,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:39.303011+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:37:39.303011+0000","last_peered":"2024-09-10T20:37:39.303011+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:37:39.303011+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12328960,"num_objects":3010,"num_object_clones":0,"num_object_copies":6020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14325,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-551}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'17495","reported_seq":17778,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:39.311440+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:37:39.311440+0000","last_peered":"2024-09-10T20:37:39.311440+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:37:39.311440+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12750848,"num_objects":3113,"num_object_clones":0,"num_object_copies":6226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14090,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-373}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'17710","reported_seq":17845,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:39.359175+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:37:39.359175+0000","last_peered":"2024-09-10T20:37:39.359175+0000","last_clean":"2024-09-10T20:37:39.359175+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:37:39.359175+0000","last_undegraded":"2024-09-10T20:37:39.359175+0000","last_fullsized":"2024-09-10T20:37:39.359175+0000","mapping_epoch":34,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12394496,"num_objects":3026,"num_object_clones":0,"num_object_copies":6052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14781,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'17378","reported_seq":17578,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:39.380686+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:37:39.380686+0000","last_peered":"2024-09-10T20:37:39.380686+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:37:39.380686+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12537856,"num_objects":3061,"num_object_clones":0,"num_object_copies":6122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3061,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15371,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-436}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318059+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318059+0000","last_peered":"2024-09-10T20:36:30.318059+0000","last_clean":"2024-09-10T20:36:30.318059+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318059+0000","last_undegraded":"2024-09-10T20:36:30.318059+0000","last_fullsized":"2024-09-10T20:36:30.318059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'17700","reported_seq":17917,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:39.341231+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:37:39.341231+0000","last_peered":"2024-09-10T20:37:39.341231+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:37:39.341231+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12411846,"num_objects":3032,"num_object_clones":0,"num_object_copies":6064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17698,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-534}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'17532","reported_seq":17724,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:39.433417+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:37:39.433417+0000","last_peered":"2024-09-10T20:37:39.433417+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:37:39.433417+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12423168,"num_objects":3033,"num_object_clones":0,"num_object_copies":6066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15360,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-573}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'17661","reported_seq":25032,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:39.357715+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:37:39.357715+0000","last_peered":"2024-09-10T20:37:39.357715+0000","last_clean":"2024-09-10T20:37:39.357715+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:37:39.357715+0000","last_undegraded":"2024-09-10T20:37:39.357715+0000","last_fullsized":"2024-09-10T20:37:39.357715+0000","mapping_epoch":48,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12300288,"num_objects":3003,"num_object_clones":0,"num_object_copies":6006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15447,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'17743","reported_seq":18000,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:39.304894+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:37:39.304894+0000","last_peered":"2024-09-10T20:37:39.304894+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:37:39.304894+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12644352,"num_objects":3087,"num_object_clones":0,"num_object_copies":6174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3087,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14466,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-505}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'17573","reported_seq":20898,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:39.329322+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:37:39.329322+0000","last_peered":"2024-09-10T20:37:39.329322+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:37:39.329322+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12333056,"num_objects":3011,"num_object_clones":0,"num_object_copies":6022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1947,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14267,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1588,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1064}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319644+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319644+0000","last_peered":"2024-09-10T20:36:30.319644+0000","last_clean":"2024-09-10T20:36:30.319644+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319644+0000","last_undegraded":"2024-09-10T20:36:30.319644+0000","last_fullsized":"2024-09-10T20:36:30.319644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'17902","reported_seq":18160,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:39.277864+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:37:39.277864+0000","last_peered":"2024-09-10T20:37:39.277864+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:37:39.277864+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12632064,"num_objects":3084,"num_object_clones":0,"num_object_copies":6168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14729,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-571}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'17546","reported_seq":24673,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:41.229072+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:37:41.229072+0000","last_peered":"2024-09-10T20:37:41.229072+0000","last_clean":"2024-09-10T20:37:41.229072+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:37:41.229072+0000","last_undegraded":"2024-09-10T20:37:41.229072+0000","last_fullsized":"2024-09-10T20:37:41.229072+0000","mapping_epoch":46,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12165603,"num_objects":2971,"num_object_clones":0,"num_object_copies":5942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14153,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":200009153,"num_objects":48834,"num_object_clones":0,"num_object_copies":97668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30464,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48834,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":246583,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12685,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":574750720,"data_stored":574684768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160887,"ondisk_log_size":160887,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182509,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612492,"kb_used_data":206004,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93759348,"statfs":{"total":96636764160,"available":96009572352,"internally_reserved":0,"allocated":210948096,"data_stored":210686148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986281,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":556892,"kb_used_data":214208,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93814948,"statfs":{"total":96636764160,"available":96066506752,"internally_reserved":0,"allocated":219348992,"data_stored":219087049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350875306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659228,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":585636,"kb_used_data":144404,"kb_used_omap":4,"kb_used_meta":441211,"kb_avail":93786204,"statfs":{"total":96636764160,"available":96037072896,"internally_reserved":0,"allocated":147869696,"data_stored":147556707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":451800750},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":209682432,"data_stored":209678819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":146911232,"data_stored":146856578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":218157056,"data_stored":218149371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:37:43.885 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:37:43.885 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'17591", 'reported_seq': 17813, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:39.276707+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:37:39.276707+0000', 'last_peered': '2024-09-10T20:37:39.276707+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:37:39.276707+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "43'7500", 'ondisk_log_start': "43'7500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12562432, 'num_objects': 3067, 'num_object_clones': 0, 'num_object_copies': 6134, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3067, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17591, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -474}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:43.885 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:37:43.885 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'17461", 'reported_seq': 17713, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:39.303011+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:37:39.303011+0000', 'last_peered': '2024-09-10T20:37:39.303011+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:37:39.303011+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "43'7400", 'ondisk_log_start': "43'7400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12328960, 'num_objects': 3010, 'num_object_clones': 0, 'num_object_copies': 6020, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3010, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14325, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -551}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:43.886 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:37:43.886 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'17495", 'reported_seq': 17778, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:39.311440+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:37:39.311440+0000', 'last_peered': '2024-09-10T20:37:39.311440+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:37:39.311440+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "43'7400", 'ondisk_log_start': "43'7400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12750848, 'num_objects': 3113, 'num_object_clones': 0, 'num_object_copies': 6226, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3113, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14090, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -373}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:43.886 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:37:43.886 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'17378", 'reported_seq': 17578, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:39.380686+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:37:39.380686+0000', 'last_peered': '2024-09-10T20:37:39.380686+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:37:39.380686+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "43'7300", 'ondisk_log_start': "43'7300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12537856, 'num_objects': 3061, 'num_object_clones': 0, 'num_object_copies': 6122, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3061, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15371, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -436}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:37:43.886 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:37:43.886 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'17700", 'reported_seq': 17917, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:39.341231+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:37:39.341231+0000', 'last_peered': '2024-09-10T20:37:39.341231+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:37:39.341231+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "43'7600", 'ondisk_log_start': "43'7600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12411846, 'num_objects': 3032, 'num_object_clones': 0, 'num_object_copies': 6064, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3032, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17698, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -534}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:43.886 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:37:43.886 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'17532", 'reported_seq': 17724, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:39.433417+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:37:39.433417+0000', 'last_peered': '2024-09-10T20:37:39.433417+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:37:39.433417+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "43'7500", 'ondisk_log_start': "43'7500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12423168, 'num_objects': 3033, 'num_object_clones': 0, 'num_object_copies': 6066, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3033, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15360, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -573}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:37:43.886 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:37:43.886 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'17743", 'reported_seq': 18000, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:39.304894+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:37:39.304894+0000', 'last_peered': '2024-09-10T20:37:39.304894+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:37:39.304894+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "43'7700", 'ondisk_log_start': "43'7700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12644352, 'num_objects': 3087, 'num_object_clones': 0, 'num_object_copies': 6174, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3087, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14466, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -505}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:43.887 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:37:43.887 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'17573", 'reported_seq': 20898, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:39.329322+0000', 'last_change': '2024-09-10T20:36:28.778006+0000', 'last_active': '2024-09-10T20:37:39.329322+0000', 'last_peered': '2024-09-10T20:37:39.329322+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:37:39.329322+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "43'7500", 'ondisk_log_start': "43'7500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12333056, 'num_objects': 3011, 'num_object_clones': 0, 'num_object_copies': 6022, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1947, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3011, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14267, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1588, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1064}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:43.887 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:37:43.887 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'17902", 'reported_seq': 18160, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:39.277864+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:37:39.277864+0000', 'last_peered': '2024-09-10T20:37:39.277864+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:37:39.277864+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "43'7900", 'ondisk_log_start': "43'7900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12632064, 'num_objects': 3084, 'num_object_clones': 0, 'num_object_copies': 6168, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3084, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14729, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -571}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:43.887 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:37:44.570 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:44.570 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 30464/97672 objects degraded (31.190%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":576,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":48836,"data_bytes":200009172,"bytes_used":1797140480,"bytes_avail":288113152000,"bytes_total":289910292480,"degraded_objects":30464,"degraded_total":97672,"degraded_ratio":0.31190105659759193,"recovering_objects_per_sec":19,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":237,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":435},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:37:44.591 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:37:44.592 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:37:45.152 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:45.154 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:37:45.177 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-09-10T20:37:45.013824+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":191038932,"num_objects":46646,"num_object_clones":0,"num_object_copies":93292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30350,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46646,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":248775,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12801,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160680,"ondisk_log_size":160680,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1768836,"kb_used_data":552076,"kb_used_omap":12,"kb_used_meta":1216691,"kb_avail":281346684,"statfs":{"total":289910292480,"available":288099004416,"internally_reserved":0,"allocated":565325824,"data_stored":564488944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1245891586},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,9,10,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29085696,"num_objects":-7101,"num_object_clones":0,"num_object_copies":-14202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-351,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7101,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":351,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003350"},"pg_stats":[{"pgid":"2.f","version":"49'17745","reported_seq":17964,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:44.414501+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:37:44.414501+0000","last_peered":"2024-09-10T20:37:44.414501+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:37:44.414501+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11931648,"num_objects":2913,"num_object_clones":0,"num_object_copies":5826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17745,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-628}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18035","reported_seq":18155,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:44.340020+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:37:44.340020+0000","last_peered":"2024-09-10T20:37:44.340020+0000","last_clean":"2024-09-10T20:37:44.340020+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:37:44.340020+0000","last_undegraded":"2024-09-10T20:37:44.340020+0000","last_fullsized":"2024-09-10T20:37:44.340020+0000","mapping_epoch":35,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11907072,"num_objects":2907,"num_object_clones":0,"num_object_copies":5814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16221,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'17735","reported_seq":25093,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:41.246946+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:37:41.246946+0000","last_peered":"2024-09-10T20:37:41.246946+0000","last_clean":"2024-09-10T20:37:41.246946+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:37:41.246946+0000","last_undegraded":"2024-09-10T20:37:41.246946+0000","last_fullsized":"2024-09-10T20:37:41.246946+0000","mapping_epoch":44,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12206104,"num_objects":2981,"num_object_clones":0,"num_object_copies":5962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2981,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15863,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'17614","reported_seq":17704,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:44.432152+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:37:44.432152+0000","last_peered":"2024-09-10T20:37:44.432152+0000","last_clean":"2024-09-10T20:37:44.432152+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:37:44.432152+0000","last_undegraded":"2024-09-10T20:37:44.432152+0000","last_fullsized":"2024-09-10T20:37:44.432152+0000","mapping_epoch":35,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12247040,"num_objects":2990,"num_object_clones":0,"num_object_copies":5980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17614,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'18111","reported_seq":18247,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:44.367206+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:37:44.367206+0000","last_peered":"2024-09-10T20:37:44.367206+0000","last_clean":"2024-09-10T20:37:44.367206+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:37:44.367206+0000","last_undegraded":"2024-09-10T20:37:44.367206+0000","last_fullsized":"2024-09-10T20:37:44.367206+0000","mapping_epoch":34,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12115968,"num_objects":2958,"num_object_clones":0,"num_object_copies":5916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15107,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'17608","reported_seq":17860,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:44.282716+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:37:44.282716+0000","last_peered":"2024-09-10T20:37:44.282716+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:37:44.282716+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11726848,"num_objects":2863,"num_object_clones":0,"num_object_copies":5726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14472,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-698}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'17655","reported_seq":17941,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:44.371050+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:37:44.371050+0000","last_peered":"2024-09-10T20:37:44.371050+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:37:44.371050+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12095488,"num_objects":2953,"num_object_clones":0,"num_object_copies":5906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14250,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-533}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'17866","reported_seq":18001,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:44.409030+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:37:44.409030+0000","last_peered":"2024-09-10T20:37:44.409030+0000","last_clean":"2024-09-10T20:37:44.409030+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:37:44.409030+0000","last_undegraded":"2024-09-10T20:37:44.409030+0000","last_fullsized":"2024-09-10T20:37:44.409030+0000","mapping_epoch":34,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11755520,"num_objects":2870,"num_object_clones":0,"num_object_copies":5740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14937,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'17515","reported_seq":17715,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:44.340816+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:37:44.340816+0000","last_peered":"2024-09-10T20:37:44.340816+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:37:44.340816+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11976704,"num_objects":2924,"num_object_clones":0,"num_object_copies":5848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15508,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-573}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318059+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318059+0000","last_peered":"2024-09-10T20:36:30.318059+0000","last_clean":"2024-09-10T20:36:30.318059+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318059+0000","last_undegraded":"2024-09-10T20:36:30.318059+0000","last_fullsized":"2024-09-10T20:36:30.318059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'17869","reported_seq":18086,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:44.383516+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:37:44.383516+0000","last_peered":"2024-09-10T20:37:44.383516+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:37:44.383516+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11719622,"num_objects":2863,"num_object_clones":0,"num_object_copies":5726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17867,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-703}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'17668","reported_seq":17861,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:44.352866+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:37:44.352866+0000","last_peered":"2024-09-10T20:37:44.352866+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:37:44.352866+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11866112,"num_objects":2897,"num_object_clones":0,"num_object_copies":5794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15496,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-709}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'17816","reported_seq":25186,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:44.442098+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:37:44.442098+0000","last_peered":"2024-09-10T20:37:44.442098+0000","last_clean":"2024-09-10T20:37:44.442098+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:37:44.442098+0000","last_undegraded":"2024-09-10T20:37:44.442098+0000","last_fullsized":"2024-09-10T20:37:44.442098+0000","mapping_epoch":48,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11665408,"num_objects":2848,"num_object_clones":0,"num_object_copies":5696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15602,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'17880","reported_seq":18133,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:44.320791+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:37:44.320791+0000","last_peered":"2024-09-10T20:37:44.320791+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:37:44.320791+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12083200,"num_objects":2950,"num_object_clones":0,"num_object_copies":5900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14603,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-642}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'17760","reported_seq":21313,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:44.399127+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:37:44.399127+0000","last_peered":"2024-09-10T20:37:44.399127+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:37:44.399127+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11567104,"num_objects":2824,"num_object_clones":0,"num_object_copies":5648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1833,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14454,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1702,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":991}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319644+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319644+0000","last_peered":"2024-09-10T20:36:30.319644+0000","last_clean":"2024-09-10T20:36:30.319644+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319644+0000","last_undegraded":"2024-09-10T20:36:30.319644+0000","last_fullsized":"2024-09-10T20:36:30.319644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18054","reported_seq":18312,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:44.310355+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:37:44.310355+0000","last_peered":"2024-09-10T20:37:44.310355+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:37:44.310355+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12009472,"num_objects":2932,"num_object_clones":0,"num_object_copies":5864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2932,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14881,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-723}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'17546","reported_seq":24673,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:41.229072+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:37:41.229072+0000","last_peered":"2024-09-10T20:37:41.229072+0000","last_clean":"2024-09-10T20:37:41.229072+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:37:41.229072+0000","last_undegraded":"2024-09-10T20:37:41.229072+0000","last_fullsized":"2024-09-10T20:37:41.229072+0000","mapping_epoch":46,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12165603,"num_objects":2971,"num_object_clones":0,"num_object_copies":5942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14153,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":191038913,"num_objects":46644,"num_object_clones":0,"num_object_copies":93288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30350,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46644,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":248773,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12799,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":561840128,"data_stored":561774176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160677,"ondisk_log_size":160677,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182510,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627064,"kb_used_data":198368,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93744776,"statfs":{"total":96636764160,"available":95994650624,"internally_reserved":0,"allocated":203128832,"data_stored":202866884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":438955690},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986282,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":556136,"kb_used_data":209304,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93815704,"statfs":{"total":96636764160,"available":96067280896,"internally_reserved":0,"allocated":214327296,"data_stored":214065353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355135146},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659228,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":585636,"kb_used_data":144404,"kb_used_omap":4,"kb_used_meta":441211,"kb_avail":93786204,"statfs":{"total":96636764160,"available":96037072896,"internally_reserved":0,"allocated":147869696,"data_stored":147556707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":451800750},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":201781248,"data_stored":201777635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":146911232,"data_stored":146856578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":213147648,"data_stored":213139963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:37:48.180 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:37:48.759 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:48.759 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:37:48.781 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-09-10T20:37:47.014428+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":189859284,"num_objects":46358,"num_object_clones":0,"num_object_copies":92716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30350,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46358,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":249063,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12801,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1749252,"kb_used_data":544968,"kb_used_omap":12,"kb_used_meta":1204211,"kb_avail":281366268,"statfs":{"total":289910292480,"available":288119058432,"internally_reserved":0,"allocated":558047232,"data_stored":557210352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1233112066},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,4,8,10,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21512192,"num_objects":-5252,"num_object_clones":0,"num_object_copies":-10504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5252,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5252,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003295"},"pg_stats":[{"pgid":"2.f","version":"49'17745","reported_seq":17964,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:44.414501+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:37:44.414501+0000","last_peered":"2024-09-10T20:37:44.414501+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:37:44.414501+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11931648,"num_objects":2913,"num_object_clones":0,"num_object_copies":5826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17745,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-628}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18035","reported_seq":18155,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:44.340020+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:37:44.340020+0000","last_peered":"2024-09-10T20:37:44.340020+0000","last_clean":"2024-09-10T20:37:44.340020+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:37:44.340020+0000","last_undegraded":"2024-09-10T20:37:44.340020+0000","last_fullsized":"2024-09-10T20:37:44.340020+0000","mapping_epoch":35,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11907072,"num_objects":2907,"num_object_clones":0,"num_object_copies":5814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16221,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'17869","reported_seq":25226,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:46.324311+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:37:46.324311+0000","last_peered":"2024-09-10T20:37:46.324311+0000","last_clean":"2024-09-10T20:37:46.324311+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:37:46.324311+0000","last_undegraded":"2024-09-10T20:37:46.324311+0000","last_fullsized":"2024-09-10T20:37:46.324311+0000","mapping_epoch":44,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11657240,"num_objects":2847,"num_object_clones":0,"num_object_copies":5694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2847,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15997,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'17614","reported_seq":17704,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:44.432152+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:37:44.432152+0000","last_peered":"2024-09-10T20:37:44.432152+0000","last_clean":"2024-09-10T20:37:44.432152+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:37:44.432152+0000","last_undegraded":"2024-09-10T20:37:44.432152+0000","last_fullsized":"2024-09-10T20:37:44.432152+0000","mapping_epoch":35,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12247040,"num_objects":2990,"num_object_clones":0,"num_object_copies":5980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17614,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'18111","reported_seq":18247,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:44.367206+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:37:44.367206+0000","last_peered":"2024-09-10T20:37:44.367206+0000","last_clean":"2024-09-10T20:37:44.367206+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:37:44.367206+0000","last_undegraded":"2024-09-10T20:37:44.367206+0000","last_fullsized":"2024-09-10T20:37:44.367206+0000","mapping_epoch":34,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12115968,"num_objects":2958,"num_object_clones":0,"num_object_copies":5916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15107,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'17608","reported_seq":17860,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:44.282716+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:37:44.282716+0000","last_peered":"2024-09-10T20:37:44.282716+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:37:44.282716+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11726848,"num_objects":2863,"num_object_clones":0,"num_object_copies":5726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14472,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-698}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'17655","reported_seq":17941,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:44.371050+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:37:44.371050+0000","last_peered":"2024-09-10T20:37:44.371050+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:37:44.371050+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12095488,"num_objects":2953,"num_object_clones":0,"num_object_copies":5906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14250,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-533}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'17866","reported_seq":18001,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:44.409030+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:37:44.409030+0000","last_peered":"2024-09-10T20:37:44.409030+0000","last_clean":"2024-09-10T20:37:44.409030+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:37:44.409030+0000","last_undegraded":"2024-09-10T20:37:44.409030+0000","last_fullsized":"2024-09-10T20:37:44.409030+0000","mapping_epoch":34,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11755520,"num_objects":2870,"num_object_clones":0,"num_object_copies":5740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14937,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'17515","reported_seq":17715,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:44.340816+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:37:44.340816+0000","last_peered":"2024-09-10T20:37:44.340816+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:37:44.340816+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11976704,"num_objects":2924,"num_object_clones":0,"num_object_copies":5848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15508,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-573}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318059+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318059+0000","last_peered":"2024-09-10T20:36:30.318059+0000","last_clean":"2024-09-10T20:36:30.318059+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318059+0000","last_undegraded":"2024-09-10T20:36:30.318059+0000","last_fullsized":"2024-09-10T20:36:30.318059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'17869","reported_seq":18086,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:44.383516+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:37:44.383516+0000","last_peered":"2024-09-10T20:37:44.383516+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:37:44.383516+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11719622,"num_objects":2863,"num_object_clones":0,"num_object_copies":5726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17867,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-703}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'17668","reported_seq":17861,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:44.352866+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:37:44.352866+0000","last_peered":"2024-09-10T20:37:44.352866+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:37:44.352866+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11866112,"num_objects":2897,"num_object_clones":0,"num_object_copies":5794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15496,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-709}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'17816","reported_seq":25186,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:44.442098+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:37:44.442098+0000","last_peered":"2024-09-10T20:37:44.442098+0000","last_clean":"2024-09-10T20:37:44.442098+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:37:44.442098+0000","last_undegraded":"2024-09-10T20:37:44.442098+0000","last_fullsized":"2024-09-10T20:37:44.442098+0000","mapping_epoch":48,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11665408,"num_objects":2848,"num_object_clones":0,"num_object_copies":5696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15602,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'17880","reported_seq":18133,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:44.320791+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:37:44.320791+0000","last_peered":"2024-09-10T20:37:44.320791+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:37:44.320791+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12083200,"num_objects":2950,"num_object_clones":0,"num_object_copies":5900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14603,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-642}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'17760","reported_seq":21313,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:44.399127+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:37:44.399127+0000","last_peered":"2024-09-10T20:37:44.399127+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:37:44.399127+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11567104,"num_objects":2824,"num_object_clones":0,"num_object_copies":5648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1833,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14454,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1702,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":991}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319644+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319644+0000","last_peered":"2024-09-10T20:36:30.319644+0000","last_clean":"2024-09-10T20:36:30.319644+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319644+0000","last_undegraded":"2024-09-10T20:36:30.319644+0000","last_fullsized":"2024-09-10T20:36:30.319644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18054","reported_seq":18312,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:44.310355+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:37:44.310355+0000","last_peered":"2024-09-10T20:37:44.310355+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:37:44.310355+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12009472,"num_objects":2932,"num_object_clones":0,"num_object_copies":5864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2932,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14881,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-723}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'17700","reported_seq":24827,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:46.276277+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:37:46.276277+0000","last_peered":"2024-09-10T20:37:46.276277+0000","last_clean":"2024-09-10T20:37:46.276277+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:37:46.276277+0000","last_undegraded":"2024-09-10T20:37:46.276277+0000","last_fullsized":"2024-09-10T20:37:46.276277+0000","mapping_epoch":46,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11534819,"num_objects":2817,"num_object_clones":0,"num_object_copies":5634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14307,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":189859265,"num_objects":46356,"num_object_clones":0,"num_object_copies":92712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30350,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46356,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":249061,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12799,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":554319872,"data_stored":554253920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160765,"ondisk_log_size":160765,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182510,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627064,"kb_used_data":198368,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93744776,"statfs":{"total":96636764160,"available":95994650624,"internally_reserved":0,"allocated":203128832,"data_stored":202866884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":438955690},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986282,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":556136,"kb_used_data":209304,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93815704,"statfs":{"total":96636764160,"available":96067280896,"internally_reserved":0,"allocated":214327296,"data_stored":214065353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355135146},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659229,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566052,"kb_used_data":137296,"kb_used_omap":4,"kb_used_meta":428731,"kb_avail":93805788,"statfs":{"total":96636764160,"available":96057126912,"internally_reserved":0,"allocated":140591104,"data_stored":140278115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":439021230},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,3,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":201781248,"data_stored":201777635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139390976,"data_stored":139336322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":213147648,"data_stored":213139963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:37:48.783 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:37:49.333 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:49.333 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:37:49.363 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":333,"stamp":"2024-09-10T20:37:49.015065+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":189859284,"num_objects":46358,"num_object_clones":0,"num_object_copies":92716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30350,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46358,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":249063,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12801,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1749252,"kb_used_data":544968,"kb_used_omap":12,"kb_used_meta":1204211,"kb_avail":281366268,"statfs":{"total":289910292480,"available":288119058432,"internally_reserved":0,"allocated":558047232,"data_stored":557210352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1233112066},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,4,8,10,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20295680,"num_objects":-4955,"num_object_clones":0,"num_object_copies":-9910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4955,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003447"},"pg_stats":[{"pgid":"2.f","version":"49'17745","reported_seq":17964,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:44.414501+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:37:44.414501+0000","last_peered":"2024-09-10T20:37:44.414501+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:37:44.414501+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11931648,"num_objects":2913,"num_object_clones":0,"num_object_copies":5826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17745,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-628}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18035","reported_seq":18155,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:44.340020+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:37:44.340020+0000","last_peered":"2024-09-10T20:37:44.340020+0000","last_clean":"2024-09-10T20:37:44.340020+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:37:44.340020+0000","last_undegraded":"2024-09-10T20:37:44.340020+0000","last_fullsized":"2024-09-10T20:37:44.340020+0000","mapping_epoch":35,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11907072,"num_objects":2907,"num_object_clones":0,"num_object_copies":5814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16221,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'17869","reported_seq":25226,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:46.324311+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:37:46.324311+0000","last_peered":"2024-09-10T20:37:46.324311+0000","last_clean":"2024-09-10T20:37:46.324311+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:37:46.324311+0000","last_undegraded":"2024-09-10T20:37:46.324311+0000","last_fullsized":"2024-09-10T20:37:46.324311+0000","mapping_epoch":44,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11657240,"num_objects":2847,"num_object_clones":0,"num_object_copies":5694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2847,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15997,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'17614","reported_seq":17704,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:44.432152+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:37:44.432152+0000","last_peered":"2024-09-10T20:37:44.432152+0000","last_clean":"2024-09-10T20:37:44.432152+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:37:44.432152+0000","last_undegraded":"2024-09-10T20:37:44.432152+0000","last_fullsized":"2024-09-10T20:37:44.432152+0000","mapping_epoch":35,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12247040,"num_objects":2990,"num_object_clones":0,"num_object_copies":5980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17614,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'18111","reported_seq":18247,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:44.367206+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:37:44.367206+0000","last_peered":"2024-09-10T20:37:44.367206+0000","last_clean":"2024-09-10T20:37:44.367206+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:37:44.367206+0000","last_undegraded":"2024-09-10T20:37:44.367206+0000","last_fullsized":"2024-09-10T20:37:44.367206+0000","mapping_epoch":34,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12115968,"num_objects":2958,"num_object_clones":0,"num_object_copies":5916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15107,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'17608","reported_seq":17860,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:44.282716+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:37:44.282716+0000","last_peered":"2024-09-10T20:37:44.282716+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:37:44.282716+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11726848,"num_objects":2863,"num_object_clones":0,"num_object_copies":5726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14472,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-698}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'17655","reported_seq":17941,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:44.371050+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:37:44.371050+0000","last_peered":"2024-09-10T20:37:44.371050+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:37:44.371050+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12095488,"num_objects":2953,"num_object_clones":0,"num_object_copies":5906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14250,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-533}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'17866","reported_seq":18001,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:44.409030+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:37:44.409030+0000","last_peered":"2024-09-10T20:37:44.409030+0000","last_clean":"2024-09-10T20:37:44.409030+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:37:44.409030+0000","last_undegraded":"2024-09-10T20:37:44.409030+0000","last_fullsized":"2024-09-10T20:37:44.409030+0000","mapping_epoch":34,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11755520,"num_objects":2870,"num_object_clones":0,"num_object_copies":5740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14937,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'17515","reported_seq":17715,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:44.340816+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:37:44.340816+0000","last_peered":"2024-09-10T20:37:44.340816+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:37:44.340816+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11976704,"num_objects":2924,"num_object_clones":0,"num_object_copies":5848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15508,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-573}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318059+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318059+0000","last_peered":"2024-09-10T20:36:30.318059+0000","last_clean":"2024-09-10T20:36:30.318059+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318059+0000","last_undegraded":"2024-09-10T20:36:30.318059+0000","last_fullsized":"2024-09-10T20:36:30.318059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'17869","reported_seq":18086,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:44.383516+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:37:44.383516+0000","last_peered":"2024-09-10T20:37:44.383516+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:37:44.383516+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11719622,"num_objects":2863,"num_object_clones":0,"num_object_copies":5726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17867,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-703}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'17668","reported_seq":17861,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:44.352866+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:37:44.352866+0000","last_peered":"2024-09-10T20:37:44.352866+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:37:44.352866+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11866112,"num_objects":2897,"num_object_clones":0,"num_object_copies":5794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15496,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-709}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'17816","reported_seq":25186,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:44.442098+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:37:44.442098+0000","last_peered":"2024-09-10T20:37:44.442098+0000","last_clean":"2024-09-10T20:37:44.442098+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:37:44.442098+0000","last_undegraded":"2024-09-10T20:37:44.442098+0000","last_fullsized":"2024-09-10T20:37:44.442098+0000","mapping_epoch":48,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11665408,"num_objects":2848,"num_object_clones":0,"num_object_copies":5696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15602,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'17880","reported_seq":18133,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:44.320791+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:37:44.320791+0000","last_peered":"2024-09-10T20:37:44.320791+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:37:44.320791+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12083200,"num_objects":2950,"num_object_clones":0,"num_object_copies":5900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14603,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-642}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'17760","reported_seq":21313,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:44.399127+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:37:44.399127+0000","last_peered":"2024-09-10T20:37:44.399127+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:37:44.399127+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11567104,"num_objects":2824,"num_object_clones":0,"num_object_copies":5648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1833,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14454,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1702,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":991}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319644+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319644+0000","last_peered":"2024-09-10T20:36:30.319644+0000","last_clean":"2024-09-10T20:36:30.319644+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319644+0000","last_undegraded":"2024-09-10T20:36:30.319644+0000","last_fullsized":"2024-09-10T20:36:30.319644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18054","reported_seq":18312,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:44.310355+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:37:44.310355+0000","last_peered":"2024-09-10T20:37:44.310355+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:37:44.310355+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12009472,"num_objects":2932,"num_object_clones":0,"num_object_copies":5864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2932,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14881,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-723}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'17700","reported_seq":24827,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:46.276277+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:37:46.276277+0000","last_peered":"2024-09-10T20:37:46.276277+0000","last_clean":"2024-09-10T20:37:46.276277+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:37:46.276277+0000","last_undegraded":"2024-09-10T20:37:46.276277+0000","last_fullsized":"2024-09-10T20:37:46.276277+0000","mapping_epoch":46,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11534819,"num_objects":2817,"num_object_clones":0,"num_object_copies":5634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14307,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":189859265,"num_objects":46356,"num_object_clones":0,"num_object_copies":92712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30350,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46356,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":249061,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12799,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":554319872,"data_stored":554253920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160765,"ondisk_log_size":160765,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182510,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627064,"kb_used_data":198368,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93744776,"statfs":{"total":96636764160,"available":95994650624,"internally_reserved":0,"allocated":203128832,"data_stored":202866884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":438955690},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986282,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":556136,"kb_used_data":209304,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93815704,"statfs":{"total":96636764160,"available":96067280896,"internally_reserved":0,"allocated":214327296,"data_stored":214065353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355135146},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659229,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566052,"kb_used_data":137296,"kb_used_omap":4,"kb_used_meta":428731,"kb_avail":93805788,"statfs":{"total":96636764160,"available":96057126912,"internally_reserved":0,"allocated":140591104,"data_stored":140278115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":439021230},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,3,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":201781248,"data_stored":201777635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139390976,"data_stored":139336322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":213147648,"data_stored":213139963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:37:49.365 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:37:49.365 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'17745", 'reported_seq': 17964, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:44.414501+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:37:44.414501+0000', 'last_peered': '2024-09-10T20:37:44.414501+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:37:44.414501+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "43'7700", 'ondisk_log_start': "43'7700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11931648, 'num_objects': 2913, 'num_object_clones': 0, 'num_object_copies': 5826, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2913, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17745, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -628}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:49.366 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:37:49.366 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'17608", 'reported_seq': 17860, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:44.282716+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:37:44.282716+0000', 'last_peered': '2024-09-10T20:37:44.282716+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:37:44.282716+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "43'7600", 'ondisk_log_start': "43'7600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11726848, 'num_objects': 2863, 'num_object_clones': 0, 'num_object_copies': 5726, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2863, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14472, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -698}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:49.366 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:37:49.366 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'17655", 'reported_seq': 17941, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:44.371050+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:37:44.371050+0000', 'last_peered': '2024-09-10T20:37:44.371050+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:37:44.371050+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "43'7600", 'ondisk_log_start': "43'7600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12095488, 'num_objects': 2953, 'num_object_clones': 0, 'num_object_copies': 5906, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2953, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14250, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -533}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:49.366 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:37:49.366 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'17515", 'reported_seq': 17715, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:44.340816+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:37:44.340816+0000', 'last_peered': '2024-09-10T20:37:44.340816+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:37:44.340816+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "43'7500", 'ondisk_log_start': "43'7500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11976704, 'num_objects': 2924, 'num_object_clones': 0, 'num_object_copies': 5848, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2924, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15508, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -573}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:37:49.366 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:37:49.366 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'17869", 'reported_seq': 18086, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:44.383516+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:37:44.383516+0000', 'last_peered': '2024-09-10T20:37:44.383516+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:37:44.383516+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "43'7800", 'ondisk_log_start': "43'7800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11719622, 'num_objects': 2863, 'num_object_clones': 0, 'num_object_copies': 5726, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2863, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17867, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -703}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:49.366 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:37:49.366 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'17668", 'reported_seq': 17861, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:44.352866+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:37:44.352866+0000', 'last_peered': '2024-09-10T20:37:44.352866+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:37:44.352866+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "43'7600", 'ondisk_log_start': "43'7600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11866112, 'num_objects': 2897, 'num_object_clones': 0, 'num_object_copies': 5794, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2897, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15496, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -709}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:37:49.367 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:37:49.368 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'17880", 'reported_seq': 18133, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:44.320791+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:37:44.320791+0000', 'last_peered': '2024-09-10T20:37:44.320791+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:37:44.320791+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "43'7800", 'ondisk_log_start': "43'7800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12083200, 'num_objects': 2950, 'num_object_clones': 0, 'num_object_copies': 5900, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2950, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14603, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -642}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:49.368 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:37:49.368 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'17760", 'reported_seq': 21313, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:44.399127+0000', 'last_change': '2024-09-10T20:36:28.778006+0000', 'last_active': '2024-09-10T20:37:44.399127+0000', 'last_peered': '2024-09-10T20:37:44.399127+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:37:44.399127+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "43'7700", 'ondisk_log_start': "43'7700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11567104, 'num_objects': 2824, 'num_object_clones': 0, 'num_object_copies': 5648, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1833, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2824, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14454, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1702, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 991}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:49.368 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:37:49.368 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'18054", 'reported_seq': 18312, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:44.310355+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:37:44.310355+0000', 'last_peered': '2024-09-10T20:37:44.310355+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:37:44.310355+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "43'8000", 'ondisk_log_start': "43'8000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12009472, 'num_objects': 2932, 'num_object_clones': 0, 'num_object_copies': 5864, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2932, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14881, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -723}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:49.368 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:37:50.050 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:50.051 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 30350/92716 objects degraded (32.734%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":581,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":46358,"data_bytes":189859284,"bytes_used":1791234048,"bytes_avail":288119058432,"bytes_total":289910292480,"degraded_objects":30350,"degraded_total":92716,"degraded_ratio":0.32734371629492215,"recovering_objects_per_sec":19,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":237,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":412},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:37:50.072 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:37:50.072 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:37:50.625 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:50.628 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:37:50.648 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":333,"stamp":"2024-09-10T20:37:49.015065+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":189859284,"num_objects":46358,"num_object_clones":0,"num_object_copies":92716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30350,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46358,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":249063,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12801,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1749252,"kb_used_data":544968,"kb_used_omap":12,"kb_used_meta":1204211,"kb_avail":281366268,"statfs":{"total":289910292480,"available":288119058432,"internally_reserved":0,"allocated":558047232,"data_stored":557210352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1233112066},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,4,8,10,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20295680,"num_objects":-4955,"num_object_clones":0,"num_object_copies":-9910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4955,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003447"},"pg_stats":[{"pgid":"2.f","version":"49'17745","reported_seq":17964,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:44.414501+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:37:44.414501+0000","last_peered":"2024-09-10T20:37:44.414501+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:37:44.414501+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11931648,"num_objects":2913,"num_object_clones":0,"num_object_copies":5826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17745,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-628}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18035","reported_seq":18155,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:44.340020+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:37:44.340020+0000","last_peered":"2024-09-10T20:37:44.340020+0000","last_clean":"2024-09-10T20:37:44.340020+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:37:44.340020+0000","last_undegraded":"2024-09-10T20:37:44.340020+0000","last_fullsized":"2024-09-10T20:37:44.340020+0000","mapping_epoch":35,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11907072,"num_objects":2907,"num_object_clones":0,"num_object_copies":5814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16221,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'17869","reported_seq":25226,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:46.324311+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:37:46.324311+0000","last_peered":"2024-09-10T20:37:46.324311+0000","last_clean":"2024-09-10T20:37:46.324311+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:37:46.324311+0000","last_undegraded":"2024-09-10T20:37:46.324311+0000","last_fullsized":"2024-09-10T20:37:46.324311+0000","mapping_epoch":44,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11657240,"num_objects":2847,"num_object_clones":0,"num_object_copies":5694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2847,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15997,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'17614","reported_seq":17704,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:44.432152+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:37:44.432152+0000","last_peered":"2024-09-10T20:37:44.432152+0000","last_clean":"2024-09-10T20:37:44.432152+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:37:44.432152+0000","last_undegraded":"2024-09-10T20:37:44.432152+0000","last_fullsized":"2024-09-10T20:37:44.432152+0000","mapping_epoch":35,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12247040,"num_objects":2990,"num_object_clones":0,"num_object_copies":5980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17614,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'18111","reported_seq":18247,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:44.367206+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:37:44.367206+0000","last_peered":"2024-09-10T20:37:44.367206+0000","last_clean":"2024-09-10T20:37:44.367206+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:37:44.367206+0000","last_undegraded":"2024-09-10T20:37:44.367206+0000","last_fullsized":"2024-09-10T20:37:44.367206+0000","mapping_epoch":34,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12115968,"num_objects":2958,"num_object_clones":0,"num_object_copies":5916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15107,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'17608","reported_seq":17860,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:44.282716+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:37:44.282716+0000","last_peered":"2024-09-10T20:37:44.282716+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:37:44.282716+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11726848,"num_objects":2863,"num_object_clones":0,"num_object_copies":5726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14472,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-698}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'17655","reported_seq":17941,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:44.371050+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:37:44.371050+0000","last_peered":"2024-09-10T20:37:44.371050+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:37:44.371050+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12095488,"num_objects":2953,"num_object_clones":0,"num_object_copies":5906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14250,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-533}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'17866","reported_seq":18001,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:44.409030+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:37:44.409030+0000","last_peered":"2024-09-10T20:37:44.409030+0000","last_clean":"2024-09-10T20:37:44.409030+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:37:44.409030+0000","last_undegraded":"2024-09-10T20:37:44.409030+0000","last_fullsized":"2024-09-10T20:37:44.409030+0000","mapping_epoch":34,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11755520,"num_objects":2870,"num_object_clones":0,"num_object_copies":5740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14937,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'17515","reported_seq":17715,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:44.340816+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:37:44.340816+0000","last_peered":"2024-09-10T20:37:44.340816+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:37:44.340816+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11976704,"num_objects":2924,"num_object_clones":0,"num_object_copies":5848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15508,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-573}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318059+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318059+0000","last_peered":"2024-09-10T20:36:30.318059+0000","last_clean":"2024-09-10T20:36:30.318059+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318059+0000","last_undegraded":"2024-09-10T20:36:30.318059+0000","last_fullsized":"2024-09-10T20:36:30.318059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'17869","reported_seq":18086,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:44.383516+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:37:44.383516+0000","last_peered":"2024-09-10T20:37:44.383516+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:37:44.383516+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11719622,"num_objects":2863,"num_object_clones":0,"num_object_copies":5726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17867,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-703}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'17668","reported_seq":17861,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:44.352866+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:37:44.352866+0000","last_peered":"2024-09-10T20:37:44.352866+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:37:44.352866+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11866112,"num_objects":2897,"num_object_clones":0,"num_object_copies":5794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15496,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-709}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'17816","reported_seq":25186,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:44.442098+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:37:44.442098+0000","last_peered":"2024-09-10T20:37:44.442098+0000","last_clean":"2024-09-10T20:37:44.442098+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:37:44.442098+0000","last_undegraded":"2024-09-10T20:37:44.442098+0000","last_fullsized":"2024-09-10T20:37:44.442098+0000","mapping_epoch":48,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11665408,"num_objects":2848,"num_object_clones":0,"num_object_copies":5696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15602,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'17880","reported_seq":18133,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:44.320791+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:37:44.320791+0000","last_peered":"2024-09-10T20:37:44.320791+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:37:44.320791+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12083200,"num_objects":2950,"num_object_clones":0,"num_object_copies":5900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14603,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-642}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'17760","reported_seq":21313,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:44.399127+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:37:44.399127+0000","last_peered":"2024-09-10T20:37:44.399127+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:37:44.399127+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11567104,"num_objects":2824,"num_object_clones":0,"num_object_copies":5648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1833,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14454,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1702,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":991}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319644+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319644+0000","last_peered":"2024-09-10T20:36:30.319644+0000","last_clean":"2024-09-10T20:36:30.319644+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319644+0000","last_undegraded":"2024-09-10T20:36:30.319644+0000","last_fullsized":"2024-09-10T20:36:30.319644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18054","reported_seq":18312,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:44.310355+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:37:44.310355+0000","last_peered":"2024-09-10T20:37:44.310355+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:37:44.310355+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12009472,"num_objects":2932,"num_object_clones":0,"num_object_copies":5864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2932,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14881,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-723}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'17700","reported_seq":24827,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:46.276277+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:37:46.276277+0000","last_peered":"2024-09-10T20:37:46.276277+0000","last_clean":"2024-09-10T20:37:46.276277+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:37:46.276277+0000","last_undegraded":"2024-09-10T20:37:46.276277+0000","last_fullsized":"2024-09-10T20:37:46.276277+0000","mapping_epoch":46,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11534819,"num_objects":2817,"num_object_clones":0,"num_object_copies":5634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14307,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":189859265,"num_objects":46356,"num_object_clones":0,"num_object_copies":92712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30350,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46356,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":249061,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12799,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":554319872,"data_stored":554253920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160765,"ondisk_log_size":160765,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182510,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627064,"kb_used_data":198368,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93744776,"statfs":{"total":96636764160,"available":95994650624,"internally_reserved":0,"allocated":203128832,"data_stored":202866884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":438955690},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986282,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":556136,"kb_used_data":209304,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93815704,"statfs":{"total":96636764160,"available":96067280896,"internally_reserved":0,"allocated":214327296,"data_stored":214065353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355135146},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659229,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566052,"kb_used_data":137296,"kb_used_omap":4,"kb_used_meta":428731,"kb_avail":93805788,"statfs":{"total":96636764160,"available":96057126912,"internally_reserved":0,"allocated":140591104,"data_stored":140278115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":439021230},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,3,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":201781248,"data_stored":201777635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139390976,"data_stored":139336322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":213147648,"data_stored":213139963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:37:53.651 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:37:54.202 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:54.203 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:37:54.229 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2024-09-10T20:37:53.016386+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":179725780,"num_objects":43884,"num_object_clones":0,"num_object_copies":87768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30224,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43884,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":251537,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12927,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160642,"ondisk_log_size":160642,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1750416,"kb_used_data":525332,"kb_used_omap":12,"kb_used_meta":1225011,"kb_avail":281365104,"statfs":{"total":289910292480,"available":288117866496,"internally_reserved":0,"allocated":537939968,"data_stored":537103088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1254411266},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,5,0,2,10,7,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21639168,"num_objects":-5283,"num_object_clones":0,"num_object_copies":-10566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-240,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5283,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003669"},"pg_stats":[{"pgid":"2.f","version":"49'17908","reported_seq":18128,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:49.397258+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:37:49.397258+0000","last_peered":"2024-09-10T20:37:49.397258+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:37:49.397258+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11264000,"num_objects":2750,"num_object_clones":0,"num_object_copies":5500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17908,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-791}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18192","reported_seq":18312,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:49.469755+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:37:49.469755+0000","last_peered":"2024-09-10T20:37:49.469755+0000","last_clean":"2024-09-10T20:37:49.469755+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:37:49.469755+0000","last_undegraded":"2024-09-10T20:37:49.469755+0000","last_fullsized":"2024-09-10T20:37:49.469755+0000","mapping_epoch":35,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11264000,"num_objects":2750,"num_object_clones":0,"num_object_copies":5500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16378,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18021","reported_seq":25379,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:51.290865+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:37:51.290865+0000","last_peered":"2024-09-10T20:37:51.290865+0000","last_clean":"2024-09-10T20:37:51.290865+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:37:51.290865+0000","last_undegraded":"2024-09-10T20:37:51.290865+0000","last_fullsized":"2024-09-10T20:37:51.290865+0000","mapping_epoch":44,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11034648,"num_objects":2695,"num_object_clones":0,"num_object_copies":5390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2695,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16149,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'17756","reported_seq":17846,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:49.363983+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:37:49.363983+0000","last_peered":"2024-09-10T20:37:49.363983+0000","last_clean":"2024-09-10T20:37:49.363983+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:37:49.363983+0000","last_undegraded":"2024-09-10T20:37:49.363983+0000","last_fullsized":"2024-09-10T20:37:49.363983+0000","mapping_epoch":35,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11665408,"num_objects":2848,"num_object_clones":0,"num_object_copies":5696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17756,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'18263","reported_seq":18399,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:49.410556+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:37:49.410556+0000","last_peered":"2024-09-10T20:37:49.410556+0000","last_clean":"2024-09-10T20:37:49.410556+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:37:49.410556+0000","last_undegraded":"2024-09-10T20:37:49.410556+0000","last_fullsized":"2024-09-10T20:37:49.410556+0000","mapping_epoch":34,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11493376,"num_objects":2806,"num_object_clones":0,"num_object_copies":5612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15259,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'17754","reported_seq":18007,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:49.420655+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:37:49.420655+0000","last_peered":"2024-09-10T20:37:49.420655+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:37:49.420655+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11128832,"num_objects":2717,"num_object_clones":0,"num_object_copies":5434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14618,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-844}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'17810","reported_seq":18096,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:49.349954+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:37:49.349954+0000","last_peered":"2024-09-10T20:37:49.349954+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:37:49.349954+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11460608,"num_objects":2798,"num_object_clones":0,"num_object_copies":5596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14405,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-688}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18019","reported_seq":18154,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:49.391861+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:37:49.391861+0000","last_peered":"2024-09-10T20:37:49.391861+0000","last_clean":"2024-09-10T20:37:49.391861+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:37:49.391861+0000","last_undegraded":"2024-09-10T20:37:49.391861+0000","last_fullsized":"2024-09-10T20:37:49.391861+0000","mapping_epoch":34,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11128832,"num_objects":2717,"num_object_clones":0,"num_object_copies":5434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15090,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'17687","reported_seq":17884,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:49.471822+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:37:49.471822+0000","last_peered":"2024-09-10T20:37:49.471822+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:37:49.471822+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11272192,"num_objects":2752,"num_object_clones":0,"num_object_copies":5504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15680,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-745}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318059+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318059+0000","last_peered":"2024-09-10T20:36:30.318059+0000","last_clean":"2024-09-10T20:36:30.318059+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318059+0000","last_undegraded":"2024-09-10T20:36:30.318059+0000","last_fullsized":"2024-09-10T20:36:30.318059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18027","reported_seq":18244,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:49.360029+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:37:49.360029+0000","last_peered":"2024-09-10T20:37:49.360029+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:37:49.360029+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11072454,"num_objects":2705,"num_object_clones":0,"num_object_copies":5410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18025,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-861}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'17829","reported_seq":18022,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:49.423180+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:37:49.423180+0000","last_peered":"2024-09-10T20:37:49.423180+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:37:49.423180+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11206656,"num_objects":2736,"num_object_clones":0,"num_object_copies":5472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15657,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-870}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'17970","reported_seq":25341,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:49.452194+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:37:49.452194+0000","last_peered":"2024-09-10T20:37:49.452194+0000","last_clean":"2024-09-10T20:37:49.452194+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:37:49.452194+0000","last_undegraded":"2024-09-10T20:37:49.452194+0000","last_fullsized":"2024-09-10T20:37:49.452194+0000","mapping_epoch":48,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11034624,"num_objects":2694,"num_object_clones":0,"num_object_copies":5388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15756,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18050","reported_seq":18302,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:49.431273+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:37:49.431273+0000","last_peered":"2024-09-10T20:37:49.431273+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:37:49.431273+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11386880,"num_objects":2780,"num_object_clones":0,"num_object_copies":5560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14773,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-812}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'17906","reported_seq":21712,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:49.420234+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:37:49.420234+0000","last_peered":"2024-09-10T20:37:49.420234+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:37:49.420234+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10969088,"num_objects":2678,"num_object_clones":0,"num_object_copies":5356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1707,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2678,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14600,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1828,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":971}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319644+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319644+0000","last_peered":"2024-09-10T20:36:30.319644+0000","last_clean":"2024-09-10T20:36:30.319644+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319644+0000","last_undegraded":"2024-09-10T20:36:30.319644+0000","last_fullsized":"2024-09-10T20:36:30.319644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18206","reported_seq":18464,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:49.292293+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:37:49.292293+0000","last_peered":"2024-09-10T20:37:49.292293+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:37:49.292293+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11386880,"num_objects":2780,"num_object_clones":0,"num_object_copies":5560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15033,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-875}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'17841","reported_seq":24968,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:51.352471+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:37:51.352471+0000","last_peered":"2024-09-10T20:37:51.352471+0000","last_clean":"2024-09-10T20:37:51.352471+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:37:51.352471+0000","last_undegraded":"2024-09-10T20:37:51.352471+0000","last_fullsized":"2024-09-10T20:37:51.352471+0000","mapping_epoch":46,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10957283,"num_objects":2676,"num_object_clones":0,"num_object_copies":5352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2676,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14448,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":179725761,"num_objects":43882,"num_object_clones":0,"num_object_copies":87764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30224,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43882,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":251535,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12925,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":534204416,"data_stored":534138464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160639,"ondisk_log_size":160639,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182511,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627868,"kb_used_data":190856,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93743972,"statfs":{"total":96636764160,"available":95993827328,"internally_reserved":0,"allocated":195436544,"data_stored":195174596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,0,1,2,3,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986283,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":555480,"kb_used_data":204488,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":93816360,"statfs":{"total":96636764160,"available":96067952640,"internally_reserved":0,"allocated":209395712,"data_stored":209133769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":359394986},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659230,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":567068,"kb_used_data":129988,"kb_used_omap":4,"kb_used_meta":437051,"kb_avail":93804772,"statfs":{"total":96636764160,"available":96056086528,"internally_reserved":0,"allocated":133107712,"data_stored":132794723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":447540910},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,6,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":194256896,"data_stored":194253283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":131829760,"data_stored":131775106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":208117760,"data_stored":208110075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:37:54.230 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:37:54.806 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:54.806 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:37:54.831 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2024-09-10T20:37:53.016386+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":179725780,"num_objects":43884,"num_object_clones":0,"num_object_copies":87768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30224,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43884,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":251537,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12927,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160642,"ondisk_log_size":160642,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1750416,"kb_used_data":525332,"kb_used_omap":12,"kb_used_meta":1225011,"kb_avail":281365104,"statfs":{"total":289910292480,"available":288117866496,"internally_reserved":0,"allocated":537939968,"data_stored":537103088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1254411266},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,5,0,2,10,7,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21639168,"num_objects":-5283,"num_object_clones":0,"num_object_copies":-10566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-240,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5283,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003669"},"pg_stats":[{"pgid":"2.f","version":"49'17908","reported_seq":18128,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:49.397258+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:37:49.397258+0000","last_peered":"2024-09-10T20:37:49.397258+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:37:49.397258+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11264000,"num_objects":2750,"num_object_clones":0,"num_object_copies":5500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17908,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-791}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18192","reported_seq":18312,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:49.469755+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:37:49.469755+0000","last_peered":"2024-09-10T20:37:49.469755+0000","last_clean":"2024-09-10T20:37:49.469755+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:37:49.469755+0000","last_undegraded":"2024-09-10T20:37:49.469755+0000","last_fullsized":"2024-09-10T20:37:49.469755+0000","mapping_epoch":35,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11264000,"num_objects":2750,"num_object_clones":0,"num_object_copies":5500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16378,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18021","reported_seq":25379,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:51.290865+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:37:51.290865+0000","last_peered":"2024-09-10T20:37:51.290865+0000","last_clean":"2024-09-10T20:37:51.290865+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:37:51.290865+0000","last_undegraded":"2024-09-10T20:37:51.290865+0000","last_fullsized":"2024-09-10T20:37:51.290865+0000","mapping_epoch":44,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11034648,"num_objects":2695,"num_object_clones":0,"num_object_copies":5390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2695,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16149,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'17756","reported_seq":17846,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:49.363983+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:37:49.363983+0000","last_peered":"2024-09-10T20:37:49.363983+0000","last_clean":"2024-09-10T20:37:49.363983+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:37:49.363983+0000","last_undegraded":"2024-09-10T20:37:49.363983+0000","last_fullsized":"2024-09-10T20:37:49.363983+0000","mapping_epoch":35,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11665408,"num_objects":2848,"num_object_clones":0,"num_object_copies":5696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17756,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'18263","reported_seq":18399,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:49.410556+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:37:49.410556+0000","last_peered":"2024-09-10T20:37:49.410556+0000","last_clean":"2024-09-10T20:37:49.410556+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:37:49.410556+0000","last_undegraded":"2024-09-10T20:37:49.410556+0000","last_fullsized":"2024-09-10T20:37:49.410556+0000","mapping_epoch":34,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11493376,"num_objects":2806,"num_object_clones":0,"num_object_copies":5612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15259,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'17754","reported_seq":18007,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:49.420655+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:37:49.420655+0000","last_peered":"2024-09-10T20:37:49.420655+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:37:49.420655+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11128832,"num_objects":2717,"num_object_clones":0,"num_object_copies":5434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14618,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-844}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'17810","reported_seq":18096,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:49.349954+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:37:49.349954+0000","last_peered":"2024-09-10T20:37:49.349954+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:37:49.349954+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11460608,"num_objects":2798,"num_object_clones":0,"num_object_copies":5596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14405,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-688}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18019","reported_seq":18154,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:49.391861+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:37:49.391861+0000","last_peered":"2024-09-10T20:37:49.391861+0000","last_clean":"2024-09-10T20:37:49.391861+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:37:49.391861+0000","last_undegraded":"2024-09-10T20:37:49.391861+0000","last_fullsized":"2024-09-10T20:37:49.391861+0000","mapping_epoch":34,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11128832,"num_objects":2717,"num_object_clones":0,"num_object_copies":5434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15090,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'17687","reported_seq":17884,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:49.471822+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:37:49.471822+0000","last_peered":"2024-09-10T20:37:49.471822+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:37:49.471822+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11272192,"num_objects":2752,"num_object_clones":0,"num_object_copies":5504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15680,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-745}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318059+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318059+0000","last_peered":"2024-09-10T20:36:30.318059+0000","last_clean":"2024-09-10T20:36:30.318059+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318059+0000","last_undegraded":"2024-09-10T20:36:30.318059+0000","last_fullsized":"2024-09-10T20:36:30.318059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18027","reported_seq":18244,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:49.360029+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:37:49.360029+0000","last_peered":"2024-09-10T20:37:49.360029+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:37:49.360029+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11072454,"num_objects":2705,"num_object_clones":0,"num_object_copies":5410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18025,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-861}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'17829","reported_seq":18022,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:49.423180+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:37:49.423180+0000","last_peered":"2024-09-10T20:37:49.423180+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:37:49.423180+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11206656,"num_objects":2736,"num_object_clones":0,"num_object_copies":5472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15657,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-870}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'17970","reported_seq":25341,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:49.452194+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:37:49.452194+0000","last_peered":"2024-09-10T20:37:49.452194+0000","last_clean":"2024-09-10T20:37:49.452194+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:37:49.452194+0000","last_undegraded":"2024-09-10T20:37:49.452194+0000","last_fullsized":"2024-09-10T20:37:49.452194+0000","mapping_epoch":48,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11034624,"num_objects":2694,"num_object_clones":0,"num_object_copies":5388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15756,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18050","reported_seq":18302,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:49.431273+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:37:49.431273+0000","last_peered":"2024-09-10T20:37:49.431273+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:37:49.431273+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11386880,"num_objects":2780,"num_object_clones":0,"num_object_copies":5560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14773,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-812}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'17906","reported_seq":21712,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:49.420234+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:37:49.420234+0000","last_peered":"2024-09-10T20:37:49.420234+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:37:49.420234+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10969088,"num_objects":2678,"num_object_clones":0,"num_object_copies":5356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1707,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2678,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14600,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1828,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":971}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319644+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319644+0000","last_peered":"2024-09-10T20:36:30.319644+0000","last_clean":"2024-09-10T20:36:30.319644+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319644+0000","last_undegraded":"2024-09-10T20:36:30.319644+0000","last_fullsized":"2024-09-10T20:36:30.319644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18206","reported_seq":18464,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:49.292293+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:37:49.292293+0000","last_peered":"2024-09-10T20:37:49.292293+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:37:49.292293+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11386880,"num_objects":2780,"num_object_clones":0,"num_object_copies":5560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15033,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-875}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'17841","reported_seq":24968,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:51.352471+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:37:51.352471+0000","last_peered":"2024-09-10T20:37:51.352471+0000","last_clean":"2024-09-10T20:37:51.352471+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:37:51.352471+0000","last_undegraded":"2024-09-10T20:37:51.352471+0000","last_fullsized":"2024-09-10T20:37:51.352471+0000","mapping_epoch":46,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10957283,"num_objects":2676,"num_object_clones":0,"num_object_copies":5352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2676,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14448,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":179725761,"num_objects":43882,"num_object_clones":0,"num_object_copies":87764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30224,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43882,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":251535,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12925,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":534204416,"data_stored":534138464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160639,"ondisk_log_size":160639,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182511,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627868,"kb_used_data":190856,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93743972,"statfs":{"total":96636764160,"available":95993827328,"internally_reserved":0,"allocated":195436544,"data_stored":195174596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,0,1,2,3,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986283,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":555480,"kb_used_data":204488,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":93816360,"statfs":{"total":96636764160,"available":96067952640,"internally_reserved":0,"allocated":209395712,"data_stored":209133769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":359394986},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659230,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":567068,"kb_used_data":129988,"kb_used_omap":4,"kb_used_meta":437051,"kb_avail":93804772,"statfs":{"total":96636764160,"available":96056086528,"internally_reserved":0,"allocated":133107712,"data_stored":132794723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":447540910},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,6,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":194256896,"data_stored":194253283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":131829760,"data_stored":131775106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":208117760,"data_stored":208110075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:37:54.832 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:37:54.833 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'17908", 'reported_seq': 18128, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:49.397258+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:37:49.397258+0000', 'last_peered': '2024-09-10T20:37:49.397258+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:37:49.397258+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "43'7900", 'ondisk_log_start': "43'7900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11264000, 'num_objects': 2750, 'num_object_clones': 0, 'num_object_copies': 5500, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2750, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17908, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -791}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:54.833 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:37:54.833 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'17754", 'reported_seq': 18007, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:49.420655+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:37:49.420655+0000', 'last_peered': '2024-09-10T20:37:49.420655+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:37:49.420655+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "43'7700", 'ondisk_log_start': "43'7700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11128832, 'num_objects': 2717, 'num_object_clones': 0, 'num_object_copies': 5434, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2717, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14618, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -844}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:54.833 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:37:54.833 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'17810", 'reported_seq': 18096, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:49.349954+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:37:49.349954+0000', 'last_peered': '2024-09-10T20:37:49.349954+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:37:49.349954+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "43'7800", 'ondisk_log_start': "43'7800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11460608, 'num_objects': 2798, 'num_object_clones': 0, 'num_object_copies': 5596, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2798, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14405, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -688}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:54.833 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:37:54.833 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'17687", 'reported_seq': 17884, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:49.471822+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:37:49.471822+0000', 'last_peered': '2024-09-10T20:37:49.471822+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:37:49.471822+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "43'7600", 'ondisk_log_start': "43'7600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11272192, 'num_objects': 2752, 'num_object_clones': 0, 'num_object_copies': 5504, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2752, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15680, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -745}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:37:54.833 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:37:54.833 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'18027", 'reported_seq': 18244, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:49.360029+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:37:49.360029+0000', 'last_peered': '2024-09-10T20:37:49.360029+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:37:49.360029+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "43'8000", 'ondisk_log_start': "43'8000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11072454, 'num_objects': 2705, 'num_object_clones': 0, 'num_object_copies': 5410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2705, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18025, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -861}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:54.833 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:37:54.834 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'17829", 'reported_seq': 18022, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:49.423180+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:37:49.423180+0000', 'last_peered': '2024-09-10T20:37:49.423180+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:37:49.423180+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "43'7800", 'ondisk_log_start': "43'7800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11206656, 'num_objects': 2736, 'num_object_clones': 0, 'num_object_copies': 5472, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2736, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15657, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -870}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:37:54.834 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:37:54.834 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'18050", 'reported_seq': 18302, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:49.431273+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:37:49.431273+0000', 'last_peered': '2024-09-10T20:37:49.431273+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:37:49.431273+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "43'8000", 'ondisk_log_start': "43'8000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11386880, 'num_objects': 2780, 'num_object_clones': 0, 'num_object_copies': 5560, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2780, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14773, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -812}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:54.834 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:37:54.834 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'17906", 'reported_seq': 21712, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:49.420234+0000', 'last_change': '2024-09-10T20:36:28.778006+0000', 'last_active': '2024-09-10T20:37:49.420234+0000', 'last_peered': '2024-09-10T20:37:49.420234+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:37:49.420234+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "43'7900", 'ondisk_log_start': "43'7900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10969088, 'num_objects': 2678, 'num_object_clones': 0, 'num_object_copies': 5356, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1707, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2678, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14600, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1828, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 971}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:54.834 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:37:54.834 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'18206", 'reported_seq': 18464, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:49.292293+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:37:49.292293+0000', 'last_peered': '2024-09-10T20:37:49.292293+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:37:49.292293+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "43'8200", 'ondisk_log_start': "43'8200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11386880, 'num_objects': 2780, 'num_object_clones': 0, 'num_object_copies': 5560, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2780, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15033, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -875}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:37:54.834 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:37:55.573 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:55.573 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 30089/83446 objects degraded (36.058%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":587,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":41723,"data_bytes":170874324,"bytes_used":1754206208,"bytes_avail":288156086272,"bytes_total":289910292480,"degraded_objects":30089,"degraded_total":83446,"degraded_ratio":0.36058049517052942,"recovering_objects_per_sec":31,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":375,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":592},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:37:55.596 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:37:55.597 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:37:56.210 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:56.210 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:37:56.236 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-09-10T20:37:55.017083+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":170874324,"num_objects":41723,"num_object_clones":0,"num_object_copies":83446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41723,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":253698,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13062,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160903,"ondisk_log_size":160903,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1713092,"kb_used_data":512964,"kb_used_omap":12,"kb_used_meta":1200051,"kb_avail":281402428,"statfs":{"total":289910292480,"available":288156086272,"internally_reserved":0,"allocated":525275136,"data_stored":524438256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1228852226},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,13,13,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29134848,"num_objects":-7113,"num_object_clones":0,"num_object_copies":-14226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-375,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7113,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":375,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003886"},"pg_stats":[{"pgid":"2.f","version":"49'18077","reported_seq":18299,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:54.429450+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:37:54.429450+0000","last_peered":"2024-09-10T20:37:54.429450+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:37:54.429450+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10571776,"num_objects":2581,"num_object_clones":0,"num_object_copies":5162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18077,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-960}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18333","reported_seq":18453,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:54.423339+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:37:54.423339+0000","last_peered":"2024-09-10T20:37:54.423339+0000","last_clean":"2024-09-10T20:37:54.423339+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:37:54.423339+0000","last_undegraded":"2024-09-10T20:37:54.423339+0000","last_fullsized":"2024-09-10T20:37:54.423339+0000","mapping_epoch":35,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10686464,"num_objects":2609,"num_object_clones":0,"num_object_copies":5218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16519,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18021","reported_seq":25379,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:51.290865+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:37:51.290865+0000","last_peered":"2024-09-10T20:37:51.290865+0000","last_clean":"2024-09-10T20:37:51.290865+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:37:51.290865+0000","last_undegraded":"2024-09-10T20:37:51.290865+0000","last_fullsized":"2024-09-10T20:37:51.290865+0000","mapping_epoch":44,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11034648,"num_objects":2695,"num_object_clones":0,"num_object_copies":5390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2695,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16149,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'17909","reported_seq":17999,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:54.472845+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:37:54.472845+0000","last_peered":"2024-09-10T20:37:54.472845+0000","last_clean":"2024-09-10T20:37:54.472845+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:37:54.472845+0000","last_undegraded":"2024-09-10T20:37:54.472845+0000","last_fullsized":"2024-09-10T20:37:54.472845+0000","mapping_epoch":35,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11038720,"num_objects":2695,"num_object_clones":0,"num_object_copies":5390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17909,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'18430","reported_seq":18566,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:54.469823+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:37:54.469823+0000","last_peered":"2024-09-10T20:37:54.469823+0000","last_clean":"2024-09-10T20:37:54.469823+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:37:54.469823+0000","last_undegraded":"2024-09-10T20:37:54.469823+0000","last_fullsized":"2024-09-10T20:37:54.469823+0000","mapping_epoch":34,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10809344,"num_objects":2639,"num_object_clones":0,"num_object_copies":5278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15426,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'17898","reported_seq":18151,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:54.457179+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:37:54.457179+0000","last_peered":"2024-09-10T20:37:54.457179+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:37:54.457179+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10539008,"num_objects":2573,"num_object_clones":0,"num_object_copies":5146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14762,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-988}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'17973","reported_seq":18259,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:54.480148+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:37:54.480148+0000","last_peered":"2024-09-10T20:37:54.480148+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:37:54.480148+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10792960,"num_objects":2635,"num_object_clones":0,"num_object_copies":5270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14568,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-851}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18166","reported_seq":18301,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:54.359957+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:37:54.359957+0000","last_peered":"2024-09-10T20:37:54.359957+0000","last_clean":"2024-09-10T20:37:54.359957+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:37:54.359957+0000","last_undegraded":"2024-09-10T20:37:54.359957+0000","last_fullsized":"2024-09-10T20:37:54.359957+0000","mapping_epoch":34,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10526720,"num_objects":2570,"num_object_clones":0,"num_object_copies":5140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15237,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'17833","reported_seq":18033,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:54.398916+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:37:54.398916+0000","last_peered":"2024-09-10T20:37:54.398916+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:37:54.398916+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10674176,"num_objects":2606,"num_object_clones":0,"num_object_copies":5212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15826,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-891}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318059+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318059+0000","last_peered":"2024-09-10T20:36:30.318059+0000","last_clean":"2024-09-10T20:36:30.318059+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318059+0000","last_undegraded":"2024-09-10T20:36:30.318059+0000","last_fullsized":"2024-09-10T20:36:30.318059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18165","reported_seq":18382,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:54.446971+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:37:54.446971+0000","last_peered":"2024-09-10T20:37:54.446971+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:37:54.446971+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10507206,"num_objects":2567,"num_object_clones":0,"num_object_copies":5134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18163,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-999}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'17983","reported_seq":18174,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:54.384406+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:37:54.384406+0000","last_peered":"2024-09-10T20:37:54.384406+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:37:54.384406+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10575872,"num_objects":2582,"num_object_clones":0,"num_object_copies":5164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15811,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18130","reported_seq":25501,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:54.414089+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:37:54.414089+0000","last_peered":"2024-09-10T20:37:54.414089+0000","last_clean":"2024-09-10T20:37:54.414089+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:37:54.414089+0000","last_undegraded":"2024-09-10T20:37:54.414089+0000","last_fullsized":"2024-09-10T20:37:54.414089+0000","mapping_epoch":48,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10379264,"num_objects":2534,"num_object_clones":0,"num_object_copies":5068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15916,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18191","reported_seq":18448,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:54.378841+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:37:54.378841+0000","last_peered":"2024-09-10T20:37:54.378841+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:37:54.378841+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10809344,"num_objects":2639,"num_object_clones":0,"num_object_copies":5278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14914,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-953}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18064","reported_seq":22140,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:54.459409+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:37:54.459409+0000","last_peered":"2024-09-10T20:37:54.459409+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:37:54.459409+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10321920,"num_objects":2520,"num_object_clones":0,"num_object_copies":5040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1572,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14758,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1963,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":948}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319644+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319644+0000","last_peered":"2024-09-10T20:36:30.319644+0000","last_clean":"2024-09-10T20:36:30.319644+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319644+0000","last_undegraded":"2024-09-10T20:36:30.319644+0000","last_fullsized":"2024-09-10T20:36:30.319644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18386","reported_seq":18644,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:54.380882+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:37:54.380882+0000","last_peered":"2024-09-10T20:37:54.380882+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:37:54.380882+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10649600,"num_objects":2600,"num_object_clones":0,"num_object_copies":5200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15213,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1055}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'17841","reported_seq":24968,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:51.352471+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:37:51.352471+0000","last_peered":"2024-09-10T20:37:51.352471+0000","last_clean":"2024-09-10T20:37:51.352471+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:37:51.352471+0000","last_undegraded":"2024-09-10T20:37:51.352471+0000","last_fullsized":"2024-09-10T20:37:51.352471+0000","mapping_epoch":46,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10957283,"num_objects":2676,"num_object_clones":0,"num_object_copies":5352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2676,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14448,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":170874305,"num_objects":41721,"num_object_clones":0,"num_object_copies":83442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41721,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":253696,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13060,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":521568256,"data_stored":521502304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160900,"ondisk_log_size":160900,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182512,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608032,"kb_used_data":183500,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93763808,"statfs":{"total":96636764160,"available":96014139392,"internally_reserved":0,"allocated":187904000,"data_stored":187642052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":434695850},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,7,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986284,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537992,"kb_used_data":199476,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93833848,"statfs":{"total":96636764160,"available":96085860352,"internally_reserved":0,"allocated":204263424,"data_stored":204001481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346615466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659230,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":567068,"kb_used_data":129988,"kb_used_omap":4,"kb_used_meta":437051,"kb_avail":93804772,"statfs":{"total":96636764160,"available":96056086528,"internally_reserved":0,"allocated":133107712,"data_stored":132794723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":447540910},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,6,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":186683392,"data_stored":186679779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":131829760,"data_stored":131775106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":203055104,"data_stored":203047419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:37:59.238 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:37:59.810 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:37:59.812 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:37:59.835 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":338,"stamp":"2024-09-10T20:37:59.018391+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":169686484,"num_objects":41433,"num_object_clones":0,"num_object_copies":82866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41433,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":253988,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13062,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160993,"ondisk_log_size":160993,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1693236,"kb_used_data":505588,"kb_used_omap":12,"kb_used_meta":1187571,"kb_avail":281422284,"statfs":{"total":289910292480,"available":288176418816,"internally_reserved":0,"allocated":517722112,"data_stored":516885232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1216072706},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,2,9,12,11,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20172800,"num_objects":-4925,"num_object_clones":0,"num_object_copies":-9850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-261,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4925,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003962"},"pg_stats":[{"pgid":"2.f","version":"49'18077","reported_seq":18299,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:54.429450+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:37:54.429450+0000","last_peered":"2024-09-10T20:37:54.429450+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:37:54.429450+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10571776,"num_objects":2581,"num_object_clones":0,"num_object_copies":5162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18077,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-960}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18333","reported_seq":18453,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:54.423339+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:37:54.423339+0000","last_peered":"2024-09-10T20:37:54.423339+0000","last_clean":"2024-09-10T20:37:54.423339+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:37:54.423339+0000","last_undegraded":"2024-09-10T20:37:54.423339+0000","last_fullsized":"2024-09-10T20:37:54.423339+0000","mapping_epoch":35,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10686464,"num_objects":2609,"num_object_clones":0,"num_object_copies":5218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16519,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18176","reported_seq":25534,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:56.165548+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:37:56.165548+0000","last_peered":"2024-09-10T20:37:56.165548+0000","last_clean":"2024-09-10T20:37:56.165548+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:37:56.165548+0000","last_undegraded":"2024-09-10T20:37:56.165548+0000","last_fullsized":"2024-09-10T20:37:56.165548+0000","mapping_epoch":44,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10399768,"num_objects":2540,"num_object_clones":0,"num_object_copies":5080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2540,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16304,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'17909","reported_seq":17999,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:54.472845+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:37:54.472845+0000","last_peered":"2024-09-10T20:37:54.472845+0000","last_clean":"2024-09-10T20:37:54.472845+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:37:54.472845+0000","last_undegraded":"2024-09-10T20:37:54.472845+0000","last_fullsized":"2024-09-10T20:37:54.472845+0000","mapping_epoch":35,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11038720,"num_objects":2695,"num_object_clones":0,"num_object_copies":5390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17909,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'18430","reported_seq":18566,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:54.469823+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:37:54.469823+0000","last_peered":"2024-09-10T20:37:54.469823+0000","last_clean":"2024-09-10T20:37:54.469823+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:37:54.469823+0000","last_undegraded":"2024-09-10T20:37:54.469823+0000","last_fullsized":"2024-09-10T20:37:54.469823+0000","mapping_epoch":34,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10809344,"num_objects":2639,"num_object_clones":0,"num_object_copies":5278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15426,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'17898","reported_seq":18151,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:54.457179+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:37:54.457179+0000","last_peered":"2024-09-10T20:37:54.457179+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:37:54.457179+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10539008,"num_objects":2573,"num_object_clones":0,"num_object_copies":5146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14762,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-988}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'17973","reported_seq":18259,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:54.480148+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:37:54.480148+0000","last_peered":"2024-09-10T20:37:54.480148+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:37:54.480148+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10792960,"num_objects":2635,"num_object_clones":0,"num_object_copies":5270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14568,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-851}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18166","reported_seq":18301,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:54.359957+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:37:54.359957+0000","last_peered":"2024-09-10T20:37:54.359957+0000","last_clean":"2024-09-10T20:37:54.359957+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:37:54.359957+0000","last_undegraded":"2024-09-10T20:37:54.359957+0000","last_fullsized":"2024-09-10T20:37:54.359957+0000","mapping_epoch":34,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10526720,"num_objects":2570,"num_object_clones":0,"num_object_copies":5140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15237,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'17833","reported_seq":18033,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:54.398916+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:37:54.398916+0000","last_peered":"2024-09-10T20:37:54.398916+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:37:54.398916+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10674176,"num_objects":2606,"num_object_clones":0,"num_object_copies":5212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15826,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-891}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318059+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318059+0000","last_peered":"2024-09-10T20:36:30.318059+0000","last_clean":"2024-09-10T20:36:30.318059+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318059+0000","last_undegraded":"2024-09-10T20:36:30.318059+0000","last_fullsized":"2024-09-10T20:36:30.318059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18165","reported_seq":18382,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:54.446971+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:37:54.446971+0000","last_peered":"2024-09-10T20:37:54.446971+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:37:54.446971+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10507206,"num_objects":2567,"num_object_clones":0,"num_object_copies":5134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18163,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-999}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'17983","reported_seq":18174,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:54.384406+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:37:54.384406+0000","last_peered":"2024-09-10T20:37:54.384406+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:37:54.384406+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10575872,"num_objects":2582,"num_object_clones":0,"num_object_copies":5164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15811,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18130","reported_seq":25501,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:54.414089+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:37:54.414089+0000","last_peered":"2024-09-10T20:37:54.414089+0000","last_clean":"2024-09-10T20:37:54.414089+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:37:54.414089+0000","last_undegraded":"2024-09-10T20:37:54.414089+0000","last_fullsized":"2024-09-10T20:37:54.414089+0000","mapping_epoch":48,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10379264,"num_objects":2534,"num_object_clones":0,"num_object_copies":5068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15916,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18191","reported_seq":18448,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:54.378841+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:37:54.378841+0000","last_peered":"2024-09-10T20:37:54.378841+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:37:54.378841+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10809344,"num_objects":2639,"num_object_clones":0,"num_object_copies":5278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14914,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-953}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18064","reported_seq":22140,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:54.459409+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:37:54.459409+0000","last_peered":"2024-09-10T20:37:54.459409+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:37:54.459409+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10321920,"num_objects":2520,"num_object_clones":0,"num_object_copies":5040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1572,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14758,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1963,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":948}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319644+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319644+0000","last_peered":"2024-09-10T20:36:30.319644+0000","last_clean":"2024-09-10T20:36:30.319644+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319644+0000","last_undegraded":"2024-09-10T20:36:30.319644+0000","last_fullsized":"2024-09-10T20:36:30.319644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18386","reported_seq":18644,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:54.380882+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:37:54.380882+0000","last_peered":"2024-09-10T20:37:54.380882+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:37:54.380882+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10649600,"num_objects":2600,"num_object_clones":0,"num_object_copies":5200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15213,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1055}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'17976","reported_seq":25103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:56.388218+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:37:56.388218+0000","last_peered":"2024-09-10T20:37:56.388218+0000","last_clean":"2024-09-10T20:37:56.388218+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:37:56.388218+0000","last_undegraded":"2024-09-10T20:37:56.388218+0000","last_fullsized":"2024-09-10T20:37:56.388218+0000","mapping_epoch":46,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10404323,"num_objects":2541,"num_object_clones":0,"num_object_copies":5082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14583,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":169686465,"num_objects":41431,"num_object_clones":0,"num_object_copies":82862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41431,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":253986,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13060,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":514093056,"data_stored":514027104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160990,"ondisk_log_size":160990,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182512,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608032,"kb_used_data":183500,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93763808,"statfs":{"total":96636764160,"available":96014139392,"internally_reserved":0,"allocated":187904000,"data_stored":187642052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":434695850},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,7,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986284,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537992,"kb_used_data":199476,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93833848,"statfs":{"total":96636764160,"available":96085860352,"internally_reserved":0,"allocated":204263424,"data_stored":204001481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346615466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659231,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547212,"kb_used_data":122612,"kb_used_omap":4,"kb_used_meta":424571,"kb_avail":93824628,"statfs":{"total":96636764160,"available":96076419072,"internally_reserved":0,"allocated":125554688,"data_stored":125241699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":434761390},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":186683392,"data_stored":186679779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":124354560,"data_stored":124299906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":203055104,"data_stored":203047419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:37:59.836 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:38:00.854 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:00.857 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:38:00.878 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":338,"stamp":"2024-09-10T20:37:59.018391+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":169686484,"num_objects":41433,"num_object_clones":0,"num_object_copies":82866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41433,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":253988,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13062,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160993,"ondisk_log_size":160993,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1693236,"kb_used_data":505588,"kb_used_omap":12,"kb_used_meta":1187571,"kb_avail":281422284,"statfs":{"total":289910292480,"available":288176418816,"internally_reserved":0,"allocated":517722112,"data_stored":516885232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1216072706},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,2,9,12,11,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20172800,"num_objects":-4925,"num_object_clones":0,"num_object_copies":-9850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-261,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4925,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003962"},"pg_stats":[{"pgid":"2.f","version":"49'18077","reported_seq":18299,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:54.429450+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:37:54.429450+0000","last_peered":"2024-09-10T20:37:54.429450+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:37:54.429450+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10571776,"num_objects":2581,"num_object_clones":0,"num_object_copies":5162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18077,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-960}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18333","reported_seq":18453,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:54.423339+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:37:54.423339+0000","last_peered":"2024-09-10T20:37:54.423339+0000","last_clean":"2024-09-10T20:37:54.423339+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:37:54.423339+0000","last_undegraded":"2024-09-10T20:37:54.423339+0000","last_fullsized":"2024-09-10T20:37:54.423339+0000","mapping_epoch":35,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10686464,"num_objects":2609,"num_object_clones":0,"num_object_copies":5218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16519,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18176","reported_seq":25534,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:56.165548+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:37:56.165548+0000","last_peered":"2024-09-10T20:37:56.165548+0000","last_clean":"2024-09-10T20:37:56.165548+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:37:56.165548+0000","last_undegraded":"2024-09-10T20:37:56.165548+0000","last_fullsized":"2024-09-10T20:37:56.165548+0000","mapping_epoch":44,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10399768,"num_objects":2540,"num_object_clones":0,"num_object_copies":5080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2540,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16304,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'17909","reported_seq":17999,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:54.472845+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:37:54.472845+0000","last_peered":"2024-09-10T20:37:54.472845+0000","last_clean":"2024-09-10T20:37:54.472845+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:37:54.472845+0000","last_undegraded":"2024-09-10T20:37:54.472845+0000","last_fullsized":"2024-09-10T20:37:54.472845+0000","mapping_epoch":35,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11038720,"num_objects":2695,"num_object_clones":0,"num_object_copies":5390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17909,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'18430","reported_seq":18566,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:54.469823+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:37:54.469823+0000","last_peered":"2024-09-10T20:37:54.469823+0000","last_clean":"2024-09-10T20:37:54.469823+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:37:54.469823+0000","last_undegraded":"2024-09-10T20:37:54.469823+0000","last_fullsized":"2024-09-10T20:37:54.469823+0000","mapping_epoch":34,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10809344,"num_objects":2639,"num_object_clones":0,"num_object_copies":5278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15426,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'17898","reported_seq":18151,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:54.457179+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:37:54.457179+0000","last_peered":"2024-09-10T20:37:54.457179+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:37:54.457179+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10539008,"num_objects":2573,"num_object_clones":0,"num_object_copies":5146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14762,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-988}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'17973","reported_seq":18259,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:54.480148+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:37:54.480148+0000","last_peered":"2024-09-10T20:37:54.480148+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:37:54.480148+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10792960,"num_objects":2635,"num_object_clones":0,"num_object_copies":5270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14568,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-851}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18166","reported_seq":18301,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:54.359957+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:37:54.359957+0000","last_peered":"2024-09-10T20:37:54.359957+0000","last_clean":"2024-09-10T20:37:54.359957+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:37:54.359957+0000","last_undegraded":"2024-09-10T20:37:54.359957+0000","last_fullsized":"2024-09-10T20:37:54.359957+0000","mapping_epoch":34,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10526720,"num_objects":2570,"num_object_clones":0,"num_object_copies":5140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15237,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'17833","reported_seq":18033,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:54.398916+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:37:54.398916+0000","last_peered":"2024-09-10T20:37:54.398916+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:37:54.398916+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10674176,"num_objects":2606,"num_object_clones":0,"num_object_copies":5212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15826,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-891}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318059+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318059+0000","last_peered":"2024-09-10T20:36:30.318059+0000","last_clean":"2024-09-10T20:36:30.318059+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318059+0000","last_undegraded":"2024-09-10T20:36:30.318059+0000","last_fullsized":"2024-09-10T20:36:30.318059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18165","reported_seq":18382,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:54.446971+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:37:54.446971+0000","last_peered":"2024-09-10T20:37:54.446971+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:37:54.446971+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10507206,"num_objects":2567,"num_object_clones":0,"num_object_copies":5134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18163,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-999}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'17983","reported_seq":18174,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:54.384406+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:37:54.384406+0000","last_peered":"2024-09-10T20:37:54.384406+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:37:54.384406+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10575872,"num_objects":2582,"num_object_clones":0,"num_object_copies":5164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15811,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18130","reported_seq":25501,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:54.414089+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:37:54.414089+0000","last_peered":"2024-09-10T20:37:54.414089+0000","last_clean":"2024-09-10T20:37:54.414089+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:37:54.414089+0000","last_undegraded":"2024-09-10T20:37:54.414089+0000","last_fullsized":"2024-09-10T20:37:54.414089+0000","mapping_epoch":48,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10379264,"num_objects":2534,"num_object_clones":0,"num_object_copies":5068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15916,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18191","reported_seq":18448,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:54.378841+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:37:54.378841+0000","last_peered":"2024-09-10T20:37:54.378841+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:37:54.378841+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10809344,"num_objects":2639,"num_object_clones":0,"num_object_copies":5278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14914,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-953}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18064","reported_seq":22140,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:54.459409+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:37:54.459409+0000","last_peered":"2024-09-10T20:37:54.459409+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:37:54.459409+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10321920,"num_objects":2520,"num_object_clones":0,"num_object_copies":5040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1572,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14758,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1963,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":948}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319644+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319644+0000","last_peered":"2024-09-10T20:36:30.319644+0000","last_clean":"2024-09-10T20:36:30.319644+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319644+0000","last_undegraded":"2024-09-10T20:36:30.319644+0000","last_fullsized":"2024-09-10T20:36:30.319644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18386","reported_seq":18644,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:54.380882+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:37:54.380882+0000","last_peered":"2024-09-10T20:37:54.380882+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:37:54.380882+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10649600,"num_objects":2600,"num_object_clones":0,"num_object_copies":5200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15213,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1055}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'17976","reported_seq":25103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:56.388218+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:37:56.388218+0000","last_peered":"2024-09-10T20:37:56.388218+0000","last_clean":"2024-09-10T20:37:56.388218+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:37:56.388218+0000","last_undegraded":"2024-09-10T20:37:56.388218+0000","last_fullsized":"2024-09-10T20:37:56.388218+0000","mapping_epoch":46,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10404323,"num_objects":2541,"num_object_clones":0,"num_object_copies":5082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14583,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":169686465,"num_objects":41431,"num_object_clones":0,"num_object_copies":82862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41431,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":253986,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13060,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":514093056,"data_stored":514027104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160990,"ondisk_log_size":160990,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182512,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608032,"kb_used_data":183500,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93763808,"statfs":{"total":96636764160,"available":96014139392,"internally_reserved":0,"allocated":187904000,"data_stored":187642052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":434695850},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,7,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986284,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537992,"kb_used_data":199476,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93833848,"statfs":{"total":96636764160,"available":96085860352,"internally_reserved":0,"allocated":204263424,"data_stored":204001481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346615466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659231,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547212,"kb_used_data":122612,"kb_used_omap":4,"kb_used_meta":424571,"kb_avail":93824628,"statfs":{"total":96636764160,"available":96076419072,"internally_reserved":0,"allocated":125554688,"data_stored":125241699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":434761390},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":186683392,"data_stored":186679779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":124354560,"data_stored":124299906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":203055104,"data_stored":203047419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:38:00.880 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:38:00.880 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'18077", 'reported_seq': 18299, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:54.429450+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:37:54.429450+0000', 'last_peered': '2024-09-10T20:37:54.429450+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:37:54.429450+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "43'8000", 'ondisk_log_start': "43'8000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10571776, 'num_objects': 2581, 'num_object_clones': 0, 'num_object_copies': 5162, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2581, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18077, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -960}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:00.880 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:38:00.880 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'17898", 'reported_seq': 18151, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:54.457179+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:37:54.457179+0000', 'last_peered': '2024-09-10T20:37:54.457179+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:37:54.457179+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "43'7800", 'ondisk_log_start': "43'7800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10539008, 'num_objects': 2573, 'num_object_clones': 0, 'num_object_copies': 5146, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2573, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14762, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -988}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:00.880 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:38:00.880 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'17973", 'reported_seq': 18259, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:54.480148+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:37:54.480148+0000', 'last_peered': '2024-09-10T20:37:54.480148+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:37:54.480148+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "43'7900", 'ondisk_log_start': "43'7900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10792960, 'num_objects': 2635, 'num_object_clones': 0, 'num_object_copies': 5270, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2635, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14568, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -851}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:00.880 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:38:00.880 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'17833", 'reported_seq': 18033, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:54.398916+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:37:54.398916+0000', 'last_peered': '2024-09-10T20:37:54.398916+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:37:54.398916+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "43'7800", 'ondisk_log_start': "43'7800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10674176, 'num_objects': 2606, 'num_object_clones': 0, 'num_object_copies': 5212, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2606, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15826, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -891}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:38:00.880 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:38:00.881 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'18165", 'reported_seq': 18382, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:54.446971+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:37:54.446971+0000', 'last_peered': '2024-09-10T20:37:54.446971+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:37:54.446971+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "43'8100", 'ondisk_log_start': "43'8100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10507206, 'num_objects': 2567, 'num_object_clones': 0, 'num_object_copies': 5134, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2567, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18163, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -999}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:00.881 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:38:00.881 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'17983", 'reported_seq': 18174, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:54.384406+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:37:54.384406+0000', 'last_peered': '2024-09-10T20:37:54.384406+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:37:54.384406+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "43'7900", 'ondisk_log_start': "43'7900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10575872, 'num_objects': 2582, 'num_object_clones': 0, 'num_object_copies': 5164, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2582, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15811, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1024}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:38:00.881 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:38:00.881 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'18191", 'reported_seq': 18448, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:54.378841+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:37:54.378841+0000', 'last_peered': '2024-09-10T20:37:54.378841+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:37:54.378841+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "43'8100", 'ondisk_log_start': "43'8100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10809344, 'num_objects': 2639, 'num_object_clones': 0, 'num_object_copies': 5278, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2639, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14914, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -953}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:00.881 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:38:00.881 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'18064", 'reported_seq': 22140, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:54.459409+0000', 'last_change': '2024-09-10T20:36:28.778006+0000', 'last_active': '2024-09-10T20:37:54.459409+0000', 'last_peered': '2024-09-10T20:37:54.459409+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:37:54.459409+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "43'8000", 'ondisk_log_start': "43'8000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10321920, 'num_objects': 2520, 'num_object_clones': 0, 'num_object_copies': 5040, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1572, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2520, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14758, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1963, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 948}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:00.881 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:38:00.881 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'18386", 'reported_seq': 18644, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:37:54.380882+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:37:54.380882+0000', 'last_peered': '2024-09-10T20:37:54.380882+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:37:54.380882+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "43'8300", 'ondisk_log_start': "43'8300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10649600, 'num_objects': 2600, 'num_object_clones': 0, 'num_object_copies': 5200, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2600, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15213, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1055}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:00.882 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:38:01.611 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:01.612 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 29971/78498 objects degraded (38.181%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":593,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":39249,"data_bytes":160740820,"bytes_used":1729744896,"bytes_avail":288180547584,"bytes_total":289910292480,"degraded_objects":29971,"degraded_total":78498,"degraded_ratio":0.38180590588295243,"recovering_objects_per_sec":31,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":379,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":592},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:38:01.640 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:38:01.640 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:38:02.059 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:02.060 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:38:02.092 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2024-09-10T20:38:01.018973+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":160740820,"num_objects":39249,"num_object_clones":0,"num_object_copies":78498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29971,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39249,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":256172,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13180,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160777,"ondisk_log_size":160777,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1689204,"kb_used_data":493240,"kb_used_omap":12,"kb_used_meta":1195891,"kb_avail":281426316,"statfs":{"total":289910292480,"available":288180547584,"internally_reserved":0,"allocated":505077760,"data_stored":504240880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1224592386},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,1,7,19,8,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29118464,"num_objects":-7109,"num_object_clones":0,"num_object_copies":-14218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-379,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7109,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":379,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003908"},"pg_stats":[{"pgid":"2.f","version":"49'18221","reported_seq":18443,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:59.492772+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:37:59.492772+0000","last_peered":"2024-09-10T20:37:59.492772+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:37:59.492772+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9981952,"num_objects":2437,"num_object_clones":0,"num_object_copies":4874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18221,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1104}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18502","reported_seq":18622,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:59.439235+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:37:59.439235+0000","last_peered":"2024-09-10T20:37:59.439235+0000","last_clean":"2024-09-10T20:37:59.439235+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:37:59.439235+0000","last_undegraded":"2024-09-10T20:37:59.439235+0000","last_fullsized":"2024-09-10T20:37:59.439235+0000","mapping_epoch":35,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9994240,"num_objects":2440,"num_object_clones":0,"num_object_copies":4880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16688,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18176","reported_seq":25534,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:56.165548+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:37:56.165548+0000","last_peered":"2024-09-10T20:37:56.165548+0000","last_clean":"2024-09-10T20:37:56.165548+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:37:56.165548+0000","last_undegraded":"2024-09-10T20:37:56.165548+0000","last_fullsized":"2024-09-10T20:37:56.165548+0000","mapping_epoch":44,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10399768,"num_objects":2540,"num_object_clones":0,"num_object_copies":5080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2540,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16304,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'18072","reported_seq":18162,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:59.465809+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:37:59.465809+0000","last_peered":"2024-09-10T20:37:59.465809+0000","last_clean":"2024-09-10T20:37:59.465809+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:37:59.465809+0000","last_undegraded":"2024-09-10T20:37:59.465809+0000","last_fullsized":"2024-09-10T20:37:59.465809+0000","mapping_epoch":35,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10371072,"num_objects":2532,"num_object_clones":0,"num_object_copies":5064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18072,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'18600","reported_seq":18737,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:59.512419+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:37:59.512419+0000","last_peered":"2024-09-10T20:37:59.512419+0000","last_clean":"2024-09-10T20:37:59.512419+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:37:59.512419+0000","last_undegraded":"2024-09-10T20:37:59.512419+0000","last_fullsized":"2024-09-10T20:37:59.512419+0000","mapping_epoch":34,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10113024,"num_objects":2469,"num_object_clones":0,"num_object_copies":4938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15596,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18057","reported_seq":18305,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:59.353672+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:37:59.353672+0000","last_peered":"2024-09-10T20:37:59.353672+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:37:59.353672+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9887744,"num_objects":2414,"num_object_clones":0,"num_object_copies":4828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14921,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'18115","reported_seq":18401,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:59.448325+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:37:59.448325+0000","last_peered":"2024-09-10T20:37:59.448325+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:37:59.448325+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10211328,"num_objects":2493,"num_object_clones":0,"num_object_copies":4986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14710,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-993}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18322","reported_seq":18457,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:59.458647+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:37:59.458647+0000","last_peered":"2024-09-10T20:37:59.458647+0000","last_clean":"2024-09-10T20:37:59.458647+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:37:59.458647+0000","last_undegraded":"2024-09-10T20:37:59.458647+0000","last_fullsized":"2024-09-10T20:37:59.458647+0000","mapping_epoch":34,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9887744,"num_objects":2414,"num_object_clones":0,"num_object_copies":4828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15393,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'17995","reported_seq":18195,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:59.467111+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:37:59.467111+0000","last_peered":"2024-09-10T20:37:59.467111+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:37:59.467111+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10010624,"num_objects":2444,"num_object_clones":0,"num_object_copies":4888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15988,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1053}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318059+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318059+0000","last_peered":"2024-09-10T20:36:30.318059+0000","last_clean":"2024-09-10T20:36:30.318059+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318059+0000","last_undegraded":"2024-09-10T20:36:30.318059+0000","last_fullsized":"2024-09-10T20:36:30.318059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18316","reported_seq":18533,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:59.458453+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:37:59.458453+0000","last_peered":"2024-09-10T20:37:59.458453+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:37:59.458453+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9888710,"num_objects":2416,"num_object_clones":0,"num_object_copies":4832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18314,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1150}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18147","reported_seq":18340,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:59.390643+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:37:59.390643+0000","last_peered":"2024-09-10T20:37:59.390643+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:37:59.390643+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9904128,"num_objects":2418,"num_object_clones":0,"num_object_copies":4836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15975,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1188}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18272","reported_seq":25643,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:59.481307+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:37:59.481307+0000","last_peered":"2024-09-10T20:37:59.481307+0000","last_clean":"2024-09-10T20:37:59.481307+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:37:59.481307+0000","last_undegraded":"2024-09-10T20:37:59.481307+0000","last_fullsized":"2024-09-10T20:37:59.481307+0000","mapping_epoch":48,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9797632,"num_objects":2392,"num_object_clones":0,"num_object_copies":4784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16058,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18345","reported_seq":18602,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:59.529493+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:37:59.529493+0000","last_peered":"2024-09-10T20:37:59.529493+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:37:59.529493+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10178560,"num_objects":2485,"num_object_clones":0,"num_object_copies":4970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15068,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1107}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18216","reported_seq":22527,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:59.536597+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:37:59.536597+0000","last_peered":"2024-09-10T20:37:59.536597+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:37:59.536597+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9699328,"num_objects":2368,"num_object_clones":0,"num_object_copies":4736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1454,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2368,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14910,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2081,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":914}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319644+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319644+0000","last_peered":"2024-09-10T20:36:30.319644+0000","last_clean":"2024-09-10T20:36:30.319644+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319644+0000","last_undegraded":"2024-09-10T20:36:30.319644+0000","last_fullsized":"2024-09-10T20:36:30.319644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18542","reported_seq":18800,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:37:59.446090+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:37:59.446090+0000","last_peered":"2024-09-10T20:37:59.446090+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:37:59.446090+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10010624,"num_objects":2444,"num_object_clones":0,"num_object_copies":4888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15369,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1211}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'17976","reported_seq":25103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:37:56.388218+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:37:56.388218+0000","last_peered":"2024-09-10T20:37:56.388218+0000","last_clean":"2024-09-10T20:37:56.388218+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:37:56.388218+0000","last_undegraded":"2024-09-10T20:37:56.388218+0000","last_fullsized":"2024-09-10T20:37:56.388218+0000","mapping_epoch":46,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10404323,"num_objects":2541,"num_object_clones":0,"num_object_copies":5082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14583,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":160740801,"num_objects":39247,"num_object_clones":0,"num_object_copies":78494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29971,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39247,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":256170,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13178,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":501186560,"data_stored":501120608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160774,"ondisk_log_size":160774,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182513,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":604940,"kb_used_data":176248,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93766900,"statfs":{"total":96636764160,"available":96017305600,"internally_reserved":0,"allocated":180477952,"data_stored":180216004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":438955690},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,0,1,10,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986285,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537052,"kb_used_data":194380,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93834788,"statfs":{"total":96636764160,"available":96086822912,"internally_reserved":0,"allocated":199045120,"data_stored":198783177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350875306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659231,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547212,"kb_used_data":122612,"kb_used_omap":4,"kb_used_meta":424571,"kb_avail":93824628,"statfs":{"total":96636764160,"available":96076419072,"internally_reserved":0,"allocated":125554688,"data_stored":125241699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":434761390},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":179040256,"data_stored":179036643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":124354560,"data_stored":124299906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":197791744,"data_stored":197784059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:38:05.094 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:38:05.653 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:05.655 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:38:05.678 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-09-10T20:38:05.020725+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":150902228,"num_objects":36847,"num_object_clones":0,"num_object_copies":73694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29819,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36847,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":258574,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13332,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160879,"ondisk_log_size":160879,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1690808,"kb_used_data":474052,"kb_used_omap":12,"kb_used_meta":1216691,"kb_avail":281424712,"statfs":{"total":289910292480,"available":288178905088,"internally_reserved":0,"allocated":485429248,"data_stored":484592368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1245891586},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,6,4,16,11],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28823552,"num_objects":-7037,"num_object_clones":0,"num_object_copies":-14074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-405,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7037,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":405,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004339"},"pg_stats":[{"pgid":"2.f","version":"49'18362","reported_seq":18585,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:04.529717+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:38:04.529717+0000","last_peered":"2024-09-10T20:38:04.529717+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:38:04.529717+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'8300","ondisk_log_start":"45'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9404416,"num_objects":2296,"num_object_clones":0,"num_object_copies":4592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18362,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1245}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18660","reported_seq":18780,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:04.460818+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:38:04.460818+0000","last_peered":"2024-09-10T20:38:04.460818+0000","last_clean":"2024-09-10T20:38:04.460818+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:38:04.460818+0000","last_undegraded":"2024-09-10T20:38:04.460818+0000","last_fullsized":"2024-09-10T20:38:04.460818+0000","mapping_epoch":35,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9347072,"num_objects":2282,"num_object_clones":0,"num_object_copies":4564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16846,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18352","reported_seq":25710,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:01.463302+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:38:01.463302+0000","last_peered":"2024-09-10T20:38:01.463302+0000","last_clean":"2024-09-10T20:38:01.463302+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:38:01.463302+0000","last_undegraded":"2024-09-10T20:38:01.463302+0000","last_fullsized":"2024-09-10T20:38:01.463302+0000","mapping_epoch":44,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9678872,"num_objects":2364,"num_object_clones":0,"num_object_copies":4728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2364,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16480,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'18235","reported_seq":18325,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:04.463465+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:38:04.463465+0000","last_peered":"2024-09-10T20:38:04.463465+0000","last_clean":"2024-09-10T20:38:04.463465+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:38:04.463465+0000","last_undegraded":"2024-09-10T20:38:04.463465+0000","last_fullsized":"2024-09-10T20:38:04.463465+0000","mapping_epoch":35,"log_start":"44'8200","ondisk_log_start":"44'8200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9703424,"num_objects":2369,"num_object_clones":0,"num_object_copies":4738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18235,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'18750","reported_seq":18887,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:04.504625+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:38:04.504625+0000","last_peered":"2024-09-10T20:38:04.504625+0000","last_clean":"2024-09-10T20:38:04.504625+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:38:04.504625+0000","last_undegraded":"2024-09-10T20:38:04.504625+0000","last_fullsized":"2024-09-10T20:38:04.504625+0000","mapping_epoch":34,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9498624,"num_objects":2319,"num_object_clones":0,"num_object_copies":4638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15746,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18190","reported_seq":18440,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:04.490785+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:38:04.490785+0000","last_peered":"2024-09-10T20:38:04.490785+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:38:04.490785+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9342976,"num_objects":2281,"num_object_clones":0,"num_object_copies":4562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15054,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1280}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'18249","reported_seq":18535,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:04.519549+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:38:04.519549+0000","last_peered":"2024-09-10T20:38:04.519549+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:38:04.519549+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9662464,"num_objects":2359,"num_object_clones":0,"num_object_copies":4718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14844,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1127}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18452","reported_seq":18587,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:04.552022+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:38:04.552022+0000","last_peered":"2024-09-10T20:38:04.552022+0000","last_clean":"2024-09-10T20:38:04.552022+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:38:04.552022+0000","last_undegraded":"2024-09-10T20:38:04.552022+0000","last_fullsized":"2024-09-10T20:38:04.552022+0000","mapping_epoch":34,"log_start":"45'8400","ondisk_log_start":"45'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9355264,"num_objects":2284,"num_object_clones":0,"num_object_copies":4568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15523,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18149","reported_seq":18350,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:04.494824+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:38:04.494824+0000","last_peered":"2024-09-10T20:38:04.494824+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:38:04.494824+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9379840,"num_objects":2290,"num_object_clones":0,"num_object_copies":4580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16142,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1207}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318059+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318059+0000","last_peered":"2024-09-10T20:36:30.318059+0000","last_clean":"2024-09-10T20:36:30.318059+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318059+0000","last_undegraded":"2024-09-10T20:36:30.318059+0000","last_fullsized":"2024-09-10T20:36:30.318059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18473","reported_seq":18690,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:04.571330+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:38:04.571330+0000","last_peered":"2024-09-10T20:38:04.571330+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:38:04.571330+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'8400","ondisk_log_start":"45'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9245638,"num_objects":2259,"num_object_clones":0,"num_object_copies":4518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18471,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1307}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18282","reported_seq":18475,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:04.510301+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:38:04.510301+0000","last_peered":"2024-09-10T20:38:04.510301+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:38:04.510301+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9351168,"num_objects":2283,"num_object_clones":0,"num_object_copies":4566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16110,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1323}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18416","reported_seq":25787,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:04.481904+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:38:04.481904+0000","last_peered":"2024-09-10T20:38:04.481904+0000","last_clean":"2024-09-10T20:38:04.481904+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:38:04.481904+0000","last_undegraded":"2024-09-10T20:38:04.481904+0000","last_fullsized":"2024-09-10T20:38:04.481904+0000","mapping_epoch":48,"log_start":"45'8400","ondisk_log_start":"45'8400","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9207808,"num_objects":2248,"num_object_clones":0,"num_object_copies":4496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16202,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18483","reported_seq":18738,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:04.530398+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:38:04.530398+0000","last_peered":"2024-09-10T20:38:04.530398+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:38:04.530398+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'8400","ondisk_log_start":"45'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9613312,"num_objects":2347,"num_object_clones":0,"num_object_copies":4694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15206,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1245}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18365","reported_seq":22981,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:04.572952+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:38:04.572952+0000","last_peered":"2024-09-10T20:38:04.572952+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:38:04.572952+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9089024,"num_objects":2219,"num_object_clones":0,"num_object_copies":4438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15059,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2233,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319644+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319644+0000","last_peered":"2024-09-10T20:36:30.319644+0000","last_clean":"2024-09-10T20:36:30.319644+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319644+0000","last_undegraded":"2024-09-10T20:36:30.319644+0000","last_fullsized":"2024-09-10T20:36:30.319644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18711","reported_seq":18969,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:04.580516+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:38:04.580516+0000","last_peered":"2024-09-10T20:38:04.580516+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:38:04.580516+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9318400,"num_objects":2275,"num_object_clones":0,"num_object_copies":4550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2275,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15538,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1380}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18147","reported_seq":25274,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:01.385238+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:38:01.385238+0000","last_peered":"2024-09-10T20:38:01.385238+0000","last_clean":"2024-09-10T20:38:01.385238+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:38:01.385238+0000","last_undegraded":"2024-09-10T20:38:01.385238+0000","last_fullsized":"2024-09-10T20:38:01.385238+0000","mapping_epoch":46,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9703907,"num_objects":2370,"num_object_clones":0,"num_object_copies":4740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14754,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":150902209,"num_objects":36845,"num_object_clones":0,"num_object_copies":73690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29819,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36845,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":258572,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13330,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":481259520,"data_stored":481193568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160876,"ondisk_log_size":160876,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182514,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":606280,"kb_used_data":169272,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93765560,"statfs":{"total":96636764160,"available":96015933440,"internally_reserved":0,"allocated":173334528,"data_stored":173072580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,4,2,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986286,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":536420,"kb_used_data":189588,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93835420,"statfs":{"total":96636764160,"available":96087470080,"internally_reserved":0,"allocated":194138112,"data_stored":193876169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355135146},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659232,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548108,"kb_used_data":115192,"kb_used_omap":4,"kb_used_meta":432891,"kb_avail":93823732,"statfs":{"total":96636764160,"available":96075501568,"internally_reserved":0,"allocated":117956608,"data_stored":117643619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":443281070},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":171671552,"data_stored":171667939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":116707328,"data_stored":116652674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":192880640,"data_stored":192872955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:38:05.679 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:38:06.231 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:06.232 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:38:06.262 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-09-10T20:38:05.020725+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":150902228,"num_objects":36847,"num_object_clones":0,"num_object_copies":73694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29819,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36847,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":258574,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13332,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160879,"ondisk_log_size":160879,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1690808,"kb_used_data":474052,"kb_used_omap":12,"kb_used_meta":1216691,"kb_avail":281424712,"statfs":{"total":289910292480,"available":288178905088,"internally_reserved":0,"allocated":485429248,"data_stored":484592368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1245891586},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,6,4,16,11],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28823552,"num_objects":-7037,"num_object_clones":0,"num_object_copies":-14074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-405,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7037,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":405,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004339"},"pg_stats":[{"pgid":"2.f","version":"49'18362","reported_seq":18585,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:04.529717+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:38:04.529717+0000","last_peered":"2024-09-10T20:38:04.529717+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:38:04.529717+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'8300","ondisk_log_start":"45'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9404416,"num_objects":2296,"num_object_clones":0,"num_object_copies":4592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18362,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1245}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18660","reported_seq":18780,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:04.460818+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:38:04.460818+0000","last_peered":"2024-09-10T20:38:04.460818+0000","last_clean":"2024-09-10T20:38:04.460818+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:38:04.460818+0000","last_undegraded":"2024-09-10T20:38:04.460818+0000","last_fullsized":"2024-09-10T20:38:04.460818+0000","mapping_epoch":35,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9347072,"num_objects":2282,"num_object_clones":0,"num_object_copies":4564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16846,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18352","reported_seq":25710,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:01.463302+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:38:01.463302+0000","last_peered":"2024-09-10T20:38:01.463302+0000","last_clean":"2024-09-10T20:38:01.463302+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:38:01.463302+0000","last_undegraded":"2024-09-10T20:38:01.463302+0000","last_fullsized":"2024-09-10T20:38:01.463302+0000","mapping_epoch":44,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9678872,"num_objects":2364,"num_object_clones":0,"num_object_copies":4728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2364,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16480,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'18235","reported_seq":18325,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:04.463465+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:38:04.463465+0000","last_peered":"2024-09-10T20:38:04.463465+0000","last_clean":"2024-09-10T20:38:04.463465+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:38:04.463465+0000","last_undegraded":"2024-09-10T20:38:04.463465+0000","last_fullsized":"2024-09-10T20:38:04.463465+0000","mapping_epoch":35,"log_start":"44'8200","ondisk_log_start":"44'8200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9703424,"num_objects":2369,"num_object_clones":0,"num_object_copies":4738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18235,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'18750","reported_seq":18887,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:04.504625+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:38:04.504625+0000","last_peered":"2024-09-10T20:38:04.504625+0000","last_clean":"2024-09-10T20:38:04.504625+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:38:04.504625+0000","last_undegraded":"2024-09-10T20:38:04.504625+0000","last_fullsized":"2024-09-10T20:38:04.504625+0000","mapping_epoch":34,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9498624,"num_objects":2319,"num_object_clones":0,"num_object_copies":4638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15746,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18190","reported_seq":18440,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:04.490785+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:38:04.490785+0000","last_peered":"2024-09-10T20:38:04.490785+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:38:04.490785+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9342976,"num_objects":2281,"num_object_clones":0,"num_object_copies":4562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15054,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1280}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'18249","reported_seq":18535,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:04.519549+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:38:04.519549+0000","last_peered":"2024-09-10T20:38:04.519549+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:38:04.519549+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9662464,"num_objects":2359,"num_object_clones":0,"num_object_copies":4718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14844,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1127}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18452","reported_seq":18587,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:04.552022+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:38:04.552022+0000","last_peered":"2024-09-10T20:38:04.552022+0000","last_clean":"2024-09-10T20:38:04.552022+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:38:04.552022+0000","last_undegraded":"2024-09-10T20:38:04.552022+0000","last_fullsized":"2024-09-10T20:38:04.552022+0000","mapping_epoch":34,"log_start":"45'8400","ondisk_log_start":"45'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9355264,"num_objects":2284,"num_object_clones":0,"num_object_copies":4568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15523,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315085+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315085+0000","last_peered":"2024-09-10T20:36:30.315085+0000","last_clean":"2024-09-10T20:36:30.315085+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315085+0000","last_undegraded":"2024-09-10T20:36:30.315085+0000","last_fullsized":"2024-09-10T20:36:30.315085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18149","reported_seq":18350,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:04.494824+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:38:04.494824+0000","last_peered":"2024-09-10T20:38:04.494824+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:38:04.494824+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9379840,"num_objects":2290,"num_object_clones":0,"num_object_copies":4580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16142,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1207}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318059+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318059+0000","last_peered":"2024-09-10T20:36:30.318059+0000","last_clean":"2024-09-10T20:36:30.318059+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318059+0000","last_undegraded":"2024-09-10T20:36:30.318059+0000","last_fullsized":"2024-09-10T20:36:30.318059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18473","reported_seq":18690,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:04.571330+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:38:04.571330+0000","last_peered":"2024-09-10T20:38:04.571330+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:38:04.571330+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'8400","ondisk_log_start":"45'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9245638,"num_objects":2259,"num_object_clones":0,"num_object_copies":4518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18471,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1307}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18282","reported_seq":18475,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:04.510301+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:38:04.510301+0000","last_peered":"2024-09-10T20:38:04.510301+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:38:04.510301+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9351168,"num_objects":2283,"num_object_clones":0,"num_object_copies":4566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16110,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1323}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18416","reported_seq":25787,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:04.481904+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:38:04.481904+0000","last_peered":"2024-09-10T20:38:04.481904+0000","last_clean":"2024-09-10T20:38:04.481904+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:38:04.481904+0000","last_undegraded":"2024-09-10T20:38:04.481904+0000","last_fullsized":"2024-09-10T20:38:04.481904+0000","mapping_epoch":48,"log_start":"45'8400","ondisk_log_start":"45'8400","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9207808,"num_objects":2248,"num_object_clones":0,"num_object_copies":4496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16202,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18483","reported_seq":18738,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:04.530398+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:38:04.530398+0000","last_peered":"2024-09-10T20:38:04.530398+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:38:04.530398+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'8400","ondisk_log_start":"45'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9613312,"num_objects":2347,"num_object_clones":0,"num_object_copies":4694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15206,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1245}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18365","reported_seq":22981,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:04.572952+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:38:04.572952+0000","last_peered":"2024-09-10T20:38:04.572952+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:38:04.572952+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9089024,"num_objects":2219,"num_object_clones":0,"num_object_copies":4438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15059,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2233,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319644+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319644+0000","last_peered":"2024-09-10T20:36:30.319644+0000","last_clean":"2024-09-10T20:36:30.319644+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319644+0000","last_undegraded":"2024-09-10T20:36:30.319644+0000","last_fullsized":"2024-09-10T20:36:30.319644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18711","reported_seq":18969,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:04.580516+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:38:04.580516+0000","last_peered":"2024-09-10T20:38:04.580516+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:38:04.580516+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9318400,"num_objects":2275,"num_object_clones":0,"num_object_copies":4550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2275,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15538,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1380}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18147","reported_seq":25274,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:01.385238+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:38:01.385238+0000","last_peered":"2024-09-10T20:38:01.385238+0000","last_clean":"2024-09-10T20:38:01.385238+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:38:01.385238+0000","last_undegraded":"2024-09-10T20:38:01.385238+0000","last_fullsized":"2024-09-10T20:38:01.385238+0000","mapping_epoch":46,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9703907,"num_objects":2370,"num_object_clones":0,"num_object_copies":4740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14754,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":150902209,"num_objects":36845,"num_object_clones":0,"num_object_copies":73690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29819,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36845,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":258572,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13330,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":481259520,"data_stored":481193568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160876,"ondisk_log_size":160876,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182514,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":606280,"kb_used_data":169272,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93765560,"statfs":{"total":96636764160,"available":96015933440,"internally_reserved":0,"allocated":173334528,"data_stored":173072580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,4,2,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986286,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":536420,"kb_used_data":189588,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93835420,"statfs":{"total":96636764160,"available":96087470080,"internally_reserved":0,"allocated":194138112,"data_stored":193876169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355135146},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659232,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548108,"kb_used_data":115192,"kb_used_omap":4,"kb_used_meta":432891,"kb_avail":93823732,"statfs":{"total":96636764160,"available":96075501568,"internally_reserved":0,"allocated":117956608,"data_stored":117643619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":443281070},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":171671552,"data_stored":171667939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":116707328,"data_stored":116652674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":192880640,"data_stored":192872955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:38:06.263 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:38:06.263 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'18362", 'reported_seq': 18585, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:04.529717+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:38:04.529717+0000', 'last_peered': '2024-09-10T20:38:04.529717+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:38:04.529717+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "45'8300", 'ondisk_log_start': "45'8300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9404416, 'num_objects': 2296, 'num_object_clones': 0, 'num_object_copies': 4592, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2296, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18362, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1245}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:06.263 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:38:06.263 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'18190", 'reported_seq': 18440, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:04.490785+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:38:04.490785+0000', 'last_peered': '2024-09-10T20:38:04.490785+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:38:04.490785+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "43'8100", 'ondisk_log_start': "43'8100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9342976, 'num_objects': 2281, 'num_object_clones': 0, 'num_object_copies': 4562, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2281, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15054, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1280}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:06.264 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:38:06.264 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'18249", 'reported_seq': 18535, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:04.519549+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:38:04.519549+0000', 'last_peered': '2024-09-10T20:38:04.519549+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:38:04.519549+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "43'8200", 'ondisk_log_start': "43'8200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9662464, 'num_objects': 2359, 'num_object_clones': 0, 'num_object_copies': 4718, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2359, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14844, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1127}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:06.264 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:38:06.264 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'18149", 'reported_seq': 18350, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:04.494824+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:38:04.494824+0000', 'last_peered': '2024-09-10T20:38:04.494824+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:38:04.494824+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "43'8100", 'ondisk_log_start': "43'8100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9379840, 'num_objects': 2290, 'num_object_clones': 0, 'num_object_copies': 4580, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2290, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16142, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1207}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:38:06.264 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:38:06.264 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'18473", 'reported_seq': 18690, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:04.571330+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:38:04.571330+0000', 'last_peered': '2024-09-10T20:38:04.571330+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:38:04.571330+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "45'8400", 'ondisk_log_start': "45'8400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9245638, 'num_objects': 2259, 'num_object_clones': 0, 'num_object_copies': 4518, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2259, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18471, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1307}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:06.264 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:38:06.264 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'18282", 'reported_seq': 18475, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:04.510301+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:38:04.510301+0000', 'last_peered': '2024-09-10T20:38:04.510301+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:38:04.510301+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "43'8200", 'ondisk_log_start': "43'8200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9351168, 'num_objects': 2283, 'num_object_clones': 0, 'num_object_copies': 4566, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2283, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16110, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1323}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:38:06.264 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:38:06.264 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'18483", 'reported_seq': 18738, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:04.530398+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:38:04.530398+0000', 'last_peered': '2024-09-10T20:38:04.530398+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:38:04.530398+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "45'8400", 'ondisk_log_start': "45'8400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9613312, 'num_objects': 2347, 'num_object_clones': 0, 'num_object_copies': 4694, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2347, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15206, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1245}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:06.265 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:38:06.265 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'18365", 'reported_seq': 22981, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:04.572952+0000', 'last_change': '2024-09-10T20:36:28.778006+0000', 'last_active': '2024-09-10T20:38:04.572952+0000', 'last_peered': '2024-09-10T20:38:04.572952+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:38:04.572952+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "43'8300", 'ondisk_log_start': "43'8300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9089024, 'num_objects': 2219, 'num_object_clones': 0, 'num_object_copies': 4438, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1302, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2219, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15059, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2233, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:06.265 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:38:06.265 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'18711", 'reported_seq': 18969, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:04.580516+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:38:04.580516+0000', 'last_peered': '2024-09-10T20:38:04.580516+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:38:04.580516+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'8700", 'ondisk_log_start': "45'8700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9318400, 'num_objects': 2275, 'num_object_clones': 0, 'num_object_copies': 4550, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2275, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15538, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1380}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:06.265 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:38:06.950 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:06.950 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 29819/73694 objects degraded (40.463%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":598,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":36847,"data_bytes":150902228,"bytes_used":1731387392,"bytes_avail":288178905088,"bytes_total":289910292480,"degraded_objects":29819,"degraded_total":73694,"degraded_ratio":0.40463267023095495,"recovering_objects_per_sec":33,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":405,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":586},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:38:06.970 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:38:06.970 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:38:07.562 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:07.564 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:38:07.584 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":342,"stamp":"2024-09-10T20:38:07.021645+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":149632468,"num_objects":36537,"num_object_clones":0,"num_object_copies":73074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29819,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36537,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":258884,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13332,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160789,"ondisk_log_size":160789,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1687844,"kb_used_data":466928,"kb_used_omap":12,"kb_used_meta":1220851,"kb_avail":281427676,"statfs":{"total":289910292480,"available":288181940224,"internally_reserved":0,"allocated":478134272,"data_stored":477297392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1250151426},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,7,3,17,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21241856,"num_objects":-5186,"num_object_clones":0,"num_object_copies":-10372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-270,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5186,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004562"},"pg_stats":[{"pgid":"2.f","version":"49'18362","reported_seq":18585,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:04.529717+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:38:04.529717+0000","last_peered":"2024-09-10T20:38:04.529717+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:38:04.529717+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'8300","ondisk_log_start":"45'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9404416,"num_objects":2296,"num_object_clones":0,"num_object_copies":4592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18362,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1245}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18660","reported_seq":18780,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:04.460818+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:38:04.460818+0000","last_peered":"2024-09-10T20:38:04.460818+0000","last_clean":"2024-09-10T20:38:04.460818+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:38:04.460818+0000","last_undegraded":"2024-09-10T20:38:04.460818+0000","last_fullsized":"2024-09-10T20:38:04.460818+0000","mapping_epoch":35,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9347072,"num_objects":2282,"num_object_clones":0,"num_object_copies":4564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16846,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18502","reported_seq":25860,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:06.515873+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:38:06.515873+0000","last_peered":"2024-09-10T20:38:06.515873+0000","last_clean":"2024-09-10T20:38:06.515873+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:38:06.515873+0000","last_undegraded":"2024-09-10T20:38:06.515873+0000","last_fullsized":"2024-09-10T20:38:06.515873+0000","mapping_epoch":44,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9064472,"num_objects":2214,"num_object_clones":0,"num_object_copies":4428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2214,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16630,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'18235","reported_seq":18325,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:04.463465+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:38:04.463465+0000","last_peered":"2024-09-10T20:38:04.463465+0000","last_clean":"2024-09-10T20:38:04.463465+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:38:04.463465+0000","last_undegraded":"2024-09-10T20:38:04.463465+0000","last_fullsized":"2024-09-10T20:38:04.463465+0000","mapping_epoch":35,"log_start":"44'8200","ondisk_log_start":"44'8200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9703424,"num_objects":2369,"num_object_clones":0,"num_object_copies":4738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18235,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'18750","reported_seq":18887,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:04.504625+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:38:04.504625+0000","last_peered":"2024-09-10T20:38:04.504625+0000","last_clean":"2024-09-10T20:38:04.504625+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:38:04.504625+0000","last_undegraded":"2024-09-10T20:38:04.504625+0000","last_fullsized":"2024-09-10T20:38:04.504625+0000","mapping_epoch":34,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9498624,"num_objects":2319,"num_object_clones":0,"num_object_copies":4638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15746,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18190","reported_seq":18440,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:04.490785+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:38:04.490785+0000","last_peered":"2024-09-10T20:38:04.490785+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:38:04.490785+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9342976,"num_objects":2281,"num_object_clones":0,"num_object_copies":4562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15054,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1280}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'18249","reported_seq":18535,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:04.519549+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:38:04.519549+0000","last_peered":"2024-09-10T20:38:04.519549+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:38:04.519549+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9662464,"num_objects":2359,"num_object_clones":0,"num_object_copies":4718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14844,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1127}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18452","reported_seq":18587,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:04.552022+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:38:04.552022+0000","last_peered":"2024-09-10T20:38:04.552022+0000","last_clean":"2024-09-10T20:38:04.552022+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:38:04.552022+0000","last_undegraded":"2024-09-10T20:38:04.552022+0000","last_fullsized":"2024-09-10T20:38:04.552022+0000","mapping_epoch":34,"log_start":"45'8400","ondisk_log_start":"45'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9355264,"num_objects":2284,"num_object_clones":0,"num_object_copies":4568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15523,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315452+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315452+0000","last_peered":"2024-09-10T20:36:30.315452+0000","last_clean":"2024-09-10T20:36:30.315452+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315452+0000","last_undegraded":"2024-09-10T20:36:30.315452+0000","last_fullsized":"2024-09-10T20:36:30.315452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18149","reported_seq":18350,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:04.494824+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:38:04.494824+0000","last_peered":"2024-09-10T20:38:04.494824+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:38:04.494824+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9379840,"num_objects":2290,"num_object_clones":0,"num_object_copies":4580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16142,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1207}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318059+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318059+0000","last_peered":"2024-09-10T20:36:30.318059+0000","last_clean":"2024-09-10T20:36:30.318059+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318059+0000","last_undegraded":"2024-09-10T20:36:30.318059+0000","last_fullsized":"2024-09-10T20:36:30.318059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18473","reported_seq":18690,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:04.571330+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:38:04.571330+0000","last_peered":"2024-09-10T20:38:04.571330+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:38:04.571330+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'8400","ondisk_log_start":"45'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9245638,"num_objects":2259,"num_object_clones":0,"num_object_copies":4518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18471,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1307}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18282","reported_seq":18475,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:04.510301+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:38:04.510301+0000","last_peered":"2024-09-10T20:38:04.510301+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:38:04.510301+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9351168,"num_objects":2283,"num_object_clones":0,"num_object_copies":4566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16110,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1323}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18416","reported_seq":25787,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:04.481904+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:38:04.481904+0000","last_peered":"2024-09-10T20:38:04.481904+0000","last_clean":"2024-09-10T20:38:04.481904+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:38:04.481904+0000","last_undegraded":"2024-09-10T20:38:04.481904+0000","last_fullsized":"2024-09-10T20:38:04.481904+0000","mapping_epoch":48,"log_start":"45'8400","ondisk_log_start":"45'8400","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9207808,"num_objects":2248,"num_object_clones":0,"num_object_copies":4496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16202,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18483","reported_seq":18738,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:04.530398+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:38:04.530398+0000","last_peered":"2024-09-10T20:38:04.530398+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:38:04.530398+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'8400","ondisk_log_start":"45'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9613312,"num_objects":2347,"num_object_clones":0,"num_object_copies":4694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15206,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1245}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18365","reported_seq":22981,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:04.572952+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:38:04.572952+0000","last_peered":"2024-09-10T20:38:04.572952+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:38:04.572952+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9089024,"num_objects":2219,"num_object_clones":0,"num_object_copies":4438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15059,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2233,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319644+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319644+0000","last_peered":"2024-09-10T20:36:30.319644+0000","last_clean":"2024-09-10T20:36:30.319644+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319644+0000","last_undegraded":"2024-09-10T20:36:30.319644+0000","last_fullsized":"2024-09-10T20:36:30.319644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18711","reported_seq":18969,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:04.580516+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:38:04.580516+0000","last_peered":"2024-09-10T20:38:04.580516+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:38:04.580516+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9318400,"num_objects":2275,"num_object_clones":0,"num_object_copies":4550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2275,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15538,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1380}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18307","reported_seq":25434,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:06.588928+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:38:06.588928+0000","last_peered":"2024-09-10T20:38:06.588928+0000","last_clean":"2024-09-10T20:38:06.588928+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:38:06.588928+0000","last_undegraded":"2024-09-10T20:38:06.588928+0000","last_fullsized":"2024-09-10T20:38:06.588928+0000","mapping_epoch":46,"log_start":"45'8300","ondisk_log_start":"45'8300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9048547,"num_objects":2210,"num_object_clones":0,"num_object_copies":4420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14914,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":149632449,"num_objects":36535,"num_object_clones":0,"num_object_copies":73070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29819,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36535,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":258882,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13330,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":473899008,"data_stored":473833056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160786,"ondisk_log_size":160786,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182514,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":606280,"kb_used_data":169272,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93765560,"statfs":{"total":96636764160,"available":96015933440,"internally_reserved":0,"allocated":173334528,"data_stored":173072580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,4,2,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986286,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":536420,"kb_used_data":189588,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93835420,"statfs":{"total":96636764160,"available":96087470080,"internally_reserved":0,"allocated":194138112,"data_stored":193876169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355135146},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659233,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545144,"kb_used_data":108068,"kb_used_omap":4,"kb_used_meta":437051,"kb_avail":93826696,"statfs":{"total":96636764160,"available":96078536704,"internally_reserved":0,"allocated":110661632,"data_stored":110348643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":447540910},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":171671552,"data_stored":171667939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":109346816,"data_stored":109292162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":192880640,"data_stored":192872955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:38:10.587 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:38:11.124 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:11.129 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:38:11.149 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2024-09-10T20:38:11.023247+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":140760532,"num_objects":34371,"num_object_clones":0,"num_object_copies":68742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29690,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34371,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":261050,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13461,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160655,"ondisk_log_size":160655,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1650496,"kb_used_data":454540,"kb_used_omap":12,"kb_used_meta":1195891,"kb_avail":281465024,"statfs":{"total":289910292480,"available":288220184576,"internally_reserved":0,"allocated":465448960,"data_stored":464612080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1224592386},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,6,1,3,9,11,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28925952,"num_objects":-7062,"num_object_clones":0,"num_object_copies":-14124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-399,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7062,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":399,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004856"},"pg_stats":[{"pgid":"2.f","version":"49'18517","reported_seq":18740,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:09.619145+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:38:09.619145+0000","last_peered":"2024-09-10T20:38:09.619145+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:38:09.619145+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8769536,"num_objects":2141,"num_object_clones":0,"num_object_copies":4282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2141,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18517,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1400}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18821","reported_seq":18940,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:09.518612+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:38:09.518612+0000","last_peered":"2024-09-10T20:38:09.518612+0000","last_clean":"2024-09-10T20:38:09.518612+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:38:09.518612+0000","last_undegraded":"2024-09-10T20:38:09.518612+0000","last_fullsized":"2024-09-10T20:38:09.518612+0000","mapping_epoch":35,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8687616,"num_objects":2121,"num_object_clones":0,"num_object_copies":4242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17007,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18502","reported_seq":25860,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:06.515873+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:38:06.515873+0000","last_peered":"2024-09-10T20:38:06.515873+0000","last_clean":"2024-09-10T20:38:06.515873+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:38:06.515873+0000","last_undegraded":"2024-09-10T20:38:06.515873+0000","last_fullsized":"2024-09-10T20:38:06.515873+0000","mapping_epoch":44,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9064472,"num_objects":2214,"num_object_clones":0,"num_object_copies":4428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2214,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16630,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'18395","reported_seq":18485,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:09.474949+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:38:09.474949+0000","last_peered":"2024-09-10T20:38:09.474949+0000","last_clean":"2024-09-10T20:38:09.474949+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:38:09.474949+0000","last_undegraded":"2024-09-10T20:38:09.474949+0000","last_fullsized":"2024-09-10T20:38:09.474949+0000","mapping_epoch":35,"log_start":"45'8300","ondisk_log_start":"45'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9048064,"num_objects":2209,"num_object_clones":0,"num_object_copies":4418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18395,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'18898","reported_seq":19034,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:09.603004+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:38:09.603004+0000","last_peered":"2024-09-10T20:38:09.603004+0000","last_clean":"2024-09-10T20:38:09.603004+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:38:09.603004+0000","last_undegraded":"2024-09-10T20:38:09.603004+0000","last_fullsized":"2024-09-10T20:38:09.603004+0000","mapping_epoch":34,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8892416,"num_objects":2171,"num_object_clones":0,"num_object_copies":4342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15894,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18342","reported_seq":18595,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:09.594862+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:38:09.594862+0000","last_peered":"2024-09-10T20:38:09.594862+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:38:09.594862+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'8300","ondisk_log_start":"45'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8720384,"num_objects":2129,"num_object_clones":0,"num_object_copies":4258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15206,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1432}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'18411","reported_seq":18697,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:09.584466+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:38:09.584466+0000","last_peered":"2024-09-10T20:38:09.584466+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:38:09.584466+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'8400","ondisk_log_start":"45'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8998912,"num_objects":2197,"num_object_clones":0,"num_object_copies":4394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2197,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15006,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1289}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18612","reported_seq":18747,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:09.551401+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:38:09.551401+0000","last_peered":"2024-09-10T20:38:09.551401+0000","last_clean":"2024-09-10T20:38:09.551401+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:38:09.551401+0000","last_undegraded":"2024-09-10T20:38:09.551401+0000","last_fullsized":"2024-09-10T20:38:09.551401+0000","mapping_epoch":34,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8699904,"num_objects":2124,"num_object_clones":0,"num_object_copies":4248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15683,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315452+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315452+0000","last_peered":"2024-09-10T20:36:30.315452+0000","last_clean":"2024-09-10T20:36:30.315452+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315452+0000","last_undegraded":"2024-09-10T20:36:30.315452+0000","last_fullsized":"2024-09-10T20:36:30.315452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18297","reported_seq":18498,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:09.512656+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:38:09.512656+0000","last_peered":"2024-09-10T20:38:09.512656+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:38:09.512656+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"44'8200","ondisk_log_start":"44'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8773632,"num_objects":2142,"num_object_clones":0,"num_object_copies":4284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16290,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1355}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318059+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318059+0000","last_peered":"2024-09-10T20:36:30.318059+0000","last_clean":"2024-09-10T20:36:30.318059+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318059+0000","last_undegraded":"2024-09-10T20:36:30.318059+0000","last_fullsized":"2024-09-10T20:36:30.318059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18624","reported_seq":18841,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:09.579630+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:38:09.579630+0000","last_peered":"2024-09-10T20:38:09.579630+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:38:09.579630+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8627142,"num_objects":2108,"num_object_clones":0,"num_object_copies":4216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18622,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1458}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18423","reported_seq":18614,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:09.482120+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:38:09.482120+0000","last_peered":"2024-09-10T20:38:09.482120+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:38:09.482120+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'8400","ondisk_log_start":"45'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8773632,"num_objects":2142,"num_object_clones":0,"num_object_copies":4284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16251,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1464}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18582","reported_seq":25953,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:09.332666+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:38:09.332666+0000","last_peered":"2024-09-10T20:38:09.332666+0000","last_clean":"2024-09-10T20:38:09.332666+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:38:09.332666+0000","last_undegraded":"2024-09-10T20:38:09.332666+0000","last_fullsized":"2024-09-10T20:38:09.332666+0000","mapping_epoch":48,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8527872,"num_objects":2082,"num_object_clones":0,"num_object_copies":4164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16368,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18644","reported_seq":18900,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:09.626323+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:38:09.626323+0000","last_peered":"2024-09-10T20:38:09.626323+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:38:09.626323+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8953856,"num_objects":2186,"num_object_clones":0,"num_object_copies":4372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15367,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1406}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18511","reported_seq":23385,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:09.605321+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:38:09.605321+0000","last_peered":"2024-09-10T20:38:09.605321+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:38:09.605321+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8491008,"num_objects":2073,"num_object_clones":0,"num_object_copies":4146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1173,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15205,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2362,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":900}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319644+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319644+0000","last_peered":"2024-09-10T20:36:30.319644+0000","last_clean":"2024-09-10T20:36:30.319644+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319644+0000","last_undegraded":"2024-09-10T20:36:30.319644+0000","last_fullsized":"2024-09-10T20:36:30.319644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18866","reported_seq":19123,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:09.621081+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:38:09.621081+0000","last_peered":"2024-09-10T20:38:09.621081+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:38:09.621081+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8683520,"num_objects":2120,"num_object_clones":0,"num_object_copies":4240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15693,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1535}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18307","reported_seq":25434,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:06.588928+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:38:06.588928+0000","last_peered":"2024-09-10T20:38:06.588928+0000","last_clean":"2024-09-10T20:38:06.588928+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:38:06.588928+0000","last_undegraded":"2024-09-10T20:38:06.588928+0000","last_fullsized":"2024-09-10T20:38:06.588928+0000","mapping_epoch":46,"log_start":"45'8300","ondisk_log_start":"45'8300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9048547,"num_objects":2210,"num_object_clones":0,"num_object_copies":4420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14914,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":140760513,"num_objects":34369,"num_object_clones":0,"num_object_copies":68738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29690,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34369,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":261048,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13459,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":461340672,"data_stored":461274720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160652,"ondisk_log_size":160652,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182515,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":586328,"kb_used_data":161800,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93785512,"statfs":{"total":96636764160,"available":96036364288,"internally_reserved":0,"allocated":165683200,"data_stored":165421252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":434695850},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,6,0,1,3,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986287,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519024,"kb_used_data":184672,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93852816,"statfs":{"total":96636764160,"available":96105283584,"internally_reserved":0,"allocated":189104128,"data_stored":188842185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342355626},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,5,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659233,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545144,"kb_used_data":108068,"kb_used_omap":4,"kb_used_meta":437051,"kb_avail":93826696,"statfs":{"total":96636764160,"available":96078536704,"internally_reserved":0,"allocated":110661632,"data_stored":110348643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":447540910},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":164077568,"data_stored":164073955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":109346816,"data_stored":109292162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":187916288,"data_stored":187908603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:38:11.151 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:38:11.718 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:11.719 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:38:11.751 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2024-09-10T20:38:11.023247+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":140760532,"num_objects":34371,"num_object_clones":0,"num_object_copies":68742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29690,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34371,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":261050,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13461,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160655,"ondisk_log_size":160655,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1650496,"kb_used_data":454540,"kb_used_omap":12,"kb_used_meta":1195891,"kb_avail":281465024,"statfs":{"total":289910292480,"available":288220184576,"internally_reserved":0,"allocated":465448960,"data_stored":464612080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1224592386},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,6,1,3,9,11,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28925952,"num_objects":-7062,"num_object_clones":0,"num_object_copies":-14124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-399,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7062,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":399,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004856"},"pg_stats":[{"pgid":"2.f","version":"49'18517","reported_seq":18740,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:09.619145+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:38:09.619145+0000","last_peered":"2024-09-10T20:38:09.619145+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:38:09.619145+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8769536,"num_objects":2141,"num_object_clones":0,"num_object_copies":4282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2141,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18517,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1400}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18821","reported_seq":18940,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:09.518612+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:38:09.518612+0000","last_peered":"2024-09-10T20:38:09.518612+0000","last_clean":"2024-09-10T20:38:09.518612+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:38:09.518612+0000","last_undegraded":"2024-09-10T20:38:09.518612+0000","last_fullsized":"2024-09-10T20:38:09.518612+0000","mapping_epoch":35,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8687616,"num_objects":2121,"num_object_clones":0,"num_object_copies":4242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17007,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18502","reported_seq":25860,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:06.515873+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:38:06.515873+0000","last_peered":"2024-09-10T20:38:06.515873+0000","last_clean":"2024-09-10T20:38:06.515873+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:38:06.515873+0000","last_undegraded":"2024-09-10T20:38:06.515873+0000","last_fullsized":"2024-09-10T20:38:06.515873+0000","mapping_epoch":44,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9064472,"num_objects":2214,"num_object_clones":0,"num_object_copies":4428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2214,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16630,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'18395","reported_seq":18485,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:09.474949+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:38:09.474949+0000","last_peered":"2024-09-10T20:38:09.474949+0000","last_clean":"2024-09-10T20:38:09.474949+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:38:09.474949+0000","last_undegraded":"2024-09-10T20:38:09.474949+0000","last_fullsized":"2024-09-10T20:38:09.474949+0000","mapping_epoch":35,"log_start":"45'8300","ondisk_log_start":"45'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9048064,"num_objects":2209,"num_object_clones":0,"num_object_copies":4418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18395,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'18898","reported_seq":19034,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:09.603004+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:38:09.603004+0000","last_peered":"2024-09-10T20:38:09.603004+0000","last_clean":"2024-09-10T20:38:09.603004+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:38:09.603004+0000","last_undegraded":"2024-09-10T20:38:09.603004+0000","last_fullsized":"2024-09-10T20:38:09.603004+0000","mapping_epoch":34,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8892416,"num_objects":2171,"num_object_clones":0,"num_object_copies":4342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15894,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18342","reported_seq":18595,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:09.594862+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:38:09.594862+0000","last_peered":"2024-09-10T20:38:09.594862+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:38:09.594862+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'8300","ondisk_log_start":"45'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8720384,"num_objects":2129,"num_object_clones":0,"num_object_copies":4258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15206,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1432}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'18411","reported_seq":18697,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:09.584466+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:38:09.584466+0000","last_peered":"2024-09-10T20:38:09.584466+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:38:09.584466+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'8400","ondisk_log_start":"45'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8998912,"num_objects":2197,"num_object_clones":0,"num_object_copies":4394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2197,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15006,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1289}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18612","reported_seq":18747,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:09.551401+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:38:09.551401+0000","last_peered":"2024-09-10T20:38:09.551401+0000","last_clean":"2024-09-10T20:38:09.551401+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:38:09.551401+0000","last_undegraded":"2024-09-10T20:38:09.551401+0000","last_fullsized":"2024-09-10T20:38:09.551401+0000","mapping_epoch":34,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8699904,"num_objects":2124,"num_object_clones":0,"num_object_copies":4248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15683,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315452+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315452+0000","last_peered":"2024-09-10T20:36:30.315452+0000","last_clean":"2024-09-10T20:36:30.315452+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315452+0000","last_undegraded":"2024-09-10T20:36:30.315452+0000","last_fullsized":"2024-09-10T20:36:30.315452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18297","reported_seq":18498,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:09.512656+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:38:09.512656+0000","last_peered":"2024-09-10T20:38:09.512656+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:38:09.512656+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"44'8200","ondisk_log_start":"44'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8773632,"num_objects":2142,"num_object_clones":0,"num_object_copies":4284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16290,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1355}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318059+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318059+0000","last_peered":"2024-09-10T20:36:30.318059+0000","last_clean":"2024-09-10T20:36:30.318059+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318059+0000","last_undegraded":"2024-09-10T20:36:30.318059+0000","last_fullsized":"2024-09-10T20:36:30.318059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18624","reported_seq":18841,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:09.579630+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:38:09.579630+0000","last_peered":"2024-09-10T20:38:09.579630+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:38:09.579630+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8627142,"num_objects":2108,"num_object_clones":0,"num_object_copies":4216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18622,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1458}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18423","reported_seq":18614,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:09.482120+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:38:09.482120+0000","last_peered":"2024-09-10T20:38:09.482120+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:38:09.482120+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'8400","ondisk_log_start":"45'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8773632,"num_objects":2142,"num_object_clones":0,"num_object_copies":4284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16251,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1464}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18582","reported_seq":25953,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:09.332666+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:38:09.332666+0000","last_peered":"2024-09-10T20:38:09.332666+0000","last_clean":"2024-09-10T20:38:09.332666+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:38:09.332666+0000","last_undegraded":"2024-09-10T20:38:09.332666+0000","last_fullsized":"2024-09-10T20:38:09.332666+0000","mapping_epoch":48,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8527872,"num_objects":2082,"num_object_clones":0,"num_object_copies":4164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16368,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18644","reported_seq":18900,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:09.626323+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:38:09.626323+0000","last_peered":"2024-09-10T20:38:09.626323+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:38:09.626323+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8953856,"num_objects":2186,"num_object_clones":0,"num_object_copies":4372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15367,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1406}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18511","reported_seq":23385,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:09.605321+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:38:09.605321+0000","last_peered":"2024-09-10T20:38:09.605321+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:38:09.605321+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8491008,"num_objects":2073,"num_object_clones":0,"num_object_copies":4146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1173,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15205,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2362,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":900}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319644+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319644+0000","last_peered":"2024-09-10T20:36:30.319644+0000","last_clean":"2024-09-10T20:36:30.319644+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319644+0000","last_undegraded":"2024-09-10T20:36:30.319644+0000","last_fullsized":"2024-09-10T20:36:30.319644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18866","reported_seq":19123,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:09.621081+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:38:09.621081+0000","last_peered":"2024-09-10T20:38:09.621081+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:38:09.621081+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8683520,"num_objects":2120,"num_object_clones":0,"num_object_copies":4240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15693,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1535}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18307","reported_seq":25434,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:06.588928+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:38:06.588928+0000","last_peered":"2024-09-10T20:38:06.588928+0000","last_clean":"2024-09-10T20:38:06.588928+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:38:06.588928+0000","last_undegraded":"2024-09-10T20:38:06.588928+0000","last_fullsized":"2024-09-10T20:38:06.588928+0000","mapping_epoch":46,"log_start":"45'8300","ondisk_log_start":"45'8300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9048547,"num_objects":2210,"num_object_clones":0,"num_object_copies":4420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14914,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":140760513,"num_objects":34369,"num_object_clones":0,"num_object_copies":68738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29690,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34369,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":261048,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13459,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":461340672,"data_stored":461274720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160652,"ondisk_log_size":160652,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182515,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":586328,"kb_used_data":161800,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93785512,"statfs":{"total":96636764160,"available":96036364288,"internally_reserved":0,"allocated":165683200,"data_stored":165421252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":434695850},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,6,0,1,3,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986287,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519024,"kb_used_data":184672,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93852816,"statfs":{"total":96636764160,"available":96105283584,"internally_reserved":0,"allocated":189104128,"data_stored":188842185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342355626},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,5,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659233,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545144,"kb_used_data":108068,"kb_used_omap":4,"kb_used_meta":437051,"kb_avail":93826696,"statfs":{"total":96636764160,"available":96078536704,"internally_reserved":0,"allocated":110661632,"data_stored":110348643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":447540910},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":164077568,"data_stored":164073955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":109346816,"data_stored":109292162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":187916288,"data_stored":187908603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:38:11.753 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:38:11.753 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'18517", 'reported_seq': 18740, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:09.619145+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:38:09.619145+0000', 'last_peered': '2024-09-10T20:38:09.619145+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:38:09.619145+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "45'8500", 'ondisk_log_start': "45'8500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8769536, 'num_objects': 2141, 'num_object_clones': 0, 'num_object_copies': 4282, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2141, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18517, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1400}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:11.753 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:38:11.754 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'18342", 'reported_seq': 18595, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:09.594862+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:38:09.594862+0000', 'last_peered': '2024-09-10T20:38:09.594862+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:38:09.594862+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'8300", 'ondisk_log_start': "45'8300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8720384, 'num_objects': 2129, 'num_object_clones': 0, 'num_object_copies': 4258, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2129, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15206, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1432}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:11.754 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:38:11.754 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'18411", 'reported_seq': 18697, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:09.584466+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:38:09.584466+0000', 'last_peered': '2024-09-10T20:38:09.584466+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:38:09.584466+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "45'8400", 'ondisk_log_start': "45'8400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8998912, 'num_objects': 2197, 'num_object_clones': 0, 'num_object_copies': 4394, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2197, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15006, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1289}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:11.754 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:38:11.754 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'18297", 'reported_seq': 18498, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:09.512656+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:38:09.512656+0000', 'last_peered': '2024-09-10T20:38:09.512656+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:38:09.512656+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "44'8200", 'ondisk_log_start': "44'8200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8773632, 'num_objects': 2142, 'num_object_clones': 0, 'num_object_copies': 4284, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2142, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16290, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1355}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:38:11.755 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:38:11.755 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'18624", 'reported_seq': 18841, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:09.579630+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:38:09.579630+0000', 'last_peered': '2024-09-10T20:38:09.579630+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:38:09.579630+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "45'8600", 'ondisk_log_start': "45'8600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8627142, 'num_objects': 2108, 'num_object_clones': 0, 'num_object_copies': 4216, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2108, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18622, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1458}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:11.755 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:38:11.755 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'18423", 'reported_seq': 18614, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:09.482120+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:38:09.482120+0000', 'last_peered': '2024-09-10T20:38:09.482120+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:38:09.482120+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "45'8400", 'ondisk_log_start': "45'8400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8773632, 'num_objects': 2142, 'num_object_clones': 0, 'num_object_copies': 4284, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2142, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16251, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1464}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:38:11.755 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:38:11.755 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'18644", 'reported_seq': 18900, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:09.626323+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:38:09.626323+0000', 'last_peered': '2024-09-10T20:38:09.626323+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:38:09.626323+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "45'8600", 'ondisk_log_start': "45'8600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8953856, 'num_objects': 2186, 'num_object_clones': 0, 'num_object_copies': 4372, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2186, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15367, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1406}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:11.756 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:38:11.756 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'18511", 'reported_seq': 23385, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:09.605321+0000', 'last_change': '2024-09-10T20:36:28.778006+0000', 'last_active': '2024-09-10T20:38:09.605321+0000', 'last_peered': '2024-09-10T20:38:09.605321+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:38:09.605321+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "45'8500", 'ondisk_log_start': "45'8500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8491008, 'num_objects': 2073, 'num_object_clones': 0, 'num_object_copies': 4146, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1173, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2073, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15205, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2362, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 900}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:11.756 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:38:11.756 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'18866", 'reported_seq': 19123, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:09.621081+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:38:09.621081+0000', 'last_peered': '2024-09-10T20:38:09.621081+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:38:09.621081+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'8800", 'ondisk_log_start': "45'8800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8683520, 'num_objects': 2120, 'num_object_clones': 0, 'num_object_copies': 4240, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2120, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15693, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1535}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:11.757 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:38:12.420 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:12.420 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 29690/68742 objects degraded (43.190%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":604,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":34371,"data_bytes":140760532,"bytes_used":1690107904,"bytes_avail":288220184576,"bytes_total":289910292480,"degraded_objects":29690,"degraded_total":68742,"degraded_ratio":0.43190480346803994,"recovering_objects_per_sec":33,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":399,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":588},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:38:12.457 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:38:12.457 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:38:13.017 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:13.021 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:38:13.043 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2024-09-10T20:38:11.023247+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":140760532,"num_objects":34371,"num_object_clones":0,"num_object_copies":68742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29690,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34371,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":261050,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13461,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160655,"ondisk_log_size":160655,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1650496,"kb_used_data":454540,"kb_used_omap":12,"kb_used_meta":1195891,"kb_avail":281465024,"statfs":{"total":289910292480,"available":288220184576,"internally_reserved":0,"allocated":465448960,"data_stored":464612080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1224592386},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,6,1,3,9,11,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28925952,"num_objects":-7062,"num_object_clones":0,"num_object_copies":-14124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-399,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7062,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":399,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004856"},"pg_stats":[{"pgid":"2.f","version":"49'18517","reported_seq":18740,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:09.619145+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:38:09.619145+0000","last_peered":"2024-09-10T20:38:09.619145+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:38:09.619145+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8769536,"num_objects":2141,"num_object_clones":0,"num_object_copies":4282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2141,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18517,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1400}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18821","reported_seq":18940,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:09.518612+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:38:09.518612+0000","last_peered":"2024-09-10T20:38:09.518612+0000","last_clean":"2024-09-10T20:38:09.518612+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:38:09.518612+0000","last_undegraded":"2024-09-10T20:38:09.518612+0000","last_fullsized":"2024-09-10T20:38:09.518612+0000","mapping_epoch":35,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8687616,"num_objects":2121,"num_object_clones":0,"num_object_copies":4242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17007,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18502","reported_seq":25860,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:06.515873+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:38:06.515873+0000","last_peered":"2024-09-10T20:38:06.515873+0000","last_clean":"2024-09-10T20:38:06.515873+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:38:06.515873+0000","last_undegraded":"2024-09-10T20:38:06.515873+0000","last_fullsized":"2024-09-10T20:38:06.515873+0000","mapping_epoch":44,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9064472,"num_objects":2214,"num_object_clones":0,"num_object_copies":4428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2214,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16630,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'18395","reported_seq":18485,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:09.474949+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:38:09.474949+0000","last_peered":"2024-09-10T20:38:09.474949+0000","last_clean":"2024-09-10T20:38:09.474949+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:38:09.474949+0000","last_undegraded":"2024-09-10T20:38:09.474949+0000","last_fullsized":"2024-09-10T20:38:09.474949+0000","mapping_epoch":35,"log_start":"45'8300","ondisk_log_start":"45'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9048064,"num_objects":2209,"num_object_clones":0,"num_object_copies":4418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18395,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'18898","reported_seq":19034,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:09.603004+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:38:09.603004+0000","last_peered":"2024-09-10T20:38:09.603004+0000","last_clean":"2024-09-10T20:38:09.603004+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:38:09.603004+0000","last_undegraded":"2024-09-10T20:38:09.603004+0000","last_fullsized":"2024-09-10T20:38:09.603004+0000","mapping_epoch":34,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8892416,"num_objects":2171,"num_object_clones":0,"num_object_copies":4342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15894,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18342","reported_seq":18595,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:09.594862+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:38:09.594862+0000","last_peered":"2024-09-10T20:38:09.594862+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:38:09.594862+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'8300","ondisk_log_start":"45'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8720384,"num_objects":2129,"num_object_clones":0,"num_object_copies":4258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15206,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1432}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'18411","reported_seq":18697,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:09.584466+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:38:09.584466+0000","last_peered":"2024-09-10T20:38:09.584466+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:38:09.584466+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'8400","ondisk_log_start":"45'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8998912,"num_objects":2197,"num_object_clones":0,"num_object_copies":4394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2197,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15006,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1289}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18612","reported_seq":18747,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:09.551401+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:38:09.551401+0000","last_peered":"2024-09-10T20:38:09.551401+0000","last_clean":"2024-09-10T20:38:09.551401+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:38:09.551401+0000","last_undegraded":"2024-09-10T20:38:09.551401+0000","last_fullsized":"2024-09-10T20:38:09.551401+0000","mapping_epoch":34,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8699904,"num_objects":2124,"num_object_clones":0,"num_object_copies":4248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15683,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315452+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315452+0000","last_peered":"2024-09-10T20:36:30.315452+0000","last_clean":"2024-09-10T20:36:30.315452+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315452+0000","last_undegraded":"2024-09-10T20:36:30.315452+0000","last_fullsized":"2024-09-10T20:36:30.315452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18297","reported_seq":18498,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:09.512656+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:38:09.512656+0000","last_peered":"2024-09-10T20:38:09.512656+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:38:09.512656+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"44'8200","ondisk_log_start":"44'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8773632,"num_objects":2142,"num_object_clones":0,"num_object_copies":4284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16290,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1355}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318059+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318059+0000","last_peered":"2024-09-10T20:36:30.318059+0000","last_clean":"2024-09-10T20:36:30.318059+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318059+0000","last_undegraded":"2024-09-10T20:36:30.318059+0000","last_fullsized":"2024-09-10T20:36:30.318059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18624","reported_seq":18841,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:09.579630+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:38:09.579630+0000","last_peered":"2024-09-10T20:38:09.579630+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:38:09.579630+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8627142,"num_objects":2108,"num_object_clones":0,"num_object_copies":4216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18622,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1458}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18423","reported_seq":18614,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:09.482120+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:38:09.482120+0000","last_peered":"2024-09-10T20:38:09.482120+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:38:09.482120+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'8400","ondisk_log_start":"45'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8773632,"num_objects":2142,"num_object_clones":0,"num_object_copies":4284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16251,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1464}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18582","reported_seq":25953,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:09.332666+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:38:09.332666+0000","last_peered":"2024-09-10T20:38:09.332666+0000","last_clean":"2024-09-10T20:38:09.332666+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:38:09.332666+0000","last_undegraded":"2024-09-10T20:38:09.332666+0000","last_fullsized":"2024-09-10T20:38:09.332666+0000","mapping_epoch":48,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8527872,"num_objects":2082,"num_object_clones":0,"num_object_copies":4164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16368,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18644","reported_seq":18900,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:09.626323+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:38:09.626323+0000","last_peered":"2024-09-10T20:38:09.626323+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:38:09.626323+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8953856,"num_objects":2186,"num_object_clones":0,"num_object_copies":4372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15367,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1406}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18511","reported_seq":23385,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:09.605321+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:38:09.605321+0000","last_peered":"2024-09-10T20:38:09.605321+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:38:09.605321+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8491008,"num_objects":2073,"num_object_clones":0,"num_object_copies":4146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1173,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15205,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2362,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":900}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319644+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319644+0000","last_peered":"2024-09-10T20:36:30.319644+0000","last_clean":"2024-09-10T20:36:30.319644+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319644+0000","last_undegraded":"2024-09-10T20:36:30.319644+0000","last_fullsized":"2024-09-10T20:36:30.319644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18866","reported_seq":19123,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:09.621081+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:38:09.621081+0000","last_peered":"2024-09-10T20:38:09.621081+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:38:09.621081+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8683520,"num_objects":2120,"num_object_clones":0,"num_object_copies":4240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15693,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1535}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18307","reported_seq":25434,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:06.588928+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:38:06.588928+0000","last_peered":"2024-09-10T20:38:06.588928+0000","last_clean":"2024-09-10T20:38:06.588928+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:38:06.588928+0000","last_undegraded":"2024-09-10T20:38:06.588928+0000","last_fullsized":"2024-09-10T20:38:06.588928+0000","mapping_epoch":46,"log_start":"45'8300","ondisk_log_start":"45'8300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9048547,"num_objects":2210,"num_object_clones":0,"num_object_copies":4420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14914,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":140760513,"num_objects":34369,"num_object_clones":0,"num_object_copies":68738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29690,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34369,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":261048,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13459,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":461340672,"data_stored":461274720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160652,"ondisk_log_size":160652,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182515,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":586328,"kb_used_data":161800,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93785512,"statfs":{"total":96636764160,"available":96036364288,"internally_reserved":0,"allocated":165683200,"data_stored":165421252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":434695850},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,6,0,1,3,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986287,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519024,"kb_used_data":184672,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93852816,"statfs":{"total":96636764160,"available":96105283584,"internally_reserved":0,"allocated":189104128,"data_stored":188842185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342355626},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,5,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659233,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545144,"kb_used_data":108068,"kb_used_omap":4,"kb_used_meta":437051,"kb_avail":93826696,"statfs":{"total":96636764160,"available":96078536704,"internally_reserved":0,"allocated":110661632,"data_stored":110348643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":447540910},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":164077568,"data_stored":164073955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":109346816,"data_stored":109292162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":187916288,"data_stored":187908603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:38:16.046 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:38:16.600 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:16.603 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:38:16.629 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-09-10T20:38:15.024901+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":130766292,"num_objects":31931,"num_object_clones":0,"num_object_copies":63862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31931,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":263490,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13605,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160995,"ondisk_log_size":160995,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1653268,"kb_used_data":435096,"kb_used_omap":12,"kb_used_meta":1218099,"kb_avail":281462252,"statfs":{"total":289910292480,"available":288217346048,"internally_reserved":0,"allocated":445538304,"data_stored":444701424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1247333378},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,0,4,6,11,13,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28553216,"num_objects":-6971,"num_object_clones":0,"num_object_copies":-13942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-425,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6971,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":425,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005061"},"pg_stats":[{"pgid":"2.f","version":"49'18643","reported_seq":18866,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:14.582299+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:38:14.582299+0000","last_peered":"2024-09-10T20:38:14.582299+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:38:14.582299+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8253440,"num_objects":2015,"num_object_clones":0,"num_object_copies":4030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18643,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1526}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18979","reported_seq":19096,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:14.445073+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:38:14.445073+0000","last_peered":"2024-09-10T20:38:14.445073+0000","last_clean":"2024-09-10T20:38:14.445073+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:38:14.445073+0000","last_undegraded":"2024-09-10T20:38:14.445073+0000","last_fullsized":"2024-09-10T20:38:14.445073+0000","mapping_epoch":35,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8040448,"num_objects":1963,"num_object_clones":0,"num_object_copies":3926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17165,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18644","reported_seq":26002,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:11.662889+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:38:11.662889+0000","last_peered":"2024-09-10T20:38:11.662889+0000","last_clean":"2024-09-10T20:38:11.662889+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:38:11.662889+0000","last_undegraded":"2024-09-10T20:38:11.662889+0000","last_fullsized":"2024-09-10T20:38:11.662889+0000","mapping_epoch":44,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8482840,"num_objects":2072,"num_object_clones":0,"num_object_copies":4144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2072,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16772,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'18559","reported_seq":18648,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:14.583170+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:38:14.583170+0000","last_peered":"2024-09-10T20:38:14.583170+0000","last_clean":"2024-09-10T20:38:14.583170+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:38:14.583170+0000","last_undegraded":"2024-09-10T20:38:14.583170+0000","last_fullsized":"2024-09-10T20:38:14.583170+0000","mapping_epoch":35,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8376320,"num_objects":2045,"num_object_clones":0,"num_object_copies":4090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18559,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19068","reported_seq":19205,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:14.639013+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:38:14.639013+0000","last_peered":"2024-09-10T20:38:14.639013+0000","last_clean":"2024-09-10T20:38:14.639013+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:38:14.639013+0000","last_undegraded":"2024-09-10T20:38:14.639013+0000","last_fullsized":"2024-09-10T20:38:14.639013+0000","mapping_epoch":34,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8196096,"num_objects":2001,"num_object_clones":0,"num_object_copies":4002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16064,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18487","reported_seq":18740,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:14.520912+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:38:14.520912+0000","last_peered":"2024-09-10T20:38:14.520912+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:38:14.520912+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'8400","ondisk_log_start":"45'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8126464,"num_objects":1984,"num_object_clones":0,"num_object_copies":3968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15351,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1577}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'18598","reported_seq":18884,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:14.591905+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:38:14.591905+0000","last_peered":"2024-09-10T20:38:14.591905+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:38:14.591905+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8232960,"num_objects":2010,"num_object_clones":0,"num_object_copies":4020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15193,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1476}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18770","reported_seq":18905,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:14.583105+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:38:14.583105+0000","last_peered":"2024-09-10T20:38:14.583105+0000","last_clean":"2024-09-10T20:38:14.583105+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:38:14.583105+0000","last_undegraded":"2024-09-10T20:38:14.583105+0000","last_fullsized":"2024-09-10T20:38:14.583105+0000","mapping_epoch":34,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8052736,"num_objects":1966,"num_object_clones":0,"num_object_copies":3932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15841,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315452+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315452+0000","last_peered":"2024-09-10T20:36:30.315452+0000","last_clean":"2024-09-10T20:36:30.315452+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315452+0000","last_undegraded":"2024-09-10T20:36:30.315452+0000","last_fullsized":"2024-09-10T20:36:30.315452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18444","reported_seq":18643,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:14.545112+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:38:14.545112+0000","last_peered":"2024-09-10T20:38:14.545112+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:38:14.545112+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'8400","ondisk_log_start":"45'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8171520,"num_objects":1995,"num_object_clones":0,"num_object_copies":3990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16437,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1502}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318059+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318059+0000","last_peered":"2024-09-10T20:36:30.318059+0000","last_clean":"2024-09-10T20:36:30.318059+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318059+0000","last_undegraded":"2024-09-10T20:36:30.318059+0000","last_fullsized":"2024-09-10T20:36:30.318059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18784","reported_seq":19001,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:14.658435+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:38:14.658435+0000","last_peered":"2024-09-10T20:38:14.658435+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:38:14.658435+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7971782,"num_objects":1948,"num_object_clones":0,"num_object_copies":3896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18782,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1618}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18577","reported_seq":18770,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:14.455185+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:38:14.455185+0000","last_peered":"2024-09-10T20:38:14.455185+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:38:14.455185+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8142848,"num_objects":1988,"num_object_clones":0,"num_object_copies":3976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16405,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1618}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18730","reported_seq":26100,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:14.567859+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:38:14.567859+0000","last_peered":"2024-09-10T20:38:14.567859+0000","last_clean":"2024-09-10T20:38:14.567859+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:38:14.567859+0000","last_undegraded":"2024-09-10T20:38:14.567859+0000","last_fullsized":"2024-09-10T20:38:14.567859+0000","mapping_epoch":48,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7921664,"num_objects":1934,"num_object_clones":0,"num_object_copies":3868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16516,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18798","reported_seq":19055,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:14.583244+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:38:14.583244+0000","last_peered":"2024-09-10T20:38:14.583244+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:38:14.583244+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8323072,"num_objects":2032,"num_object_clones":0,"num_object_copies":4064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15521,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1560}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18653","reported_seq":23815,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:14.635174+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:38:14.635174+0000","last_peered":"2024-09-10T20:38:14.635174+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:38:14.635174+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7909376,"num_objects":1931,"num_object_clones":0,"num_object_copies":3862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1029,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15347,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2506,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":902}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319644+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319644+0000","last_peered":"2024-09-10T20:36:30.319644+0000","last_clean":"2024-09-10T20:36:30.319644+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319644+0000","last_undegraded":"2024-09-10T20:36:30.319644+0000","last_fullsized":"2024-09-10T20:36:30.319644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19004","reported_seq":19262,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:14.549776+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:38:14.549776+0000","last_peered":"2024-09-10T20:38:14.549776+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:38:14.549776+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8118272,"num_objects":1982,"num_object_clones":0,"num_object_copies":3964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15831,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1673}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18454","reported_seq":25581,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:11.690966+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:38:11.690966+0000","last_peered":"2024-09-10T20:38:11.690966+0000","last_clean":"2024-09-10T20:38:11.690966+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:38:11.690966+0000","last_undegraded":"2024-09-10T20:38:11.690966+0000","last_fullsized":"2024-09-10T20:38:11.690966+0000","mapping_epoch":46,"log_start":"45'8400","ondisk_log_start":"45'8400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8446435,"num_objects":2063,"num_object_clones":0,"num_object_copies":4126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2063,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15061,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":130766273,"num_objects":31929,"num_object_clones":0,"num_object_copies":63858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31929,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":263488,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13603,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":441040896,"data_stored":440974944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160992,"ondisk_log_size":160992,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182516,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":587372,"kb_used_data":154520,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93784468,"statfs":{"total":96636764160,"available":96035295232,"internally_reserved":0,"allocated":158228480,"data_stored":157966532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":443215530},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,0,1,3,7,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986288,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518372,"kb_used_data":179852,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93853468,"statfs":{"total":96636764160,"available":96105951232,"internally_reserved":0,"allocated":184168448,"data_stored":183906505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346615466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659234,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547524,"kb_used_data":100724,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93824316,"statfs":{"total":96636764160,"available":96076099584,"internally_reserved":0,"allocated":103141376,"data_stored":102828387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":457502382},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,3,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":156545024,"data_stored":156541411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":101703680,"data_stored":101649026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":182792192,"data_stored":182784507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:38:16.630 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:38:17.172 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:17.174 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:38:17.194 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-09-10T20:38:17.025658+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":129607124,"num_objects":31648,"num_object_clones":0,"num_object_copies":63296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31648,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":263773,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13605,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161078,"ondisk_log_size":161078,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1650092,"kb_used_data":427740,"kb_used_omap":12,"kb_used_meta":1222259,"kb_avail":281465428,"statfs":{"total":289910292480,"available":288220598272,"internally_reserved":0,"allocated":438005760,"data_stored":437168880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1251593218},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,2,3,7,10,11],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21295104,"num_objects":-5199,"num_object_clones":0,"num_object_copies":-10398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-273,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5199,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":273,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004932"},"pg_stats":[{"pgid":"2.f","version":"49'18643","reported_seq":18866,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:14.582299+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:38:14.582299+0000","last_peered":"2024-09-10T20:38:14.582299+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:38:14.582299+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8253440,"num_objects":2015,"num_object_clones":0,"num_object_copies":4030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18643,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1526}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18979","reported_seq":19096,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:14.445073+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:38:14.445073+0000","last_peered":"2024-09-10T20:38:14.445073+0000","last_clean":"2024-09-10T20:38:14.445073+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:38:14.445073+0000","last_undegraded":"2024-09-10T20:38:14.445073+0000","last_fullsized":"2024-09-10T20:38:14.445073+0000","mapping_epoch":35,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8040448,"num_objects":1963,"num_object_clones":0,"num_object_copies":3926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17165,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18800","reported_seq":26158,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:16.399395+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:38:16.399395+0000","last_peered":"2024-09-10T20:38:16.399395+0000","last_clean":"2024-09-10T20:38:16.399395+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:38:16.399395+0000","last_undegraded":"2024-09-10T20:38:16.399395+0000","last_fullsized":"2024-09-10T20:38:16.399395+0000","mapping_epoch":44,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7843864,"num_objects":1916,"num_object_clones":0,"num_object_copies":3832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1916,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16928,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'18559","reported_seq":18648,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:14.583170+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:38:14.583170+0000","last_peered":"2024-09-10T20:38:14.583170+0000","last_clean":"2024-09-10T20:38:14.583170+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:38:14.583170+0000","last_undegraded":"2024-09-10T20:38:14.583170+0000","last_fullsized":"2024-09-10T20:38:14.583170+0000","mapping_epoch":35,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8376320,"num_objects":2045,"num_object_clones":0,"num_object_copies":4090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18559,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19068","reported_seq":19205,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:14.639013+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:38:14.639013+0000","last_peered":"2024-09-10T20:38:14.639013+0000","last_clean":"2024-09-10T20:38:14.639013+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:38:14.639013+0000","last_undegraded":"2024-09-10T20:38:14.639013+0000","last_fullsized":"2024-09-10T20:38:14.639013+0000","mapping_epoch":34,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8196096,"num_objects":2001,"num_object_clones":0,"num_object_copies":4002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16064,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18487","reported_seq":18740,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:14.520912+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:38:14.520912+0000","last_peered":"2024-09-10T20:38:14.520912+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:38:14.520912+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'8400","ondisk_log_start":"45'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8126464,"num_objects":1984,"num_object_clones":0,"num_object_copies":3968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15351,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1577}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'18598","reported_seq":18884,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:14.591905+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:38:14.591905+0000","last_peered":"2024-09-10T20:38:14.591905+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:38:14.591905+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8232960,"num_objects":2010,"num_object_clones":0,"num_object_copies":4020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15193,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1476}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18770","reported_seq":18905,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:14.583105+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:38:14.583105+0000","last_peered":"2024-09-10T20:38:14.583105+0000","last_clean":"2024-09-10T20:38:14.583105+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:38:14.583105+0000","last_undegraded":"2024-09-10T20:38:14.583105+0000","last_fullsized":"2024-09-10T20:38:14.583105+0000","mapping_epoch":34,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8052736,"num_objects":1966,"num_object_clones":0,"num_object_copies":3932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15841,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315452+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315452+0000","last_peered":"2024-09-10T20:36:30.315452+0000","last_clean":"2024-09-10T20:36:30.315452+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315452+0000","last_undegraded":"2024-09-10T20:36:30.315452+0000","last_fullsized":"2024-09-10T20:36:30.315452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18444","reported_seq":18643,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:14.545112+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:38:14.545112+0000","last_peered":"2024-09-10T20:38:14.545112+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:38:14.545112+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'8400","ondisk_log_start":"45'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8171520,"num_objects":1995,"num_object_clones":0,"num_object_copies":3990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16437,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1502}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318059+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318059+0000","last_peered":"2024-09-10T20:36:30.318059+0000","last_clean":"2024-09-10T20:36:30.318059+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318059+0000","last_undegraded":"2024-09-10T20:36:30.318059+0000","last_fullsized":"2024-09-10T20:36:30.318059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18784","reported_seq":19001,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:14.658435+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:38:14.658435+0000","last_peered":"2024-09-10T20:38:14.658435+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:38:14.658435+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7971782,"num_objects":1948,"num_object_clones":0,"num_object_copies":3896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18782,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1618}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18577","reported_seq":18770,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:14.455185+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:38:14.455185+0000","last_peered":"2024-09-10T20:38:14.455185+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:38:14.455185+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8142848,"num_objects":1988,"num_object_clones":0,"num_object_copies":3976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16405,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1618}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18730","reported_seq":26100,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:14.567859+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:38:14.567859+0000","last_peered":"2024-09-10T20:38:14.567859+0000","last_clean":"2024-09-10T20:38:14.567859+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:38:14.567859+0000","last_undegraded":"2024-09-10T20:38:14.567859+0000","last_fullsized":"2024-09-10T20:38:14.567859+0000","mapping_epoch":48,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7921664,"num_objects":1934,"num_object_clones":0,"num_object_copies":3868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16516,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18798","reported_seq":19055,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:14.583244+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:38:14.583244+0000","last_peered":"2024-09-10T20:38:14.583244+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:38:14.583244+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8323072,"num_objects":2032,"num_object_clones":0,"num_object_copies":4064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15521,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1560}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18653","reported_seq":23815,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:14.635174+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:38:14.635174+0000","last_peered":"2024-09-10T20:38:14.635174+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:38:14.635174+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7909376,"num_objects":1931,"num_object_clones":0,"num_object_copies":3862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1029,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15347,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2506,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":902}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319644+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319644+0000","last_peered":"2024-09-10T20:36:30.319644+0000","last_clean":"2024-09-10T20:36:30.319644+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319644+0000","last_undegraded":"2024-09-10T20:36:30.319644+0000","last_fullsized":"2024-09-10T20:36:30.319644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19004","reported_seq":19262,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:14.549776+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:38:14.549776+0000","last_peered":"2024-09-10T20:38:14.549776+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:38:14.549776+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8118272,"num_objects":1982,"num_object_clones":0,"num_object_copies":3964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15831,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1673}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18581","reported_seq":25708,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:16.625772+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:38:16.625772+0000","last_peered":"2024-09-10T20:38:16.625772+0000","last_clean":"2024-09-10T20:38:16.625772+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:38:16.625772+0000","last_undegraded":"2024-09-10T20:38:16.625772+0000","last_fullsized":"2024-09-10T20:38:16.625772+0000","mapping_epoch":46,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7926243,"num_objects":1936,"num_object_clones":0,"num_object_copies":3872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15188,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":129607105,"num_objects":31646,"num_object_clones":0,"num_object_copies":63292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31646,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":263771,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13603,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":433569792,"data_stored":433503840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161075,"ondisk_log_size":161075,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182516,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":587372,"kb_used_data":154520,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93784468,"statfs":{"total":96636764160,"available":96035295232,"internally_reserved":0,"allocated":158228480,"data_stored":157966532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":443215530},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,0,1,3,7,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986288,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518372,"kb_used_data":179852,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93853468,"statfs":{"total":96636764160,"available":96105951232,"internally_reserved":0,"allocated":184168448,"data_stored":183906505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346615466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659235,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":544348,"kb_used_data":93368,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93827492,"statfs":{"total":96636764160,"available":96079351808,"internally_reserved":0,"allocated":95608832,"data_stored":95295843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":461762222},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,4,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":156545024,"data_stored":156541411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94232576,"data_stored":94177922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":182792192,"data_stored":182784507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:38:17.196 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:38:17.196 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'18643", 'reported_seq': 18866, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:14.582299+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:38:14.582299+0000', 'last_peered': '2024-09-10T20:38:14.582299+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:38:14.582299+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "45'8600", 'ondisk_log_start': "45'8600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8253440, 'num_objects': 2015, 'num_object_clones': 0, 'num_object_copies': 4030, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2015, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18643, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1526}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:17.196 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:38:17.196 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'18487", 'reported_seq': 18740, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:14.520912+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:38:14.520912+0000', 'last_peered': '2024-09-10T20:38:14.520912+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:38:14.520912+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'8400", 'ondisk_log_start': "45'8400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8126464, 'num_objects': 1984, 'num_object_clones': 0, 'num_object_copies': 3968, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1984, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15351, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1577}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:17.196 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:38:17.196 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'18598", 'reported_seq': 18884, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:14.591905+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:38:14.591905+0000', 'last_peered': '2024-09-10T20:38:14.591905+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:38:14.591905+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "45'8500", 'ondisk_log_start': "45'8500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8232960, 'num_objects': 2010, 'num_object_clones': 0, 'num_object_copies': 4020, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2010, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15193, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1476}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:17.196 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:38:17.196 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'18444", 'reported_seq': 18643, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:14.545112+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:38:14.545112+0000', 'last_peered': '2024-09-10T20:38:14.545112+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:38:14.545112+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "45'8400", 'ondisk_log_start': "45'8400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8171520, 'num_objects': 1995, 'num_object_clones': 0, 'num_object_copies': 3990, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1995, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16437, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1502}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:38:17.197 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:38:17.197 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'18784", 'reported_seq': 19001, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:14.658435+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:38:14.658435+0000', 'last_peered': '2024-09-10T20:38:14.658435+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:38:14.658435+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "45'8700", 'ondisk_log_start': "45'8700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7971782, 'num_objects': 1948, 'num_object_clones': 0, 'num_object_copies': 3896, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1948, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18782, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1618}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:17.197 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:38:17.197 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'18577", 'reported_seq': 18770, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:14.455185+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:38:14.455185+0000', 'last_peered': '2024-09-10T20:38:14.455185+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:38:14.455185+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "45'8500", 'ondisk_log_start': "45'8500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8142848, 'num_objects': 1988, 'num_object_clones': 0, 'num_object_copies': 3976, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1988, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16405, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1618}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:38:17.197 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:38:17.197 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'18798", 'reported_seq': 19055, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:14.583244+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:38:14.583244+0000', 'last_peered': '2024-09-10T20:38:14.583244+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:38:14.583244+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "45'8700", 'ondisk_log_start': "45'8700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8323072, 'num_objects': 2032, 'num_object_clones': 0, 'num_object_copies': 4064, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2032, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15521, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1560}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:17.197 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:38:17.197 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'18653", 'reported_seq': 23815, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:14.635174+0000', 'last_change': '2024-09-10T20:36:28.778006+0000', 'last_active': '2024-09-10T20:38:14.635174+0000', 'last_peered': '2024-09-10T20:38:14.635174+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:38:14.635174+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "45'8600", 'ondisk_log_start': "45'8600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7909376, 'num_objects': 1931, 'num_object_clones': 0, 'num_object_copies': 3862, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1029, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1931, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15347, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2506, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 902}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:17.197 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:38:17.197 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19004", 'reported_seq': 19262, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:14.549776+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:38:14.549776+0000', 'last_peered': '2024-09-10T20:38:14.549776+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:38:14.549776+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'9000", 'ondisk_log_start': "45'9000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8118272, 'num_objects': 1982, 'num_object_clones': 0, 'num_object_copies': 3964, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1982, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15831, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1673}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:17.198 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:38:17.894 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:17.894 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 29546/63296 objects degraded (46.679%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":609,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":31648,"data_bytes":129607124,"bytes_used":1689694208,"bytes_avail":288220598272,"bytes_total":289910292480,"degraded_objects":29546,"degraded_total":63296,"degraded_ratio":0.46679095045500513,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":273,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":433},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:38:17.925 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:38:17.926 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:38:18.487 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:18.490 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:38:18.509 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-09-10T20:38:17.025658+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":129607124,"num_objects":31648,"num_object_clones":0,"num_object_copies":63296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31648,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":263773,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13605,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161078,"ondisk_log_size":161078,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1650092,"kb_used_data":427740,"kb_used_omap":12,"kb_used_meta":1222259,"kb_avail":281465428,"statfs":{"total":289910292480,"available":288220598272,"internally_reserved":0,"allocated":438005760,"data_stored":437168880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1251593218},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,2,3,7,10,11],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21295104,"num_objects":-5199,"num_object_clones":0,"num_object_copies":-10398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-273,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5199,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":273,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004932"},"pg_stats":[{"pgid":"2.f","version":"49'18643","reported_seq":18866,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:14.582299+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:38:14.582299+0000","last_peered":"2024-09-10T20:38:14.582299+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:38:14.582299+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8253440,"num_objects":2015,"num_object_clones":0,"num_object_copies":4030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18643,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1526}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18979","reported_seq":19096,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:14.445073+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:38:14.445073+0000","last_peered":"2024-09-10T20:38:14.445073+0000","last_clean":"2024-09-10T20:38:14.445073+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:38:14.445073+0000","last_undegraded":"2024-09-10T20:38:14.445073+0000","last_fullsized":"2024-09-10T20:38:14.445073+0000","mapping_epoch":35,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8040448,"num_objects":1963,"num_object_clones":0,"num_object_copies":3926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17165,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18800","reported_seq":26158,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:16.399395+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:38:16.399395+0000","last_peered":"2024-09-10T20:38:16.399395+0000","last_clean":"2024-09-10T20:38:16.399395+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:38:16.399395+0000","last_undegraded":"2024-09-10T20:38:16.399395+0000","last_fullsized":"2024-09-10T20:38:16.399395+0000","mapping_epoch":44,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7843864,"num_objects":1916,"num_object_clones":0,"num_object_copies":3832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1916,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16928,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'18559","reported_seq":18648,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:14.583170+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:38:14.583170+0000","last_peered":"2024-09-10T20:38:14.583170+0000","last_clean":"2024-09-10T20:38:14.583170+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:38:14.583170+0000","last_undegraded":"2024-09-10T20:38:14.583170+0000","last_fullsized":"2024-09-10T20:38:14.583170+0000","mapping_epoch":35,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8376320,"num_objects":2045,"num_object_clones":0,"num_object_copies":4090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18559,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19068","reported_seq":19205,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:14.639013+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:38:14.639013+0000","last_peered":"2024-09-10T20:38:14.639013+0000","last_clean":"2024-09-10T20:38:14.639013+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:38:14.639013+0000","last_undegraded":"2024-09-10T20:38:14.639013+0000","last_fullsized":"2024-09-10T20:38:14.639013+0000","mapping_epoch":34,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8196096,"num_objects":2001,"num_object_clones":0,"num_object_copies":4002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16064,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18487","reported_seq":18740,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:14.520912+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:38:14.520912+0000","last_peered":"2024-09-10T20:38:14.520912+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:38:14.520912+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'8400","ondisk_log_start":"45'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8126464,"num_objects":1984,"num_object_clones":0,"num_object_copies":3968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15351,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1577}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'18598","reported_seq":18884,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:14.591905+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:38:14.591905+0000","last_peered":"2024-09-10T20:38:14.591905+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:38:14.591905+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8232960,"num_objects":2010,"num_object_clones":0,"num_object_copies":4020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15193,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1476}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18770","reported_seq":18905,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:14.583105+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:38:14.583105+0000","last_peered":"2024-09-10T20:38:14.583105+0000","last_clean":"2024-09-10T20:38:14.583105+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:38:14.583105+0000","last_undegraded":"2024-09-10T20:38:14.583105+0000","last_fullsized":"2024-09-10T20:38:14.583105+0000","mapping_epoch":34,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8052736,"num_objects":1966,"num_object_clones":0,"num_object_copies":3932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15841,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315452+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315452+0000","last_peered":"2024-09-10T20:36:30.315452+0000","last_clean":"2024-09-10T20:36:30.315452+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315452+0000","last_undegraded":"2024-09-10T20:36:30.315452+0000","last_fullsized":"2024-09-10T20:36:30.315452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18444","reported_seq":18643,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:14.545112+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:38:14.545112+0000","last_peered":"2024-09-10T20:38:14.545112+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:38:14.545112+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'8400","ondisk_log_start":"45'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8171520,"num_objects":1995,"num_object_clones":0,"num_object_copies":3990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16437,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1502}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318059+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318059+0000","last_peered":"2024-09-10T20:36:30.318059+0000","last_clean":"2024-09-10T20:36:30.318059+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318059+0000","last_undegraded":"2024-09-10T20:36:30.318059+0000","last_fullsized":"2024-09-10T20:36:30.318059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18784","reported_seq":19001,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:14.658435+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:38:14.658435+0000","last_peered":"2024-09-10T20:38:14.658435+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:38:14.658435+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7971782,"num_objects":1948,"num_object_clones":0,"num_object_copies":3896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18782,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1618}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18577","reported_seq":18770,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:14.455185+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:38:14.455185+0000","last_peered":"2024-09-10T20:38:14.455185+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:38:14.455185+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8142848,"num_objects":1988,"num_object_clones":0,"num_object_copies":3976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16405,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1618}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18730","reported_seq":26100,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:14.567859+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:38:14.567859+0000","last_peered":"2024-09-10T20:38:14.567859+0000","last_clean":"2024-09-10T20:38:14.567859+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:38:14.567859+0000","last_undegraded":"2024-09-10T20:38:14.567859+0000","last_fullsized":"2024-09-10T20:38:14.567859+0000","mapping_epoch":48,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7921664,"num_objects":1934,"num_object_clones":0,"num_object_copies":3868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16516,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18798","reported_seq":19055,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:14.583244+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:38:14.583244+0000","last_peered":"2024-09-10T20:38:14.583244+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:38:14.583244+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8323072,"num_objects":2032,"num_object_clones":0,"num_object_copies":4064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15521,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1560}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18653","reported_seq":23815,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:14.635174+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:38:14.635174+0000","last_peered":"2024-09-10T20:38:14.635174+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:38:14.635174+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7909376,"num_objects":1931,"num_object_clones":0,"num_object_copies":3862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1029,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15347,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2506,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":902}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319644+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319644+0000","last_peered":"2024-09-10T20:36:30.319644+0000","last_clean":"2024-09-10T20:36:30.319644+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319644+0000","last_undegraded":"2024-09-10T20:36:30.319644+0000","last_fullsized":"2024-09-10T20:36:30.319644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19004","reported_seq":19262,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:14.549776+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:38:14.549776+0000","last_peered":"2024-09-10T20:38:14.549776+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:38:14.549776+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8118272,"num_objects":1982,"num_object_clones":0,"num_object_copies":3964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15831,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1673}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18581","reported_seq":25708,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:16.625772+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:38:16.625772+0000","last_peered":"2024-09-10T20:38:16.625772+0000","last_clean":"2024-09-10T20:38:16.625772+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:38:16.625772+0000","last_undegraded":"2024-09-10T20:38:16.625772+0000","last_fullsized":"2024-09-10T20:38:16.625772+0000","mapping_epoch":46,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7926243,"num_objects":1936,"num_object_clones":0,"num_object_copies":3872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15188,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":129607105,"num_objects":31646,"num_object_clones":0,"num_object_copies":63292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31646,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":263771,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13603,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":433569792,"data_stored":433503840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161075,"ondisk_log_size":161075,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182516,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":587372,"kb_used_data":154520,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93784468,"statfs":{"total":96636764160,"available":96035295232,"internally_reserved":0,"allocated":158228480,"data_stored":157966532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":443215530},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,0,1,3,7,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986288,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518372,"kb_used_data":179852,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93853468,"statfs":{"total":96636764160,"available":96105951232,"internally_reserved":0,"allocated":184168448,"data_stored":183906505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346615466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659235,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":544348,"kb_used_data":93368,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93827492,"statfs":{"total":96636764160,"available":96079351808,"internally_reserved":0,"allocated":95608832,"data_stored":95295843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":461762222},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,4,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":156545024,"data_stored":156541411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94232576,"data_stored":94177922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":182792192,"data_stored":182784507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:38:21.511 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:38:22.078 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:22.079 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:38:22.107 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2024-09-10T20:38:21.027252+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":121886164,"num_objects":29763,"num_object_clones":0,"num_object_copies":59526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29763,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":265658,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13731,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160763,"ondisk_log_size":160763,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1646092,"kb_used_data":415420,"kb_used_omap":12,"kb_used_meta":1230579,"kb_avail":281469428,"statfs":{"total":289910292480,"available":288224694272,"internally_reserved":0,"allocated":425390080,"data_stored":424553200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1260112898},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,4,5,3,10,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27746304,"num_objects":-6774,"num_object_clones":0,"num_object_copies":-13548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-399,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6774,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":399,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004755"},"pg_stats":[{"pgid":"2.f","version":"49'18800","reported_seq":19023,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:19.048609+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:38:19.048609+0000","last_peered":"2024-09-10T20:38:19.048609+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:38:19.048609+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7610368,"num_objects":1858,"num_object_clones":0,"num_object_copies":3716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18800,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1683}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19113","reported_seq":19234,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:19.099055+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:38:19.099055+0000","last_peered":"2024-09-10T20:38:19.099055+0000","last_clean":"2024-09-10T20:38:19.099055+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:38:19.099055+0000","last_undegraded":"2024-09-10T20:38:19.099055+0000","last_fullsized":"2024-09-10T20:38:19.099055+0000","mapping_epoch":35,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7491584,"num_objects":1829,"num_object_clones":0,"num_object_copies":3658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17299,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18800","reported_seq":26158,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:16.399395+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:38:16.399395+0000","last_peered":"2024-09-10T20:38:16.399395+0000","last_clean":"2024-09-10T20:38:16.399395+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:38:16.399395+0000","last_undegraded":"2024-09-10T20:38:16.399395+0000","last_fullsized":"2024-09-10T20:38:16.399395+0000","mapping_epoch":44,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7843864,"num_objects":1916,"num_object_clones":0,"num_object_copies":3832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1916,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16928,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'18702","reported_seq":18793,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:19.105391+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:38:19.105391+0000","last_peered":"2024-09-10T20:38:19.105391+0000","last_clean":"2024-09-10T20:38:19.105391+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:38:19.105391+0000","last_undegraded":"2024-09-10T20:38:19.105391+0000","last_fullsized":"2024-09-10T20:38:19.105391+0000","mapping_epoch":35,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7790592,"num_objects":1902,"num_object_clones":0,"num_object_copies":3804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18702,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19204","reported_seq":19341,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:19.083467+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:38:19.083467+0000","last_peered":"2024-09-10T20:38:19.083467+0000","last_clean":"2024-09-10T20:38:19.083467+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:38:19.083467+0000","last_undegraded":"2024-09-10T20:38:19.083467+0000","last_fullsized":"2024-09-10T20:38:19.083467+0000","mapping_epoch":34,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7639040,"num_objects":1865,"num_object_clones":0,"num_object_copies":3730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16200,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18620","reported_seq":18871,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:19.068046+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:38:19.068046+0000","last_peered":"2024-09-10T20:38:19.068046+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:38:19.068046+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7581696,"num_objects":1851,"num_object_clones":0,"num_object_copies":3702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15484,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1710}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'18736","reported_seq":19022,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:19.052465+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:38:19.052465+0000","last_peered":"2024-09-10T20:38:19.052465+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:38:19.052465+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7667712,"num_objects":1872,"num_object_clones":0,"num_object_copies":3744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1872,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15331,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18890","reported_seq":19025,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:19.089705+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:38:19.089705+0000","last_peered":"2024-09-10T20:38:19.089705+0000","last_clean":"2024-09-10T20:38:19.089705+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:38:19.089705+0000","last_undegraded":"2024-09-10T20:38:19.089705+0000","last_fullsized":"2024-09-10T20:38:19.089705+0000","mapping_epoch":34,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7561216,"num_objects":1846,"num_object_clones":0,"num_object_copies":3692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15961,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315452+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315452+0000","last_peered":"2024-09-10T20:36:30.315452+0000","last_clean":"2024-09-10T20:36:30.315452+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315452+0000","last_undegraded":"2024-09-10T20:36:30.315452+0000","last_fullsized":"2024-09-10T20:36:30.315452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18578","reported_seq":18779,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:19.046165+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:38:19.046165+0000","last_peered":"2024-09-10T20:38:19.046165+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:38:19.046165+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7622656,"num_objects":1861,"num_object_clones":0,"num_object_copies":3722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16571,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1636}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318059+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318059+0000","last_peered":"2024-09-10T20:36:30.318059+0000","last_clean":"2024-09-10T20:36:30.318059+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318059+0000","last_undegraded":"2024-09-10T20:36:30.318059+0000","last_fullsized":"2024-09-10T20:36:30.318059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18910","reported_seq":19126,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:19.065646+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:38:19.065646+0000","last_peered":"2024-09-10T20:38:19.065646+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:38:19.065646+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7455686,"num_objects":1822,"num_object_clones":0,"num_object_copies":3644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18908,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1744}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18737","reported_seq":18930,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:19.070807+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:38:19.070807+0000","last_peered":"2024-09-10T20:38:19.070807+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:38:19.070807+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7487488,"num_objects":1828,"num_object_clones":0,"num_object_copies":3656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16565,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1778}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18851","reported_seq":26222,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:19.089861+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:38:19.089861+0000","last_peered":"2024-09-10T20:38:19.089861+0000","last_clean":"2024-09-10T20:38:19.089861+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:38:19.089861+0000","last_undegraded":"2024-09-10T20:38:19.089861+0000","last_fullsized":"2024-09-10T20:38:19.089861+0000","mapping_epoch":48,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7426048,"num_objects":1813,"num_object_clones":0,"num_object_copies":3626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16637,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18928","reported_seq":19185,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:18.991007+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:38:18.991007+0000","last_peered":"2024-09-10T20:38:18.991007+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:38:18.991007+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7790592,"num_objects":1902,"num_object_clones":0,"num_object_copies":3804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15651,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1690}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18774","reported_seq":24188,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:19.054281+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:38:19.054281+0000","last_peered":"2024-09-10T20:38:19.054281+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:38:19.054281+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7413760,"num_objects":1810,"num_object_clones":0,"num_object_copies":3620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":903,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15468,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2632,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":907}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319644+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319644+0000","last_peered":"2024-09-10T20:36:30.319644+0000","last_clean":"2024-09-10T20:36:30.319644+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319644+0000","last_undegraded":"2024-09-10T20:36:30.319644+0000","last_fullsized":"2024-09-10T20:36:30.319644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19136","reported_seq":19393,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:19.038360+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:38:19.038360+0000","last_peered":"2024-09-10T20:38:19.038360+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:38:19.038360+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7577600,"num_objects":1850,"num_object_clones":0,"num_object_copies":3700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15963,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1805}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18581","reported_seq":25708,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:16.625772+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:38:16.625772+0000","last_peered":"2024-09-10T20:38:16.625772+0000","last_clean":"2024-09-10T20:38:16.625772+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:38:16.625772+0000","last_undegraded":"2024-09-10T20:38:16.625772+0000","last_fullsized":"2024-09-10T20:38:16.625772+0000","mapping_epoch":46,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7926243,"num_objects":1936,"num_object_clones":0,"num_object_copies":3872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15188,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":121886145,"num_objects":29761,"num_object_clones":0,"num_object_copies":59522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29761,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":265656,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13729,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":422547456,"data_stored":422481504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160760,"ondisk_log_size":160760,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182517,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":584216,"kb_used_data":147204,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93787624,"statfs":{"total":96636764160,"available":96038526976,"internally_reserved":0,"allocated":150736896,"data_stored":150474948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,1,0,7,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986289,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517528,"kb_used_data":174848,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93854312,"statfs":{"total":96636764160,"available":96106815488,"internally_reserved":0,"allocated":179044352,"data_stored":178782409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350875306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659235,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":544348,"kb_used_data":93368,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93827492,"statfs":{"total":96636764160,"available":96079351808,"internally_reserved":0,"allocated":95608832,"data_stored":95295843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":461762222},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,4,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":149975040,"data_stored":149971427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94232576,"data_stored":94177922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":178339840,"data_stored":178332155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:38:22.109 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:38:22.659 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:22.660 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:38:22.684 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2024-09-10T20:38:21.027252+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":121886164,"num_objects":29763,"num_object_clones":0,"num_object_copies":59526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29763,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":265658,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13731,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160763,"ondisk_log_size":160763,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1646092,"kb_used_data":415420,"kb_used_omap":12,"kb_used_meta":1230579,"kb_avail":281469428,"statfs":{"total":289910292480,"available":288224694272,"internally_reserved":0,"allocated":425390080,"data_stored":424553200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1260112898},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,4,5,3,10,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27746304,"num_objects":-6774,"num_object_clones":0,"num_object_copies":-13548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-399,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6774,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":399,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004755"},"pg_stats":[{"pgid":"2.f","version":"49'18800","reported_seq":19023,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:19.048609+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:38:19.048609+0000","last_peered":"2024-09-10T20:38:19.048609+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:38:19.048609+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7610368,"num_objects":1858,"num_object_clones":0,"num_object_copies":3716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18800,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1683}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19113","reported_seq":19234,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:19.099055+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:38:19.099055+0000","last_peered":"2024-09-10T20:38:19.099055+0000","last_clean":"2024-09-10T20:38:19.099055+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:38:19.099055+0000","last_undegraded":"2024-09-10T20:38:19.099055+0000","last_fullsized":"2024-09-10T20:38:19.099055+0000","mapping_epoch":35,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7491584,"num_objects":1829,"num_object_clones":0,"num_object_copies":3658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17299,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18800","reported_seq":26158,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:16.399395+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:38:16.399395+0000","last_peered":"2024-09-10T20:38:16.399395+0000","last_clean":"2024-09-10T20:38:16.399395+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:38:16.399395+0000","last_undegraded":"2024-09-10T20:38:16.399395+0000","last_fullsized":"2024-09-10T20:38:16.399395+0000","mapping_epoch":44,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7843864,"num_objects":1916,"num_object_clones":0,"num_object_copies":3832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1916,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16928,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'18702","reported_seq":18793,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:19.105391+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:38:19.105391+0000","last_peered":"2024-09-10T20:38:19.105391+0000","last_clean":"2024-09-10T20:38:19.105391+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:38:19.105391+0000","last_undegraded":"2024-09-10T20:38:19.105391+0000","last_fullsized":"2024-09-10T20:38:19.105391+0000","mapping_epoch":35,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7790592,"num_objects":1902,"num_object_clones":0,"num_object_copies":3804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18702,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19204","reported_seq":19341,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:19.083467+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:38:19.083467+0000","last_peered":"2024-09-10T20:38:19.083467+0000","last_clean":"2024-09-10T20:38:19.083467+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:38:19.083467+0000","last_undegraded":"2024-09-10T20:38:19.083467+0000","last_fullsized":"2024-09-10T20:38:19.083467+0000","mapping_epoch":34,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7639040,"num_objects":1865,"num_object_clones":0,"num_object_copies":3730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16200,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18620","reported_seq":18871,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:19.068046+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:38:19.068046+0000","last_peered":"2024-09-10T20:38:19.068046+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:38:19.068046+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7581696,"num_objects":1851,"num_object_clones":0,"num_object_copies":3702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15484,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1710}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'18736","reported_seq":19022,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:19.052465+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:38:19.052465+0000","last_peered":"2024-09-10T20:38:19.052465+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:38:19.052465+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7667712,"num_objects":1872,"num_object_clones":0,"num_object_copies":3744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1872,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15331,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18890","reported_seq":19025,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:19.089705+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:38:19.089705+0000","last_peered":"2024-09-10T20:38:19.089705+0000","last_clean":"2024-09-10T20:38:19.089705+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:38:19.089705+0000","last_undegraded":"2024-09-10T20:38:19.089705+0000","last_fullsized":"2024-09-10T20:38:19.089705+0000","mapping_epoch":34,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7561216,"num_objects":1846,"num_object_clones":0,"num_object_copies":3692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15961,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315452+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315452+0000","last_peered":"2024-09-10T20:36:30.315452+0000","last_clean":"2024-09-10T20:36:30.315452+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315452+0000","last_undegraded":"2024-09-10T20:36:30.315452+0000","last_fullsized":"2024-09-10T20:36:30.315452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18578","reported_seq":18779,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:19.046165+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:38:19.046165+0000","last_peered":"2024-09-10T20:38:19.046165+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:38:19.046165+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7622656,"num_objects":1861,"num_object_clones":0,"num_object_copies":3722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16571,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1636}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318059+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318059+0000","last_peered":"2024-09-10T20:36:30.318059+0000","last_clean":"2024-09-10T20:36:30.318059+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318059+0000","last_undegraded":"2024-09-10T20:36:30.318059+0000","last_fullsized":"2024-09-10T20:36:30.318059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18910","reported_seq":19126,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:19.065646+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:38:19.065646+0000","last_peered":"2024-09-10T20:38:19.065646+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:38:19.065646+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7455686,"num_objects":1822,"num_object_clones":0,"num_object_copies":3644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18908,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1744}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18737","reported_seq":18930,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:19.070807+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:38:19.070807+0000","last_peered":"2024-09-10T20:38:19.070807+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:38:19.070807+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7487488,"num_objects":1828,"num_object_clones":0,"num_object_copies":3656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16565,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1778}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18851","reported_seq":26222,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:19.089861+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:38:19.089861+0000","last_peered":"2024-09-10T20:38:19.089861+0000","last_clean":"2024-09-10T20:38:19.089861+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:38:19.089861+0000","last_undegraded":"2024-09-10T20:38:19.089861+0000","last_fullsized":"2024-09-10T20:38:19.089861+0000","mapping_epoch":48,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7426048,"num_objects":1813,"num_object_clones":0,"num_object_copies":3626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16637,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18928","reported_seq":19185,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:18.991007+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:38:18.991007+0000","last_peered":"2024-09-10T20:38:18.991007+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:38:18.991007+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7790592,"num_objects":1902,"num_object_clones":0,"num_object_copies":3804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15651,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1690}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18774","reported_seq":24188,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:19.054281+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:38:19.054281+0000","last_peered":"2024-09-10T20:38:19.054281+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:38:19.054281+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7413760,"num_objects":1810,"num_object_clones":0,"num_object_copies":3620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":903,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15468,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2632,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":907}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319644+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319644+0000","last_peered":"2024-09-10T20:36:30.319644+0000","last_clean":"2024-09-10T20:36:30.319644+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319644+0000","last_undegraded":"2024-09-10T20:36:30.319644+0000","last_fullsized":"2024-09-10T20:36:30.319644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19136","reported_seq":19393,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:19.038360+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:38:19.038360+0000","last_peered":"2024-09-10T20:38:19.038360+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:38:19.038360+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7577600,"num_objects":1850,"num_object_clones":0,"num_object_copies":3700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15963,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1805}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18581","reported_seq":25708,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:16.625772+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:38:16.625772+0000","last_peered":"2024-09-10T20:38:16.625772+0000","last_clean":"2024-09-10T20:38:16.625772+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:38:16.625772+0000","last_undegraded":"2024-09-10T20:38:16.625772+0000","last_fullsized":"2024-09-10T20:38:16.625772+0000","mapping_epoch":46,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7926243,"num_objects":1936,"num_object_clones":0,"num_object_copies":3872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15188,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":121886145,"num_objects":29761,"num_object_clones":0,"num_object_copies":59522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29761,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":265656,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13729,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":422547456,"data_stored":422481504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160760,"ondisk_log_size":160760,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182517,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":584216,"kb_used_data":147204,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93787624,"statfs":{"total":96636764160,"available":96038526976,"internally_reserved":0,"allocated":150736896,"data_stored":150474948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,1,0,7,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986289,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517528,"kb_used_data":174848,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93854312,"statfs":{"total":96636764160,"available":96106815488,"internally_reserved":0,"allocated":179044352,"data_stored":178782409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350875306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659235,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":544348,"kb_used_data":93368,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93827492,"statfs":{"total":96636764160,"available":96079351808,"internally_reserved":0,"allocated":95608832,"data_stored":95295843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":461762222},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,4,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":149975040,"data_stored":149971427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94232576,"data_stored":94177922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":178339840,"data_stored":178332155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:38:22.686 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:38:22.686 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'18800", 'reported_seq': 19023, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:19.048609+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:38:19.048609+0000', 'last_peered': '2024-09-10T20:38:19.048609+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:38:19.048609+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "45'8700", 'ondisk_log_start': "45'8700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7610368, 'num_objects': 1858, 'num_object_clones': 0, 'num_object_copies': 3716, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1858, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18800, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1683}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:22.686 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:38:22.686 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'18620", 'reported_seq': 18871, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:19.068046+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:38:19.068046+0000', 'last_peered': '2024-09-10T20:38:19.068046+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:38:19.068046+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'8600", 'ondisk_log_start': "45'8600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7581696, 'num_objects': 1851, 'num_object_clones': 0, 'num_object_copies': 3702, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1851, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15484, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1710}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:22.686 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:38:22.686 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'18736", 'reported_seq': 19022, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:19.052465+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:38:19.052465+0000', 'last_peered': '2024-09-10T20:38:19.052465+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:38:19.052465+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "45'8700", 'ondisk_log_start': "45'8700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7667712, 'num_objects': 1872, 'num_object_clones': 0, 'num_object_copies': 3744, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1872, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15331, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:22.686 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:38:22.686 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'18578", 'reported_seq': 18779, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:19.046165+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:38:19.046165+0000', 'last_peered': '2024-09-10T20:38:19.046165+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:38:19.046165+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "45'8500", 'ondisk_log_start': "45'8500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7622656, 'num_objects': 1861, 'num_object_clones': 0, 'num_object_copies': 3722, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1861, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16571, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1636}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:38:22.687 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:38:22.687 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'18910", 'reported_seq': 19126, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:19.065646+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:38:19.065646+0000', 'last_peered': '2024-09-10T20:38:19.065646+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:38:19.065646+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "45'8900", 'ondisk_log_start': "45'8900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7455686, 'num_objects': 1822, 'num_object_clones': 0, 'num_object_copies': 3644, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1822, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18908, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1744}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:22.687 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:38:22.687 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'18737", 'reported_seq': 18930, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:19.070807+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:38:19.070807+0000', 'last_peered': '2024-09-10T20:38:19.070807+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:38:19.070807+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "45'8700", 'ondisk_log_start': "45'8700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7487488, 'num_objects': 1828, 'num_object_clones': 0, 'num_object_copies': 3656, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1828, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16565, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1778}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:38:22.687 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:38:22.687 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'18928", 'reported_seq': 19185, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:18.991007+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:38:18.991007+0000', 'last_peered': '2024-09-10T20:38:18.991007+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:38:18.991007+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "45'8900", 'ondisk_log_start': "45'8900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7790592, 'num_objects': 1902, 'num_object_clones': 0, 'num_object_copies': 3804, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1902, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15651, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1690}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:22.687 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:38:22.687 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'18774", 'reported_seq': 24188, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:19.054281+0000', 'last_change': '2024-09-10T20:36:28.778006+0000', 'last_active': '2024-09-10T20:38:19.054281+0000', 'last_peered': '2024-09-10T20:38:19.054281+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:38:19.054281+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "45'8700", 'ondisk_log_start': "45'8700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7413760, 'num_objects': 1810, 'num_object_clones': 0, 'num_object_copies': 3620, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 903, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1810, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15468, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2632, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 907}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:22.687 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:38:22.687 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19136", 'reported_seq': 19393, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:19.038360+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:38:19.038360+0000', 'last_peered': '2024-09-10T20:38:19.038360+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:38:19.038360+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'9100", 'ondisk_log_start': "45'9100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7577600, 'num_objects': 1850, 'num_object_clones': 0, 'num_object_copies': 3700, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1850, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15963, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1805}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:22.688 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:38:23.369 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:23.369 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 29420/59526 objects degraded (49.424%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":614,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":29763,"data_bytes":121886164,"bytes_used":1685598208,"bytes_avail":288224694272,"bytes_total":289910292480,"degraded_objects":29420,"degraded_total":59526,"degraded_ratio":0.49423781204851663,"recovering_objects_per_sec":33,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":399,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":564},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:38:23.399 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:38:23.399 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:38:23.977 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:23.978 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:38:23.999 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":350,"stamp":"2024-09-10T20:38:23.029058+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":120882644,"num_objects":29518,"num_object_clones":0,"num_object_copies":59036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29518,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":265903,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13731,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160608,"ondisk_log_size":160608,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1645192,"kb_used_data":410380,"kb_used_omap":12,"kb_used_meta":1234739,"kb_avail":281470328,"statfs":{"total":289910292480,"available":288225615872,"internally_reserved":0,"allocated":420229120,"data_stored":419392240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1264372738},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,3,4,12,9,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19877888,"num_objects":-4853,"num_object_clones":0,"num_object_copies":-9706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-270,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4853,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005811"},"pg_stats":[{"pgid":"2.f","version":"49'18800","reported_seq":19023,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:19.048609+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:38:19.048609+0000","last_peered":"2024-09-10T20:38:19.048609+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:38:19.048609+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7610368,"num_objects":1858,"num_object_clones":0,"num_object_copies":3716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18800,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1683}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19113","reported_seq":19234,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:19.099055+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:38:19.099055+0000","last_peered":"2024-09-10T20:38:19.099055+0000","last_clean":"2024-09-10T20:38:19.099055+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:38:19.099055+0000","last_undegraded":"2024-09-10T20:38:19.099055+0000","last_fullsized":"2024-09-10T20:38:19.099055+0000","mapping_epoch":35,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7491584,"num_objects":1829,"num_object_clones":0,"num_object_copies":3658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17299,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18920","reported_seq":26278,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:21.730832+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:38:21.730832+0000","last_peered":"2024-09-10T20:38:21.730832+0000","last_clean":"2024-09-10T20:38:21.730832+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:38:21.730832+0000","last_undegraded":"2024-09-10T20:38:21.730832+0000","last_fullsized":"2024-09-10T20:38:21.730832+0000","mapping_epoch":44,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7352344,"num_objects":1796,"num_object_clones":0,"num_object_copies":3592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1796,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17048,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'18702","reported_seq":18793,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:19.105391+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:38:19.105391+0000","last_peered":"2024-09-10T20:38:19.105391+0000","last_clean":"2024-09-10T20:38:19.105391+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:38:19.105391+0000","last_undegraded":"2024-09-10T20:38:19.105391+0000","last_fullsized":"2024-09-10T20:38:19.105391+0000","mapping_epoch":35,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7790592,"num_objects":1902,"num_object_clones":0,"num_object_copies":3804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18702,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19204","reported_seq":19341,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:19.083467+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:38:19.083467+0000","last_peered":"2024-09-10T20:38:19.083467+0000","last_clean":"2024-09-10T20:38:19.083467+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:38:19.083467+0000","last_undegraded":"2024-09-10T20:38:19.083467+0000","last_fullsized":"2024-09-10T20:38:19.083467+0000","mapping_epoch":34,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7639040,"num_objects":1865,"num_object_clones":0,"num_object_copies":3730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16200,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18620","reported_seq":18871,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:19.068046+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:38:19.068046+0000","last_peered":"2024-09-10T20:38:19.068046+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:38:19.068046+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7581696,"num_objects":1851,"num_object_clones":0,"num_object_copies":3702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15484,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1710}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'18736","reported_seq":19022,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:19.052465+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:38:19.052465+0000","last_peered":"2024-09-10T20:38:19.052465+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:38:19.052465+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7667712,"num_objects":1872,"num_object_clones":0,"num_object_copies":3744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1872,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15331,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18890","reported_seq":19025,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:19.089705+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:38:19.089705+0000","last_peered":"2024-09-10T20:38:19.089705+0000","last_clean":"2024-09-10T20:38:19.089705+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:38:19.089705+0000","last_undegraded":"2024-09-10T20:38:19.089705+0000","last_fullsized":"2024-09-10T20:38:19.089705+0000","mapping_epoch":34,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7561216,"num_objects":1846,"num_object_clones":0,"num_object_copies":3692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15961,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315452+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315452+0000","last_peered":"2024-09-10T20:36:30.315452+0000","last_clean":"2024-09-10T20:36:30.315452+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315452+0000","last_undegraded":"2024-09-10T20:36:30.315452+0000","last_fullsized":"2024-09-10T20:36:30.315452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18578","reported_seq":18779,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:19.046165+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:38:19.046165+0000","last_peered":"2024-09-10T20:38:19.046165+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:38:19.046165+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7622656,"num_objects":1861,"num_object_clones":0,"num_object_copies":3722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16571,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1636}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318059+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318059+0000","last_peered":"2024-09-10T20:36:30.318059+0000","last_clean":"2024-09-10T20:36:30.318059+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318059+0000","last_undegraded":"2024-09-10T20:36:30.318059+0000","last_fullsized":"2024-09-10T20:36:30.318059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18910","reported_seq":19126,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:19.065646+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:38:19.065646+0000","last_peered":"2024-09-10T20:38:19.065646+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:38:19.065646+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7455686,"num_objects":1822,"num_object_clones":0,"num_object_copies":3644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18908,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1744}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18737","reported_seq":18930,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:19.070807+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:38:19.070807+0000","last_peered":"2024-09-10T20:38:19.070807+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:38:19.070807+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7487488,"num_objects":1828,"num_object_clones":0,"num_object_copies":3656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16565,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1778}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316202+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316202+0000","last_peered":"2024-09-10T20:36:30.316202+0000","last_clean":"2024-09-10T20:36:30.316202+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316202+0000","last_undegraded":"2024-09-10T20:36:30.316202+0000","last_fullsized":"2024-09-10T20:36:30.316202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18851","reported_seq":26222,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:19.089861+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:38:19.089861+0000","last_peered":"2024-09-10T20:38:19.089861+0000","last_clean":"2024-09-10T20:38:19.089861+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:38:19.089861+0000","last_undegraded":"2024-09-10T20:38:19.089861+0000","last_fullsized":"2024-09-10T20:38:19.089861+0000","mapping_epoch":48,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7426048,"num_objects":1813,"num_object_clones":0,"num_object_copies":3626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16637,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18928","reported_seq":19185,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:18.991007+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:38:18.991007+0000","last_peered":"2024-09-10T20:38:18.991007+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:38:18.991007+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7790592,"num_objects":1902,"num_object_clones":0,"num_object_copies":3804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15651,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1690}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316573+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316573+0000","last_peered":"2024-09-10T20:36:30.316573+0000","last_clean":"2024-09-10T20:36:30.316573+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316573+0000","last_undegraded":"2024-09-10T20:36:30.316573+0000","last_fullsized":"2024-09-10T20:36:30.316573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18774","reported_seq":24188,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:19.054281+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:38:19.054281+0000","last_peered":"2024-09-10T20:38:19.054281+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:38:19.054281+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7413760,"num_objects":1810,"num_object_clones":0,"num_object_copies":3620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":903,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15468,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2632,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":907}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319644+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319644+0000","last_peered":"2024-09-10T20:36:30.319644+0000","last_clean":"2024-09-10T20:36:30.319644+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319644+0000","last_undegraded":"2024-09-10T20:36:30.319644+0000","last_fullsized":"2024-09-10T20:36:30.319644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19136","reported_seq":19393,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:19.038360+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:38:19.038360+0000","last_peered":"2024-09-10T20:38:19.038360+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:38:19.038360+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7577600,"num_objects":1850,"num_object_clones":0,"num_object_copies":3700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15963,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1805}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18706","reported_seq":25833,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:21.683420+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:38:21.683420+0000","last_peered":"2024-09-10T20:38:21.683420+0000","last_clean":"2024-09-10T20:38:21.683420+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:38:21.683420+0000","last_undegraded":"2024-09-10T20:38:21.683420+0000","last_fullsized":"2024-09-10T20:38:21.683420+0000","mapping_epoch":46,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7414243,"num_objects":1811,"num_object_clones":0,"num_object_copies":3622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15313,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":120882625,"num_objects":29516,"num_object_clones":0,"num_object_copies":59032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29516,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":265901,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13729,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417103872,"data_stored":417037920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160605,"ondisk_log_size":160605,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182517,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":584216,"kb_used_data":147204,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93787624,"statfs":{"total":96636764160,"available":96038526976,"internally_reserved":0,"allocated":150736896,"data_stored":150474948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,1,0,7,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986289,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517528,"kb_used_data":174848,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93854312,"statfs":{"total":96636764160,"available":96106815488,"internally_reserved":0,"allocated":179044352,"data_stored":178782409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350875306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659236,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":543448,"kb_used_data":88328,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93828392,"statfs":{"total":96636764160,"available":96080273408,"internally_reserved":0,"allocated":90447872,"data_stored":90134883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":466022062},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,3,11,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":149975040,"data_stored":149971427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":88788992,"data_stored":88734338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":178339840,"data_stored":178332155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:38:27.001 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:38:27.579 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:27.582 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:38:27.612 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2024-09-10T20:38:27.030316+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":112596436,"num_objects":27495,"num_object_clones":0,"num_object_copies":54990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29286,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27495,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267926,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13865,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160831,"ondisk_log_size":160831,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1612792,"kb_used_data":394616,"kb_used_omap":12,"kb_used_meta":1218099,"kb_avail":281502728,"statfs":{"total":289910292480,"available":288258793472,"internally_reserved":0,"allocated":404086784,"data_stored":403249904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1247333378},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,8,8,10,7,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18169856,"num_objects":-4436,"num_object_clones":0,"num_object_copies":-8872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-260,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4436,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005415"},"pg_stats":[{"pgid":"2.f","version":"49'18940","reported_seq":19163,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:24.707499+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:38:24.707499+0000","last_peered":"2024-09-10T20:38:24.707499+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:38:24.707499+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7036928,"num_objects":1718,"num_object_clones":0,"num_object_copies":3436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18940,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1823}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19230","reported_seq":19351,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:24.472066+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:38:24.472066+0000","last_peered":"2024-09-10T20:38:24.472066+0000","last_clean":"2024-09-10T20:38:24.472066+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:38:24.472066+0000","last_undegraded":"2024-09-10T20:38:24.472066+0000","last_fullsized":"2024-09-10T20:38:24.472066+0000","mapping_epoch":35,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7012352,"num_objects":1712,"num_object_clones":0,"num_object_copies":3424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17416,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19068","reported_seq":26423,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:26.764189+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:38:26.764189+0000","last_peered":"2024-09-10T20:38:26.764189+0000","last_clean":"2024-09-10T20:38:26.764189+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:38:26.764189+0000","last_undegraded":"2024-09-10T20:38:26.764189+0000","last_fullsized":"2024-09-10T20:38:26.764189+0000","mapping_epoch":44,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6746136,"num_objects":1648,"num_object_clones":0,"num_object_copies":3296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1648,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17196,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'18822","reported_seq":18913,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:24.578743+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:38:24.578743+0000","last_peered":"2024-09-10T20:38:24.578743+0000","last_clean":"2024-09-10T20:38:24.578743+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:38:24.578743+0000","last_undegraded":"2024-09-10T20:38:24.578743+0000","last_fullsized":"2024-09-10T20:38:24.578743+0000","mapping_epoch":35,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7299072,"num_objects":1782,"num_object_clones":0,"num_object_copies":3564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18822,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19323","reported_seq":19460,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:24.667534+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:38:24.667534+0000","last_peered":"2024-09-10T20:38:24.667534+0000","last_clean":"2024-09-10T20:38:24.667534+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:38:24.667534+0000","last_undegraded":"2024-09-10T20:38:24.667534+0000","last_fullsized":"2024-09-10T20:38:24.667534+0000","mapping_epoch":34,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7151616,"num_objects":1746,"num_object_clones":0,"num_object_copies":3492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16319,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18745","reported_seq":18998,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:24.507844+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:38:24.507844+0000","last_peered":"2024-09-10T20:38:24.507844+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:38:24.507844+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7069696,"num_objects":1726,"num_object_clones":0,"num_object_copies":3452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15609,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1835}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'18881","reported_seq":19167,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:24.692440+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:38:24.692440+0000","last_peered":"2024-09-10T20:38:24.692440+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:38:24.692440+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7073792,"num_objects":1727,"num_object_clones":0,"num_object_copies":3454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15476,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1759}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19008","reported_seq":19142,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:24.675016+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:38:24.675016+0000","last_peered":"2024-09-10T20:38:24.675016+0000","last_clean":"2024-09-10T20:38:24.675016+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:38:24.675016+0000","last_undegraded":"2024-09-10T20:38:24.675016+0000","last_fullsized":"2024-09-10T20:38:24.675016+0000","mapping_epoch":34,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7077888,"num_objects":1728,"num_object_clones":0,"num_object_copies":3456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16079,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315452+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315452+0000","last_peered":"2024-09-10T20:36:30.315452+0000","last_clean":"2024-09-10T20:36:30.315452+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315452+0000","last_undegraded":"2024-09-10T20:36:30.315452+0000","last_fullsized":"2024-09-10T20:36:30.315452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18697","reported_seq":18898,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:24.553658+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:38:24.553658+0000","last_peered":"2024-09-10T20:38:24.553658+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:38:24.553658+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7135232,"num_objects":1742,"num_object_clones":0,"num_object_copies":3484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16690,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1755}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318059+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318059+0000","last_peered":"2024-09-10T20:36:30.318059+0000","last_clean":"2024-09-10T20:36:30.318059+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318059+0000","last_undegraded":"2024-09-10T20:36:30.318059+0000","last_fullsized":"2024-09-10T20:36:30.318059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19051","reported_seq":19268,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:24.711359+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:38:24.711359+0000","last_peered":"2024-09-10T20:38:24.711359+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:38:24.711359+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6878150,"num_objects":1681,"num_object_clones":0,"num_object_copies":3362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19049,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1885}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18857","reported_seq":19050,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:24.527585+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:38:24.527585+0000","last_peered":"2024-09-10T20:38:24.527585+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:38:24.527585+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6995968,"num_objects":1708,"num_object_clones":0,"num_object_copies":3416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16685,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1898}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316250+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316250+0000","last_peered":"2024-09-10T20:36:30.316250+0000","last_clean":"2024-09-10T20:36:30.316250+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316250+0000","last_undegraded":"2024-09-10T20:36:30.316250+0000","last_fullsized":"2024-09-10T20:36:30.316250+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18964","reported_seq":26334,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:24.545713+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:38:24.545713+0000","last_peered":"2024-09-10T20:38:24.545713+0000","last_clean":"2024-09-10T20:38:24.545713+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:38:24.545713+0000","last_undegraded":"2024-09-10T20:38:24.545713+0000","last_fullsized":"2024-09-10T20:38:24.545713+0000","mapping_epoch":48,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6963200,"num_objects":1700,"num_object_clones":0,"num_object_copies":3400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16750,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19061","reported_seq":19318,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:24.706594+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:38:24.706594+0000","last_peered":"2024-09-10T20:38:24.706594+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:38:24.706594+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7245824,"num_objects":1769,"num_object_clones":0,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15784,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1823}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316613+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316613+0000","last_peered":"2024-09-10T20:36:30.316613+0000","last_clean":"2024-09-10T20:36:30.316613+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316613+0000","last_undegraded":"2024-09-10T20:36:30.316613+0000","last_fullsized":"2024-09-10T20:36:30.316613+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18887","reported_seq":24569,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:24.722809+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:38:24.722809+0000","last_peered":"2024-09-10T20:38:24.722809+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:38:24.722809+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6950912,"num_objects":1697,"num_object_clones":0,"num_object_copies":3394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":769,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15581,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2766,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":928}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319644+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319644+0000","last_peered":"2024-09-10T20:36:30.319644+0000","last_clean":"2024-09-10T20:36:30.319644+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319644+0000","last_undegraded":"2024-09-10T20:36:30.319644+0000","last_fullsized":"2024-09-10T20:36:30.319644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19256","reported_seq":19514,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:24.553870+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:38:24.553870+0000","last_peered":"2024-09-10T20:38:24.553870+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:38:24.553870+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7086080,"num_objects":1730,"num_object_clones":0,"num_object_copies":3460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16083,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1925}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18838","reported_seq":25965,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:26.744080+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:38:26.744080+0000","last_peered":"2024-09-10T20:38:26.744080+0000","last_clean":"2024-09-10T20:38:26.744080+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:38:26.744080+0000","last_undegraded":"2024-09-10T20:38:26.744080+0000","last_fullsized":"2024-09-10T20:38:26.744080+0000","mapping_epoch":46,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6873571,"num_objects":1679,"num_object_clones":0,"num_object_copies":3358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15445,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":112596417,"num_objects":27493,"num_object_clones":0,"num_object_copies":54986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29286,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27493,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267924,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13863,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":399523840,"data_stored":399457888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160828,"ondisk_log_size":160828,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182518,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566496,"kb_used_data":141960,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93805344,"statfs":{"total":96636764160,"available":96056672256,"internally_reserved":0,"allocated":145367040,"data_stored":145105092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":434695850},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,5,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986290,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518340,"kb_used_data":171508,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93853500,"statfs":{"total":96636764160,"available":96105984000,"internally_reserved":0,"allocated":175624192,"data_stored":175362249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355135146},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,0,1,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659237,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527956,"kb_used_data":81148,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93843884,"statfs":{"total":96636764160,"available":96096137216,"internally_reserved":0,"allocated":83095552,"data_stored":82782563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":457502382},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,3,3,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":143769600,"data_stored":143765987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81285120,"data_stored":81230466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":174469120,"data_stored":174461435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:38:27.613 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:38:28.198 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:28.199 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:38:28.224 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2024-09-10T20:38:27.030316+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":112596436,"num_objects":27495,"num_object_clones":0,"num_object_copies":54990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29286,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27495,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267926,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13865,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160831,"ondisk_log_size":160831,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1612792,"kb_used_data":394616,"kb_used_omap":12,"kb_used_meta":1218099,"kb_avail":281502728,"statfs":{"total":289910292480,"available":288258793472,"internally_reserved":0,"allocated":404086784,"data_stored":403249904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1247333378},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,8,8,10,7,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18169856,"num_objects":-4436,"num_object_clones":0,"num_object_copies":-8872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-260,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4436,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005415"},"pg_stats":[{"pgid":"2.f","version":"49'18940","reported_seq":19163,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:24.707499+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:38:24.707499+0000","last_peered":"2024-09-10T20:38:24.707499+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:38:24.707499+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7036928,"num_objects":1718,"num_object_clones":0,"num_object_copies":3436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18940,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1823}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19230","reported_seq":19351,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:24.472066+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:38:24.472066+0000","last_peered":"2024-09-10T20:38:24.472066+0000","last_clean":"2024-09-10T20:38:24.472066+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:38:24.472066+0000","last_undegraded":"2024-09-10T20:38:24.472066+0000","last_fullsized":"2024-09-10T20:38:24.472066+0000","mapping_epoch":35,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7012352,"num_objects":1712,"num_object_clones":0,"num_object_copies":3424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17416,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19068","reported_seq":26423,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:26.764189+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:38:26.764189+0000","last_peered":"2024-09-10T20:38:26.764189+0000","last_clean":"2024-09-10T20:38:26.764189+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:38:26.764189+0000","last_undegraded":"2024-09-10T20:38:26.764189+0000","last_fullsized":"2024-09-10T20:38:26.764189+0000","mapping_epoch":44,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6746136,"num_objects":1648,"num_object_clones":0,"num_object_copies":3296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1648,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17196,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'18822","reported_seq":18913,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:24.578743+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:38:24.578743+0000","last_peered":"2024-09-10T20:38:24.578743+0000","last_clean":"2024-09-10T20:38:24.578743+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:38:24.578743+0000","last_undegraded":"2024-09-10T20:38:24.578743+0000","last_fullsized":"2024-09-10T20:38:24.578743+0000","mapping_epoch":35,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7299072,"num_objects":1782,"num_object_clones":0,"num_object_copies":3564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18822,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19323","reported_seq":19460,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:24.667534+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:38:24.667534+0000","last_peered":"2024-09-10T20:38:24.667534+0000","last_clean":"2024-09-10T20:38:24.667534+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:38:24.667534+0000","last_undegraded":"2024-09-10T20:38:24.667534+0000","last_fullsized":"2024-09-10T20:38:24.667534+0000","mapping_epoch":34,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7151616,"num_objects":1746,"num_object_clones":0,"num_object_copies":3492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16319,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18745","reported_seq":18998,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:24.507844+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:38:24.507844+0000","last_peered":"2024-09-10T20:38:24.507844+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:38:24.507844+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7069696,"num_objects":1726,"num_object_clones":0,"num_object_copies":3452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15609,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1835}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'18881","reported_seq":19167,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:24.692440+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:38:24.692440+0000","last_peered":"2024-09-10T20:38:24.692440+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:38:24.692440+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7073792,"num_objects":1727,"num_object_clones":0,"num_object_copies":3454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15476,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1759}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19008","reported_seq":19142,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:24.675016+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:38:24.675016+0000","last_peered":"2024-09-10T20:38:24.675016+0000","last_clean":"2024-09-10T20:38:24.675016+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:38:24.675016+0000","last_undegraded":"2024-09-10T20:38:24.675016+0000","last_fullsized":"2024-09-10T20:38:24.675016+0000","mapping_epoch":34,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7077888,"num_objects":1728,"num_object_clones":0,"num_object_copies":3456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16079,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315452+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315452+0000","last_peered":"2024-09-10T20:36:30.315452+0000","last_clean":"2024-09-10T20:36:30.315452+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315452+0000","last_undegraded":"2024-09-10T20:36:30.315452+0000","last_fullsized":"2024-09-10T20:36:30.315452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18697","reported_seq":18898,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:24.553658+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:38:24.553658+0000","last_peered":"2024-09-10T20:38:24.553658+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:38:24.553658+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7135232,"num_objects":1742,"num_object_clones":0,"num_object_copies":3484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16690,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1755}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318059+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318059+0000","last_peered":"2024-09-10T20:36:30.318059+0000","last_clean":"2024-09-10T20:36:30.318059+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318059+0000","last_undegraded":"2024-09-10T20:36:30.318059+0000","last_fullsized":"2024-09-10T20:36:30.318059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19051","reported_seq":19268,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:24.711359+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:38:24.711359+0000","last_peered":"2024-09-10T20:38:24.711359+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:38:24.711359+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6878150,"num_objects":1681,"num_object_clones":0,"num_object_copies":3362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19049,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1885}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18857","reported_seq":19050,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:24.527585+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:38:24.527585+0000","last_peered":"2024-09-10T20:38:24.527585+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:38:24.527585+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6995968,"num_objects":1708,"num_object_clones":0,"num_object_copies":3416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16685,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1898}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316250+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316250+0000","last_peered":"2024-09-10T20:36:30.316250+0000","last_clean":"2024-09-10T20:36:30.316250+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316250+0000","last_undegraded":"2024-09-10T20:36:30.316250+0000","last_fullsized":"2024-09-10T20:36:30.316250+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18964","reported_seq":26334,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:24.545713+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:38:24.545713+0000","last_peered":"2024-09-10T20:38:24.545713+0000","last_clean":"2024-09-10T20:38:24.545713+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:38:24.545713+0000","last_undegraded":"2024-09-10T20:38:24.545713+0000","last_fullsized":"2024-09-10T20:38:24.545713+0000","mapping_epoch":48,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6963200,"num_objects":1700,"num_object_clones":0,"num_object_copies":3400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16750,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19061","reported_seq":19318,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:24.706594+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:38:24.706594+0000","last_peered":"2024-09-10T20:38:24.706594+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:38:24.706594+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7245824,"num_objects":1769,"num_object_clones":0,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15784,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1823}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316613+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316613+0000","last_peered":"2024-09-10T20:36:30.316613+0000","last_clean":"2024-09-10T20:36:30.316613+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316613+0000","last_undegraded":"2024-09-10T20:36:30.316613+0000","last_fullsized":"2024-09-10T20:36:30.316613+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18887","reported_seq":24569,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:24.722809+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:38:24.722809+0000","last_peered":"2024-09-10T20:38:24.722809+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:38:24.722809+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6950912,"num_objects":1697,"num_object_clones":0,"num_object_copies":3394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":769,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15581,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2766,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":928}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319644+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319644+0000","last_peered":"2024-09-10T20:36:30.319644+0000","last_clean":"2024-09-10T20:36:30.319644+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319644+0000","last_undegraded":"2024-09-10T20:36:30.319644+0000","last_fullsized":"2024-09-10T20:36:30.319644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19256","reported_seq":19514,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:24.553870+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:38:24.553870+0000","last_peered":"2024-09-10T20:38:24.553870+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:38:24.553870+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7086080,"num_objects":1730,"num_object_clones":0,"num_object_copies":3460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16083,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1925}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18838","reported_seq":25965,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:26.744080+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:38:26.744080+0000","last_peered":"2024-09-10T20:38:26.744080+0000","last_clean":"2024-09-10T20:38:26.744080+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:38:26.744080+0000","last_undegraded":"2024-09-10T20:38:26.744080+0000","last_fullsized":"2024-09-10T20:38:26.744080+0000","mapping_epoch":46,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6873571,"num_objects":1679,"num_object_clones":0,"num_object_copies":3358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15445,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":112596417,"num_objects":27493,"num_object_clones":0,"num_object_copies":54986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29286,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27493,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267924,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13863,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":399523840,"data_stored":399457888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160828,"ondisk_log_size":160828,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182518,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566496,"kb_used_data":141960,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93805344,"statfs":{"total":96636764160,"available":96056672256,"internally_reserved":0,"allocated":145367040,"data_stored":145105092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":434695850},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,5,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986290,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518340,"kb_used_data":171508,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93853500,"statfs":{"total":96636764160,"available":96105984000,"internally_reserved":0,"allocated":175624192,"data_stored":175362249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355135146},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,0,1,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659237,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527956,"kb_used_data":81148,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93843884,"statfs":{"total":96636764160,"available":96096137216,"internally_reserved":0,"allocated":83095552,"data_stored":82782563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":457502382},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,3,3,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":143769600,"data_stored":143765987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81285120,"data_stored":81230466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":174469120,"data_stored":174461435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:38:28.225 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:38:28.225 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'18940", 'reported_seq': 19163, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:24.707499+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:38:24.707499+0000', 'last_peered': '2024-09-10T20:38:24.707499+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:38:24.707499+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "45'8900", 'ondisk_log_start': "45'8900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7036928, 'num_objects': 1718, 'num_object_clones': 0, 'num_object_copies': 3436, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1718, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18940, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1823}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:28.226 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:38:28.226 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'18745", 'reported_seq': 18998, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:24.507844+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:38:24.507844+0000', 'last_peered': '2024-09-10T20:38:24.507844+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:38:24.507844+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'8700", 'ondisk_log_start': "45'8700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7069696, 'num_objects': 1726, 'num_object_clones': 0, 'num_object_copies': 3452, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1726, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15609, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1835}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:28.226 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:38:28.226 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'18881", 'reported_seq': 19167, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:24.692440+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:38:24.692440+0000', 'last_peered': '2024-09-10T20:38:24.692440+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:38:24.692440+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "45'8800", 'ondisk_log_start': "45'8800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7073792, 'num_objects': 1727, 'num_object_clones': 0, 'num_object_copies': 3454, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1727, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15476, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1759}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:28.226 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:38:28.226 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'18697", 'reported_seq': 18898, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:24.553658+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:38:24.553658+0000', 'last_peered': '2024-09-10T20:38:24.553658+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:38:24.553658+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "45'8600", 'ondisk_log_start': "45'8600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7135232, 'num_objects': 1742, 'num_object_clones': 0, 'num_object_copies': 3484, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1742, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16690, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1755}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:38:28.226 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:38:28.226 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19051", 'reported_seq': 19268, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:24.711359+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:38:24.711359+0000', 'last_peered': '2024-09-10T20:38:24.711359+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:38:24.711359+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "45'9000", 'ondisk_log_start': "45'9000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6878150, 'num_objects': 1681, 'num_object_clones': 0, 'num_object_copies': 3362, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1681, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19049, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1885}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:28.226 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:38:28.226 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'18857", 'reported_seq': 19050, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:24.527585+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:38:24.527585+0000', 'last_peered': '2024-09-10T20:38:24.527585+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:38:24.527585+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "45'8800", 'ondisk_log_start': "45'8800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6995968, 'num_objects': 1708, 'num_object_clones': 0, 'num_object_copies': 3416, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1708, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16685, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1898}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:38:28.227 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:38:28.227 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'19061", 'reported_seq': 19318, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:24.706594+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:38:24.706594+0000', 'last_peered': '2024-09-10T20:38:24.706594+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:38:24.706594+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "45'9000", 'ondisk_log_start': "45'9000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7245824, 'num_objects': 1769, 'num_object_clones': 0, 'num_object_copies': 3538, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1769, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15784, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1823}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:28.227 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:38:28.227 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'18887", 'reported_seq': 24569, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:24.722809+0000', 'last_change': '2024-09-10T20:36:28.778006+0000', 'last_active': '2024-09-10T20:38:24.722809+0000', 'last_peered': '2024-09-10T20:38:24.722809+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:38:24.722809+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "45'8800", 'ondisk_log_start': "45'8800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6950912, 'num_objects': 1697, 'num_object_clones': 0, 'num_object_copies': 3394, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 769, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1697, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15581, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2766, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 928}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:28.227 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:38:28.227 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19256", 'reported_seq': 19514, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:24.553870+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:38:24.553870+0000', 'last_peered': '2024-09-10T20:38:24.553870+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:38:24.553870+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'9200", 'ondisk_log_start': "45'9200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7086080, 'num_objects': 1730, 'num_object_clones': 0, 'num_object_copies': 3460, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1730, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16083, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1925}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:28.227 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:38:28.947 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:28.947 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 29286/54990 objects degraded (53.257%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":620,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":27495,"data_bytes":112596436,"bytes_used":1651499008,"bytes_avail":288258793472,"bytes_total":289910292480,"degraded_objects":29286,"degraded_total":54990,"degraded_ratio":0.532569558101473,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":260,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":369},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:38:28.967 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:38:28.967 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:38:29.520 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:29.521 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:38:29.542 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":353,"stamp":"2024-09-10T20:38:29.031941+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":112596436,"num_objects":27495,"num_object_clones":0,"num_object_copies":54990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29286,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27495,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267926,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13865,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160831,"ondisk_log_size":160831,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1612792,"kb_used_data":394616,"kb_used_omap":12,"kb_used_meta":1218099,"kb_avail":281502728,"statfs":{"total":289910292480,"available":288258793472,"internally_reserved":0,"allocated":404086784,"data_stored":403249904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1247333378},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,8,8,10,7,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17010688,"num_objects":-4153,"num_object_clones":0,"num_object_copies":-8306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-260,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4153,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4153,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006282"},"pg_stats":[{"pgid":"2.f","version":"49'18940","reported_seq":19163,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:24.707499+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:38:24.707499+0000","last_peered":"2024-09-10T20:38:24.707499+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:38:24.707499+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7036928,"num_objects":1718,"num_object_clones":0,"num_object_copies":3436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18940,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1823}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19230","reported_seq":19351,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:24.472066+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:38:24.472066+0000","last_peered":"2024-09-10T20:38:24.472066+0000","last_clean":"2024-09-10T20:38:24.472066+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:38:24.472066+0000","last_undegraded":"2024-09-10T20:38:24.472066+0000","last_fullsized":"2024-09-10T20:38:24.472066+0000","mapping_epoch":35,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7012352,"num_objects":1712,"num_object_clones":0,"num_object_copies":3424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17416,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19068","reported_seq":26423,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:26.764189+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:38:26.764189+0000","last_peered":"2024-09-10T20:38:26.764189+0000","last_clean":"2024-09-10T20:38:26.764189+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:38:26.764189+0000","last_undegraded":"2024-09-10T20:38:26.764189+0000","last_fullsized":"2024-09-10T20:38:26.764189+0000","mapping_epoch":44,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6746136,"num_objects":1648,"num_object_clones":0,"num_object_copies":3296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1648,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17196,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'18822","reported_seq":18913,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:24.578743+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:38:24.578743+0000","last_peered":"2024-09-10T20:38:24.578743+0000","last_clean":"2024-09-10T20:38:24.578743+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:38:24.578743+0000","last_undegraded":"2024-09-10T20:38:24.578743+0000","last_fullsized":"2024-09-10T20:38:24.578743+0000","mapping_epoch":35,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7299072,"num_objects":1782,"num_object_clones":0,"num_object_copies":3564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18822,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19323","reported_seq":19460,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:24.667534+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:38:24.667534+0000","last_peered":"2024-09-10T20:38:24.667534+0000","last_clean":"2024-09-10T20:38:24.667534+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:38:24.667534+0000","last_undegraded":"2024-09-10T20:38:24.667534+0000","last_fullsized":"2024-09-10T20:38:24.667534+0000","mapping_epoch":34,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7151616,"num_objects":1746,"num_object_clones":0,"num_object_copies":3492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16319,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18745","reported_seq":18998,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:24.507844+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:38:24.507844+0000","last_peered":"2024-09-10T20:38:24.507844+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:38:24.507844+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7069696,"num_objects":1726,"num_object_clones":0,"num_object_copies":3452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15609,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1835}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'18881","reported_seq":19167,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:24.692440+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:38:24.692440+0000","last_peered":"2024-09-10T20:38:24.692440+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:38:24.692440+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7073792,"num_objects":1727,"num_object_clones":0,"num_object_copies":3454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15476,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1759}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19008","reported_seq":19142,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:24.675016+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:38:24.675016+0000","last_peered":"2024-09-10T20:38:24.675016+0000","last_clean":"2024-09-10T20:38:24.675016+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:38:24.675016+0000","last_undegraded":"2024-09-10T20:38:24.675016+0000","last_fullsized":"2024-09-10T20:38:24.675016+0000","mapping_epoch":34,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7077888,"num_objects":1728,"num_object_clones":0,"num_object_copies":3456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16079,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315452+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315452+0000","last_peered":"2024-09-10T20:36:30.315452+0000","last_clean":"2024-09-10T20:36:30.315452+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315452+0000","last_undegraded":"2024-09-10T20:36:30.315452+0000","last_fullsized":"2024-09-10T20:36:30.315452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18697","reported_seq":18898,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:24.553658+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:38:24.553658+0000","last_peered":"2024-09-10T20:38:24.553658+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:38:24.553658+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7135232,"num_objects":1742,"num_object_clones":0,"num_object_copies":3484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16690,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1755}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318059+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318059+0000","last_peered":"2024-09-10T20:36:30.318059+0000","last_clean":"2024-09-10T20:36:30.318059+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318059+0000","last_undegraded":"2024-09-10T20:36:30.318059+0000","last_fullsized":"2024-09-10T20:36:30.318059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19051","reported_seq":19268,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:24.711359+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:38:24.711359+0000","last_peered":"2024-09-10T20:38:24.711359+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:38:24.711359+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6878150,"num_objects":1681,"num_object_clones":0,"num_object_copies":3362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19049,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1885}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18857","reported_seq":19050,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:24.527585+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:38:24.527585+0000","last_peered":"2024-09-10T20:38:24.527585+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:38:24.527585+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6995968,"num_objects":1708,"num_object_clones":0,"num_object_copies":3416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16685,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1898}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316250+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316250+0000","last_peered":"2024-09-10T20:36:30.316250+0000","last_clean":"2024-09-10T20:36:30.316250+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316250+0000","last_undegraded":"2024-09-10T20:36:30.316250+0000","last_fullsized":"2024-09-10T20:36:30.316250+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18964","reported_seq":26334,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:24.545713+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:38:24.545713+0000","last_peered":"2024-09-10T20:38:24.545713+0000","last_clean":"2024-09-10T20:38:24.545713+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:38:24.545713+0000","last_undegraded":"2024-09-10T20:38:24.545713+0000","last_fullsized":"2024-09-10T20:38:24.545713+0000","mapping_epoch":48,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6963200,"num_objects":1700,"num_object_clones":0,"num_object_copies":3400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16750,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19061","reported_seq":19318,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:24.706594+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:38:24.706594+0000","last_peered":"2024-09-10T20:38:24.706594+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:38:24.706594+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7245824,"num_objects":1769,"num_object_clones":0,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15784,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1823}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316613+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316613+0000","last_peered":"2024-09-10T20:36:30.316613+0000","last_clean":"2024-09-10T20:36:30.316613+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316613+0000","last_undegraded":"2024-09-10T20:36:30.316613+0000","last_fullsized":"2024-09-10T20:36:30.316613+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18887","reported_seq":24569,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:24.722809+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:38:24.722809+0000","last_peered":"2024-09-10T20:38:24.722809+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:38:24.722809+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6950912,"num_objects":1697,"num_object_clones":0,"num_object_copies":3394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":769,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15581,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2766,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":928}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319644+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319644+0000","last_peered":"2024-09-10T20:36:30.319644+0000","last_clean":"2024-09-10T20:36:30.319644+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319644+0000","last_undegraded":"2024-09-10T20:36:30.319644+0000","last_fullsized":"2024-09-10T20:36:30.319644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19256","reported_seq":19514,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:24.553870+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:38:24.553870+0000","last_peered":"2024-09-10T20:38:24.553870+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:38:24.553870+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7086080,"num_objects":1730,"num_object_clones":0,"num_object_copies":3460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16083,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1925}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316959+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316959+0000","last_peered":"2024-09-10T20:36:30.316959+0000","last_clean":"2024-09-10T20:36:30.316959+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316959+0000","last_undegraded":"2024-09-10T20:36:30.316959+0000","last_fullsized":"2024-09-10T20:36:30.316959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18838","reported_seq":25965,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:26.744080+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:38:26.744080+0000","last_peered":"2024-09-10T20:38:26.744080+0000","last_clean":"2024-09-10T20:38:26.744080+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:38:26.744080+0000","last_undegraded":"2024-09-10T20:38:26.744080+0000","last_fullsized":"2024-09-10T20:38:26.744080+0000","mapping_epoch":46,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6873571,"num_objects":1679,"num_object_clones":0,"num_object_copies":3358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15445,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":112596417,"num_objects":27493,"num_object_clones":0,"num_object_copies":54986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29286,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27493,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267924,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13863,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":399523840,"data_stored":399457888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160828,"ondisk_log_size":160828,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182518,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566496,"kb_used_data":141960,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93805344,"statfs":{"total":96636764160,"available":96056672256,"internally_reserved":0,"allocated":145367040,"data_stored":145105092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":434695850},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,5,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986290,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518340,"kb_used_data":171508,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93853500,"statfs":{"total":96636764160,"available":96105984000,"internally_reserved":0,"allocated":175624192,"data_stored":175362249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355135146},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,0,1,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659237,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527956,"kb_used_data":81148,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93843884,"statfs":{"total":96636764160,"available":96096137216,"internally_reserved":0,"allocated":83095552,"data_stored":82782563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":457502382},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,3,3,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":143769600,"data_stored":143765987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81285120,"data_stored":81230466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":174469120,"data_stored":174461435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:38:32.545 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:38:33.109 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:33.112 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:38:33.141 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-09-10T20:38:33.033413+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":102512084,"num_objects":25033,"num_object_clones":0,"num_object_copies":50066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25033,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":270388,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14019,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160793,"ondisk_log_size":160793,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1605448,"kb_used_data":374796,"kb_used_omap":12,"kb_used_meta":1230579,"kb_avail":281510072,"statfs":{"total":289910292480,"available":288266313728,"internally_reserved":0,"allocated":383791104,"data_stored":382954224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1260112898},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,2,2,6,11,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19374080,"num_objects":-4730,"num_object_clones":0,"num_object_copies":-9460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-288,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4730,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006161"},"pg_stats":[{"pgid":"2.f","version":"49'19101","reported_seq":19324,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:29.644824+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:38:29.644824+0000","last_peered":"2024-09-10T20:38:29.644824+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:38:29.644824+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6377472,"num_objects":1557,"num_object_clones":0,"num_object_copies":3114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1557,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19101,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19381","reported_seq":19502,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:29.594611+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:38:29.594611+0000","last_peered":"2024-09-10T20:38:29.594611+0000","last_clean":"2024-09-10T20:38:29.594611+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:38:29.594611+0000","last_undegraded":"2024-09-10T20:38:29.594611+0000","last_fullsized":"2024-09-10T20:38:29.594611+0000","mapping_epoch":35,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6393856,"num_objects":1561,"num_object_clones":0,"num_object_copies":3122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17567,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19223","reported_seq":26581,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:31.804904+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:38:31.804904+0000","last_peered":"2024-09-10T20:38:31.804904+0000","last_clean":"2024-09-10T20:38:31.804904+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:38:31.804904+0000","last_undegraded":"2024-09-10T20:38:31.804904+0000","last_fullsized":"2024-09-10T20:38:31.804904+0000","mapping_epoch":44,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6111256,"num_objects":1493,"num_object_clones":0,"num_object_copies":2986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1493,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17351,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'18954","reported_seq":19045,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:29.618174+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:38:29.618174+0000","last_peered":"2024-09-10T20:38:29.618174+0000","last_clean":"2024-09-10T20:38:29.618174+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:38:29.618174+0000","last_undegraded":"2024-09-10T20:38:29.618174+0000","last_fullsized":"2024-09-10T20:38:29.618174+0000","mapping_epoch":35,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6758400,"num_objects":1650,"num_object_clones":0,"num_object_copies":3300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18954,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19472","reported_seq":19609,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:29.638532+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:38:29.638532+0000","last_peered":"2024-09-10T20:38:29.638532+0000","last_clean":"2024-09-10T20:38:29.638532+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:38:29.638532+0000","last_undegraded":"2024-09-10T20:38:29.638532+0000","last_fullsized":"2024-09-10T20:38:29.638532+0000","mapping_epoch":34,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6541312,"num_objects":1597,"num_object_clones":0,"num_object_copies":3194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16468,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18883","reported_seq":19136,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:29.678001+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:38:29.678001+0000","last_peered":"2024-09-10T20:38:29.678001+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:38:29.678001+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6504448,"num_objects":1588,"num_object_clones":0,"num_object_copies":3176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15747,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1973}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19033","reported_seq":19318,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:29.722610+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:38:29.722610+0000","last_peered":"2024-09-10T20:38:29.722610+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:38:29.722610+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6451200,"num_objects":1575,"num_object_clones":0,"num_object_copies":3150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15628,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1911}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19169","reported_seq":19304,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:29.685154+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:38:29.685154+0000","last_peered":"2024-09-10T20:38:29.685154+0000","last_clean":"2024-09-10T20:38:29.685154+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:38:29.685154+0000","last_undegraded":"2024-09-10T20:38:29.685154+0000","last_fullsized":"2024-09-10T20:38:29.685154+0000","mapping_epoch":34,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6418432,"num_objects":1567,"num_object_clones":0,"num_object_copies":3134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16240,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315452+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315452+0000","last_peered":"2024-09-10T20:36:30.315452+0000","last_clean":"2024-09-10T20:36:30.315452+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315452+0000","last_undegraded":"2024-09-10T20:36:30.315452+0000","last_fullsized":"2024-09-10T20:36:30.315452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18864","reported_seq":19063,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:29.536221+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:38:29.536221+0000","last_peered":"2024-09-10T20:38:29.536221+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:38:29.536221+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6451200,"num_objects":1575,"num_object_clones":0,"num_object_copies":3150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16857,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1922}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318059+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318059+0000","last_peered":"2024-09-10T20:36:30.318059+0000","last_clean":"2024-09-10T20:36:30.318059+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318059+0000","last_undegraded":"2024-09-10T20:36:30.318059+0000","last_fullsized":"2024-09-10T20:36:30.318059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19177","reported_seq":19394,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:29.694974+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:38:29.694974+0000","last_peered":"2024-09-10T20:38:29.694974+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:38:29.694974+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6362054,"num_objects":1555,"num_object_clones":0,"num_object_copies":3110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19175,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2011}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19006","reported_seq":19199,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:29.570167+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:38:29.570167+0000","last_peered":"2024-09-10T20:38:29.570167+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:38:29.570167+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6385664,"num_objects":1559,"num_object_clones":0,"num_object_copies":3118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16834,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2047}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316250+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316250+0000","last_peered":"2024-09-10T20:36:30.316250+0000","last_clean":"2024-09-10T20:36:30.316250+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316250+0000","last_undegraded":"2024-09-10T20:36:30.316250+0000","last_fullsized":"2024-09-10T20:36:30.316250+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19136","reported_seq":26507,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:29.553239+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:38:29.553239+0000","last_peered":"2024-09-10T20:38:29.553239+0000","last_clean":"2024-09-10T20:38:29.553239+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:38:29.553239+0000","last_undegraded":"2024-09-10T20:38:29.553239+0000","last_fullsized":"2024-09-10T20:38:29.553239+0000","mapping_epoch":48,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6258688,"num_objects":1528,"num_object_clones":0,"num_object_copies":3056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16922,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19239","reported_seq":19492,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:29.728338+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:38:29.728338+0000","last_peered":"2024-09-10T20:38:29.728338+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:38:29.728338+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6516736,"num_objects":1591,"num_object_clones":0,"num_object_copies":3182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15962,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2001}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316613+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316613+0000","last_peered":"2024-09-10T20:36:30.316613+0000","last_clean":"2024-09-10T20:36:30.316613+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316613+0000","last_undegraded":"2024-09-10T20:36:30.316613+0000","last_fullsized":"2024-09-10T20:36:30.316613+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19034","reported_seq":25020,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:29.738138+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:38:29.738138+0000","last_peered":"2024-09-10T20:38:29.738138+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:38:29.738138+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6348800,"num_objects":1550,"num_object_clones":0,"num_object_copies":3100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":615,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1550,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15728,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":935}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319644+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319644+0000","last_peered":"2024-09-10T20:36:30.319644+0000","last_clean":"2024-09-10T20:36:30.319644+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319644+0000","last_undegraded":"2024-09-10T20:36:30.319644+0000","last_fullsized":"2024-09-10T20:36:30.319644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19437","reported_seq":19695,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:29.743507+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:38:29.743507+0000","last_peered":"2024-09-10T20:38:29.743507+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:38:29.743507+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6344704,"num_objects":1549,"num_object_clones":0,"num_object_copies":3098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16264,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2106}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316994+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316994+0000","last_peered":"2024-09-10T20:36:30.316994+0000","last_clean":"2024-09-10T20:36:30.316994+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316994+0000","last_undegraded":"2024-09-10T20:36:30.316994+0000","last_fullsized":"2024-09-10T20:36:30.316994+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18981","reported_seq":26108,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:31.743027+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:38:31.743027+0000","last_peered":"2024-09-10T20:38:31.743027+0000","last_clean":"2024-09-10T20:38:31.743027+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:38:31.743027+0000","last_undegraded":"2024-09-10T20:38:31.743027+0000","last_fullsized":"2024-09-10T20:38:31.743027+0000","mapping_epoch":46,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6287843,"num_objects":1536,"num_object_clones":0,"num_object_copies":3072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15588,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":102512065,"num_objects":25031,"num_object_clones":0,"num_object_copies":50062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25031,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":270386,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14017,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":379559936,"data_stored":379493984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160790,"ondisk_log_size":160790,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182519,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":563432,"kb_used_data":134736,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93808408,"statfs":{"total":96636764160,"available":96059809792,"internally_reserved":0,"allocated":137969664,"data_stored":137707716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":438955690},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,1,2,5,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986291,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517464,"kb_used_data":166468,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":93854376,"statfs":{"total":96636764160,"available":96106881024,"internally_reserved":0,"allocated":170463232,"data_stored":170201289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":359394986},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659238,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":524552,"kb_used_data":73592,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93847288,"statfs":{"total":96636764160,"available":96099622912,"internally_reserved":0,"allocated":75358208,"data_stored":75045219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":461762222},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,3,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":136413184,"data_stored":136409571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73723904,"data_stored":73669250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":169422848,"data_stored":169415163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:38:33.142 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:38:33.590 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:33.591 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:38:33.617 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-09-10T20:38:33.033413+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":102512084,"num_objects":25033,"num_object_clones":0,"num_object_copies":50066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25033,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":270388,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14019,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160793,"ondisk_log_size":160793,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1605448,"kb_used_data":374796,"kb_used_omap":12,"kb_used_meta":1230579,"kb_avail":281510072,"statfs":{"total":289910292480,"available":288266313728,"internally_reserved":0,"allocated":383791104,"data_stored":382954224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1260112898},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,2,2,6,11,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19374080,"num_objects":-4730,"num_object_clones":0,"num_object_copies":-9460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-288,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4730,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006161"},"pg_stats":[{"pgid":"2.f","version":"49'19101","reported_seq":19324,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:29.644824+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:38:29.644824+0000","last_peered":"2024-09-10T20:38:29.644824+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:38:29.644824+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6377472,"num_objects":1557,"num_object_clones":0,"num_object_copies":3114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1557,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19101,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19381","reported_seq":19502,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:29.594611+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:38:29.594611+0000","last_peered":"2024-09-10T20:38:29.594611+0000","last_clean":"2024-09-10T20:38:29.594611+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:38:29.594611+0000","last_undegraded":"2024-09-10T20:38:29.594611+0000","last_fullsized":"2024-09-10T20:38:29.594611+0000","mapping_epoch":35,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6393856,"num_objects":1561,"num_object_clones":0,"num_object_copies":3122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17567,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19223","reported_seq":26581,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:31.804904+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:38:31.804904+0000","last_peered":"2024-09-10T20:38:31.804904+0000","last_clean":"2024-09-10T20:38:31.804904+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:38:31.804904+0000","last_undegraded":"2024-09-10T20:38:31.804904+0000","last_fullsized":"2024-09-10T20:38:31.804904+0000","mapping_epoch":44,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6111256,"num_objects":1493,"num_object_clones":0,"num_object_copies":2986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1493,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17351,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'18954","reported_seq":19045,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:29.618174+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:38:29.618174+0000","last_peered":"2024-09-10T20:38:29.618174+0000","last_clean":"2024-09-10T20:38:29.618174+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:38:29.618174+0000","last_undegraded":"2024-09-10T20:38:29.618174+0000","last_fullsized":"2024-09-10T20:38:29.618174+0000","mapping_epoch":35,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6758400,"num_objects":1650,"num_object_clones":0,"num_object_copies":3300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18954,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19472","reported_seq":19609,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:29.638532+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:38:29.638532+0000","last_peered":"2024-09-10T20:38:29.638532+0000","last_clean":"2024-09-10T20:38:29.638532+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:38:29.638532+0000","last_undegraded":"2024-09-10T20:38:29.638532+0000","last_fullsized":"2024-09-10T20:38:29.638532+0000","mapping_epoch":34,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6541312,"num_objects":1597,"num_object_clones":0,"num_object_copies":3194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16468,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18883","reported_seq":19136,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:29.678001+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:38:29.678001+0000","last_peered":"2024-09-10T20:38:29.678001+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:38:29.678001+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6504448,"num_objects":1588,"num_object_clones":0,"num_object_copies":3176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15747,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1973}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19033","reported_seq":19318,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:29.722610+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:38:29.722610+0000","last_peered":"2024-09-10T20:38:29.722610+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:38:29.722610+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6451200,"num_objects":1575,"num_object_clones":0,"num_object_copies":3150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15628,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1911}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19169","reported_seq":19304,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:29.685154+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:38:29.685154+0000","last_peered":"2024-09-10T20:38:29.685154+0000","last_clean":"2024-09-10T20:38:29.685154+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:38:29.685154+0000","last_undegraded":"2024-09-10T20:38:29.685154+0000","last_fullsized":"2024-09-10T20:38:29.685154+0000","mapping_epoch":34,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6418432,"num_objects":1567,"num_object_clones":0,"num_object_copies":3134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16240,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315452+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315452+0000","last_peered":"2024-09-10T20:36:30.315452+0000","last_clean":"2024-09-10T20:36:30.315452+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315452+0000","last_undegraded":"2024-09-10T20:36:30.315452+0000","last_fullsized":"2024-09-10T20:36:30.315452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18864","reported_seq":19063,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:29.536221+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:38:29.536221+0000","last_peered":"2024-09-10T20:38:29.536221+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:38:29.536221+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6451200,"num_objects":1575,"num_object_clones":0,"num_object_copies":3150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16857,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1922}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318059+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318059+0000","last_peered":"2024-09-10T20:36:30.318059+0000","last_clean":"2024-09-10T20:36:30.318059+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318059+0000","last_undegraded":"2024-09-10T20:36:30.318059+0000","last_fullsized":"2024-09-10T20:36:30.318059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19177","reported_seq":19394,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:29.694974+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:38:29.694974+0000","last_peered":"2024-09-10T20:38:29.694974+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:38:29.694974+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6362054,"num_objects":1555,"num_object_clones":0,"num_object_copies":3110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19175,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2011}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19006","reported_seq":19199,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:29.570167+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:38:29.570167+0000","last_peered":"2024-09-10T20:38:29.570167+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:38:29.570167+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6385664,"num_objects":1559,"num_object_clones":0,"num_object_copies":3118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16834,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2047}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316250+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316250+0000","last_peered":"2024-09-10T20:36:30.316250+0000","last_clean":"2024-09-10T20:36:30.316250+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316250+0000","last_undegraded":"2024-09-10T20:36:30.316250+0000","last_fullsized":"2024-09-10T20:36:30.316250+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19136","reported_seq":26507,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:29.553239+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:38:29.553239+0000","last_peered":"2024-09-10T20:38:29.553239+0000","last_clean":"2024-09-10T20:38:29.553239+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:38:29.553239+0000","last_undegraded":"2024-09-10T20:38:29.553239+0000","last_fullsized":"2024-09-10T20:38:29.553239+0000","mapping_epoch":48,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6258688,"num_objects":1528,"num_object_clones":0,"num_object_copies":3056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16922,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19239","reported_seq":19492,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:29.728338+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:38:29.728338+0000","last_peered":"2024-09-10T20:38:29.728338+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:38:29.728338+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6516736,"num_objects":1591,"num_object_clones":0,"num_object_copies":3182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15962,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2001}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316613+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316613+0000","last_peered":"2024-09-10T20:36:30.316613+0000","last_clean":"2024-09-10T20:36:30.316613+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316613+0000","last_undegraded":"2024-09-10T20:36:30.316613+0000","last_fullsized":"2024-09-10T20:36:30.316613+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19034","reported_seq":25020,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:29.738138+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:38:29.738138+0000","last_peered":"2024-09-10T20:38:29.738138+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:38:29.738138+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6348800,"num_objects":1550,"num_object_clones":0,"num_object_copies":3100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":615,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1550,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15728,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":935}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319644+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319644+0000","last_peered":"2024-09-10T20:36:30.319644+0000","last_clean":"2024-09-10T20:36:30.319644+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319644+0000","last_undegraded":"2024-09-10T20:36:30.319644+0000","last_fullsized":"2024-09-10T20:36:30.319644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19437","reported_seq":19695,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:29.743507+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:38:29.743507+0000","last_peered":"2024-09-10T20:38:29.743507+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:38:29.743507+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6344704,"num_objects":1549,"num_object_clones":0,"num_object_copies":3098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16264,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2106}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316994+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316994+0000","last_peered":"2024-09-10T20:36:30.316994+0000","last_clean":"2024-09-10T20:36:30.316994+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316994+0000","last_undegraded":"2024-09-10T20:36:30.316994+0000","last_fullsized":"2024-09-10T20:36:30.316994+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18981","reported_seq":26108,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:31.743027+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:38:31.743027+0000","last_peered":"2024-09-10T20:38:31.743027+0000","last_clean":"2024-09-10T20:38:31.743027+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:38:31.743027+0000","last_undegraded":"2024-09-10T20:38:31.743027+0000","last_fullsized":"2024-09-10T20:38:31.743027+0000","mapping_epoch":46,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6287843,"num_objects":1536,"num_object_clones":0,"num_object_copies":3072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15588,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":102512065,"num_objects":25031,"num_object_clones":0,"num_object_copies":50062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25031,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":270386,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14017,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":379559936,"data_stored":379493984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160790,"ondisk_log_size":160790,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182519,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":563432,"kb_used_data":134736,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93808408,"statfs":{"total":96636764160,"available":96059809792,"internally_reserved":0,"allocated":137969664,"data_stored":137707716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":438955690},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,1,2,5,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986291,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517464,"kb_used_data":166468,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":93854376,"statfs":{"total":96636764160,"available":96106881024,"internally_reserved":0,"allocated":170463232,"data_stored":170201289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":359394986},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659238,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":524552,"kb_used_data":73592,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93847288,"statfs":{"total":96636764160,"available":96099622912,"internally_reserved":0,"allocated":75358208,"data_stored":75045219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":461762222},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,3,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":136413184,"data_stored":136409571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73723904,"data_stored":73669250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":169422848,"data_stored":169415163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:38:33.618 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:38:33.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'19101", 'reported_seq': 19324, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:29.644824+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:38:29.644824+0000', 'last_peered': '2024-09-10T20:38:29.644824+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:38:29.644824+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "45'9100", 'ondisk_log_start': "45'9100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6377472, 'num_objects': 1557, 'num_object_clones': 0, 'num_object_copies': 3114, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1557, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19101, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:33.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:38:33.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'18883", 'reported_seq': 19136, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:29.678001+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:38:29.678001+0000', 'last_peered': '2024-09-10T20:38:29.678001+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:38:29.678001+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'8800", 'ondisk_log_start': "45'8800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6504448, 'num_objects': 1588, 'num_object_clones': 0, 'num_object_copies': 3176, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1588, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15747, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1973}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:33.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:38:33.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19033", 'reported_seq': 19318, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:29.722610+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:38:29.722610+0000', 'last_peered': '2024-09-10T20:38:29.722610+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:38:29.722610+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "45'9000", 'ondisk_log_start': "45'9000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6451200, 'num_objects': 1575, 'num_object_clones': 0, 'num_object_copies': 3150, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1575, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15628, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1911}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:33.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:38:33.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'18864", 'reported_seq': 19063, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:29.536221+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:38:29.536221+0000', 'last_peered': '2024-09-10T20:38:29.536221+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:38:29.536221+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "45'8800", 'ondisk_log_start': "45'8800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6451200, 'num_objects': 1575, 'num_object_clones': 0, 'num_object_copies': 3150, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1575, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16857, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1922}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:38:33.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:38:33.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19177", 'reported_seq': 19394, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:29.694974+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:38:29.694974+0000', 'last_peered': '2024-09-10T20:38:29.694974+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:38:29.694974+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "45'9100", 'ondisk_log_start': "45'9100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6362054, 'num_objects': 1555, 'num_object_clones': 0, 'num_object_copies': 3110, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1555, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19175, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2011}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:33.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:38:33.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'19006", 'reported_seq': 19199, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:29.570167+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:38:29.570167+0000', 'last_peered': '2024-09-10T20:38:29.570167+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:38:29.570167+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "45'9000", 'ondisk_log_start': "45'9000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6385664, 'num_objects': 1559, 'num_object_clones': 0, 'num_object_copies': 3118, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1559, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16834, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2047}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:38:33.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:38:33.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'19239", 'reported_seq': 19492, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:29.728338+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:38:29.728338+0000', 'last_peered': '2024-09-10T20:38:29.728338+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:38:29.728338+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "45'9200", 'ondisk_log_start': "45'9200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6516736, 'num_objects': 1591, 'num_object_clones': 0, 'num_object_copies': 3182, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1591, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15962, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2001}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:33.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:38:33.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19034", 'reported_seq': 25020, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:29.738138+0000', 'last_change': '2024-09-10T20:36:28.778006+0000', 'last_active': '2024-09-10T20:38:29.738138+0000', 'last_peered': '2024-09-10T20:38:29.738138+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:38:29.738138+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "45'9000", 'ondisk_log_start': "45'9000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6348800, 'num_objects': 1550, 'num_object_clones': 0, 'num_object_copies': 3100, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 615, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1550, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15728, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2920, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 935}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:33.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:38:33.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19437", 'reported_seq': 19695, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:29.743507+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:38:29.743507+0000', 'last_peered': '2024-09-10T20:38:29.743507+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:38:29.743507+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'9400", 'ondisk_log_start': "45'9400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6344704, 'num_objects': 1549, 'num_object_clones': 0, 'num_object_copies': 3098, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1549, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16264, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2106}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:33.621 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:38:34.328 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:34.328 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 29132/50066 objects degraded (58.187%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":625,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":25033,"data_bytes":102512084,"bytes_used":1643978752,"bytes_avail":288266313728,"bytes_total":289910292480,"degraded_objects":29132,"degraded_total":50066,"degraded_ratio":0.58187192905364915,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":288,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":393},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:38:34.357 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:38:34.358 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:38:34.890 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:34.891 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:38:34.915 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-09-10T20:38:33.033413+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":102512084,"num_objects":25033,"num_object_clones":0,"num_object_copies":50066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25033,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":270388,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14019,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160793,"ondisk_log_size":160793,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1605448,"kb_used_data":374796,"kb_used_omap":12,"kb_used_meta":1230579,"kb_avail":281510072,"statfs":{"total":289910292480,"available":288266313728,"internally_reserved":0,"allocated":383791104,"data_stored":382954224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1260112898},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,2,2,6,11,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19374080,"num_objects":-4730,"num_object_clones":0,"num_object_copies":-9460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-288,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4730,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006161"},"pg_stats":[{"pgid":"2.f","version":"49'19101","reported_seq":19324,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:29.644824+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:38:29.644824+0000","last_peered":"2024-09-10T20:38:29.644824+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:38:29.644824+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6377472,"num_objects":1557,"num_object_clones":0,"num_object_copies":3114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1557,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19101,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19381","reported_seq":19502,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:29.594611+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:38:29.594611+0000","last_peered":"2024-09-10T20:38:29.594611+0000","last_clean":"2024-09-10T20:38:29.594611+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:38:29.594611+0000","last_undegraded":"2024-09-10T20:38:29.594611+0000","last_fullsized":"2024-09-10T20:38:29.594611+0000","mapping_epoch":35,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6393856,"num_objects":1561,"num_object_clones":0,"num_object_copies":3122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17567,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19223","reported_seq":26581,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:31.804904+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:38:31.804904+0000","last_peered":"2024-09-10T20:38:31.804904+0000","last_clean":"2024-09-10T20:38:31.804904+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:38:31.804904+0000","last_undegraded":"2024-09-10T20:38:31.804904+0000","last_fullsized":"2024-09-10T20:38:31.804904+0000","mapping_epoch":44,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6111256,"num_objects":1493,"num_object_clones":0,"num_object_copies":2986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1493,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17351,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'18954","reported_seq":19045,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:29.618174+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:38:29.618174+0000","last_peered":"2024-09-10T20:38:29.618174+0000","last_clean":"2024-09-10T20:38:29.618174+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:38:29.618174+0000","last_undegraded":"2024-09-10T20:38:29.618174+0000","last_fullsized":"2024-09-10T20:38:29.618174+0000","mapping_epoch":35,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6758400,"num_objects":1650,"num_object_clones":0,"num_object_copies":3300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18954,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19472","reported_seq":19609,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:29.638532+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:38:29.638532+0000","last_peered":"2024-09-10T20:38:29.638532+0000","last_clean":"2024-09-10T20:38:29.638532+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:38:29.638532+0000","last_undegraded":"2024-09-10T20:38:29.638532+0000","last_fullsized":"2024-09-10T20:38:29.638532+0000","mapping_epoch":34,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6541312,"num_objects":1597,"num_object_clones":0,"num_object_copies":3194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16468,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18883","reported_seq":19136,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:29.678001+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:38:29.678001+0000","last_peered":"2024-09-10T20:38:29.678001+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:38:29.678001+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6504448,"num_objects":1588,"num_object_clones":0,"num_object_copies":3176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15747,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1973}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19033","reported_seq":19318,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:29.722610+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:38:29.722610+0000","last_peered":"2024-09-10T20:38:29.722610+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:38:29.722610+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6451200,"num_objects":1575,"num_object_clones":0,"num_object_copies":3150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15628,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1911}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19169","reported_seq":19304,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:29.685154+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:38:29.685154+0000","last_peered":"2024-09-10T20:38:29.685154+0000","last_clean":"2024-09-10T20:38:29.685154+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:38:29.685154+0000","last_undegraded":"2024-09-10T20:38:29.685154+0000","last_fullsized":"2024-09-10T20:38:29.685154+0000","mapping_epoch":34,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6418432,"num_objects":1567,"num_object_clones":0,"num_object_copies":3134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16240,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315452+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315452+0000","last_peered":"2024-09-10T20:36:30.315452+0000","last_clean":"2024-09-10T20:36:30.315452+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315452+0000","last_undegraded":"2024-09-10T20:36:30.315452+0000","last_fullsized":"2024-09-10T20:36:30.315452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18864","reported_seq":19063,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:29.536221+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:38:29.536221+0000","last_peered":"2024-09-10T20:38:29.536221+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:38:29.536221+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6451200,"num_objects":1575,"num_object_clones":0,"num_object_copies":3150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16857,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1922}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318059+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318059+0000","last_peered":"2024-09-10T20:36:30.318059+0000","last_clean":"2024-09-10T20:36:30.318059+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318059+0000","last_undegraded":"2024-09-10T20:36:30.318059+0000","last_fullsized":"2024-09-10T20:36:30.318059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19177","reported_seq":19394,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:29.694974+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:38:29.694974+0000","last_peered":"2024-09-10T20:38:29.694974+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:38:29.694974+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6362054,"num_objects":1555,"num_object_clones":0,"num_object_copies":3110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19175,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2011}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19006","reported_seq":19199,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:29.570167+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:38:29.570167+0000","last_peered":"2024-09-10T20:38:29.570167+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:38:29.570167+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6385664,"num_objects":1559,"num_object_clones":0,"num_object_copies":3118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16834,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2047}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316250+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316250+0000","last_peered":"2024-09-10T20:36:30.316250+0000","last_clean":"2024-09-10T20:36:30.316250+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316250+0000","last_undegraded":"2024-09-10T20:36:30.316250+0000","last_fullsized":"2024-09-10T20:36:30.316250+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19136","reported_seq":26507,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:29.553239+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:38:29.553239+0000","last_peered":"2024-09-10T20:38:29.553239+0000","last_clean":"2024-09-10T20:38:29.553239+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:38:29.553239+0000","last_undegraded":"2024-09-10T20:38:29.553239+0000","last_fullsized":"2024-09-10T20:38:29.553239+0000","mapping_epoch":48,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6258688,"num_objects":1528,"num_object_clones":0,"num_object_copies":3056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16922,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19239","reported_seq":19492,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:29.728338+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:38:29.728338+0000","last_peered":"2024-09-10T20:38:29.728338+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:38:29.728338+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6516736,"num_objects":1591,"num_object_clones":0,"num_object_copies":3182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15962,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2001}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316613+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316613+0000","last_peered":"2024-09-10T20:36:30.316613+0000","last_clean":"2024-09-10T20:36:30.316613+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316613+0000","last_undegraded":"2024-09-10T20:36:30.316613+0000","last_fullsized":"2024-09-10T20:36:30.316613+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19034","reported_seq":25020,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:29.738138+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:38:29.738138+0000","last_peered":"2024-09-10T20:38:29.738138+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:38:29.738138+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6348800,"num_objects":1550,"num_object_clones":0,"num_object_copies":3100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":615,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1550,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15728,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":935}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319644+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319644+0000","last_peered":"2024-09-10T20:36:30.319644+0000","last_clean":"2024-09-10T20:36:30.319644+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319644+0000","last_undegraded":"2024-09-10T20:36:30.319644+0000","last_fullsized":"2024-09-10T20:36:30.319644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19437","reported_seq":19695,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:29.743507+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:38:29.743507+0000","last_peered":"2024-09-10T20:38:29.743507+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:38:29.743507+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6344704,"num_objects":1549,"num_object_clones":0,"num_object_copies":3098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16264,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2106}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316994+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316994+0000","last_peered":"2024-09-10T20:36:30.316994+0000","last_clean":"2024-09-10T20:36:30.316994+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316994+0000","last_undegraded":"2024-09-10T20:36:30.316994+0000","last_fullsized":"2024-09-10T20:36:30.316994+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18981","reported_seq":26108,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:31.743027+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:38:31.743027+0000","last_peered":"2024-09-10T20:38:31.743027+0000","last_clean":"2024-09-10T20:38:31.743027+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:38:31.743027+0000","last_undegraded":"2024-09-10T20:38:31.743027+0000","last_fullsized":"2024-09-10T20:38:31.743027+0000","mapping_epoch":46,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6287843,"num_objects":1536,"num_object_clones":0,"num_object_copies":3072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15588,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":102512065,"num_objects":25031,"num_object_clones":0,"num_object_copies":50062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25031,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":270386,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14017,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":379559936,"data_stored":379493984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160790,"ondisk_log_size":160790,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182519,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":563432,"kb_used_data":134736,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93808408,"statfs":{"total":96636764160,"available":96059809792,"internally_reserved":0,"allocated":137969664,"data_stored":137707716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":438955690},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,1,2,5,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986291,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517464,"kb_used_data":166468,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":93854376,"statfs":{"total":96636764160,"available":96106881024,"internally_reserved":0,"allocated":170463232,"data_stored":170201289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":359394986},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659238,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":524552,"kb_used_data":73592,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93847288,"statfs":{"total":96636764160,"available":96099622912,"internally_reserved":0,"allocated":75358208,"data_stored":75045219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":461762222},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,3,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":136413184,"data_stored":136409571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73723904,"data_stored":73669250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":169422848,"data_stored":169415163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:38:37.917 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:38:38.488 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:38.490 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:38:38.520 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":357,"stamp":"2024-09-10T20:38:37.034932+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":92706260,"num_objects":22639,"num_object_clones":0,"num_object_copies":45278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28978,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22639,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":272782,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14173,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160787,"ondisk_log_size":160787,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1573780,"kb_used_data":355608,"kb_used_omap":12,"kb_used_meta":1218099,"kb_avail":281541740,"statfs":{"total":289910292480,"available":288298741760,"internally_reserved":0,"allocated":364142592,"data_stored":363305712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1247333378},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,3,3,6,11,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21037056,"num_objects":-5136,"num_object_clones":0,"num_object_copies":-10272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-308,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5136,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005196"},"pg_stats":[{"pgid":"2.f","version":"49'19246","reported_seq":19469,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:34.671272+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:38:34.671272+0000","last_peered":"2024-09-10T20:38:34.671272+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:38:34.671272+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5783552,"num_objects":1412,"num_object_clones":0,"num_object_copies":2824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19246,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2129}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19526","reported_seq":19647,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:34.498206+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:38:34.498206+0000","last_peered":"2024-09-10T20:38:34.498206+0000","last_clean":"2024-09-10T20:38:34.498206+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:38:34.498206+0000","last_undegraded":"2024-09-10T20:38:34.498206+0000","last_fullsized":"2024-09-10T20:38:34.498206+0000","mapping_epoch":35,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5799936,"num_objects":1416,"num_object_clones":0,"num_object_copies":2832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17712,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19360","reported_seq":26717,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:36.727316+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:38:36.727316+0000","last_peered":"2024-09-10T20:38:36.727316+0000","last_clean":"2024-09-10T20:38:36.727316+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:38:36.727316+0000","last_undegraded":"2024-09-10T20:38:36.727316+0000","last_fullsized":"2024-09-10T20:38:36.727316+0000","mapping_epoch":44,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5550104,"num_objects":1356,"num_object_clones":0,"num_object_copies":2712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1356,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17488,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19113","reported_seq":19204,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:34.571253+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:38:34.571253+0000","last_peered":"2024-09-10T20:38:34.571253+0000","last_clean":"2024-09-10T20:38:34.571253+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:38:34.571253+0000","last_undegraded":"2024-09-10T20:38:34.571253+0000","last_fullsized":"2024-09-10T20:38:34.571253+0000","mapping_epoch":35,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6107136,"num_objects":1491,"num_object_clones":0,"num_object_copies":2982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19113,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19620","reported_seq":19757,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:34.814525+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:38:34.814525+0000","last_peered":"2024-09-10T20:38:34.814525+0000","last_clean":"2024-09-10T20:38:34.814525+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:38:34.814525+0000","last_undegraded":"2024-09-10T20:38:34.814525+0000","last_fullsized":"2024-09-10T20:38:34.814525+0000","mapping_epoch":34,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5935104,"num_objects":1449,"num_object_clones":0,"num_object_copies":2898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16616,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'19038","reported_seq":19289,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:34.750029+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:38:34.750029+0000","last_peered":"2024-09-10T20:38:34.750029+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:38:34.750029+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5869568,"num_objects":1433,"num_object_clones":0,"num_object_copies":2866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15902,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2128}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19198","reported_seq":19484,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:34.792818+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:38:34.792818+0000","last_peered":"2024-09-10T20:38:34.792818+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:38:34.792818+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5775360,"num_objects":1410,"num_object_clones":0,"num_object_copies":2820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15793,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19338","reported_seq":19473,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:34.760019+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:38:34.760019+0000","last_peered":"2024-09-10T20:38:34.760019+0000","last_clean":"2024-09-10T20:38:34.760019+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:38:34.760019+0000","last_undegraded":"2024-09-10T20:38:34.760019+0000","last_fullsized":"2024-09-10T20:38:34.760019+0000","mapping_epoch":34,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5726208,"num_objects":1398,"num_object_clones":0,"num_object_copies":2796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1398,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16409,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315452+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315452+0000","last_peered":"2024-09-10T20:36:30.315452+0000","last_clean":"2024-09-10T20:36:30.315452+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315452+0000","last_undegraded":"2024-09-10T20:36:30.315452+0000","last_fullsized":"2024-09-10T20:36:30.315452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19017","reported_seq":19218,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:34.590046+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:38:34.590046+0000","last_peered":"2024-09-10T20:38:34.590046+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:38:34.590046+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5824512,"num_objects":1422,"num_object_clones":0,"num_object_copies":2844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1422,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17010,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2075}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318059+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318059+0000","last_peered":"2024-09-10T20:36:30.318059+0000","last_clean":"2024-09-10T20:36:30.318059+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318059+0000","last_undegraded":"2024-09-10T20:36:30.318059+0000","last_fullsized":"2024-09-10T20:36:30.318059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19322","reported_seq":19539,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:34.748511+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:38:34.748511+0000","last_peered":"2024-09-10T20:38:34.748511+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:38:34.748511+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5768134,"num_objects":1410,"num_object_clones":0,"num_object_copies":2820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19320,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2156}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19160","reported_seq":19353,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:34.496859+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:38:34.496859+0000","last_peered":"2024-09-10T20:38:34.496859+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:38:34.496859+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5754880,"num_objects":1405,"num_object_clones":0,"num_object_copies":2810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2201}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316250+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316250+0000","last_peered":"2024-09-10T20:36:30.316250+0000","last_clean":"2024-09-10T20:36:30.316250+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316250+0000","last_undegraded":"2024-09-10T20:36:30.316250+0000","last_fullsized":"2024-09-10T20:36:30.316250+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19283","reported_seq":26654,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:34.608072+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:38:34.608072+0000","last_peered":"2024-09-10T20:38:34.608072+0000","last_clean":"2024-09-10T20:38:34.608072+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:38:34.608072+0000","last_undegraded":"2024-09-10T20:38:34.608072+0000","last_fullsized":"2024-09-10T20:38:34.608072+0000","mapping_epoch":48,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5656576,"num_objects":1381,"num_object_clones":0,"num_object_copies":2762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17069,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19392","reported_seq":19649,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:34.672070+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:38:34.672070+0000","last_peered":"2024-09-10T20:38:34.672070+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:38:34.672070+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5890048,"num_objects":1438,"num_object_clones":0,"num_object_copies":2876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1438,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16115,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316613+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316613+0000","last_peered":"2024-09-10T20:36:30.316613+0000","last_clean":"2024-09-10T20:36:30.316613+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316613+0000","last_undegraded":"2024-09-10T20:36:30.316613+0000","last_fullsized":"2024-09-10T20:36:30.316613+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19180","reported_seq":25478,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:34.794751+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:38:34.794751+0000","last_peered":"2024-09-10T20:38:34.794751+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:38:34.794751+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5750784,"num_objects":1404,"num_object_clones":0,"num_object_copies":2808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":461,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15874,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3074,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":943}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319644+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319644+0000","last_peered":"2024-09-10T20:36:30.319644+0000","last_clean":"2024-09-10T20:36:30.319644+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319644+0000","last_undegraded":"2024-09-10T20:36:30.319644+0000","last_fullsized":"2024-09-10T20:36:30.319644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19560","reported_seq":19818,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:34.671803+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:38:34.671803+0000","last_peered":"2024-09-10T20:38:34.671803+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:38:34.671803+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5840896,"num_objects":1426,"num_object_clones":0,"num_object_copies":2852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16387,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2229}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316994+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316994+0000","last_peered":"2024-09-10T20:36:30.316994+0000","last_clean":"2024-09-10T20:36:30.316994+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316994+0000","last_undegraded":"2024-09-10T20:36:30.316994+0000","last_fullsized":"2024-09-10T20:36:30.316994+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19131","reported_seq":26257,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:36.825332+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:38:36.825332+0000","last_peered":"2024-09-10T20:38:36.825332+0000","last_clean":"2024-09-10T20:38:36.825332+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:38:36.825332+0000","last_undegraded":"2024-09-10T20:38:36.825332+0000","last_fullsized":"2024-09-10T20:38:36.825332+0000","mapping_epoch":46,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5673443,"num_objects":1386,"num_object_clones":0,"num_object_copies":2772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15738,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":92706241,"num_objects":22637,"num_object_clones":0,"num_object_copies":45274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28978,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22637,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":272780,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14171,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":359854080,"data_stored":359788128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160784,"ondisk_log_size":160784,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182520,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":564396,"kb_used_data":127384,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93807444,"statfs":{"total":96636764160,"available":96058822656,"internally_reserved":0,"allocated":130441216,"data_stored":130179268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,4,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986292,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":500124,"kb_used_data":161612,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93871716,"statfs":{"total":96636764160,"available":96124637184,"internally_reserved":0,"allocated":165490688,"data_stored":165228745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346615466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,0,2,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659239,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509260,"kb_used_data":66612,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93862580,"statfs":{"total":96636764160,"available":96115281920,"internally_reserved":0,"allocated":68210688,"data_stored":67897699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":453242542},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,3,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":128942080,"data_stored":128938467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":66449408,"data_stored":66394754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":164462592,"data_stored":164454907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:38:38.521 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:38:39.073 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:39.074 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:38:39.097 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":358,"stamp":"2024-09-10T20:38:39.035674+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":92706260,"num_objects":22639,"num_object_clones":0,"num_object_copies":45278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28978,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22639,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":272782,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14173,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160787,"ondisk_log_size":160787,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1573780,"kb_used_data":355608,"kb_used_omap":12,"kb_used_meta":1218099,"kb_avail":281541740,"statfs":{"total":289910292480,"available":288298741760,"internally_reserved":0,"allocated":364142592,"data_stored":363305712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1247333378},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,3,3,6,11,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19890176,"num_objects":-4856,"num_object_clones":0,"num_object_copies":-9712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-308,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4856,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005357"},"pg_stats":[{"pgid":"2.f","version":"49'19246","reported_seq":19469,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:34.671272+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:38:34.671272+0000","last_peered":"2024-09-10T20:38:34.671272+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:38:34.671272+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5783552,"num_objects":1412,"num_object_clones":0,"num_object_copies":2824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19246,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2129}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19526","reported_seq":19647,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:34.498206+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:38:34.498206+0000","last_peered":"2024-09-10T20:38:34.498206+0000","last_clean":"2024-09-10T20:38:34.498206+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:38:34.498206+0000","last_undegraded":"2024-09-10T20:38:34.498206+0000","last_fullsized":"2024-09-10T20:38:34.498206+0000","mapping_epoch":35,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5799936,"num_objects":1416,"num_object_clones":0,"num_object_copies":2832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17712,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19360","reported_seq":26717,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:36.727316+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:38:36.727316+0000","last_peered":"2024-09-10T20:38:36.727316+0000","last_clean":"2024-09-10T20:38:36.727316+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:38:36.727316+0000","last_undegraded":"2024-09-10T20:38:36.727316+0000","last_fullsized":"2024-09-10T20:38:36.727316+0000","mapping_epoch":44,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5550104,"num_objects":1356,"num_object_clones":0,"num_object_copies":2712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1356,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17488,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19113","reported_seq":19204,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:34.571253+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:38:34.571253+0000","last_peered":"2024-09-10T20:38:34.571253+0000","last_clean":"2024-09-10T20:38:34.571253+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:38:34.571253+0000","last_undegraded":"2024-09-10T20:38:34.571253+0000","last_fullsized":"2024-09-10T20:38:34.571253+0000","mapping_epoch":35,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6107136,"num_objects":1491,"num_object_clones":0,"num_object_copies":2982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19113,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19620","reported_seq":19757,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:34.814525+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:38:34.814525+0000","last_peered":"2024-09-10T20:38:34.814525+0000","last_clean":"2024-09-10T20:38:34.814525+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:38:34.814525+0000","last_undegraded":"2024-09-10T20:38:34.814525+0000","last_fullsized":"2024-09-10T20:38:34.814525+0000","mapping_epoch":34,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5935104,"num_objects":1449,"num_object_clones":0,"num_object_copies":2898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16616,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'19038","reported_seq":19289,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:34.750029+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:38:34.750029+0000","last_peered":"2024-09-10T20:38:34.750029+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:38:34.750029+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5869568,"num_objects":1433,"num_object_clones":0,"num_object_copies":2866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15902,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2128}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19198","reported_seq":19484,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:34.792818+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:38:34.792818+0000","last_peered":"2024-09-10T20:38:34.792818+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:38:34.792818+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5775360,"num_objects":1410,"num_object_clones":0,"num_object_copies":2820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15793,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19338","reported_seq":19473,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:34.760019+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:38:34.760019+0000","last_peered":"2024-09-10T20:38:34.760019+0000","last_clean":"2024-09-10T20:38:34.760019+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:38:34.760019+0000","last_undegraded":"2024-09-10T20:38:34.760019+0000","last_fullsized":"2024-09-10T20:38:34.760019+0000","mapping_epoch":34,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5726208,"num_objects":1398,"num_object_clones":0,"num_object_copies":2796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1398,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16409,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315452+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315452+0000","last_peered":"2024-09-10T20:36:30.315452+0000","last_clean":"2024-09-10T20:36:30.315452+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315452+0000","last_undegraded":"2024-09-10T20:36:30.315452+0000","last_fullsized":"2024-09-10T20:36:30.315452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19017","reported_seq":19218,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:34.590046+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:38:34.590046+0000","last_peered":"2024-09-10T20:38:34.590046+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:38:34.590046+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5824512,"num_objects":1422,"num_object_clones":0,"num_object_copies":2844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1422,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17010,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2075}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318059+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318059+0000","last_peered":"2024-09-10T20:36:30.318059+0000","last_clean":"2024-09-10T20:36:30.318059+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318059+0000","last_undegraded":"2024-09-10T20:36:30.318059+0000","last_fullsized":"2024-09-10T20:36:30.318059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19322","reported_seq":19539,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:34.748511+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:38:34.748511+0000","last_peered":"2024-09-10T20:38:34.748511+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:38:34.748511+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5768134,"num_objects":1410,"num_object_clones":0,"num_object_copies":2820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19320,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2156}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19160","reported_seq":19353,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:34.496859+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:38:34.496859+0000","last_peered":"2024-09-10T20:38:34.496859+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:38:34.496859+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5754880,"num_objects":1405,"num_object_clones":0,"num_object_copies":2810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2201}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316250+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316250+0000","last_peered":"2024-09-10T20:36:30.316250+0000","last_clean":"2024-09-10T20:36:30.316250+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316250+0000","last_undegraded":"2024-09-10T20:36:30.316250+0000","last_fullsized":"2024-09-10T20:36:30.316250+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19283","reported_seq":26654,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:34.608072+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:38:34.608072+0000","last_peered":"2024-09-10T20:38:34.608072+0000","last_clean":"2024-09-10T20:38:34.608072+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:38:34.608072+0000","last_undegraded":"2024-09-10T20:38:34.608072+0000","last_fullsized":"2024-09-10T20:38:34.608072+0000","mapping_epoch":48,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5656576,"num_objects":1381,"num_object_clones":0,"num_object_copies":2762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17069,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19392","reported_seq":19649,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:34.672070+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:38:34.672070+0000","last_peered":"2024-09-10T20:38:34.672070+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:38:34.672070+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5890048,"num_objects":1438,"num_object_clones":0,"num_object_copies":2876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1438,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16115,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316613+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316613+0000","last_peered":"2024-09-10T20:36:30.316613+0000","last_clean":"2024-09-10T20:36:30.316613+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316613+0000","last_undegraded":"2024-09-10T20:36:30.316613+0000","last_fullsized":"2024-09-10T20:36:30.316613+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19180","reported_seq":25478,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:34.794751+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:38:34.794751+0000","last_peered":"2024-09-10T20:38:34.794751+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:38:34.794751+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5750784,"num_objects":1404,"num_object_clones":0,"num_object_copies":2808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":461,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15874,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3074,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":943}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319644+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319644+0000","last_peered":"2024-09-10T20:36:30.319644+0000","last_clean":"2024-09-10T20:36:30.319644+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319644+0000","last_undegraded":"2024-09-10T20:36:30.319644+0000","last_fullsized":"2024-09-10T20:36:30.319644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19560","reported_seq":19818,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:34.671803+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:38:34.671803+0000","last_peered":"2024-09-10T20:38:34.671803+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:38:34.671803+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5840896,"num_objects":1426,"num_object_clones":0,"num_object_copies":2852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16387,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2229}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316994+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316994+0000","last_peered":"2024-09-10T20:36:30.316994+0000","last_clean":"2024-09-10T20:36:30.316994+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316994+0000","last_undegraded":"2024-09-10T20:36:30.316994+0000","last_fullsized":"2024-09-10T20:36:30.316994+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19131","reported_seq":26257,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:36.825332+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:38:36.825332+0000","last_peered":"2024-09-10T20:38:36.825332+0000","last_clean":"2024-09-10T20:38:36.825332+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:38:36.825332+0000","last_undegraded":"2024-09-10T20:38:36.825332+0000","last_fullsized":"2024-09-10T20:38:36.825332+0000","mapping_epoch":46,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5673443,"num_objects":1386,"num_object_clones":0,"num_object_copies":2772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15738,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":92706241,"num_objects":22637,"num_object_clones":0,"num_object_copies":45274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28978,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22637,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":272780,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14171,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":359854080,"data_stored":359788128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160784,"ondisk_log_size":160784,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182520,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":564396,"kb_used_data":127384,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93807444,"statfs":{"total":96636764160,"available":96058822656,"internally_reserved":0,"allocated":130441216,"data_stored":130179268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,4,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986292,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":500124,"kb_used_data":161612,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93871716,"statfs":{"total":96636764160,"available":96124637184,"internally_reserved":0,"allocated":165490688,"data_stored":165228745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346615466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,0,2,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659239,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509260,"kb_used_data":66612,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93862580,"statfs":{"total":96636764160,"available":96115281920,"internally_reserved":0,"allocated":68210688,"data_stored":67897699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":453242542},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,3,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":128942080,"data_stored":128938467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":66449408,"data_stored":66394754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":164462592,"data_stored":164454907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:38:39.098 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:38:39.098 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'19246", 'reported_seq': 19469, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:34.671272+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:38:34.671272+0000', 'last_peered': '2024-09-10T20:38:34.671272+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:38:34.671272+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "45'9200", 'ondisk_log_start': "45'9200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5783552, 'num_objects': 1412, 'num_object_clones': 0, 'num_object_copies': 2824, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1412, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19246, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2129}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:39.098 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:38:39.098 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'19038", 'reported_seq': 19289, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:34.750029+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:38:34.750029+0000', 'last_peered': '2024-09-10T20:38:34.750029+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:38:34.750029+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'9000", 'ondisk_log_start': "45'9000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5869568, 'num_objects': 1433, 'num_object_clones': 0, 'num_object_copies': 2866, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1433, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15902, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2128}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:39.098 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:38:39.099 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19198", 'reported_seq': 19484, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:34.792818+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:38:34.792818+0000', 'last_peered': '2024-09-10T20:38:34.792818+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:38:34.792818+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "45'9100", 'ondisk_log_start': "45'9100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5775360, 'num_objects': 1410, 'num_object_clones': 0, 'num_object_copies': 2820, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1410, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15793, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:39.099 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:38:39.099 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'19017", 'reported_seq': 19218, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:34.590046+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:38:34.590046+0000', 'last_peered': '2024-09-10T20:38:34.590046+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:38:34.590046+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "45'9000", 'ondisk_log_start': "45'9000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5824512, 'num_objects': 1422, 'num_object_clones': 0, 'num_object_copies': 2844, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1422, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17010, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2075}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:38:39.099 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:38:39.099 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19322", 'reported_seq': 19539, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:34.748511+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:38:34.748511+0000', 'last_peered': '2024-09-10T20:38:34.748511+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:38:34.748511+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "45'9300", 'ondisk_log_start': "45'9300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5768134, 'num_objects': 1410, 'num_object_clones': 0, 'num_object_copies': 2820, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1410, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19320, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2156}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:39.099 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:38:39.099 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'19160", 'reported_seq': 19353, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:34.496859+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:38:34.496859+0000', 'last_peered': '2024-09-10T20:38:34.496859+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:38:34.496859+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "45'9100", 'ondisk_log_start': "45'9100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5754880, 'num_objects': 1405, 'num_object_clones': 0, 'num_object_copies': 2810, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1405, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16988, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2201}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:38:39.099 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:38:39.099 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'19392", 'reported_seq': 19649, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:34.672070+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:38:34.672070+0000', 'last_peered': '2024-09-10T20:38:34.672070+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:38:34.672070+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "45'9300", 'ondisk_log_start': "45'9300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5890048, 'num_objects': 1438, 'num_object_clones': 0, 'num_object_copies': 2876, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1438, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16115, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2154}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:39.099 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:38:39.100 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19180", 'reported_seq': 25478, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:34.794751+0000', 'last_change': '2024-09-10T20:36:28.778006+0000', 'last_active': '2024-09-10T20:38:34.794751+0000', 'last_peered': '2024-09-10T20:38:34.794751+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:38:34.794751+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "45'9100", 'ondisk_log_start': "45'9100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5750784, 'num_objects': 1404, 'num_object_clones': 0, 'num_object_copies': 2808, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 461, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1404, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15874, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3074, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 943}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:39.100 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:38:39.100 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19560", 'reported_seq': 19818, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:34.671803+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:38:34.671803+0000', 'last_peered': '2024-09-10T20:38:34.671803+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:38:34.671803+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5840896, 'num_objects': 1426, 'num_object_clones': 0, 'num_object_copies': 2852, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1426, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16387, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2229}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:39.100 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:38:39.788 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:39.789 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 28978/45278 objects degraded (64.000%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":631,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":22639,"data_bytes":92706260,"bytes_used":1611550720,"bytes_avail":288298741760,"bytes_total":289910292480,"degraded_objects":28978,"degraded_total":45278,"degraded_ratio":0.64000176686249388,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":308,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":404},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:38:39.811 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:38:39.811 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:38:40.388 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:40.391 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:38:40.418 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":358,"stamp":"2024-09-10T20:38:39.035674+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":92706260,"num_objects":22639,"num_object_clones":0,"num_object_copies":45278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28978,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22639,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":272782,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14173,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160787,"ondisk_log_size":160787,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1573780,"kb_used_data":355608,"kb_used_omap":12,"kb_used_meta":1218099,"kb_avail":281541740,"statfs":{"total":289910292480,"available":288298741760,"internally_reserved":0,"allocated":364142592,"data_stored":363305712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1247333378},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,3,3,6,11,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19890176,"num_objects":-4856,"num_object_clones":0,"num_object_copies":-9712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-308,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4856,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005357"},"pg_stats":[{"pgid":"2.f","version":"49'19246","reported_seq":19469,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:34.671272+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:38:34.671272+0000","last_peered":"2024-09-10T20:38:34.671272+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:38:34.671272+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5783552,"num_objects":1412,"num_object_clones":0,"num_object_copies":2824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19246,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2129}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19526","reported_seq":19647,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:34.498206+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:38:34.498206+0000","last_peered":"2024-09-10T20:38:34.498206+0000","last_clean":"2024-09-10T20:38:34.498206+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:38:34.498206+0000","last_undegraded":"2024-09-10T20:38:34.498206+0000","last_fullsized":"2024-09-10T20:38:34.498206+0000","mapping_epoch":35,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5799936,"num_objects":1416,"num_object_clones":0,"num_object_copies":2832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17712,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19360","reported_seq":26717,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:36.727316+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:38:36.727316+0000","last_peered":"2024-09-10T20:38:36.727316+0000","last_clean":"2024-09-10T20:38:36.727316+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:38:36.727316+0000","last_undegraded":"2024-09-10T20:38:36.727316+0000","last_fullsized":"2024-09-10T20:38:36.727316+0000","mapping_epoch":44,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5550104,"num_objects":1356,"num_object_clones":0,"num_object_copies":2712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1356,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17488,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19113","reported_seq":19204,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:34.571253+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:38:34.571253+0000","last_peered":"2024-09-10T20:38:34.571253+0000","last_clean":"2024-09-10T20:38:34.571253+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:38:34.571253+0000","last_undegraded":"2024-09-10T20:38:34.571253+0000","last_fullsized":"2024-09-10T20:38:34.571253+0000","mapping_epoch":35,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6107136,"num_objects":1491,"num_object_clones":0,"num_object_copies":2982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19113,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19620","reported_seq":19757,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:34.814525+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:38:34.814525+0000","last_peered":"2024-09-10T20:38:34.814525+0000","last_clean":"2024-09-10T20:38:34.814525+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:38:34.814525+0000","last_undegraded":"2024-09-10T20:38:34.814525+0000","last_fullsized":"2024-09-10T20:38:34.814525+0000","mapping_epoch":34,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5935104,"num_objects":1449,"num_object_clones":0,"num_object_copies":2898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16616,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'19038","reported_seq":19289,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:34.750029+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:38:34.750029+0000","last_peered":"2024-09-10T20:38:34.750029+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:38:34.750029+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5869568,"num_objects":1433,"num_object_clones":0,"num_object_copies":2866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15902,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2128}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19198","reported_seq":19484,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:34.792818+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:38:34.792818+0000","last_peered":"2024-09-10T20:38:34.792818+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:38:34.792818+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5775360,"num_objects":1410,"num_object_clones":0,"num_object_copies":2820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15793,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19338","reported_seq":19473,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:34.760019+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:38:34.760019+0000","last_peered":"2024-09-10T20:38:34.760019+0000","last_clean":"2024-09-10T20:38:34.760019+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:38:34.760019+0000","last_undegraded":"2024-09-10T20:38:34.760019+0000","last_fullsized":"2024-09-10T20:38:34.760019+0000","mapping_epoch":34,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5726208,"num_objects":1398,"num_object_clones":0,"num_object_copies":2796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1398,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16409,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315452+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315452+0000","last_peered":"2024-09-10T20:36:30.315452+0000","last_clean":"2024-09-10T20:36:30.315452+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315452+0000","last_undegraded":"2024-09-10T20:36:30.315452+0000","last_fullsized":"2024-09-10T20:36:30.315452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19017","reported_seq":19218,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:34.590046+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:38:34.590046+0000","last_peered":"2024-09-10T20:38:34.590046+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:38:34.590046+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5824512,"num_objects":1422,"num_object_clones":0,"num_object_copies":2844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1422,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17010,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2075}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318059+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318059+0000","last_peered":"2024-09-10T20:36:30.318059+0000","last_clean":"2024-09-10T20:36:30.318059+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318059+0000","last_undegraded":"2024-09-10T20:36:30.318059+0000","last_fullsized":"2024-09-10T20:36:30.318059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19322","reported_seq":19539,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:34.748511+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:38:34.748511+0000","last_peered":"2024-09-10T20:38:34.748511+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:38:34.748511+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5768134,"num_objects":1410,"num_object_clones":0,"num_object_copies":2820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19320,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2156}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19160","reported_seq":19353,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:34.496859+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:38:34.496859+0000","last_peered":"2024-09-10T20:38:34.496859+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:38:34.496859+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5754880,"num_objects":1405,"num_object_clones":0,"num_object_copies":2810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2201}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316250+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316250+0000","last_peered":"2024-09-10T20:36:30.316250+0000","last_clean":"2024-09-10T20:36:30.316250+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316250+0000","last_undegraded":"2024-09-10T20:36:30.316250+0000","last_fullsized":"2024-09-10T20:36:30.316250+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19283","reported_seq":26654,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:34.608072+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:38:34.608072+0000","last_peered":"2024-09-10T20:38:34.608072+0000","last_clean":"2024-09-10T20:38:34.608072+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:38:34.608072+0000","last_undegraded":"2024-09-10T20:38:34.608072+0000","last_fullsized":"2024-09-10T20:38:34.608072+0000","mapping_epoch":48,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5656576,"num_objects":1381,"num_object_clones":0,"num_object_copies":2762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17069,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19392","reported_seq":19649,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:34.672070+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:38:34.672070+0000","last_peered":"2024-09-10T20:38:34.672070+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:38:34.672070+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5890048,"num_objects":1438,"num_object_clones":0,"num_object_copies":2876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1438,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16115,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316613+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316613+0000","last_peered":"2024-09-10T20:36:30.316613+0000","last_clean":"2024-09-10T20:36:30.316613+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316613+0000","last_undegraded":"2024-09-10T20:36:30.316613+0000","last_fullsized":"2024-09-10T20:36:30.316613+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19180","reported_seq":25478,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:34.794751+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:38:34.794751+0000","last_peered":"2024-09-10T20:38:34.794751+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:38:34.794751+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5750784,"num_objects":1404,"num_object_clones":0,"num_object_copies":2808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":461,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15874,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3074,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":943}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319644+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319644+0000","last_peered":"2024-09-10T20:36:30.319644+0000","last_clean":"2024-09-10T20:36:30.319644+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319644+0000","last_undegraded":"2024-09-10T20:36:30.319644+0000","last_fullsized":"2024-09-10T20:36:30.319644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19560","reported_seq":19818,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:34.671803+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:38:34.671803+0000","last_peered":"2024-09-10T20:38:34.671803+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:38:34.671803+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5840896,"num_objects":1426,"num_object_clones":0,"num_object_copies":2852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16387,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2229}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316994+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316994+0000","last_peered":"2024-09-10T20:36:30.316994+0000","last_clean":"2024-09-10T20:36:30.316994+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316994+0000","last_undegraded":"2024-09-10T20:36:30.316994+0000","last_fullsized":"2024-09-10T20:36:30.316994+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19131","reported_seq":26257,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:36.825332+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:38:36.825332+0000","last_peered":"2024-09-10T20:38:36.825332+0000","last_clean":"2024-09-10T20:38:36.825332+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:38:36.825332+0000","last_undegraded":"2024-09-10T20:38:36.825332+0000","last_fullsized":"2024-09-10T20:38:36.825332+0000","mapping_epoch":46,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5673443,"num_objects":1386,"num_object_clones":0,"num_object_copies":2772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15738,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":92706241,"num_objects":22637,"num_object_clones":0,"num_object_copies":45274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28978,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22637,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":272780,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14171,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":359854080,"data_stored":359788128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160784,"ondisk_log_size":160784,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182520,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":564396,"kb_used_data":127384,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93807444,"statfs":{"total":96636764160,"available":96058822656,"internally_reserved":0,"allocated":130441216,"data_stored":130179268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,4,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986292,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":500124,"kb_used_data":161612,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93871716,"statfs":{"total":96636764160,"available":96124637184,"internally_reserved":0,"allocated":165490688,"data_stored":165228745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346615466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,0,2,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659239,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509260,"kb_used_data":66612,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93862580,"statfs":{"total":96636764160,"available":96115281920,"internally_reserved":0,"allocated":68210688,"data_stored":67897699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":453242542},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,3,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":128942080,"data_stored":128938467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":66449408,"data_stored":66394754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":164462592,"data_stored":164454907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:38:43.421 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:38:43.984 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:43.984 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:38:44.005 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-09-10T20:38:43.036965+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":82433492,"num_objects":20131,"num_object_clones":0,"num_object_copies":40262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28818,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20131,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":275290,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14333,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160895,"ondisk_log_size":160895,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1568020,"kb_used_data":335960,"kb_used_omap":12,"kb_used_meta":1231987,"kb_avail":281547500,"statfs":{"total":289910292480,"available":288304640000,"internally_reserved":0,"allocated":344023040,"data_stored":343186160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1261554690},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,9,6,9,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21299200,"num_objects":-5200,"num_object_clones":0,"num_object_copies":-10400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-314,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5200,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":314,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004217"},"pg_stats":[{"pgid":"2.f","version":"49'19390","reported_seq":19613,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:39.851585+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:38:39.851585+0000","last_peered":"2024-09-10T20:38:39.851585+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:38:39.851585+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5193728,"num_objects":1268,"num_object_clones":0,"num_object_copies":2536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19390,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2273}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19683","reported_seq":19804,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:39.589227+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:38:39.589227+0000","last_peered":"2024-09-10T20:38:39.589227+0000","last_clean":"2024-09-10T20:38:39.589227+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:38:39.589227+0000","last_undegraded":"2024-09-10T20:38:39.589227+0000","last_fullsized":"2024-09-10T20:38:39.589227+0000","mapping_epoch":35,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5156864,"num_objects":1259,"num_object_clones":0,"num_object_copies":2518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17869,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19512","reported_seq":26870,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:41.878287+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:38:41.878287+0000","last_peered":"2024-09-10T20:38:41.878287+0000","last_clean":"2024-09-10T20:38:41.878287+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:38:41.878287+0000","last_undegraded":"2024-09-10T20:38:41.878287+0000","last_fullsized":"2024-09-10T20:38:41.878287+0000","mapping_epoch":44,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4927512,"num_objects":1204,"num_object_clones":0,"num_object_copies":2408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1204,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17640,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19296","reported_seq":19387,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:39.573514+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:38:39.573514+0000","last_peered":"2024-09-10T20:38:39.573514+0000","last_clean":"2024-09-10T20:38:39.573514+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:38:39.573514+0000","last_undegraded":"2024-09-10T20:38:39.573514+0000","last_fullsized":"2024-09-10T20:38:39.573514+0000","mapping_epoch":35,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5357568,"num_objects":1308,"num_object_clones":0,"num_object_copies":2616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19296,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19761","reported_seq":19898,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:39.732963+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:38:39.732963+0000","last_peered":"2024-09-10T20:38:39.732963+0000","last_clean":"2024-09-10T20:38:39.732963+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:38:39.732963+0000","last_undegraded":"2024-09-10T20:38:39.732963+0000","last_fullsized":"2024-09-10T20:38:39.732963+0000","mapping_epoch":34,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5357568,"num_objects":1308,"num_object_clones":0,"num_object_copies":2616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16757,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'19212","reported_seq":19465,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:39.798471+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:38:39.798471+0000","last_peered":"2024-09-10T20:38:39.798471+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:38:39.798471+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5156864,"num_objects":1259,"num_object_clones":0,"num_object_copies":2518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16076,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2302}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19354","reported_seq":19640,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:39.816952+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:38:39.816952+0000","last_peered":"2024-09-10T20:38:39.816952+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:38:39.816952+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5136384,"num_objects":1254,"num_object_clones":0,"num_object_copies":2508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15949,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2232}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19497","reported_seq":19632,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:39.792994+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:38:39.792994+0000","last_peered":"2024-09-10T20:38:39.792994+0000","last_clean":"2024-09-10T20:38:39.792994+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:38:39.792994+0000","last_undegraded":"2024-09-10T20:38:39.792994+0000","last_fullsized":"2024-09-10T20:38:39.792994+0000","mapping_epoch":34,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5074944,"num_objects":1239,"num_object_clones":0,"num_object_copies":2478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315452+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315452+0000","last_peered":"2024-09-10T20:36:30.315452+0000","last_clean":"2024-09-10T20:36:30.315452+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315452+0000","last_undegraded":"2024-09-10T20:36:30.315452+0000","last_fullsized":"2024-09-10T20:36:30.315452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19167","reported_seq":19368,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:39.503914+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:38:39.503914+0000","last_peered":"2024-09-10T20:38:39.503914+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:38:39.503914+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5210112,"num_objects":1272,"num_object_clones":0,"num_object_copies":2544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17160,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2225}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318059+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318059+0000","last_peered":"2024-09-10T20:36:30.318059+0000","last_clean":"2024-09-10T20:36:30.318059+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318059+0000","last_undegraded":"2024-09-10T20:36:30.318059+0000","last_fullsized":"2024-09-10T20:36:30.318059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19469","reported_seq":19684,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:39.865274+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:38:39.865274+0000","last_peered":"2024-09-10T20:38:39.865274+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:38:39.865274+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5166022,"num_objects":1263,"num_object_clones":0,"num_object_copies":2526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19467,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2303}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19327","reported_seq":19520,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:39.607875+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:38:39.607875+0000","last_peered":"2024-09-10T20:38:39.607875+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:38:39.607875+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5070848,"num_objects":1238,"num_object_clones":0,"num_object_copies":2476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2368}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316250+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316250+0000","last_peered":"2024-09-10T20:36:30.316250+0000","last_clean":"2024-09-10T20:36:30.316250+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316250+0000","last_undegraded":"2024-09-10T20:36:30.316250+0000","last_fullsized":"2024-09-10T20:36:30.316250+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19418","reported_seq":26789,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:39.621455+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:38:39.621455+0000","last_peered":"2024-09-10T20:38:39.621455+0000","last_clean":"2024-09-10T20:38:39.621455+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:38:39.621455+0000","last_undegraded":"2024-09-10T20:38:39.621455+0000","last_fullsized":"2024-09-10T20:38:39.621455+0000","mapping_epoch":48,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5103616,"num_objects":1246,"num_object_clones":0,"num_object_copies":2492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1246,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17204,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19568","reported_seq":19825,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:39.871831+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:38:39.871831+0000","last_peered":"2024-09-10T20:38:39.871831+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:38:39.871831+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5169152,"num_objects":1262,"num_object_clones":0,"num_object_copies":2524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16291,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2330}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316613+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316613+0000","last_peered":"2024-09-10T20:36:30.316613+0000","last_clean":"2024-09-10T20:36:30.316613+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316613+0000","last_undegraded":"2024-09-10T20:36:30.316613+0000","last_fullsized":"2024-09-10T20:36:30.316613+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19331","reported_seq":25948,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:39.855018+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:38:39.855018+0000","last_peered":"2024-09-10T20:38:39.855018+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:38:39.855018+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5132288,"num_objects":1253,"num_object_clones":0,"num_object_copies":2506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":301,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3234,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":952}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319644+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319644+0000","last_peered":"2024-09-10T20:36:30.319644+0000","last_clean":"2024-09-10T20:36:30.319644+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319644+0000","last_undegraded":"2024-09-10T20:36:30.319644+0000","last_fullsized":"2024-09-10T20:36:30.319644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19715","reported_seq":19973,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:39.649190+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:38:39.649190+0000","last_peered":"2024-09-10T20:38:39.649190+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:38:39.649190+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5206016,"num_objects":1271,"num_object_clones":0,"num_object_copies":2542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16542,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2384}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316994+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316994+0000","last_peered":"2024-09-10T20:36:30.316994+0000","last_clean":"2024-09-10T20:36:30.316994+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316994+0000","last_undegraded":"2024-09-10T20:36:30.316994+0000","last_fullsized":"2024-09-10T20:36:30.316994+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19292","reported_seq":26418,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:41.864785+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:38:41.864785+0000","last_peered":"2024-09-10T20:38:41.864785+0000","last_clean":"2024-09-10T20:38:41.864785+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:38:41.864785+0000","last_undegraded":"2024-09-10T20:38:41.864785+0000","last_fullsized":"2024-09-10T20:38:41.864785+0000","mapping_epoch":46,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5013987,"num_objects":1225,"num_object_clones":0,"num_object_copies":2450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15899,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":82433473,"num_objects":20129,"num_object_clones":0,"num_object_copies":40258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28818,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20129,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":275288,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14331,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":339492864,"data_stored":339426912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160892,"ondisk_log_size":160892,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182521,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562488,"kb_used_data":119912,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93809352,"statfs":{"total":96636764160,"available":96060776448,"internally_reserved":0,"allocated":122789888,"data_stored":122527940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,2,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986293,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499380,"kb_used_data":156704,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93872460,"statfs":{"total":96636764160,"available":96125399040,"internally_reserved":0,"allocated":160464896,"data_stored":160202953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350875306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659240,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506152,"kb_used_data":59344,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93865688,"statfs":{"total":96636764160,"available":96118464512,"internally_reserved":0,"allocated":60768256,"data_stored":60455267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":457502382},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":121171968,"data_stored":121168355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":58933248,"data_stored":58878594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":159387648,"data_stored":159379963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:38:44.007 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:38:44.575 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:44.575 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:38:44.613 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-09-10T20:38:43.036965+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":82433492,"num_objects":20131,"num_object_clones":0,"num_object_copies":40262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28818,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20131,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":275290,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14333,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160895,"ondisk_log_size":160895,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1568020,"kb_used_data":335960,"kb_used_omap":12,"kb_used_meta":1231987,"kb_avail":281547500,"statfs":{"total":289910292480,"available":288304640000,"internally_reserved":0,"allocated":344023040,"data_stored":343186160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1261554690},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,9,6,9,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21299200,"num_objects":-5200,"num_object_clones":0,"num_object_copies":-10400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-314,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5200,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":314,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004217"},"pg_stats":[{"pgid":"2.f","version":"49'19390","reported_seq":19613,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:39.851585+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:38:39.851585+0000","last_peered":"2024-09-10T20:38:39.851585+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:38:39.851585+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5193728,"num_objects":1268,"num_object_clones":0,"num_object_copies":2536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19390,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2273}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19683","reported_seq":19804,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:39.589227+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:38:39.589227+0000","last_peered":"2024-09-10T20:38:39.589227+0000","last_clean":"2024-09-10T20:38:39.589227+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:38:39.589227+0000","last_undegraded":"2024-09-10T20:38:39.589227+0000","last_fullsized":"2024-09-10T20:38:39.589227+0000","mapping_epoch":35,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5156864,"num_objects":1259,"num_object_clones":0,"num_object_copies":2518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17869,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19512","reported_seq":26870,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:41.878287+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:38:41.878287+0000","last_peered":"2024-09-10T20:38:41.878287+0000","last_clean":"2024-09-10T20:38:41.878287+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:38:41.878287+0000","last_undegraded":"2024-09-10T20:38:41.878287+0000","last_fullsized":"2024-09-10T20:38:41.878287+0000","mapping_epoch":44,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4927512,"num_objects":1204,"num_object_clones":0,"num_object_copies":2408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1204,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17640,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19296","reported_seq":19387,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:39.573514+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:38:39.573514+0000","last_peered":"2024-09-10T20:38:39.573514+0000","last_clean":"2024-09-10T20:38:39.573514+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:38:39.573514+0000","last_undegraded":"2024-09-10T20:38:39.573514+0000","last_fullsized":"2024-09-10T20:38:39.573514+0000","mapping_epoch":35,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5357568,"num_objects":1308,"num_object_clones":0,"num_object_copies":2616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19296,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19761","reported_seq":19898,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:39.732963+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:38:39.732963+0000","last_peered":"2024-09-10T20:38:39.732963+0000","last_clean":"2024-09-10T20:38:39.732963+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:38:39.732963+0000","last_undegraded":"2024-09-10T20:38:39.732963+0000","last_fullsized":"2024-09-10T20:38:39.732963+0000","mapping_epoch":34,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5357568,"num_objects":1308,"num_object_clones":0,"num_object_copies":2616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16757,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'19212","reported_seq":19465,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:39.798471+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:38:39.798471+0000","last_peered":"2024-09-10T20:38:39.798471+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:38:39.798471+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5156864,"num_objects":1259,"num_object_clones":0,"num_object_copies":2518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16076,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2302}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19354","reported_seq":19640,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:39.816952+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:38:39.816952+0000","last_peered":"2024-09-10T20:38:39.816952+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:38:39.816952+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5136384,"num_objects":1254,"num_object_clones":0,"num_object_copies":2508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15949,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2232}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19497","reported_seq":19632,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:39.792994+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:38:39.792994+0000","last_peered":"2024-09-10T20:38:39.792994+0000","last_clean":"2024-09-10T20:38:39.792994+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:38:39.792994+0000","last_undegraded":"2024-09-10T20:38:39.792994+0000","last_fullsized":"2024-09-10T20:38:39.792994+0000","mapping_epoch":34,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5074944,"num_objects":1239,"num_object_clones":0,"num_object_copies":2478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315452+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315452+0000","last_peered":"2024-09-10T20:36:30.315452+0000","last_clean":"2024-09-10T20:36:30.315452+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315452+0000","last_undegraded":"2024-09-10T20:36:30.315452+0000","last_fullsized":"2024-09-10T20:36:30.315452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19167","reported_seq":19368,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:39.503914+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:38:39.503914+0000","last_peered":"2024-09-10T20:38:39.503914+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:38:39.503914+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5210112,"num_objects":1272,"num_object_clones":0,"num_object_copies":2544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17160,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2225}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318059+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318059+0000","last_peered":"2024-09-10T20:36:30.318059+0000","last_clean":"2024-09-10T20:36:30.318059+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318059+0000","last_undegraded":"2024-09-10T20:36:30.318059+0000","last_fullsized":"2024-09-10T20:36:30.318059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19469","reported_seq":19684,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:39.865274+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:38:39.865274+0000","last_peered":"2024-09-10T20:38:39.865274+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:38:39.865274+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5166022,"num_objects":1263,"num_object_clones":0,"num_object_copies":2526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19467,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2303}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19327","reported_seq":19520,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:39.607875+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:38:39.607875+0000","last_peered":"2024-09-10T20:38:39.607875+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:38:39.607875+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5070848,"num_objects":1238,"num_object_clones":0,"num_object_copies":2476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2368}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316250+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316250+0000","last_peered":"2024-09-10T20:36:30.316250+0000","last_clean":"2024-09-10T20:36:30.316250+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316250+0000","last_undegraded":"2024-09-10T20:36:30.316250+0000","last_fullsized":"2024-09-10T20:36:30.316250+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19418","reported_seq":26789,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:39.621455+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:38:39.621455+0000","last_peered":"2024-09-10T20:38:39.621455+0000","last_clean":"2024-09-10T20:38:39.621455+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:38:39.621455+0000","last_undegraded":"2024-09-10T20:38:39.621455+0000","last_fullsized":"2024-09-10T20:38:39.621455+0000","mapping_epoch":48,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5103616,"num_objects":1246,"num_object_clones":0,"num_object_copies":2492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1246,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17204,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19568","reported_seq":19825,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:39.871831+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:38:39.871831+0000","last_peered":"2024-09-10T20:38:39.871831+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:38:39.871831+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5169152,"num_objects":1262,"num_object_clones":0,"num_object_copies":2524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16291,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2330}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316613+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316613+0000","last_peered":"2024-09-10T20:36:30.316613+0000","last_clean":"2024-09-10T20:36:30.316613+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316613+0000","last_undegraded":"2024-09-10T20:36:30.316613+0000","last_fullsized":"2024-09-10T20:36:30.316613+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19331","reported_seq":25948,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:39.855018+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:38:39.855018+0000","last_peered":"2024-09-10T20:38:39.855018+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:38:39.855018+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5132288,"num_objects":1253,"num_object_clones":0,"num_object_copies":2506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":301,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3234,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":952}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319644+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319644+0000","last_peered":"2024-09-10T20:36:30.319644+0000","last_clean":"2024-09-10T20:36:30.319644+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319644+0000","last_undegraded":"2024-09-10T20:36:30.319644+0000","last_fullsized":"2024-09-10T20:36:30.319644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19715","reported_seq":19973,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:39.649190+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:38:39.649190+0000","last_peered":"2024-09-10T20:38:39.649190+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:38:39.649190+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5206016,"num_objects":1271,"num_object_clones":0,"num_object_copies":2542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16542,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2384}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316994+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316994+0000","last_peered":"2024-09-10T20:36:30.316994+0000","last_clean":"2024-09-10T20:36:30.316994+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316994+0000","last_undegraded":"2024-09-10T20:36:30.316994+0000","last_fullsized":"2024-09-10T20:36:30.316994+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19292","reported_seq":26418,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:41.864785+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:38:41.864785+0000","last_peered":"2024-09-10T20:38:41.864785+0000","last_clean":"2024-09-10T20:38:41.864785+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:38:41.864785+0000","last_undegraded":"2024-09-10T20:38:41.864785+0000","last_fullsized":"2024-09-10T20:38:41.864785+0000","mapping_epoch":46,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5013987,"num_objects":1225,"num_object_clones":0,"num_object_copies":2450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15899,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":82433473,"num_objects":20129,"num_object_clones":0,"num_object_copies":40258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28818,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20129,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":275288,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14331,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":339492864,"data_stored":339426912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160892,"ondisk_log_size":160892,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182521,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562488,"kb_used_data":119912,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93809352,"statfs":{"total":96636764160,"available":96060776448,"internally_reserved":0,"allocated":122789888,"data_stored":122527940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,2,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986293,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499380,"kb_used_data":156704,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93872460,"statfs":{"total":96636764160,"available":96125399040,"internally_reserved":0,"allocated":160464896,"data_stored":160202953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350875306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659240,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506152,"kb_used_data":59344,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93865688,"statfs":{"total":96636764160,"available":96118464512,"internally_reserved":0,"allocated":60768256,"data_stored":60455267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":457502382},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":121171968,"data_stored":121168355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":58933248,"data_stored":58878594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":159387648,"data_stored":159379963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:38:44.615 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:38:44.615 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'19390", 'reported_seq': 19613, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:39.851585+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:38:39.851585+0000', 'last_peered': '2024-09-10T20:38:39.851585+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:38:39.851585+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "45'9300", 'ondisk_log_start': "45'9300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5193728, 'num_objects': 1268, 'num_object_clones': 0, 'num_object_copies': 2536, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1268, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19390, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2273}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:44.615 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:38:44.615 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'19212", 'reported_seq': 19465, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:39.798471+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:38:39.798471+0000', 'last_peered': '2024-09-10T20:38:39.798471+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:38:39.798471+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'9200", 'ondisk_log_start': "45'9200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5156864, 'num_objects': 1259, 'num_object_clones': 0, 'num_object_copies': 2518, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1259, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16076, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2302}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:44.615 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:38:44.615 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19354", 'reported_seq': 19640, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:39.816952+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:38:39.816952+0000', 'last_peered': '2024-09-10T20:38:39.816952+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:38:39.816952+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "45'9300", 'ondisk_log_start': "45'9300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5136384, 'num_objects': 1254, 'num_object_clones': 0, 'num_object_copies': 2508, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1254, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15949, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2232}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:44.615 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:38:44.616 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'19167", 'reported_seq': 19368, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:39.503914+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:38:39.503914+0000', 'last_peered': '2024-09-10T20:38:39.503914+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:38:39.503914+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "45'9100", 'ondisk_log_start': "45'9100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5210112, 'num_objects': 1272, 'num_object_clones': 0, 'num_object_copies': 2544, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1272, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17160, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2225}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:38:44.616 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:38:44.616 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19469", 'reported_seq': 19684, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:39.865274+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:38:39.865274+0000', 'last_peered': '2024-09-10T20:38:39.865274+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:38:39.865274+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "45'9400", 'ondisk_log_start': "45'9400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5166022, 'num_objects': 1263, 'num_object_clones': 0, 'num_object_copies': 2526, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1263, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19467, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2303}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:44.616 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:38:44.616 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'19327", 'reported_seq': 19520, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:39.607875+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:38:39.607875+0000', 'last_peered': '2024-09-10T20:38:39.607875+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:38:39.607875+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "45'9300", 'ondisk_log_start': "45'9300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5070848, 'num_objects': 1238, 'num_object_clones': 0, 'num_object_copies': 2476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1238, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17155, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2368}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:38:44.616 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:38:44.616 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'19568", 'reported_seq': 19825, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:39.871831+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:38:39.871831+0000', 'last_peered': '2024-09-10T20:38:39.871831+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:38:39.871831+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5169152, 'num_objects': 1262, 'num_object_clones': 0, 'num_object_copies': 2524, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1262, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16291, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2330}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:44.616 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:38:44.616 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19331", 'reported_seq': 25948, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:39.855018+0000', 'last_change': '2024-09-10T20:36:28.778006+0000', 'last_active': '2024-09-10T20:38:39.855018+0000', 'last_peered': '2024-09-10T20:38:39.855018+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:38:39.855018+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "45'9300", 'ondisk_log_start': "45'9300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5132288, 'num_objects': 1253, 'num_object_clones': 0, 'num_object_copies': 2506, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 301, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1253, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3234, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 952}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:44.616 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:38:44.616 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19715", 'reported_seq': 19973, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:39.649190+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:38:39.649190+0000', 'last_peered': '2024-09-10T20:38:39.649190+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:38:39.649190+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5206016, 'num_objects': 1271, 'num_object_clones': 0, 'num_object_copies': 2542, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1271, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16542, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2384}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:44.617 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:38:45.332 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:45.333 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 28818/40262 objects degraded (71.576%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":636,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":20131,"data_bytes":82433492,"bytes_used":1605652480,"bytes_avail":288304640000,"bytes_total":289910292480,"degraded_objects":28818,"degraded_total":40262,"degraded_ratio":0.71576176046892859,"recovering_objects_per_sec":26,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":314,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":433},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:38:45.356 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:38:45.357 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:38:45.917 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:45.919 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:38:45.945 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":361,"stamp":"2024-09-10T20:38:45.037695+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":73422292,"num_objects":17931,"num_object_clones":0,"num_object_copies":35862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28666,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17931,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":277490,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14485,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160695,"ondisk_log_size":160695,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1568004,"kb_used_data":323456,"kb_used_omap":12,"kb_used_meta":1244467,"kb_avail":281547516,"statfs":{"total":289910292480,"available":288304656384,"internally_reserved":0,"allocated":331218944,"data_stored":330382064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1274334210},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,9,9,10,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29089792,"num_objects":-7102,"num_object_clones":0,"num_object_copies":-14204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-466,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7102,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":466,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004282"},"pg_stats":[{"pgid":"2.f","version":"49'19535","reported_seq":19758,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:44.825766+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:38:44.825766+0000","last_peered":"2024-09-10T20:38:44.825766+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:38:44.825766+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4599808,"num_objects":1123,"num_object_clones":0,"num_object_copies":2246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19535,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2418}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19849","reported_seq":19970,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:44.540847+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:38:44.540847+0000","last_peered":"2024-09-10T20:38:44.540847+0000","last_clean":"2024-09-10T20:38:44.540847+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:38:44.540847+0000","last_undegraded":"2024-09-10T20:38:44.540847+0000","last_fullsized":"2024-09-10T20:38:44.540847+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4476928,"num_objects":1093,"num_object_clones":0,"num_object_copies":2186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18035,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19512","reported_seq":26870,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:41.878287+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:38:41.878287+0000","last_peered":"2024-09-10T20:38:41.878287+0000","last_clean":"2024-09-10T20:38:41.878287+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:38:41.878287+0000","last_undegraded":"2024-09-10T20:38:41.878287+0000","last_fullsized":"2024-09-10T20:38:41.878287+0000","mapping_epoch":44,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4927512,"num_objects":1204,"num_object_clones":0,"num_object_copies":2408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1204,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17640,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19462","reported_seq":19553,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:44.600622+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:38:44.600622+0000","last_peered":"2024-09-10T20:38:44.600622+0000","last_clean":"2024-09-10T20:38:44.600622+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:38:44.600622+0000","last_undegraded":"2024-09-10T20:38:44.600622+0000","last_fullsized":"2024-09-10T20:38:44.600622+0000","mapping_epoch":35,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4677632,"num_objects":1142,"num_object_clones":0,"num_object_copies":2284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19462,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19938","reported_seq":20075,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:44.877697+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:38:44.877697+0000","last_peered":"2024-09-10T20:38:44.877697+0000","last_clean":"2024-09-10T20:38:44.877697+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:38:44.877697+0000","last_undegraded":"2024-09-10T20:38:44.877697+0000","last_fullsized":"2024-09-10T20:38:44.877697+0000","mapping_epoch":34,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4632576,"num_objects":1131,"num_object_clones":0,"num_object_copies":2262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16934,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'19378","reported_seq":19631,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:44.822393+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:38:44.822393+0000","last_peered":"2024-09-10T20:38:44.822393+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:38:44.822393+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4476928,"num_objects":1093,"num_object_clones":0,"num_object_copies":2186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16242,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19512","reported_seq":19797,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:44.888095+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:38:44.888095+0000","last_peered":"2024-09-10T20:38:44.888095+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:38:44.888095+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4489216,"num_objects":1096,"num_object_clones":0,"num_object_copies":2192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16107,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2390}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19637","reported_seq":19771,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:44.866442+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:38:44.866442+0000","last_peered":"2024-09-10T20:38:44.866442+0000","last_clean":"2024-09-10T20:38:44.866442+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:38:44.866442+0000","last_undegraded":"2024-09-10T20:38:44.866442+0000","last_fullsized":"2024-09-10T20:38:44.866442+0000","mapping_epoch":34,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4501504,"num_objects":1099,"num_object_clones":0,"num_object_copies":2198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1099,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16708,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315452+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315452+0000","last_peered":"2024-09-10T20:36:30.315452+0000","last_clean":"2024-09-10T20:36:30.315452+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315452+0000","last_undegraded":"2024-09-10T20:36:30.315452+0000","last_fullsized":"2024-09-10T20:36:30.315452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19322","reported_seq":19521,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:44.590154+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:38:44.590154+0000","last_peered":"2024-09-10T20:38:44.590154+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:38:44.590154+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4575232,"num_objects":1117,"num_object_clones":0,"num_object_copies":2234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17315,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2380}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318059+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318059+0000","last_peered":"2024-09-10T20:36:30.318059+0000","last_clean":"2024-09-10T20:36:30.318059+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318059+0000","last_undegraded":"2024-09-10T20:36:30.318059+0000","last_fullsized":"2024-09-10T20:36:30.318059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19612","reported_seq":19829,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:44.784359+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:38:44.784359+0000","last_peered":"2024-09-10T20:38:44.784359+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:38:44.784359+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4580294,"num_objects":1120,"num_object_clones":0,"num_object_copies":2240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19610,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2446}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19470","reported_seq":19663,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:44.592916+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:38:44.592916+0000","last_peered":"2024-09-10T20:38:44.592916+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:38:44.592916+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4485120,"num_objects":1095,"num_object_clones":0,"num_object_copies":2190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1095,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17298,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2511}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316250+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316250+0000","last_peered":"2024-09-10T20:36:30.316250+0000","last_clean":"2024-09-10T20:36:30.316250+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316250+0000","last_undegraded":"2024-09-10T20:36:30.316250+0000","last_fullsized":"2024-09-10T20:36:30.316250+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19571","reported_seq":26942,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:44.595627+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:38:44.595627+0000","last_peered":"2024-09-10T20:38:44.595627+0000","last_clean":"2024-09-10T20:38:44.595627+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:38:44.595627+0000","last_undegraded":"2024-09-10T20:38:44.595627+0000","last_fullsized":"2024-09-10T20:38:44.595627+0000","mapping_epoch":48,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4476928,"num_objects":1093,"num_object_clones":0,"num_object_copies":2186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17357,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19723","reported_seq":19980,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:44.936718+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:38:44.936718+0000","last_peered":"2024-09-10T20:38:44.936718+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:38:44.936718+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4534272,"num_objects":1107,"num_object_clones":0,"num_object_copies":2214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16446,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2485}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316613+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316613+0000","last_peered":"2024-09-10T20:36:30.316613+0000","last_clean":"2024-09-10T20:36:30.316613+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316613+0000","last_undegraded":"2024-09-10T20:36:30.316613+0000","last_fullsized":"2024-09-10T20:36:30.316613+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19514","reported_seq":26435,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:44.880513+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:38:44.880513+0000","last_peered":"2024-09-10T20:38:44.880513+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:38:44.880513+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4382720,"num_objects":1070,"num_object_clones":0,"num_object_copies":2140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1070,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16208,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3386,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":921}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319644+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319644+0000","last_peered":"2024-09-10T20:36:30.319644+0000","last_clean":"2024-09-10T20:36:30.319644+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319644+0000","last_undegraded":"2024-09-10T20:36:30.319644+0000","last_fullsized":"2024-09-10T20:36:30.319644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19865","reported_seq":20122,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:44.851121+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:38:44.851121+0000","last_peered":"2024-09-10T20:38:44.851121+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:38:44.851121+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4591616,"num_objects":1121,"num_object_clones":0,"num_object_copies":2242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16692,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2534}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316994+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316994+0000","last_peered":"2024-09-10T20:36:30.316994+0000","last_clean":"2024-09-10T20:36:30.316994+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316994+0000","last_undegraded":"2024-09-10T20:36:30.316994+0000","last_fullsized":"2024-09-10T20:36:30.316994+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.317624+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:30.317624+0000","last_peered":"2024-09-10T20:36:30.317624+0000","last_clean":"2024-09-10T20:36:30.317624+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:30.317624+0000","last_undegraded":"2024-09-10T20:36:30.317624+0000","last_fullsized":"2024-09-10T20:36:30.317624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19292","reported_seq":26418,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:41.864785+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:38:41.864785+0000","last_peered":"2024-09-10T20:38:41.864785+0000","last_clean":"2024-09-10T20:38:41.864785+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:38:41.864785+0000","last_undegraded":"2024-09-10T20:38:41.864785+0000","last_fullsized":"2024-09-10T20:38:41.864785+0000","mapping_epoch":46,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5013987,"num_objects":1225,"num_object_clones":0,"num_object_copies":2450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15899,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":73422273,"num_objects":17929,"num_object_clones":0,"num_object_copies":35858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28666,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17929,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":277488,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14483,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":326496256,"data_stored":326430304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160692,"ondisk_log_size":160692,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182522,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":563224,"kb_used_data":112320,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93808616,"statfs":{"total":96636764160,"available":96060022784,"internally_reserved":0,"allocated":115015680,"data_stored":114753732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":461696682},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,2,2,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986294,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":498628,"kb_used_data":151792,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93873212,"statfs":{"total":96636764160,"available":96126169088,"internally_reserved":0,"allocated":155435008,"data_stored":155173065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355135146},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,0,5,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659240,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506152,"kb_used_data":59344,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93865688,"statfs":{"total":96636764160,"available":96118464512,"internally_reserved":0,"allocated":60768256,"data_stored":60455267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":457502382},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":113324032,"data_stored":113320419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":58933248,"data_stored":58878594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":154238976,"data_stored":154231291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:38:48.948 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:38:49.532 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:49.533 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:38:49.559 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-09-10T20:38:49.039389+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":72103380,"num_objects":17609,"num_object_clones":0,"num_object_copies":35218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28666,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17609,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":277812,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14485,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160717,"ondisk_log_size":160717,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1569060,"kb_used_data":316192,"kb_used_omap":12,"kb_used_meta":1252787,"kb_avail":281546460,"statfs":{"total":289910292480,"available":288303575040,"internally_reserved":0,"allocated":323780608,"data_stored":322943728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1282853890},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,4,6,9,6,3,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20602880,"num_objects":-5030,"num_object_clones":0,"num_object_copies":-10060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-312,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5030,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004457"},"pg_stats":[{"pgid":"2.f","version":"49'19535","reported_seq":19758,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:44.825766+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:38:44.825766+0000","last_peered":"2024-09-10T20:38:44.825766+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:38:44.825766+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4599808,"num_objects":1123,"num_object_clones":0,"num_object_copies":2246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19535,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2418}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19849","reported_seq":19970,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:44.540847+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:38:44.540847+0000","last_peered":"2024-09-10T20:38:44.540847+0000","last_clean":"2024-09-10T20:38:44.540847+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:38:44.540847+0000","last_undegraded":"2024-09-10T20:38:44.540847+0000","last_fullsized":"2024-09-10T20:38:44.540847+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4476928,"num_objects":1093,"num_object_clones":0,"num_object_copies":2186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18035,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19675","reported_seq":27033,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:46.937897+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:38:46.937897+0000","last_peered":"2024-09-10T20:38:46.937897+0000","last_clean":"2024-09-10T20:38:46.937897+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:38:46.937897+0000","last_undegraded":"2024-09-10T20:38:46.937897+0000","last_fullsized":"2024-09-10T20:38:46.937897+0000","mapping_epoch":44,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4259864,"num_objects":1041,"num_object_clones":0,"num_object_copies":2082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1041,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17803,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19462","reported_seq":19553,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:44.600622+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:38:44.600622+0000","last_peered":"2024-09-10T20:38:44.600622+0000","last_clean":"2024-09-10T20:38:44.600622+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:38:44.600622+0000","last_undegraded":"2024-09-10T20:38:44.600622+0000","last_fullsized":"2024-09-10T20:38:44.600622+0000","mapping_epoch":35,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4677632,"num_objects":1142,"num_object_clones":0,"num_object_copies":2284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19462,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19938","reported_seq":20075,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:44.877697+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:38:44.877697+0000","last_peered":"2024-09-10T20:38:44.877697+0000","last_clean":"2024-09-10T20:38:44.877697+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:38:44.877697+0000","last_undegraded":"2024-09-10T20:38:44.877697+0000","last_fullsized":"2024-09-10T20:38:44.877697+0000","mapping_epoch":34,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4632576,"num_objects":1131,"num_object_clones":0,"num_object_copies":2262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16934,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'19378","reported_seq":19631,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:44.822393+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:38:44.822393+0000","last_peered":"2024-09-10T20:38:44.822393+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:38:44.822393+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4476928,"num_objects":1093,"num_object_clones":0,"num_object_copies":2186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16242,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19512","reported_seq":19797,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:44.888095+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:38:44.888095+0000","last_peered":"2024-09-10T20:38:44.888095+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:38:44.888095+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4489216,"num_objects":1096,"num_object_clones":0,"num_object_copies":2192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16107,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2390}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19637","reported_seq":19771,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:44.866442+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:38:44.866442+0000","last_peered":"2024-09-10T20:38:44.866442+0000","last_clean":"2024-09-10T20:38:44.866442+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:38:44.866442+0000","last_undegraded":"2024-09-10T20:38:44.866442+0000","last_fullsized":"2024-09-10T20:38:44.866442+0000","mapping_epoch":34,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4501504,"num_objects":1099,"num_object_clones":0,"num_object_copies":2198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1099,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16708,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315452+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315452+0000","last_peered":"2024-09-10T20:36:30.315452+0000","last_clean":"2024-09-10T20:36:30.315452+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315452+0000","last_undegraded":"2024-09-10T20:36:30.315452+0000","last_fullsized":"2024-09-10T20:36:30.315452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19322","reported_seq":19521,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:44.590154+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:38:44.590154+0000","last_peered":"2024-09-10T20:38:44.590154+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:38:44.590154+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4575232,"num_objects":1117,"num_object_clones":0,"num_object_copies":2234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17315,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2380}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318059+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318059+0000","last_peered":"2024-09-10T20:36:30.318059+0000","last_clean":"2024-09-10T20:36:30.318059+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318059+0000","last_undegraded":"2024-09-10T20:36:30.318059+0000","last_fullsized":"2024-09-10T20:36:30.318059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19612","reported_seq":19829,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:44.784359+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:38:44.784359+0000","last_peered":"2024-09-10T20:38:44.784359+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:38:44.784359+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4580294,"num_objects":1120,"num_object_clones":0,"num_object_copies":2240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19610,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2446}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19470","reported_seq":19663,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:44.592916+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:38:44.592916+0000","last_peered":"2024-09-10T20:38:44.592916+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:38:44.592916+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4485120,"num_objects":1095,"num_object_clones":0,"num_object_copies":2190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1095,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17298,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2511}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316250+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316250+0000","last_peered":"2024-09-10T20:36:30.316250+0000","last_clean":"2024-09-10T20:36:30.316250+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316250+0000","last_undegraded":"2024-09-10T20:36:30.316250+0000","last_fullsized":"2024-09-10T20:36:30.316250+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19571","reported_seq":26942,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:44.595627+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:38:44.595627+0000","last_peered":"2024-09-10T20:38:44.595627+0000","last_clean":"2024-09-10T20:38:44.595627+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:38:44.595627+0000","last_undegraded":"2024-09-10T20:38:44.595627+0000","last_fullsized":"2024-09-10T20:38:44.595627+0000","mapping_epoch":48,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4476928,"num_objects":1093,"num_object_clones":0,"num_object_copies":2186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17357,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19723","reported_seq":19980,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:44.936718+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:38:44.936718+0000","last_peered":"2024-09-10T20:38:44.936718+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:38:44.936718+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4534272,"num_objects":1107,"num_object_clones":0,"num_object_copies":2214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16446,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2485}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316613+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316613+0000","last_peered":"2024-09-10T20:36:30.316613+0000","last_clean":"2024-09-10T20:36:30.316613+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316613+0000","last_undegraded":"2024-09-10T20:36:30.316613+0000","last_fullsized":"2024-09-10T20:36:30.316613+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19514","reported_seq":26435,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:44.880513+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:38:44.880513+0000","last_peered":"2024-09-10T20:38:44.880513+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:38:44.880513+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4382720,"num_objects":1070,"num_object_clones":0,"num_object_copies":2140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1070,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16208,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3386,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":921}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319644+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319644+0000","last_peered":"2024-09-10T20:36:30.319644+0000","last_clean":"2024-09-10T20:36:30.319644+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319644+0000","last_undegraded":"2024-09-10T20:36:30.319644+0000","last_fullsized":"2024-09-10T20:36:30.319644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19865","reported_seq":20122,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:44.851121+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:38:44.851121+0000","last_peered":"2024-09-10T20:38:44.851121+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:38:44.851121+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4591616,"num_objects":1121,"num_object_clones":0,"num_object_copies":2242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16692,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2534}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316994+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316994+0000","last_peered":"2024-09-10T20:36:30.316994+0000","last_clean":"2024-09-10T20:36:30.316994+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316994+0000","last_undegraded":"2024-09-10T20:36:30.316994+0000","last_fullsized":"2024-09-10T20:36:30.316994+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:35.648960+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:35.648960+0000","last_peered":"2024-09-10T20:36:35.648960+0000","last_clean":"2024-09-10T20:36:35.648960+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:35.648960+0000","last_undegraded":"2024-09-10T20:36:35.648960+0000","last_fullsized":"2024-09-10T20:36:35.648960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19451","reported_seq":26579,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:46.857078+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:38:46.857078+0000","last_peered":"2024-09-10T20:38:46.857078+0000","last_clean":"2024-09-10T20:38:46.857078+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:38:46.857078+0000","last_undegraded":"2024-09-10T20:38:46.857078+0000","last_fullsized":"2024-09-10T20:38:46.857078+0000","mapping_epoch":46,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4362723,"num_objects":1066,"num_object_clones":0,"num_object_copies":2132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16058,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":72103361,"num_objects":17607,"num_object_clones":0,"num_object_copies":35214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28666,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17607,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":277810,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14483,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":318857216,"data_stored":318791264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160714,"ondisk_log_size":160714,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182522,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":563224,"kb_used_data":112320,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93808616,"statfs":{"total":96636764160,"available":96060022784,"internally_reserved":0,"allocated":115015680,"data_stored":114753732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":461696682},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,2,2,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986294,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":498628,"kb_used_data":151792,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93873212,"statfs":{"total":96636764160,"available":96126169088,"internally_reserved":0,"allocated":155435008,"data_stored":155173065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355135146},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,0,5,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659241,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":507208,"kb_used_data":52080,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93864632,"statfs":{"total":96636764160,"available":96117383168,"internally_reserved":0,"allocated":53329920,"data_stored":53016931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":466022062},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,0,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":113324032,"data_stored":113320419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":51294208,"data_stored":51239554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":154238976,"data_stored":154231291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:38:49.560 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:38:50.118 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:50.118 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:38:50.146 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2024-09-10T20:38:49.649254+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":72103380,"num_objects":17609,"num_object_clones":0,"num_object_copies":35218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28666,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17609,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":277812,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14485,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160717,"ondisk_log_size":160717,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1569060,"kb_used_data":316192,"kb_used_omap":12,"kb_used_meta":1252787,"kb_avail":281546460,"statfs":{"total":289910292480,"available":288303575040,"internally_reserved":0,"allocated":323780608,"data_stored":322943728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1282853890},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,4,6,9,6,3,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20602880,"num_objects":-5030,"num_object_clones":0,"num_object_copies":-10060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-312,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5030,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.613579"},"pg_stats":[{"pgid":"2.f","version":"49'19535","reported_seq":19758,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:44.825766+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:38:44.825766+0000","last_peered":"2024-09-10T20:38:44.825766+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:38:44.825766+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4599808,"num_objects":1123,"num_object_clones":0,"num_object_copies":2246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19535,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2418}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19849","reported_seq":19970,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:44.540847+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:38:44.540847+0000","last_peered":"2024-09-10T20:38:44.540847+0000","last_clean":"2024-09-10T20:38:44.540847+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:38:44.540847+0000","last_undegraded":"2024-09-10T20:38:44.540847+0000","last_fullsized":"2024-09-10T20:38:44.540847+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4476928,"num_objects":1093,"num_object_clones":0,"num_object_copies":2186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18035,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19675","reported_seq":27033,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:46.937897+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:38:46.937897+0000","last_peered":"2024-09-10T20:38:46.937897+0000","last_clean":"2024-09-10T20:38:46.937897+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:38:46.937897+0000","last_undegraded":"2024-09-10T20:38:46.937897+0000","last_fullsized":"2024-09-10T20:38:46.937897+0000","mapping_epoch":44,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4259864,"num_objects":1041,"num_object_clones":0,"num_object_copies":2082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1041,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17803,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19462","reported_seq":19553,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:44.600622+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:38:44.600622+0000","last_peered":"2024-09-10T20:38:44.600622+0000","last_clean":"2024-09-10T20:38:44.600622+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:38:44.600622+0000","last_undegraded":"2024-09-10T20:38:44.600622+0000","last_fullsized":"2024-09-10T20:38:44.600622+0000","mapping_epoch":35,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4677632,"num_objects":1142,"num_object_clones":0,"num_object_copies":2284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19462,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19938","reported_seq":20075,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:44.877697+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:38:44.877697+0000","last_peered":"2024-09-10T20:38:44.877697+0000","last_clean":"2024-09-10T20:38:44.877697+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:38:44.877697+0000","last_undegraded":"2024-09-10T20:38:44.877697+0000","last_fullsized":"2024-09-10T20:38:44.877697+0000","mapping_epoch":34,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4632576,"num_objects":1131,"num_object_clones":0,"num_object_copies":2262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16934,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'19378","reported_seq":19631,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:44.822393+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:38:44.822393+0000","last_peered":"2024-09-10T20:38:44.822393+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:38:44.822393+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4476928,"num_objects":1093,"num_object_clones":0,"num_object_copies":2186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16242,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19512","reported_seq":19797,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:44.888095+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:38:44.888095+0000","last_peered":"2024-09-10T20:38:44.888095+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:38:44.888095+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4489216,"num_objects":1096,"num_object_clones":0,"num_object_copies":2192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16107,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2390}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19637","reported_seq":19771,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:44.866442+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:38:44.866442+0000","last_peered":"2024-09-10T20:38:44.866442+0000","last_clean":"2024-09-10T20:38:44.866442+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:38:44.866442+0000","last_undegraded":"2024-09-10T20:38:44.866442+0000","last_fullsized":"2024-09-10T20:38:44.866442+0000","mapping_epoch":34,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4501504,"num_objects":1099,"num_object_clones":0,"num_object_copies":2198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1099,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16708,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315452+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315452+0000","last_peered":"2024-09-10T20:36:30.315452+0000","last_clean":"2024-09-10T20:36:30.315452+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315452+0000","last_undegraded":"2024-09-10T20:36:30.315452+0000","last_fullsized":"2024-09-10T20:36:30.315452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19322","reported_seq":19521,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:44.590154+0000","last_change":"2024-09-10T20:30:05.721572+0000","last_active":"2024-09-10T20:38:44.590154+0000","last_peered":"2024-09-10T20:38:44.590154+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:38:44.590154+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4575232,"num_objects":1117,"num_object_clones":0,"num_object_copies":2234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17315,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2380}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.318059+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:36:30.318059+0000","last_peered":"2024-09-10T20:36:30.318059+0000","last_clean":"2024-09-10T20:36:30.318059+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:36:30.318059+0000","last_undegraded":"2024-09-10T20:36:30.318059+0000","last_fullsized":"2024-09-10T20:36:30.318059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19612","reported_seq":19829,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:44.784359+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:38:44.784359+0000","last_peered":"2024-09-10T20:38:44.784359+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:38:44.784359+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4580294,"num_objects":1120,"num_object_clones":0,"num_object_copies":2240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19610,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2446}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315295+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:36:30.315295+0000","last_peered":"2024-09-10T20:36:30.315295+0000","last_clean":"2024-09-10T20:36:30.315295+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:36:30.315295+0000","last_undegraded":"2024-09-10T20:36:30.315295+0000","last_fullsized":"2024-09-10T20:36:30.315295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19470","reported_seq":19663,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:44.592916+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:38:44.592916+0000","last_peered":"2024-09-10T20:38:44.592916+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:38:44.592916+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4485120,"num_objects":1095,"num_object_clones":0,"num_object_copies":2190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1095,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17298,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2511}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316250+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316250+0000","last_peered":"2024-09-10T20:36:30.316250+0000","last_clean":"2024-09-10T20:36:30.316250+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316250+0000","last_undegraded":"2024-09-10T20:36:30.316250+0000","last_fullsized":"2024-09-10T20:36:30.316250+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19571","reported_seq":26942,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:44.595627+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:38:44.595627+0000","last_peered":"2024-09-10T20:38:44.595627+0000","last_clean":"2024-09-10T20:38:44.595627+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:38:44.595627+0000","last_undegraded":"2024-09-10T20:38:44.595627+0000","last_fullsized":"2024-09-10T20:38:44.595627+0000","mapping_epoch":48,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4476928,"num_objects":1093,"num_object_clones":0,"num_object_copies":2186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17357,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19723","reported_seq":19980,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:44.936718+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:38:44.936718+0000","last_peered":"2024-09-10T20:38:44.936718+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:38:44.936718+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4534272,"num_objects":1107,"num_object_clones":0,"num_object_copies":2214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16446,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2485}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316613+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316613+0000","last_peered":"2024-09-10T20:36:30.316613+0000","last_clean":"2024-09-10T20:36:30.316613+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316613+0000","last_undegraded":"2024-09-10T20:36:30.316613+0000","last_fullsized":"2024-09-10T20:36:30.316613+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19514","reported_seq":26435,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:44.880513+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:38:44.880513+0000","last_peered":"2024-09-10T20:38:44.880513+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:38:44.880513+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4382720,"num_objects":1070,"num_object_clones":0,"num_object_copies":2140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1070,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16208,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3386,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":921}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.319644+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:36:30.319644+0000","last_peered":"2024-09-10T20:36:30.319644+0000","last_clean":"2024-09-10T20:36:30.319644+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:36:30.319644+0000","last_undegraded":"2024-09-10T20:36:30.319644+0000","last_fullsized":"2024-09-10T20:36:30.319644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19865","reported_seq":20122,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:44.851121+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:38:44.851121+0000","last_peered":"2024-09-10T20:38:44.851121+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:38:44.851121+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4591616,"num_objects":1121,"num_object_clones":0,"num_object_copies":2242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16692,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2534}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316994+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316994+0000","last_peered":"2024-09-10T20:36:30.316994+0000","last_clean":"2024-09-10T20:36:30.316994+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316994+0000","last_undegraded":"2024-09-10T20:36:30.316994+0000","last_fullsized":"2024-09-10T20:36:30.316994+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:35.648960+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:35.648960+0000","last_peered":"2024-09-10T20:36:35.648960+0000","last_clean":"2024-09-10T20:36:35.648960+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:35.648960+0000","last_undegraded":"2024-09-10T20:36:35.648960+0000","last_fullsized":"2024-09-10T20:36:35.648960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19451","reported_seq":26579,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:46.857078+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:38:46.857078+0000","last_peered":"2024-09-10T20:38:46.857078+0000","last_clean":"2024-09-10T20:38:46.857078+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:38:46.857078+0000","last_undegraded":"2024-09-10T20:38:46.857078+0000","last_fullsized":"2024-09-10T20:38:46.857078+0000","mapping_epoch":46,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4362723,"num_objects":1066,"num_object_clones":0,"num_object_copies":2132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16058,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":72103361,"num_objects":17607,"num_object_clones":0,"num_object_copies":35214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28666,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17607,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":277810,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14483,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":318857216,"data_stored":318791264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160714,"ondisk_log_size":160714,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182522,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":563224,"kb_used_data":112320,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93808616,"statfs":{"total":96636764160,"available":96060022784,"internally_reserved":0,"allocated":115015680,"data_stored":114753732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":461696682},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,2,2,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986294,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":498628,"kb_used_data":151792,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93873212,"statfs":{"total":96636764160,"available":96126169088,"internally_reserved":0,"allocated":155435008,"data_stored":155173065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355135146},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,0,5,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659241,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":507208,"kb_used_data":52080,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93864632,"statfs":{"total":96636764160,"available":96117383168,"internally_reserved":0,"allocated":53329920,"data_stored":53016931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":466022062},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,0,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":113324032,"data_stored":113320419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":51294208,"data_stored":51239554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":154238976,"data_stored":154231291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:38:50.147 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:38:50.147 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'19535", 'reported_seq': 19758, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:44.825766+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:38:44.825766+0000', 'last_peered': '2024-09-10T20:38:44.825766+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:38:44.825766+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4599808, 'num_objects': 1123, 'num_object_clones': 0, 'num_object_copies': 2246, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1123, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19535, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2418}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:50.147 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:38:50.147 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'19378", 'reported_seq': 19631, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:44.822393+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:38:44.822393+0000', 'last_peered': '2024-09-10T20:38:44.822393+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:38:44.822393+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'9300", 'ondisk_log_start': "45'9300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4476928, 'num_objects': 1093, 'num_object_clones': 0, 'num_object_copies': 2186, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1093, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16242, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2468}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:50.147 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:38:50.147 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19512", 'reported_seq': 19797, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:44.888095+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:38:44.888095+0000', 'last_peered': '2024-09-10T20:38:44.888095+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:38:44.888095+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4489216, 'num_objects': 1096, 'num_object_clones': 0, 'num_object_copies': 2192, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1096, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16107, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2390}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:50.147 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:38:50.148 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'19322", 'reported_seq': 19521, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:44.590154+0000', 'last_change': '2024-09-10T20:30:05.721572+0000', 'last_active': '2024-09-10T20:38:44.590154+0000', 'last_peered': '2024-09-10T20:38:44.590154+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:38:44.590154+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "45'9300", 'ondisk_log_start': "45'9300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4575232, 'num_objects': 1117, 'num_object_clones': 0, 'num_object_copies': 2234, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1117, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17315, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2380}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:38:50.148 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:38:50.148 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19612", 'reported_seq': 19829, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:44.784359+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:38:44.784359+0000', 'last_peered': '2024-09-10T20:38:44.784359+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:38:44.784359+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4580294, 'num_objects': 1120, 'num_object_clones': 0, 'num_object_copies': 2240, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1120, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19610, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2446}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:50.148 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:38:50.148 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'19470", 'reported_seq': 19663, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:44.592916+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:38:44.592916+0000', 'last_peered': '2024-09-10T20:38:44.592916+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:38:44.592916+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "45'9400", 'ondisk_log_start': "45'9400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4485120, 'num_objects': 1095, 'num_object_clones': 0, 'num_object_copies': 2190, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1095, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17298, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2511}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:38:50.148 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:38:50.148 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'19723", 'reported_seq': 19980, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:44.936718+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:38:44.936718+0000', 'last_peered': '2024-09-10T20:38:44.936718+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:38:44.936718+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4534272, 'num_objects': 1107, 'num_object_clones': 0, 'num_object_copies': 2214, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1107, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16446, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2485}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:50.148 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T20:38:50.148 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19514", 'reported_seq': 26435, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:44.880513+0000', 'last_change': '2024-09-10T20:36:28.778006+0000', 'last_active': '2024-09-10T20:38:44.880513+0000', 'last_peered': '2024-09-10T20:38:44.880513+0000', 'last_clean': '2024-09-10T20:30:03.290990+0000', 'last_became_active': '2024-09-10T20:30:05.703208+0000', 'last_became_peered': '2024-09-10T20:30:05.703208+0000', 'last_unstale': '2024-09-10T20:38:44.880513+0000', 'last_undegraded': '2024-09-10T20:30:05.380322+0000', 'last_fullsized': '2024-09-10T20:30:05.306241+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3352", 'last_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:00.666616+0000', 'objects_scrubbed': 3298, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1096, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4382720, 'num_objects': 1070, 'num_object_clones': 0, 'num_object_copies': 2140, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1070, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16208, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3386, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 921}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:50.148 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:38:50.149 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19865", 'reported_seq': 20122, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:44.851121+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:38:44.851121+0000', 'last_peered': '2024-09-10T20:38:44.851121+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:38:44.851121+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4591616, 'num_objects': 1121, 'num_object_clones': 0, 'num_object_copies': 2242, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1121, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16692, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2534}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:50.149 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:38:50.795 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:50.795 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 28666/35218 objects degraded (81.396%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":642,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":17609,"data_bytes":72103380,"bytes_used":1606717440,"bytes_avail":288303575040,"bytes_total":289910292480,"degraded_objects":28666,"degraded_total":35218,"degraded_ratio":0.81395877108296899,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":312,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":419},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:38:50.821 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:38:50.822 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:38:51.422 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:51.425 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:38:51.452 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":366,"stamp":"2024-09-10T20:38:51.040516+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":63686100,"num_objects":15554,"num_object_clones":0,"num_object_copies":31108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28661,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15554,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":279867,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14490,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160672,"ondisk_log_size":160672,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1548348,"kb_used_data":303804,"kb_used_omap":12,"kb_used_meta":1244467,"kb_avail":281567172,"statfs":{"total":289910292480,"available":288324784128,"internally_reserved":0,"allocated":311095296,"data_stored":310258416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1274334210},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,0,2,2,8,7,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18747392,"num_objects":-4577,"num_object_clones":0,"num_object_copies":-9154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4577,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003550"},"pg_stats":[{"pgid":"2.f","version":"50'19705","reported_seq":19929,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:49.921145+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:38:49.921145+0000","last_peered":"2024-09-10T20:38:49.921145+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:38:49.921145+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3903488,"num_objects":953,"num_object_clones":0,"num_object_copies":1906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19705,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2588}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20005","reported_seq":20128,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T20:38:49.672160+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:38:49.672160+0000","last_peered":"2024-09-10T20:38:49.672160+0000","last_clean":"2024-09-10T20:38:49.672160+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:38:49.672160+0000","last_undegraded":"2024-09-10T20:38:49.672160+0000","last_fullsized":"2024-09-10T20:38:49.672160+0000","mapping_epoch":35,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3837952,"num_objects":937,"num_object_clones":0,"num_object_copies":1874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18191,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19675","reported_seq":27033,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:46.937897+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:38:46.937897+0000","last_peered":"2024-09-10T20:38:46.937897+0000","last_clean":"2024-09-10T20:38:46.937897+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:38:46.937897+0000","last_undegraded":"2024-09-10T20:38:46.937897+0000","last_fullsized":"2024-09-10T20:38:46.937897+0000","mapping_epoch":44,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4259864,"num_objects":1041,"num_object_clones":0,"num_object_copies":2082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1041,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17803,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19602","reported_seq":19695,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T20:38:49.670905+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:38:49.670905+0000","last_peered":"2024-09-10T20:38:49.670905+0000","last_clean":"2024-09-10T20:38:49.670905+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:38:49.670905+0000","last_undegraded":"2024-09-10T20:38:49.670905+0000","last_fullsized":"2024-09-10T20:38:49.670905+0000","mapping_epoch":35,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4104192,"num_objects":1002,"num_object_clones":0,"num_object_copies":2004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19602,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20099","reported_seq":20238,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T20:38:49.903688+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:38:49.903688+0000","last_peered":"2024-09-10T20:38:49.903688+0000","last_clean":"2024-09-10T20:38:49.903688+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:38:49.903688+0000","last_undegraded":"2024-09-10T20:38:49.903688+0000","last_fullsized":"2024-09-10T20:38:49.903688+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3973120,"num_objects":970,"num_object_clones":0,"num_object_copies":1940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17095,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'19561","reported_seq":19816,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:49.902440+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:38:49.902440+0000","last_peered":"2024-09-10T20:38:49.902440+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:38:49.902440+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3727360,"num_objects":910,"num_object_clones":0,"num_object_copies":1820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2651}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'19676","reported_seq":19965,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:49.951122+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:38:49.951122+0000","last_peered":"2024-09-10T20:38:49.951122+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:38:49.951122+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3817472,"num_objects":932,"num_object_clones":0,"num_object_copies":1864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":932,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16271,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2554}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'19808","reported_seq":19945,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T20:38:49.945301+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:38:49.945301+0000","last_peered":"2024-09-10T20:38:49.945301+0000","last_clean":"2024-09-10T20:38:49.945301+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:38:49.945301+0000","last_undegraded":"2024-09-10T20:38:49.945301+0000","last_fullsized":"2024-09-10T20:38:49.945301+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3801088,"num_objects":928,"num_object_clones":0,"num_object_copies":1856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16879,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.315452+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:36:30.315452+0000","last_peered":"2024-09-10T20:36:30.315452+0000","last_clean":"2024-09-10T20:36:30.315452+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:36:30.315452+0000","last_undegraded":"2024-09-10T20:36:30.315452+0000","last_fullsized":"2024-09-10T20:36:30.315452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'19490","reported_seq":19703,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:49.682521+0000","last_change":"2024-09-10T20:38:49.349021+0000","last_active":"2024-09-10T20:38:49.682521+0000","last_peered":"2024-09-10T20:38:49.682521+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:38:49.682521+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3887104,"num_objects":949,"num_object_clones":0,"num_object_copies":1898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3494,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17483,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2545}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T20:38:49.671978+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:38:49.671978+0000","last_peered":"2024-09-10T20:38:49.671978+0000","last_clean":"2024-09-10T20:38:49.671978+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:38:49.671978+0000","last_undegraded":"2024-09-10T20:38:49.671978+0000","last_fullsized":"2024-09-10T20:38:49.671978+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'19761","reported_seq":19980,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:49.839712+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:38:49.839712+0000","last_peered":"2024-09-10T20:38:49.839712+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:38:49.839712+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3969990,"num_objects":971,"num_object_clones":0,"num_object_copies":1942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19759,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2595}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":113,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T20:38:49.664762+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:38:49.664762+0000","last_peered":"2024-09-10T20:38:49.664762+0000","last_clean":"2024-09-10T20:38:49.664762+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:38:49.664762+0000","last_undegraded":"2024-09-10T20:38:49.664762+0000","last_fullsized":"2024-09-10T20:38:49.664762+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'19617","reported_seq":19811,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:49.674932+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:38:49.674932+0000","last_peered":"2024-09-10T20:38:49.674932+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:38:49.674932+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3883008,"num_objects":948,"num_object_clones":0,"num_object_copies":1896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17445,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2658}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316250+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:36:30.316250+0000","last_peered":"2024-09-10T20:36:30.316250+0000","last_clean":"2024-09-10T20:36:30.316250+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:36:30.316250+0000","last_undegraded":"2024-09-10T20:36:30.316250+0000","last_fullsized":"2024-09-10T20:36:30.316250+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19730","reported_seq":27103,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T20:38:49.671131+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:38:49.671131+0000","last_peered":"2024-09-10T20:38:49.671131+0000","last_clean":"2024-09-10T20:38:49.671131+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:38:49.671131+0000","last_undegraded":"2024-09-10T20:38:49.671131+0000","last_fullsized":"2024-09-10T20:38:49.671131+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3825664,"num_objects":934,"num_object_clones":0,"num_object_copies":1868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17516,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'19862","reported_seq":20122,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:49.869862+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:38:49.869862+0000","last_peered":"2024-09-10T20:38:49.869862+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:38:49.869862+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3964928,"num_objects":968,"num_object_clones":0,"num_object_copies":1936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16585,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2624}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316613+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:36:30.316613+0000","last_peered":"2024-09-10T20:36:30.316613+0000","last_clean":"2024-09-10T20:36:30.316613+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:36:30.316613+0000","last_undegraded":"2024-09-10T20:36:30.316613+0000","last_fullsized":"2024-09-10T20:36:30.316613+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19514","reported_seq":26435,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:44.880513+0000","last_change":"2024-09-10T20:36:28.778006+0000","last_active":"2024-09-10T20:38:44.880513+0000","last_peered":"2024-09-10T20:38:44.880513+0000","last_clean":"2024-09-10T20:30:03.290990+0000","last_became_active":"2024-09-10T20:30:05.703208+0000","last_became_peered":"2024-09-10T20:30:05.703208+0000","last_unstale":"2024-09-10T20:38:44.880513+0000","last_undegraded":"2024-09-10T20:30:05.380322+0000","last_fullsized":"2024-09-10T20:30:05.306241+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4382720,"num_objects":1070,"num_object_clones":0,"num_object_copies":2140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1070,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16208,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3386,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":921}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T20:38:49.671887+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:38:49.671887+0000","last_peered":"2024-09-10T20:38:49.671887+0000","last_clean":"2024-09-10T20:38:49.671887+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:38:49.671887+0000","last_undegraded":"2024-09-10T20:38:49.671887+0000","last_fullsized":"2024-09-10T20:38:49.671887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20013","reported_seq":20273,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:49.853644+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:38:49.853644+0000","last_peered":"2024-09-10T20:38:49.853644+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:38:49.853644+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3985408,"num_objects":973,"num_object_clones":0,"num_object_copies":1946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16840,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2682}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:30.316994+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:36:30.316994+0000","last_peered":"2024-09-10T20:36:30.316994+0000","last_clean":"2024-09-10T20:36:30.316994+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:36:30.316994+0000","last_undegraded":"2024-09-10T20:36:30.316994+0000","last_fullsized":"2024-09-10T20:36:30.316994+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:36:35.648960+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:36:35.648960+0000","last_peered":"2024-09-10T20:36:35.648960+0000","last_clean":"2024-09-10T20:36:35.648960+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:36:35.648960+0000","last_undegraded":"2024-09-10T20:36:35.648960+0000","last_fullsized":"2024-09-10T20:36:35.648960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19451","reported_seq":26579,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T20:38:46.857078+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:38:46.857078+0000","last_peered":"2024-09-10T20:38:46.857078+0000","last_clean":"2024-09-10T20:38:46.857078+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:38:46.857078+0000","last_undegraded":"2024-09-10T20:38:46.857078+0000","last_fullsized":"2024-09-10T20:38:46.857078+0000","mapping_epoch":46,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4362723,"num_objects":1066,"num_object_clones":0,"num_object_copies":2132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16058,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":63686081,"num_objects":15552,"num_object_clones":0,"num_object_copies":31104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28661,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15552,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":279865,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14488,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":306085888,"data_stored":306019936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160669,"ondisk_log_size":160669,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182523,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":559992,"kb_used_data":104932,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93811848,"statfs":{"total":96636764160,"available":96063332352,"internally_reserved":0,"allocated":107450368,"data_stored":107188420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":465956522},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,1,1,5,7,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986295,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481148,"kb_used_data":146792,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93890692,"statfs":{"total":96636764160,"available":96144068608,"internally_reserved":0,"allocated":150315008,"data_stored":150053065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342355626},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659241,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":507208,"kb_used_data":52080,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93864632,"statfs":{"total":96636764160,"available":96117383168,"internally_reserved":0,"allocated":53329920,"data_stored":53016931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":466022062},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,0,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":105713664,"data_stored":105710051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":51294208,"data_stored":51239554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":149078016,"data_stored":149070331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:38:54.454 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:38:55.047 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:55.053 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:38:55.074 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":367,"stamp":"2024-09-10T20:38:53.041162+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":61760980,"num_objects":15084,"num_object_clones":0,"num_object_copies":30168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28512,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15084,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":280337,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14639,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1531560,"kb_used_data":295340,"kb_used_omap":12,"kb_used_meta":1236147,"kb_avail":281583960,"statfs":{"total":289910292480,"available":288341975040,"internally_reserved":0,"allocated":302428160,"data_stored":301588801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1265814530},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,1,1,5,6,14,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11661312,"num_objects":-2847,"num_object_clones":0,"num_object_copies":-5694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2847,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2847,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003466"},"pg_stats":[{"pgid":"2.f","version":"50'19705","reported_seq":19929,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:49.921145+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:38:49.921145+0000","last_peered":"2024-09-10T20:38:49.921145+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:38:49.921145+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3903488,"num_objects":953,"num_object_clones":0,"num_object_copies":1906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19705,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2588}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20005","reported_seq":20128,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T20:38:49.672160+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:38:49.672160+0000","last_peered":"2024-09-10T20:38:49.672160+0000","last_clean":"2024-09-10T20:38:49.672160+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:38:49.672160+0000","last_undegraded":"2024-09-10T20:38:49.672160+0000","last_fullsized":"2024-09-10T20:38:49.672160+0000","mapping_epoch":35,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3837952,"num_objects":937,"num_object_clones":0,"num_object_copies":1874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18191,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19819","reported_seq":27181,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:51.997197+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:38:51.997197+0000","last_peered":"2024-09-10T20:38:51.997197+0000","last_clean":"2024-09-10T20:38:51.997197+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:38:51.997197+0000","last_undegraded":"2024-09-10T20:38:51.997197+0000","last_fullsized":"2024-09-10T20:38:51.997197+0000","mapping_epoch":44,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3670040,"num_objects":897,"num_object_clones":0,"num_object_copies":1794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":897,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17947,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19602","reported_seq":19695,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T20:38:49.670905+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:38:49.670905+0000","last_peered":"2024-09-10T20:38:49.670905+0000","last_clean":"2024-09-10T20:38:49.670905+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:38:49.670905+0000","last_undegraded":"2024-09-10T20:38:49.670905+0000","last_fullsized":"2024-09-10T20:38:49.670905+0000","mapping_epoch":35,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4104192,"num_objects":1002,"num_object_clones":0,"num_object_copies":2004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19602,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20099","reported_seq":20238,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T20:38:49.903688+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:38:49.903688+0000","last_peered":"2024-09-10T20:38:49.903688+0000","last_clean":"2024-09-10T20:38:49.903688+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:38:49.903688+0000","last_undegraded":"2024-09-10T20:38:49.903688+0000","last_fullsized":"2024-09-10T20:38:49.903688+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3973120,"num_objects":970,"num_object_clones":0,"num_object_copies":1940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17095,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'19561","reported_seq":19816,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:49.902440+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:38:49.902440+0000","last_peered":"2024-09-10T20:38:49.902440+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:38:49.902440+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3727360,"num_objects":910,"num_object_clones":0,"num_object_copies":1820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2651}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'19676","reported_seq":19965,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:49.951122+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:38:49.951122+0000","last_peered":"2024-09-10T20:38:49.951122+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:38:49.951122+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3817472,"num_objects":932,"num_object_clones":0,"num_object_copies":1864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":932,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16271,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2554}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'19808","reported_seq":19945,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T20:38:49.945301+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:38:49.945301+0000","last_peered":"2024-09-10T20:38:49.945301+0000","last_clean":"2024-09-10T20:38:49.945301+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:38:49.945301+0000","last_undegraded":"2024-09-10T20:38:49.945301+0000","last_fullsized":"2024-09-10T20:38:49.945301+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3801088,"num_objects":928,"num_object_clones":0,"num_object_copies":1856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16879,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.652747+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:38:50.652747+0000","last_peered":"2024-09-10T20:38:50.652747+0000","last_clean":"2024-09-10T20:38:50.652747+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:38:50.652747+0000","last_undegraded":"2024-09-10T20:38:50.652747+0000","last_fullsized":"2024-09-10T20:38:50.652747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'19490","reported_seq":19703,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:49.682521+0000","last_change":"2024-09-10T20:38:49.349021+0000","last_active":"2024-09-10T20:38:49.682521+0000","last_peered":"2024-09-10T20:38:49.682521+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:38:49.682521+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3887104,"num_objects":949,"num_object_clones":0,"num_object_copies":1898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3494,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17483,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2545}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T20:38:49.671978+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:38:49.671978+0000","last_peered":"2024-09-10T20:38:49.671978+0000","last_clean":"2024-09-10T20:38:49.671978+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:38:49.671978+0000","last_undegraded":"2024-09-10T20:38:49.671978+0000","last_fullsized":"2024-09-10T20:38:49.671978+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'19761","reported_seq":19980,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:49.839712+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:38:49.839712+0000","last_peered":"2024-09-10T20:38:49.839712+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:38:49.839712+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3969990,"num_objects":971,"num_object_clones":0,"num_object_copies":1942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19759,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2595}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":113,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T20:38:49.664762+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:38:49.664762+0000","last_peered":"2024-09-10T20:38:49.664762+0000","last_clean":"2024-09-10T20:38:49.664762+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:38:49.664762+0000","last_undegraded":"2024-09-10T20:38:49.664762+0000","last_fullsized":"2024-09-10T20:38:49.664762+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'19617","reported_seq":19811,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:49.674932+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:38:49.674932+0000","last_peered":"2024-09-10T20:38:49.674932+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:38:49.674932+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3883008,"num_objects":948,"num_object_clones":0,"num_object_copies":1896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17445,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2658}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651037+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:38:50.651037+0000","last_peered":"2024-09-10T20:38:50.651037+0000","last_clean":"2024-09-10T20:38:50.651037+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:38:50.651037+0000","last_undegraded":"2024-09-10T20:38:50.651037+0000","last_fullsized":"2024-09-10T20:38:50.651037+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19730","reported_seq":27103,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T20:38:49.671131+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:38:49.671131+0000","last_peered":"2024-09-10T20:38:49.671131+0000","last_clean":"2024-09-10T20:38:49.671131+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:38:49.671131+0000","last_undegraded":"2024-09-10T20:38:49.671131+0000","last_fullsized":"2024-09-10T20:38:49.671131+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3825664,"num_objects":934,"num_object_clones":0,"num_object_copies":1868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17516,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'19862","reported_seq":20122,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:49.869862+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:38:49.869862+0000","last_peered":"2024-09-10T20:38:49.869862+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:38:49.869862+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3964928,"num_objects":968,"num_object_clones":0,"num_object_copies":1936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16585,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2624}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651168+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:38:50.651168+0000","last_peered":"2024-09-10T20:38:50.651168+0000","last_clean":"2024-09-10T20:38:50.651168+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:38:50.651168+0000","last_undegraded":"2024-09-10T20:38:50.651168+0000","last_fullsized":"2024-09-10T20:38:50.651168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'19692","reported_seq":26925,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:51.965526+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:38:51.965526+0000","last_peered":"2024-09-10T20:38:51.965526+0000","last_clean":"2024-09-10T20:38:51.965526+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:38:51.965526+0000","last_undegraded":"2024-09-10T20:38:51.965526+0000","last_fullsized":"2024-09-10T20:38:51.965526+0000","mapping_epoch":50,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3653632,"num_objects":892,"num_object_clones":0,"num_object_copies":1784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16386,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T20:38:49.671887+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:38:49.671887+0000","last_peered":"2024-09-10T20:38:49.671887+0000","last_clean":"2024-09-10T20:38:49.671887+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:38:49.671887+0000","last_undegraded":"2024-09-10T20:38:49.671887+0000","last_fullsized":"2024-09-10T20:38:49.671887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20013","reported_seq":20273,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:49.853644+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:38:49.853644+0000","last_peered":"2024-09-10T20:38:49.853644+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:38:49.853644+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3985408,"num_objects":973,"num_object_clones":0,"num_object_copies":1946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16840,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2682}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651258+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:38:50.651258+0000","last_peered":"2024-09-10T20:38:50.651258+0000","last_clean":"2024-09-10T20:38:50.651258+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:38:50.651258+0000","last_undegraded":"2024-09-10T20:38:50.651258+0000","last_fullsized":"2024-09-10T20:38:50.651258+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651173+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:38:50.651173+0000","last_peered":"2024-09-10T20:38:50.651173+0000","last_clean":"2024-09-10T20:38:50.651173+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:38:50.651173+0000","last_undegraded":"2024-09-10T20:38:50.651173+0000","last_fullsized":"2024-09-10T20:38:50.651173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'19599","reported_seq":26731,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:51.819892+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:38:51.819892+0000","last_peered":"2024-09-10T20:38:51.819892+0000","last_clean":"2024-09-10T20:38:51.819892+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:38:51.819892+0000","last_undegraded":"2024-09-10T20:38:51.819892+0000","last_fullsized":"2024-09-10T20:38:51.819892+0000","mapping_epoch":46,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3756515,"num_objects":918,"num_object_clones":0,"num_object_copies":1836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16206,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":61760961,"num_objects":15082,"num_object_clones":0,"num_object_copies":30164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28512,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15082,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":280335,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14637,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":298840064,"data_stored":298781338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160739,"ondisk_log_size":160739,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182523,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":559992,"kb_used_data":104932,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93811848,"statfs":{"total":96636764160,"available":96063332352,"internally_reserved":0,"allocated":107450368,"data_stored":107188420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":465956522},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,1,1,5,7,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986295,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481148,"kb_used_data":146792,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93890692,"statfs":{"total":96636764160,"available":96144068608,"internally_reserved":0,"allocated":150315008,"data_stored":150053065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342355626},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659242,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490420,"kb_used_data":43616,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93881420,"statfs":{"total":96636764160,"available":96134574080,"internally_reserved":0,"allocated":44662784,"data_stored":44347316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":457502382},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,4,0,7],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":105713664,"data_stored":105710051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":44048384,"data_stored":44000956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":149078016,"data_stored":149070331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:38:55.075 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:38:55.619 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:55.619 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:38:55.644 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2024-09-10T20:38:55.042106+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":53806548,"num_objects":13142,"num_object_clones":0,"num_object_copies":26284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28355,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13142,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282279,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14796,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160784,"ondisk_log_size":160784,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1529308,"kb_used_data":283360,"kb_used_omap":12,"kb_used_meta":1245875,"kb_avail":281586212,"statfs":{"total":289910292480,"available":288344281088,"internally_reserved":0,"allocated":290160640,"data_stored":289301871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1275776002},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,5,5,7,9,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18296832,"num_objects":-4467,"num_object_clones":0,"num_object_copies":-8934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-311,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4467,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003617"},"pg_stats":[{"pgid":"2.f","version":"51'19856","reported_seq":20083,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:54.887732+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:38:54.887732+0000","last_peered":"2024-09-10T20:38:54.887732+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:38:54.887732+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3284992,"num_objects":802,"num_object_clones":0,"num_object_copies":1604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19856,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2739}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20162","reported_seq":20287,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:54.643685+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:38:54.643685+0000","last_peered":"2024-09-10T20:38:54.643685+0000","last_clean":"2024-09-10T20:38:54.643685+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:38:54.643685+0000","last_undegraded":"2024-09-10T20:38:54.643685+0000","last_fullsized":"2024-09-10T20:38:54.643685+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3194880,"num_objects":780,"num_object_clones":0,"num_object_copies":1560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18348,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19819","reported_seq":27181,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:51.997197+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:38:51.997197+0000","last_peered":"2024-09-10T20:38:51.997197+0000","last_clean":"2024-09-10T20:38:51.997197+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:38:51.997197+0000","last_undegraded":"2024-09-10T20:38:51.997197+0000","last_fullsized":"2024-09-10T20:38:51.997197+0000","mapping_epoch":44,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3670040,"num_objects":897,"num_object_clones":0,"num_object_copies":1794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":897,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17947,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'19761","reported_seq":19856,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:54.612259+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:38:54.612259+0000","last_peered":"2024-09-10T20:38:54.612259+0000","last_clean":"2024-09-10T20:38:54.612259+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:38:54.612259+0000","last_undegraded":"2024-09-10T20:38:54.612259+0000","last_fullsized":"2024-09-10T20:38:54.612259+0000","mapping_epoch":35,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3452928,"num_objects":843,"num_object_clones":0,"num_object_copies":1686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19761,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20246","reported_seq":20387,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:54.868314+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:38:54.868314+0000","last_peered":"2024-09-10T20:38:54.868314+0000","last_clean":"2024-09-10T20:38:54.868314+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:38:54.868314+0000","last_undegraded":"2024-09-10T20:38:54.868314+0000","last_fullsized":"2024-09-10T20:38:54.868314+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3371008,"num_objects":823,"num_object_clones":0,"num_object_copies":1646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17242,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'19709","reported_seq":19964,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:54.963029+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:38:54.963029+0000","last_peered":"2024-09-10T20:38:54.963029+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:38:54.963029+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3121152,"num_objects":762,"num_object_clones":0,"num_object_copies":1524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16573,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2799}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19835","reported_seq":20125,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:54.963599+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:38:54.963599+0000","last_peered":"2024-09-10T20:38:54.963599+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:38:54.963599+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3166208,"num_objects":773,"num_object_clones":0,"num_object_copies":1546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16430,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'19968","reported_seq":20107,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:54.976967+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:38:54.976967+0000","last_peered":"2024-09-10T20:38:54.976967+0000","last_clean":"2024-09-10T20:38:54.976967+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:38:54.976967+0000","last_undegraded":"2024-09-10T20:38:54.976967+0000","last_fullsized":"2024-09-10T20:38:54.976967+0000","mapping_epoch":34,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":768,"num_object_clones":0,"num_object_copies":1536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.652747+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:38:50.652747+0000","last_peered":"2024-09-10T20:38:50.652747+0000","last_clean":"2024-09-10T20:38:50.652747+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:38:50.652747+0000","last_undegraded":"2024-09-10T20:38:50.652747+0000","last_fullsized":"2024-09-10T20:38:50.652747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'19620","reported_seq":20150,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:54.695860+0000","last_change":"2024-09-10T20:38:49.349021+0000","last_active":"2024-09-10T20:38:54.695860+0000","last_peered":"2024-09-10T20:38:54.695860+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:38:54.695860+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3354624,"num_objects":819,"num_object_clones":0,"num_object_copies":1638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3337,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17613,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":191,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2518}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.654924+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:38:50.654924+0000","last_peered":"2024-09-10T20:38:50.654924+0000","last_clean":"2024-09-10T20:38:50.654924+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:38:50.654924+0000","last_undegraded":"2024-09-10T20:38:50.654924+0000","last_fullsized":"2024-09-10T20:38:50.654924+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'19917","reported_seq":20138,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:54.772612+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:38:54.772612+0000","last_peered":"2024-09-10T20:38:54.772612+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:38:54.772612+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3331014,"num_objects":815,"num_object_clones":0,"num_object_copies":1630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19915,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2751}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.648348+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:38:50.648348+0000","last_peered":"2024-09-10T20:38:50.648348+0000","last_clean":"2024-09-10T20:38:50.648348+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:38:50.648348+0000","last_undegraded":"2024-09-10T20:38:50.648348+0000","last_fullsized":"2024-09-10T20:38:50.648348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19761","reported_seq":19958,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:54.653518+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:38:54.653518+0000","last_peered":"2024-09-10T20:38:54.653518+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:38:54.653518+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3293184,"num_objects":804,"num_object_clones":0,"num_object_copies":1608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17589,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2802}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651037+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:38:50.651037+0000","last_peered":"2024-09-10T20:38:50.651037+0000","last_clean":"2024-09-10T20:38:50.651037+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:38:50.651037+0000","last_undegraded":"2024-09-10T20:38:50.651037+0000","last_fullsized":"2024-09-10T20:38:50.651037+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19854","reported_seq":27229,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:54.626431+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:38:54.626431+0000","last_peered":"2024-09-10T20:38:54.626431+0000","last_clean":"2024-09-10T20:38:54.626431+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:38:54.626431+0000","last_undegraded":"2024-09-10T20:38:54.626431+0000","last_fullsized":"2024-09-10T20:38:54.626431+0000","mapping_epoch":48,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3317760,"num_objects":810,"num_object_clones":0,"num_object_copies":1620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20013","reported_seq":20275,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:54.863951+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:38:54.863951+0000","last_peered":"2024-09-10T20:38:54.863951+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:38:54.863951+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3346432,"num_objects":817,"num_object_clones":0,"num_object_copies":1634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16736,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2775}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651168+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:38:50.651168+0000","last_peered":"2024-09-10T20:38:50.651168+0000","last_clean":"2024-09-10T20:38:50.651168+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:38:50.651168+0000","last_undegraded":"2024-09-10T20:38:50.651168+0000","last_fullsized":"2024-09-10T20:38:50.651168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'19692","reported_seq":26925,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:51.965526+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:38:51.965526+0000","last_peered":"2024-09-10T20:38:51.965526+0000","last_clean":"2024-09-10T20:38:51.965526+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:38:51.965526+0000","last_undegraded":"2024-09-10T20:38:51.965526+0000","last_fullsized":"2024-09-10T20:38:51.965526+0000","mapping_epoch":50,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3653632,"num_objects":892,"num_object_clones":0,"num_object_copies":1784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16386,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.656849+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:38:50.656849+0000","last_peered":"2024-09-10T20:38:50.656849+0000","last_clean":"2024-09-10T20:38:50.656849+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:38:50.656849+0000","last_undegraded":"2024-09-10T20:38:50.656849+0000","last_fullsized":"2024-09-10T20:38:50.656849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20169","reported_seq":20429,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:54.970359+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:38:54.970359+0000","last_peered":"2024-09-10T20:38:54.970359+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:38:54.970359+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3346432,"num_objects":817,"num_object_clones":0,"num_object_copies":1634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16996,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2838}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651258+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:38:50.651258+0000","last_peered":"2024-09-10T20:38:50.651258+0000","last_clean":"2024-09-10T20:38:50.651258+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:38:50.651258+0000","last_undegraded":"2024-09-10T20:38:50.651258+0000","last_fullsized":"2024-09-10T20:38:50.651258+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651173+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:38:50.651173+0000","last_peered":"2024-09-10T20:38:50.651173+0000","last_clean":"2024-09-10T20:38:50.651173+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:38:50.651173+0000","last_undegraded":"2024-09-10T20:38:50.651173+0000","last_fullsized":"2024-09-10T20:38:50.651173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'19599","reported_seq":26731,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:51.819892+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:38:51.819892+0000","last_peered":"2024-09-10T20:38:51.819892+0000","last_clean":"2024-09-10T20:38:51.819892+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:38:51.819892+0000","last_undegraded":"2024-09-10T20:38:51.819892+0000","last_fullsized":"2024-09-10T20:38:51.819892+0000","mapping_epoch":46,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3756515,"num_objects":918,"num_object_clones":0,"num_object_copies":1836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16206,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":53806529,"num_objects":13140,"num_object_clones":0,"num_object_copies":26280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28355,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13140,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282277,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14794,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":286474240,"data_stored":286415514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160781,"ondisk_log_size":160781,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182524,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":540184,"kb_used_data":97608,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93831656,"statfs":{"total":96636764160,"available":96083615744,"internally_reserved":0,"allocated":99950592,"data_stored":99678939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,5,1,3,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986296,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":498704,"kb_used_data":142136,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93873136,"statfs":{"total":96636764160,"available":96126091264,"internally_reserved":0,"allocated":145547264,"data_stored":145275616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365096618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,4,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659242,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490420,"kb_used_data":43616,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93881420,"statfs":{"total":96636764160,"available":96134574080,"internally_reserved":0,"allocated":44662784,"data_stored":44347316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":457502382},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,4,0,7],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":98111488,"data_stored":98107875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":44048384,"data_stored":44000956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":144314368,"data_stored":144306683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:38:55.645 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:38:55.646 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'19856", 'reported_seq': 20083, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:54.887732+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:38:54.887732+0000', 'last_peered': '2024-09-10T20:38:54.887732+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:38:54.887732+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3284992, 'num_objects': 802, 'num_object_clones': 0, 'num_object_copies': 1604, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 802, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19856, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2739}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:55.646 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:38:55.646 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'19709", 'reported_seq': 19964, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:54.963029+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:38:54.963029+0000', 'last_peered': '2024-09-10T20:38:54.963029+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:38:54.963029+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3121152, 'num_objects': 762, 'num_object_clones': 0, 'num_object_copies': 1524, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 762, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16573, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2799}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:55.646 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:38:55.646 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19835", 'reported_seq': 20125, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:54.963599+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:38:54.963599+0000', 'last_peered': '2024-09-10T20:38:54.963599+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:38:54.963599+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3166208, 'num_objects': 773, 'num_object_clones': 0, 'num_object_copies': 1546, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 773, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16430, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:55.646 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:38:55.646 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'19620", 'reported_seq': 20150, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:54.695860+0000', 'last_change': '2024-09-10T20:38:49.349021+0000', 'last_active': '2024-09-10T20:38:54.695860+0000', 'last_peered': '2024-09-10T20:38:54.695860+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:38:54.695860+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3354624, 'num_objects': 819, 'num_object_clones': 0, 'num_object_copies': 1638, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3337, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 819, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17613, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 191, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2518}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:38:55.646 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:38:55.646 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'19917", 'reported_seq': 20138, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:54.772612+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:38:54.772612+0000', 'last_peered': '2024-09-10T20:38:54.772612+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:38:54.772612+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3331014, 'num_objects': 815, 'num_object_clones': 0, 'num_object_copies': 1630, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 815, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19915, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2751}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:55.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:38:55.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "51'19761", 'reported_seq': 19958, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:54.653518+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:38:54.653518+0000', 'last_peered': '2024-09-10T20:38:54.653518+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:38:54.653518+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3293184, 'num_objects': 804, 'num_object_clones': 0, 'num_object_copies': 1608, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 804, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17589, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2802}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:38:55.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:38:55.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'20013", 'reported_seq': 20275, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:54.863951+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:38:54.863951+0000', 'last_peered': '2024-09-10T20:38:54.863951+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:38:54.863951+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3346432, 'num_objects': 817, 'num_object_clones': 0, 'num_object_copies': 1634, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 817, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16736, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2775}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:55.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:38:55.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "51'20169", 'reported_seq': 20429, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:54.970359+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:38:54.970359+0000', 'last_peered': '2024-09-10T20:38:54.970359+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:38:54.970359+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3346432, 'num_objects': 817, 'num_object_clones': 0, 'num_object_copies': 1634, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 817, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16996, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2838}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:38:55.647 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:38:56.338 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:56.338 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 28355/26284 objects degraded (107.879%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":647,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":13142,"data_bytes":53806548,"bytes_used":1566011392,"bytes_avail":288344281088,"bytes_total":289910292480,"degraded_objects":28355,"degraded_total":26284,"degraded_ratio":1.0787931821640542,"recovering_objects_per_sec":38,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":311,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":558},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:38:56.356 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:38:56.356 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:38:56.913 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:38:56.913 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:38:56.935 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2024-09-10T20:38:55.042106+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":53806548,"num_objects":13142,"num_object_clones":0,"num_object_copies":26284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28355,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13142,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282279,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14796,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160784,"ondisk_log_size":160784,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1529308,"kb_used_data":283360,"kb_used_omap":12,"kb_used_meta":1245875,"kb_avail":281586212,"statfs":{"total":289910292480,"available":288344281088,"internally_reserved":0,"allocated":290160640,"data_stored":289301871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1275776002},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,5,5,7,9,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18296832,"num_objects":-4467,"num_object_clones":0,"num_object_copies":-8934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-311,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4467,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003617"},"pg_stats":[{"pgid":"2.f","version":"51'19856","reported_seq":20083,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:54.887732+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:38:54.887732+0000","last_peered":"2024-09-10T20:38:54.887732+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:38:54.887732+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3284992,"num_objects":802,"num_object_clones":0,"num_object_copies":1604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19856,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2739}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20162","reported_seq":20287,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:54.643685+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:38:54.643685+0000","last_peered":"2024-09-10T20:38:54.643685+0000","last_clean":"2024-09-10T20:38:54.643685+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:38:54.643685+0000","last_undegraded":"2024-09-10T20:38:54.643685+0000","last_fullsized":"2024-09-10T20:38:54.643685+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3194880,"num_objects":780,"num_object_clones":0,"num_object_copies":1560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18348,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19819","reported_seq":27181,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:51.997197+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:38:51.997197+0000","last_peered":"2024-09-10T20:38:51.997197+0000","last_clean":"2024-09-10T20:38:51.997197+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:38:51.997197+0000","last_undegraded":"2024-09-10T20:38:51.997197+0000","last_fullsized":"2024-09-10T20:38:51.997197+0000","mapping_epoch":44,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3670040,"num_objects":897,"num_object_clones":0,"num_object_copies":1794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":897,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17947,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'19761","reported_seq":19856,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:54.612259+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:38:54.612259+0000","last_peered":"2024-09-10T20:38:54.612259+0000","last_clean":"2024-09-10T20:38:54.612259+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:38:54.612259+0000","last_undegraded":"2024-09-10T20:38:54.612259+0000","last_fullsized":"2024-09-10T20:38:54.612259+0000","mapping_epoch":35,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3452928,"num_objects":843,"num_object_clones":0,"num_object_copies":1686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19761,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20246","reported_seq":20387,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:54.868314+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:38:54.868314+0000","last_peered":"2024-09-10T20:38:54.868314+0000","last_clean":"2024-09-10T20:38:54.868314+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:38:54.868314+0000","last_undegraded":"2024-09-10T20:38:54.868314+0000","last_fullsized":"2024-09-10T20:38:54.868314+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3371008,"num_objects":823,"num_object_clones":0,"num_object_copies":1646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17242,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'19709","reported_seq":19964,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:54.963029+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:38:54.963029+0000","last_peered":"2024-09-10T20:38:54.963029+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:38:54.963029+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3121152,"num_objects":762,"num_object_clones":0,"num_object_copies":1524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16573,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2799}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19835","reported_seq":20125,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:54.963599+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:38:54.963599+0000","last_peered":"2024-09-10T20:38:54.963599+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:38:54.963599+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3166208,"num_objects":773,"num_object_clones":0,"num_object_copies":1546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16430,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'19968","reported_seq":20107,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:54.976967+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:38:54.976967+0000","last_peered":"2024-09-10T20:38:54.976967+0000","last_clean":"2024-09-10T20:38:54.976967+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:38:54.976967+0000","last_undegraded":"2024-09-10T20:38:54.976967+0000","last_fullsized":"2024-09-10T20:38:54.976967+0000","mapping_epoch":34,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":768,"num_object_clones":0,"num_object_copies":1536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.652747+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:38:50.652747+0000","last_peered":"2024-09-10T20:38:50.652747+0000","last_clean":"2024-09-10T20:38:50.652747+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:38:50.652747+0000","last_undegraded":"2024-09-10T20:38:50.652747+0000","last_fullsized":"2024-09-10T20:38:50.652747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'19620","reported_seq":20150,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:54.695860+0000","last_change":"2024-09-10T20:38:49.349021+0000","last_active":"2024-09-10T20:38:54.695860+0000","last_peered":"2024-09-10T20:38:54.695860+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:38:54.695860+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3354624,"num_objects":819,"num_object_clones":0,"num_object_copies":1638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3337,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17613,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":191,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2518}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.654924+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:38:50.654924+0000","last_peered":"2024-09-10T20:38:50.654924+0000","last_clean":"2024-09-10T20:38:50.654924+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:38:50.654924+0000","last_undegraded":"2024-09-10T20:38:50.654924+0000","last_fullsized":"2024-09-10T20:38:50.654924+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'19917","reported_seq":20138,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:54.772612+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:38:54.772612+0000","last_peered":"2024-09-10T20:38:54.772612+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:38:54.772612+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3331014,"num_objects":815,"num_object_clones":0,"num_object_copies":1630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19915,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2751}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.648348+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:38:50.648348+0000","last_peered":"2024-09-10T20:38:50.648348+0000","last_clean":"2024-09-10T20:38:50.648348+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:38:50.648348+0000","last_undegraded":"2024-09-10T20:38:50.648348+0000","last_fullsized":"2024-09-10T20:38:50.648348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19761","reported_seq":19958,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:54.653518+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:38:54.653518+0000","last_peered":"2024-09-10T20:38:54.653518+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:38:54.653518+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3293184,"num_objects":804,"num_object_clones":0,"num_object_copies":1608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17589,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2802}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651037+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:38:50.651037+0000","last_peered":"2024-09-10T20:38:50.651037+0000","last_clean":"2024-09-10T20:38:50.651037+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:38:50.651037+0000","last_undegraded":"2024-09-10T20:38:50.651037+0000","last_fullsized":"2024-09-10T20:38:50.651037+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19854","reported_seq":27229,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:54.626431+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:38:54.626431+0000","last_peered":"2024-09-10T20:38:54.626431+0000","last_clean":"2024-09-10T20:38:54.626431+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:38:54.626431+0000","last_undegraded":"2024-09-10T20:38:54.626431+0000","last_fullsized":"2024-09-10T20:38:54.626431+0000","mapping_epoch":48,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3317760,"num_objects":810,"num_object_clones":0,"num_object_copies":1620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20013","reported_seq":20275,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:54.863951+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:38:54.863951+0000","last_peered":"2024-09-10T20:38:54.863951+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:38:54.863951+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3346432,"num_objects":817,"num_object_clones":0,"num_object_copies":1634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16736,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2775}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651168+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:38:50.651168+0000","last_peered":"2024-09-10T20:38:50.651168+0000","last_clean":"2024-09-10T20:38:50.651168+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:38:50.651168+0000","last_undegraded":"2024-09-10T20:38:50.651168+0000","last_fullsized":"2024-09-10T20:38:50.651168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'19692","reported_seq":26925,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:51.965526+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:38:51.965526+0000","last_peered":"2024-09-10T20:38:51.965526+0000","last_clean":"2024-09-10T20:38:51.965526+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:38:51.965526+0000","last_undegraded":"2024-09-10T20:38:51.965526+0000","last_fullsized":"2024-09-10T20:38:51.965526+0000","mapping_epoch":50,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3653632,"num_objects":892,"num_object_clones":0,"num_object_copies":1784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16386,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.656849+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:38:50.656849+0000","last_peered":"2024-09-10T20:38:50.656849+0000","last_clean":"2024-09-10T20:38:50.656849+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:38:50.656849+0000","last_undegraded":"2024-09-10T20:38:50.656849+0000","last_fullsized":"2024-09-10T20:38:50.656849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20169","reported_seq":20429,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:54.970359+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:38:54.970359+0000","last_peered":"2024-09-10T20:38:54.970359+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:38:54.970359+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3346432,"num_objects":817,"num_object_clones":0,"num_object_copies":1634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16996,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2838}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651258+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:38:50.651258+0000","last_peered":"2024-09-10T20:38:50.651258+0000","last_clean":"2024-09-10T20:38:50.651258+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:38:50.651258+0000","last_undegraded":"2024-09-10T20:38:50.651258+0000","last_fullsized":"2024-09-10T20:38:50.651258+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651173+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:38:50.651173+0000","last_peered":"2024-09-10T20:38:50.651173+0000","last_clean":"2024-09-10T20:38:50.651173+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:38:50.651173+0000","last_undegraded":"2024-09-10T20:38:50.651173+0000","last_fullsized":"2024-09-10T20:38:50.651173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'19599","reported_seq":26731,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:51.819892+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:38:51.819892+0000","last_peered":"2024-09-10T20:38:51.819892+0000","last_clean":"2024-09-10T20:38:51.819892+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:38:51.819892+0000","last_undegraded":"2024-09-10T20:38:51.819892+0000","last_fullsized":"2024-09-10T20:38:51.819892+0000","mapping_epoch":46,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3756515,"num_objects":918,"num_object_clones":0,"num_object_copies":1836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16206,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":53806529,"num_objects":13140,"num_object_clones":0,"num_object_copies":26280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28355,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13140,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282277,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14794,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":286474240,"data_stored":286415514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160781,"ondisk_log_size":160781,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182524,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":540184,"kb_used_data":97608,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93831656,"statfs":{"total":96636764160,"available":96083615744,"internally_reserved":0,"allocated":99950592,"data_stored":99678939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,5,1,3,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986296,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":498704,"kb_used_data":142136,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93873136,"statfs":{"total":96636764160,"available":96126091264,"internally_reserved":0,"allocated":145547264,"data_stored":145275616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365096618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,4,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659242,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490420,"kb_used_data":43616,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93881420,"statfs":{"total":96636764160,"available":96134574080,"internally_reserved":0,"allocated":44662784,"data_stored":44347316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":457502382},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,4,0,7],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":98111488,"data_stored":98107875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":44048384,"data_stored":44000956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":144314368,"data_stored":144306683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:38:59.937 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:39:00.467 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:00.468 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:39:00.500 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":370,"stamp":"2024-09-10T20:38:59.043666+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":51844564,"num_objects":12663,"num_object_clones":0,"num_object_copies":25326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28355,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12663,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282758,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14796,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160763,"ondisk_log_size":160763,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1543992,"kb_used_data":275828,"kb_used_omap":12,"kb_used_meta":1268083,"kb_avail":281571528,"statfs":{"total":289910292480,"available":288329244672,"internally_reserved":0,"allocated":282447872,"data_stored":281589103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1298516994},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,5,2,9,8,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20258816,"num_objects":-4946,"num_object_clones":0,"num_object_copies":-9892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-311,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4946,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.394412"},"pg_stats":[{"pgid":"2.f","version":"51'19856","reported_seq":20083,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:54.887732+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:38:54.887732+0000","last_peered":"2024-09-10T20:38:54.887732+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:38:54.887732+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3284992,"num_objects":802,"num_object_clones":0,"num_object_copies":1604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19856,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2739}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20162","reported_seq":20287,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:54.643685+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:38:54.643685+0000","last_peered":"2024-09-10T20:38:54.643685+0000","last_clean":"2024-09-10T20:38:54.643685+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:38:54.643685+0000","last_undegraded":"2024-09-10T20:38:54.643685+0000","last_fullsized":"2024-09-10T20:38:54.643685+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3194880,"num_objects":780,"num_object_clones":0,"num_object_copies":1560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18348,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19963","reported_seq":27325,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:56.997927+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:38:56.997927+0000","last_peered":"2024-09-10T20:38:56.997927+0000","last_clean":"2024-09-10T20:38:56.997927+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:38:56.997927+0000","last_undegraded":"2024-09-10T20:38:56.997927+0000","last_fullsized":"2024-09-10T20:38:56.997927+0000","mapping_epoch":44,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3080216,"num_objects":753,"num_object_clones":0,"num_object_copies":1506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":753,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18091,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'19761","reported_seq":19856,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:54.612259+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:38:54.612259+0000","last_peered":"2024-09-10T20:38:54.612259+0000","last_clean":"2024-09-10T20:38:54.612259+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:38:54.612259+0000","last_undegraded":"2024-09-10T20:38:54.612259+0000","last_fullsized":"2024-09-10T20:38:54.612259+0000","mapping_epoch":35,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3452928,"num_objects":843,"num_object_clones":0,"num_object_copies":1686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19761,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20246","reported_seq":20387,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:54.868314+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:38:54.868314+0000","last_peered":"2024-09-10T20:38:54.868314+0000","last_clean":"2024-09-10T20:38:54.868314+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:38:54.868314+0000","last_undegraded":"2024-09-10T20:38:54.868314+0000","last_fullsized":"2024-09-10T20:38:54.868314+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3371008,"num_objects":823,"num_object_clones":0,"num_object_copies":1646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17242,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'19709","reported_seq":19964,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:54.963029+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:38:54.963029+0000","last_peered":"2024-09-10T20:38:54.963029+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:38:54.963029+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3121152,"num_objects":762,"num_object_clones":0,"num_object_copies":1524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16573,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2799}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19835","reported_seq":20125,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:54.963599+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:38:54.963599+0000","last_peered":"2024-09-10T20:38:54.963599+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:38:54.963599+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3166208,"num_objects":773,"num_object_clones":0,"num_object_copies":1546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16430,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'19968","reported_seq":20107,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:54.976967+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:38:54.976967+0000","last_peered":"2024-09-10T20:38:54.976967+0000","last_clean":"2024-09-10T20:38:54.976967+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:38:54.976967+0000","last_undegraded":"2024-09-10T20:38:54.976967+0000","last_fullsized":"2024-09-10T20:38:54.976967+0000","mapping_epoch":34,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":768,"num_object_clones":0,"num_object_copies":1536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.652747+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:38:50.652747+0000","last_peered":"2024-09-10T20:38:50.652747+0000","last_clean":"2024-09-10T20:38:50.652747+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:38:50.652747+0000","last_undegraded":"2024-09-10T20:38:50.652747+0000","last_fullsized":"2024-09-10T20:38:50.652747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'19620","reported_seq":20150,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:54.695860+0000","last_change":"2024-09-10T20:38:49.349021+0000","last_active":"2024-09-10T20:38:54.695860+0000","last_peered":"2024-09-10T20:38:54.695860+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:38:54.695860+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3354624,"num_objects":819,"num_object_clones":0,"num_object_copies":1638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3337,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17613,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":191,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2518}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.654924+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:38:50.654924+0000","last_peered":"2024-09-10T20:38:50.654924+0000","last_clean":"2024-09-10T20:38:50.654924+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:38:50.654924+0000","last_undegraded":"2024-09-10T20:38:50.654924+0000","last_fullsized":"2024-09-10T20:38:50.654924+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'19917","reported_seq":20138,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:54.772612+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:38:54.772612+0000","last_peered":"2024-09-10T20:38:54.772612+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:38:54.772612+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3331014,"num_objects":815,"num_object_clones":0,"num_object_copies":1630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19915,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2751}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.648348+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:38:50.648348+0000","last_peered":"2024-09-10T20:38:50.648348+0000","last_clean":"2024-09-10T20:38:50.648348+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:38:50.648348+0000","last_undegraded":"2024-09-10T20:38:50.648348+0000","last_fullsized":"2024-09-10T20:38:50.648348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19761","reported_seq":19958,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:54.653518+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:38:54.653518+0000","last_peered":"2024-09-10T20:38:54.653518+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:38:54.653518+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3293184,"num_objects":804,"num_object_clones":0,"num_object_copies":1608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17589,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2802}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651037+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:38:50.651037+0000","last_peered":"2024-09-10T20:38:50.651037+0000","last_clean":"2024-09-10T20:38:50.651037+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:38:50.651037+0000","last_undegraded":"2024-09-10T20:38:50.651037+0000","last_fullsized":"2024-09-10T20:38:50.651037+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19854","reported_seq":27229,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:54.626431+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:38:54.626431+0000","last_peered":"2024-09-10T20:38:54.626431+0000","last_clean":"2024-09-10T20:38:54.626431+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:38:54.626431+0000","last_undegraded":"2024-09-10T20:38:54.626431+0000","last_fullsized":"2024-09-10T20:38:54.626431+0000","mapping_epoch":48,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3317760,"num_objects":810,"num_object_clones":0,"num_object_copies":1620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20013","reported_seq":20275,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:54.863951+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:38:54.863951+0000","last_peered":"2024-09-10T20:38:54.863951+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:38:54.863951+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3346432,"num_objects":817,"num_object_clones":0,"num_object_copies":1634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16736,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2775}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651168+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:38:50.651168+0000","last_peered":"2024-09-10T20:38:50.651168+0000","last_clean":"2024-09-10T20:38:50.651168+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:38:50.651168+0000","last_undegraded":"2024-09-10T20:38:50.651168+0000","last_fullsized":"2024-09-10T20:38:50.651168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'19869","reported_seq":27103,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:56.960256+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:38:56.960256+0000","last_peered":"2024-09-10T20:38:56.960256+0000","last_clean":"2024-09-10T20:38:56.960256+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:38:56.960256+0000","last_undegraded":"2024-09-10T20:38:56.960256+0000","last_fullsized":"2024-09-10T20:38:56.960256+0000","mapping_epoch":50,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2928640,"num_objects":715,"num_object_clones":0,"num_object_copies":1430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16563,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.656849+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:38:50.656849+0000","last_peered":"2024-09-10T20:38:50.656849+0000","last_clean":"2024-09-10T20:38:50.656849+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:38:50.656849+0000","last_undegraded":"2024-09-10T20:38:50.656849+0000","last_fullsized":"2024-09-10T20:38:50.656849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20169","reported_seq":20429,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:54.970359+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:38:54.970359+0000","last_peered":"2024-09-10T20:38:54.970359+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:38:54.970359+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3346432,"num_objects":817,"num_object_clones":0,"num_object_copies":1634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16996,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2838}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651258+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:38:50.651258+0000","last_peered":"2024-09-10T20:38:50.651258+0000","last_clean":"2024-09-10T20:38:50.651258+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:38:50.651258+0000","last_undegraded":"2024-09-10T20:38:50.651258+0000","last_fullsized":"2024-09-10T20:38:50.651258+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651173+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:38:50.651173+0000","last_peered":"2024-09-10T20:38:50.651173+0000","last_clean":"2024-09-10T20:38:50.651173+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:38:50.651173+0000","last_undegraded":"2024-09-10T20:38:50.651173+0000","last_fullsized":"2024-09-10T20:38:50.651173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'19757","reported_seq":26889,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:56.899505+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:38:56.899505+0000","last_peered":"2024-09-10T20:38:56.899505+0000","last_clean":"2024-09-10T20:38:56.899505+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:38:56.899505+0000","last_undegraded":"2024-09-10T20:38:56.899505+0000","last_fullsized":"2024-09-10T20:38:56.899505+0000","mapping_epoch":46,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3109347,"num_objects":760,"num_object_clones":0,"num_object_copies":1520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16364,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":51844545,"num_objects":12661,"num_object_clones":0,"num_object_copies":25322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28355,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12661,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282756,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14794,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278822912,"data_stored":278764186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160760,"ondisk_log_size":160760,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182524,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":540184,"kb_used_data":97608,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93831656,"statfs":{"total":96636764160,"available":96083615744,"internally_reserved":0,"allocated":99950592,"data_stored":99678939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,5,1,3,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986296,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":498704,"kb_used_data":142136,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93873136,"statfs":{"total":96636764160,"available":96126091264,"internally_reserved":0,"allocated":145547264,"data_stored":145275616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365096618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,4,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659243,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505104,"kb_used_data":36084,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93866736,"statfs":{"total":96636764160,"available":96119537664,"internally_reserved":0,"allocated":36950016,"data_stored":36634548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480243374},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,6,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":98111488,"data_stored":98107875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36397056,"data_stored":36349628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":144314368,"data_stored":144306683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:39:00.501 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:39:01.117 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:01.117 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:39:01.151 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":371,"stamp":"2024-09-10T20:39:01.044323+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":43693524,"num_objects":10673,"num_object_clones":0,"num_object_copies":21346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28242,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10673,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":284748,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14909,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160753,"ondisk_log_size":160753,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1544420,"kb_used_data":263780,"kb_used_omap":12,"kb_used_meta":1280563,"kb_avail":281571100,"statfs":{"total":289910292480,"available":288328806400,"internally_reserved":0,"allocated":270110720,"data_stored":269251951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1311296514},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,6,6,11,12,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19992576,"num_objects":-4881,"num_object_clones":0,"num_object_copies":-9762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-419,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4881,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":419,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.397592"},"pg_stats":[{"pgid":"2.f","version":"51'20021","reported_seq":20248,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:59.934770+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:38:59.934770+0000","last_peered":"2024-09-10T20:38:59.934770+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:38:59.934770+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2609152,"num_objects":637,"num_object_clones":0,"num_object_copies":1274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20021,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2904}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20321","reported_seq":20446,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:59.678908+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:38:59.678908+0000","last_peered":"2024-09-10T20:38:59.678908+0000","last_clean":"2024-09-10T20:38:59.678908+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:38:59.678908+0000","last_undegraded":"2024-09-10T20:38:59.678908+0000","last_fullsized":"2024-09-10T20:38:59.678908+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2543616,"num_objects":621,"num_object_clones":0,"num_object_copies":1242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18507,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19963","reported_seq":27325,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:56.997927+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:38:56.997927+0000","last_peered":"2024-09-10T20:38:56.997927+0000","last_clean":"2024-09-10T20:38:56.997927+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:38:56.997927+0000","last_undegraded":"2024-09-10T20:38:56.997927+0000","last_fullsized":"2024-09-10T20:38:56.997927+0000","mapping_epoch":44,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3080216,"num_objects":753,"num_object_clones":0,"num_object_copies":1506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":753,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18091,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'19915","reported_seq":20010,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:59.650806+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:38:59.650806+0000","last_peered":"2024-09-10T20:38:59.650806+0000","last_clean":"2024-09-10T20:38:59.650806+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:38:59.650806+0000","last_undegraded":"2024-09-10T20:38:59.650806+0000","last_fullsized":"2024-09-10T20:38:59.650806+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2822144,"num_objects":689,"num_object_clones":0,"num_object_copies":1378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19915,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20381","reported_seq":20522,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:00.013805+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:00.013805+0000","last_peered":"2024-09-10T20:39:00.013805+0000","last_clean":"2024-09-10T20:39:00.013805+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:00.013805+0000","last_undegraded":"2024-09-10T20:39:00.013805+0000","last_fullsized":"2024-09-10T20:39:00.013805+0000","mapping_epoch":34,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":688,"num_object_clones":0,"num_object_copies":1376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'19867","reported_seq":20122,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:00.012089+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:00.012089+0000","last_peered":"2024-09-10T20:39:00.012089+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:00.012089+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2473984,"num_objects":604,"num_object_clones":0,"num_object_copies":1208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16731,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2957}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19959","reported_seq":20250,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:59.957264+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:38:59.957264+0000","last_peered":"2024-09-10T20:38:59.957264+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:38:59.957264+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2658304,"num_objects":649,"num_object_clones":0,"num_object_copies":1298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16554,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2837}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20109","reported_seq":20248,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:59.899188+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:38:59.899188+0000","last_peered":"2024-09-10T20:38:59.899188+0000","last_clean":"2024-09-10T20:38:59.899188+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:38:59.899188+0000","last_undegraded":"2024-09-10T20:38:59.899188+0000","last_fullsized":"2024-09-10T20:38:59.899188+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2568192,"num_objects":627,"num_object_clones":0,"num_object_copies":1254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17180,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.652747+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:38:50.652747+0000","last_peered":"2024-09-10T20:38:50.652747+0000","last_clean":"2024-09-10T20:38:50.652747+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:38:50.652747+0000","last_undegraded":"2024-09-10T20:38:50.652747+0000","last_fullsized":"2024-09-10T20:38:50.652747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'19775","reported_seq":20531,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:59.677620+0000","last_change":"2024-09-10T20:38:49.349021+0000","last_active":"2024-09-10T20:38:59.677620+0000","last_peered":"2024-09-10T20:38:59.677620+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:38:59.677620+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2719744,"num_objects":664,"num_object_clones":0,"num_object_copies":1328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3224,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17768,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2560}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.654924+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:38:50.654924+0000","last_peered":"2024-09-10T20:38:50.654924+0000","last_clean":"2024-09-10T20:38:50.654924+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:38:50.654924+0000","last_undegraded":"2024-09-10T20:38:50.654924+0000","last_fullsized":"2024-09-10T20:38:50.654924+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20088","reported_seq":20306,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:00.019026+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:00.019026+0000","last_peered":"2024-09-10T20:39:00.019026+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:00.019026+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2630598,"num_objects":644,"num_object_clones":0,"num_object_copies":1288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20086,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2922}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.648348+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:38:50.648348+0000","last_peered":"2024-09-10T20:38:50.648348+0000","last_clean":"2024-09-10T20:38:50.648348+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:38:50.648348+0000","last_undegraded":"2024-09-10T20:38:50.648348+0000","last_fullsized":"2024-09-10T20:38:50.648348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19931","reported_seq":20128,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:59.637912+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:38:59.637912+0000","last_peered":"2024-09-10T20:38:59.637912+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:38:59.637912+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2596864,"num_objects":634,"num_object_clones":0,"num_object_copies":1268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17759,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2972}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651037+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:38:50.651037+0000","last_peered":"2024-09-10T20:38:50.651037+0000","last_clean":"2024-09-10T20:38:50.651037+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:38:50.651037+0000","last_undegraded":"2024-09-10T20:38:50.651037+0000","last_fullsized":"2024-09-10T20:38:50.651037+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20009","reported_seq":27384,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:59.656879+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:38:59.656879+0000","last_peered":"2024-09-10T20:38:59.656879+0000","last_clean":"2024-09-10T20:38:59.656879+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:38:59.656879+0000","last_undegraded":"2024-09-10T20:38:59.656879+0000","last_fullsized":"2024-09-10T20:38:59.656879+0000","mapping_epoch":48,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2682880,"num_objects":655,"num_object_clones":0,"num_object_copies":1310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17795,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20156","reported_seq":20418,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:59.993363+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:38:59.993363+0000","last_peered":"2024-09-10T20:38:59.993363+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:38:59.993363+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2760704,"num_objects":674,"num_object_clones":0,"num_object_copies":1348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16879,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2918}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651168+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:38:50.651168+0000","last_peered":"2024-09-10T20:38:50.651168+0000","last_clean":"2024-09-10T20:38:50.651168+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:38:50.651168+0000","last_undegraded":"2024-09-10T20:38:50.651168+0000","last_fullsized":"2024-09-10T20:38:50.651168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'19869","reported_seq":27103,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:56.960256+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:38:56.960256+0000","last_peered":"2024-09-10T20:38:56.960256+0000","last_clean":"2024-09-10T20:38:56.960256+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:38:56.960256+0000","last_undegraded":"2024-09-10T20:38:56.960256+0000","last_fullsized":"2024-09-10T20:38:56.960256+0000","mapping_epoch":50,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2928640,"num_objects":715,"num_object_clones":0,"num_object_copies":1430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16563,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.656849+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:38:50.656849+0000","last_peered":"2024-09-10T20:38:50.656849+0000","last_clean":"2024-09-10T20:38:50.656849+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:38:50.656849+0000","last_undegraded":"2024-09-10T20:38:50.656849+0000","last_fullsized":"2024-09-10T20:38:50.656849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20329","reported_seq":20590,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:59.974359+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:38:59.974359+0000","last_peered":"2024-09-10T20:38:59.974359+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:38:59.974359+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2691072,"num_objects":657,"num_object_clones":0,"num_object_copies":1314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17156,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2998}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651258+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:38:50.651258+0000","last_peered":"2024-09-10T20:38:50.651258+0000","last_clean":"2024-09-10T20:38:50.651258+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:38:50.651258+0000","last_undegraded":"2024-09-10T20:38:50.651258+0000","last_fullsized":"2024-09-10T20:38:50.651258+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651173+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:38:50.651173+0000","last_peered":"2024-09-10T20:38:50.651173+0000","last_clean":"2024-09-10T20:38:50.651173+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:38:50.651173+0000","last_undegraded":"2024-09-10T20:38:50.651173+0000","last_fullsized":"2024-09-10T20:38:50.651173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'19757","reported_seq":26889,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:56.899505+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:38:56.899505+0000","last_peered":"2024-09-10T20:38:56.899505+0000","last_clean":"2024-09-10T20:38:56.899505+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:38:56.899505+0000","last_undegraded":"2024-09-10T20:38:56.899505+0000","last_fullsized":"2024-09-10T20:38:56.899505+0000","mapping_epoch":46,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3109347,"num_objects":760,"num_object_clones":0,"num_object_copies":1520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16364,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":43693505,"num_objects":10671,"num_object_clones":0,"num_object_copies":21342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28242,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10671,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":284746,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14907,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":266280960,"data_stored":266222234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160750,"ondisk_log_size":160750,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182525,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":541312,"kb_used_data":90412,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93830528,"statfs":{"total":96636764160,"available":96082460672,"internally_reserved":0,"allocated":92581888,"data_stored":92310235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":461696682},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,4,1,3,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986297,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":498004,"kb_used_data":137284,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93873836,"statfs":{"total":96636764160,"available":96126808064,"internally_reserved":0,"allocated":140578816,"data_stored":140307168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369356458},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659243,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505104,"kb_used_data":36084,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93866736,"statfs":{"total":96636764160,"available":96119537664,"internally_reserved":0,"allocated":36950016,"data_stored":36634548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480243374},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,6,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":90583040,"data_stored":90579427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36397056,"data_stored":36349628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":139300864,"data_stored":139293179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:39:01.153 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:39:01.153 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'20021", 'reported_seq': 20248, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:59.934770+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:38:59.934770+0000', 'last_peered': '2024-09-10T20:38:59.934770+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:38:59.934770+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2609152, 'num_objects': 637, 'num_object_clones': 0, 'num_object_copies': 1274, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 637, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20021, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2904}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:01.154 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:39:01.154 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'19867", 'reported_seq': 20122, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:00.012089+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:39:00.012089+0000', 'last_peered': '2024-09-10T20:39:00.012089+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:39:00.012089+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2473984, 'num_objects': 604, 'num_object_clones': 0, 'num_object_copies': 1208, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 604, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16731, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2957}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:01.154 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:39:01.154 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19959", 'reported_seq': 20250, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:59.957264+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:38:59.957264+0000', 'last_peered': '2024-09-10T20:38:59.957264+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:38:59.957264+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2658304, 'num_objects': 649, 'num_object_clones': 0, 'num_object_copies': 1298, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 649, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16554, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2837}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:01.154 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:39:01.155 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'19775", 'reported_seq': 20531, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:59.677620+0000', 'last_change': '2024-09-10T20:38:49.349021+0000', 'last_active': '2024-09-10T20:38:59.677620+0000', 'last_peered': '2024-09-10T20:38:59.677620+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:38:59.677620+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2719744, 'num_objects': 664, 'num_object_clones': 0, 'num_object_copies': 1328, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3224, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 664, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17768, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 304, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2560}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:39:01.155 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:39:01.155 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20088", 'reported_seq': 20306, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:00.019026+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:39:00.019026+0000', 'last_peered': '2024-09-10T20:39:00.019026+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:39:00.019026+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2630598, 'num_objects': 644, 'num_object_clones': 0, 'num_object_copies': 1288, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 644, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20086, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2922}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:01.155 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:39:01.156 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "51'19931", 'reported_seq': 20128, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:59.637912+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:38:59.637912+0000', 'last_peered': '2024-09-10T20:38:59.637912+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:38:59.637912+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2596864, 'num_objects': 634, 'num_object_clones': 0, 'num_object_copies': 1268, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 634, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17759, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2972}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:39:01.156 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:39:01.156 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'20156", 'reported_seq': 20418, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:59.993363+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:38:59.993363+0000', 'last_peered': '2024-09-10T20:38:59.993363+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:38:59.993363+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2760704, 'num_objects': 674, 'num_object_clones': 0, 'num_object_copies': 1348, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 674, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16879, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2918}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:01.157 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:39:01.157 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "51'20329", 'reported_seq': 20590, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:59.974359+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:38:59.974359+0000', 'last_peered': '2024-09-10T20:38:59.974359+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:38:59.974359+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'10300", 'ondisk_log_start': "45'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2691072, 'num_objects': 657, 'num_object_clones': 0, 'num_object_copies': 1314, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 657, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17156, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2998}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:01.158 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:39:01.849 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:01.850 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 28242/21346 objects degraded (132.306%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":653,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":10673,"data_bytes":43693524,"bytes_used":1581486080,"bytes_avail":288328806400,"bytes_total":289910292480,"degraded_objects":28242,"degraded_total":21346,"degraded_ratio":1.3230581842031297,"recovering_objects_per_sec":40,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":419,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":469},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:39:01.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:39:01.874 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:39:02.419 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:02.420 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:39:02.453 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":371,"stamp":"2024-09-10T20:39:01.044323+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":43693524,"num_objects":10673,"num_object_clones":0,"num_object_copies":21346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28242,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10673,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":284748,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14909,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160753,"ondisk_log_size":160753,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1544420,"kb_used_data":263780,"kb_used_omap":12,"kb_used_meta":1280563,"kb_avail":281571100,"statfs":{"total":289910292480,"available":288328806400,"internally_reserved":0,"allocated":270110720,"data_stored":269251951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1311296514},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,6,6,11,12,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19992576,"num_objects":-4881,"num_object_clones":0,"num_object_copies":-9762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-419,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4881,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":419,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.397592"},"pg_stats":[{"pgid":"2.f","version":"51'20021","reported_seq":20248,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:59.934770+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:38:59.934770+0000","last_peered":"2024-09-10T20:38:59.934770+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:38:59.934770+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2609152,"num_objects":637,"num_object_clones":0,"num_object_copies":1274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20021,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2904}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20321","reported_seq":20446,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:59.678908+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:38:59.678908+0000","last_peered":"2024-09-10T20:38:59.678908+0000","last_clean":"2024-09-10T20:38:59.678908+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:38:59.678908+0000","last_undegraded":"2024-09-10T20:38:59.678908+0000","last_fullsized":"2024-09-10T20:38:59.678908+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2543616,"num_objects":621,"num_object_clones":0,"num_object_copies":1242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18507,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19963","reported_seq":27325,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:56.997927+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:38:56.997927+0000","last_peered":"2024-09-10T20:38:56.997927+0000","last_clean":"2024-09-10T20:38:56.997927+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:38:56.997927+0000","last_undegraded":"2024-09-10T20:38:56.997927+0000","last_fullsized":"2024-09-10T20:38:56.997927+0000","mapping_epoch":44,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3080216,"num_objects":753,"num_object_clones":0,"num_object_copies":1506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":753,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18091,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'19915","reported_seq":20010,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:59.650806+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:38:59.650806+0000","last_peered":"2024-09-10T20:38:59.650806+0000","last_clean":"2024-09-10T20:38:59.650806+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:38:59.650806+0000","last_undegraded":"2024-09-10T20:38:59.650806+0000","last_fullsized":"2024-09-10T20:38:59.650806+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2822144,"num_objects":689,"num_object_clones":0,"num_object_copies":1378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19915,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20381","reported_seq":20522,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:00.013805+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:00.013805+0000","last_peered":"2024-09-10T20:39:00.013805+0000","last_clean":"2024-09-10T20:39:00.013805+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:00.013805+0000","last_undegraded":"2024-09-10T20:39:00.013805+0000","last_fullsized":"2024-09-10T20:39:00.013805+0000","mapping_epoch":34,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":688,"num_object_clones":0,"num_object_copies":1376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'19867","reported_seq":20122,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:00.012089+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:00.012089+0000","last_peered":"2024-09-10T20:39:00.012089+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:00.012089+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2473984,"num_objects":604,"num_object_clones":0,"num_object_copies":1208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16731,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2957}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19959","reported_seq":20250,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:59.957264+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:38:59.957264+0000","last_peered":"2024-09-10T20:38:59.957264+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:38:59.957264+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2658304,"num_objects":649,"num_object_clones":0,"num_object_copies":1298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16554,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2837}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20109","reported_seq":20248,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:59.899188+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:38:59.899188+0000","last_peered":"2024-09-10T20:38:59.899188+0000","last_clean":"2024-09-10T20:38:59.899188+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:38:59.899188+0000","last_undegraded":"2024-09-10T20:38:59.899188+0000","last_fullsized":"2024-09-10T20:38:59.899188+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2568192,"num_objects":627,"num_object_clones":0,"num_object_copies":1254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17180,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.652747+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:38:50.652747+0000","last_peered":"2024-09-10T20:38:50.652747+0000","last_clean":"2024-09-10T20:38:50.652747+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:38:50.652747+0000","last_undegraded":"2024-09-10T20:38:50.652747+0000","last_fullsized":"2024-09-10T20:38:50.652747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'19775","reported_seq":20531,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:59.677620+0000","last_change":"2024-09-10T20:38:49.349021+0000","last_active":"2024-09-10T20:38:59.677620+0000","last_peered":"2024-09-10T20:38:59.677620+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:38:59.677620+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2719744,"num_objects":664,"num_object_clones":0,"num_object_copies":1328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3224,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17768,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2560}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.654924+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:38:50.654924+0000","last_peered":"2024-09-10T20:38:50.654924+0000","last_clean":"2024-09-10T20:38:50.654924+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:38:50.654924+0000","last_undegraded":"2024-09-10T20:38:50.654924+0000","last_fullsized":"2024-09-10T20:38:50.654924+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20088","reported_seq":20306,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:00.019026+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:00.019026+0000","last_peered":"2024-09-10T20:39:00.019026+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:00.019026+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2630598,"num_objects":644,"num_object_clones":0,"num_object_copies":1288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20086,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2922}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.648348+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:38:50.648348+0000","last_peered":"2024-09-10T20:38:50.648348+0000","last_clean":"2024-09-10T20:38:50.648348+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:38:50.648348+0000","last_undegraded":"2024-09-10T20:38:50.648348+0000","last_fullsized":"2024-09-10T20:38:50.648348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19931","reported_seq":20128,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:59.637912+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:38:59.637912+0000","last_peered":"2024-09-10T20:38:59.637912+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:38:59.637912+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2596864,"num_objects":634,"num_object_clones":0,"num_object_copies":1268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17759,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2972}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651037+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:38:50.651037+0000","last_peered":"2024-09-10T20:38:50.651037+0000","last_clean":"2024-09-10T20:38:50.651037+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:38:50.651037+0000","last_undegraded":"2024-09-10T20:38:50.651037+0000","last_fullsized":"2024-09-10T20:38:50.651037+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20009","reported_seq":27384,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:59.656879+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:38:59.656879+0000","last_peered":"2024-09-10T20:38:59.656879+0000","last_clean":"2024-09-10T20:38:59.656879+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:38:59.656879+0000","last_undegraded":"2024-09-10T20:38:59.656879+0000","last_fullsized":"2024-09-10T20:38:59.656879+0000","mapping_epoch":48,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2682880,"num_objects":655,"num_object_clones":0,"num_object_copies":1310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17795,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20156","reported_seq":20418,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:59.993363+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:38:59.993363+0000","last_peered":"2024-09-10T20:38:59.993363+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:38:59.993363+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2760704,"num_objects":674,"num_object_clones":0,"num_object_copies":1348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16879,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2918}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651168+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:38:50.651168+0000","last_peered":"2024-09-10T20:38:50.651168+0000","last_clean":"2024-09-10T20:38:50.651168+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:38:50.651168+0000","last_undegraded":"2024-09-10T20:38:50.651168+0000","last_fullsized":"2024-09-10T20:38:50.651168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'19869","reported_seq":27103,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:56.960256+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:38:56.960256+0000","last_peered":"2024-09-10T20:38:56.960256+0000","last_clean":"2024-09-10T20:38:56.960256+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:38:56.960256+0000","last_undegraded":"2024-09-10T20:38:56.960256+0000","last_fullsized":"2024-09-10T20:38:56.960256+0000","mapping_epoch":50,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2928640,"num_objects":715,"num_object_clones":0,"num_object_copies":1430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16563,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.656849+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:38:50.656849+0000","last_peered":"2024-09-10T20:38:50.656849+0000","last_clean":"2024-09-10T20:38:50.656849+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:38:50.656849+0000","last_undegraded":"2024-09-10T20:38:50.656849+0000","last_fullsized":"2024-09-10T20:38:50.656849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20329","reported_seq":20590,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:59.974359+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:38:59.974359+0000","last_peered":"2024-09-10T20:38:59.974359+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:38:59.974359+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2691072,"num_objects":657,"num_object_clones":0,"num_object_copies":1314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17156,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2998}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651258+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:38:50.651258+0000","last_peered":"2024-09-10T20:38:50.651258+0000","last_clean":"2024-09-10T20:38:50.651258+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:38:50.651258+0000","last_undegraded":"2024-09-10T20:38:50.651258+0000","last_fullsized":"2024-09-10T20:38:50.651258+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651173+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:38:50.651173+0000","last_peered":"2024-09-10T20:38:50.651173+0000","last_clean":"2024-09-10T20:38:50.651173+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:38:50.651173+0000","last_undegraded":"2024-09-10T20:38:50.651173+0000","last_fullsized":"2024-09-10T20:38:50.651173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'19757","reported_seq":26889,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:56.899505+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:38:56.899505+0000","last_peered":"2024-09-10T20:38:56.899505+0000","last_clean":"2024-09-10T20:38:56.899505+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:38:56.899505+0000","last_undegraded":"2024-09-10T20:38:56.899505+0000","last_fullsized":"2024-09-10T20:38:56.899505+0000","mapping_epoch":46,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3109347,"num_objects":760,"num_object_clones":0,"num_object_copies":1520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16364,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":43693505,"num_objects":10671,"num_object_clones":0,"num_object_copies":21342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28242,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10671,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":284746,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14907,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":266280960,"data_stored":266222234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160750,"ondisk_log_size":160750,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182525,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":541312,"kb_used_data":90412,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93830528,"statfs":{"total":96636764160,"available":96082460672,"internally_reserved":0,"allocated":92581888,"data_stored":92310235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":461696682},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,4,1,3,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986297,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":498004,"kb_used_data":137284,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93873836,"statfs":{"total":96636764160,"available":96126808064,"internally_reserved":0,"allocated":140578816,"data_stored":140307168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369356458},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659243,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505104,"kb_used_data":36084,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93866736,"statfs":{"total":96636764160,"available":96119537664,"internally_reserved":0,"allocated":36950016,"data_stored":36634548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480243374},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,6,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":90583040,"data_stored":90579427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36397056,"data_stored":36349628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":139300864,"data_stored":139293179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:39:05.455 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:39:06.011 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:06.011 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:39:06.037 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":373,"stamp":"2024-09-10T20:39:05.045602+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":38905300,"num_objects":9504,"num_object_clones":0,"num_object_copies":19008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28125,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9504,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":285917,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15026,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160722,"ondisk_log_size":160722,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1545004,"kb_used_data":251888,"kb_used_omap":12,"kb_used_meta":1293043,"kb_avail":281570516,"statfs":{"total":289910292480,"available":288328208384,"internally_reserved":0,"allocated":257933312,"data_stored":257074543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1324076034},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,9,5,12,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22855680,"num_objects":-5580,"num_object_clones":0,"num_object_copies":-11160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-387,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5580,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":387,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004440"},"pg_stats":[{"pgid":"2.f","version":"51'20021","reported_seq":20248,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:59.934770+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:38:59.934770+0000","last_peered":"2024-09-10T20:38:59.934770+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:38:59.934770+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2609152,"num_objects":637,"num_object_clones":0,"num_object_copies":1274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20021,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2904}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20437","reported_seq":20562,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:04.427079+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:04.427079+0000","last_peered":"2024-09-10T20:39:04.427079+0000","last_clean":"2024-09-10T20:39:04.427079+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:04.427079+0000","last_undegraded":"2024-09-10T20:39:04.427079+0000","last_fullsized":"2024-09-10T20:39:04.427079+0000","mapping_epoch":35,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2068480,"num_objects":505,"num_object_clones":0,"num_object_copies":1010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18623,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20121","reported_seq":27483,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:02.013269+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:02.013269+0000","last_peered":"2024-09-10T20:39:02.013269+0000","last_clean":"2024-09-10T20:39:02.013269+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:02.013269+0000","last_undegraded":"2024-09-10T20:39:02.013269+0000","last_fullsized":"2024-09-10T20:39:02.013269+0000","mapping_epoch":44,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2433048,"num_objects":595,"num_object_clones":0,"num_object_copies":1190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":595,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18249,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20095","reported_seq":20190,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:04.666142+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:04.666142+0000","last_peered":"2024-09-10T20:39:04.666142+0000","last_clean":"2024-09-10T20:39:04.666142+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:04.666142+0000","last_undegraded":"2024-09-10T20:39:04.666142+0000","last_fullsized":"2024-09-10T20:39:04.666142+0000","mapping_epoch":35,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2084864,"num_objects":509,"num_object_clones":0,"num_object_copies":1018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20095,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20381","reported_seq":20522,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:00.013805+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:00.013805+0000","last_peered":"2024-09-10T20:39:00.013805+0000","last_clean":"2024-09-10T20:39:00.013805+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:00.013805+0000","last_undegraded":"2024-09-10T20:39:00.013805+0000","last_fullsized":"2024-09-10T20:39:00.013805+0000","mapping_epoch":34,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":688,"num_object_clones":0,"num_object_copies":1376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'19867","reported_seq":20122,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:00.012089+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:00.012089+0000","last_peered":"2024-09-10T20:39:00.012089+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:00.012089+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2473984,"num_objects":604,"num_object_clones":0,"num_object_copies":1208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16731,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2957}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19959","reported_seq":20250,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:59.957264+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:38:59.957264+0000","last_peered":"2024-09-10T20:38:59.957264+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:38:59.957264+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2658304,"num_objects":649,"num_object_clones":0,"num_object_copies":1298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16554,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2837}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20109","reported_seq":20248,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:59.899188+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:38:59.899188+0000","last_peered":"2024-09-10T20:38:59.899188+0000","last_clean":"2024-09-10T20:38:59.899188+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:38:59.899188+0000","last_undegraded":"2024-09-10T20:38:59.899188+0000","last_fullsized":"2024-09-10T20:38:59.899188+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2568192,"num_objects":627,"num_object_clones":0,"num_object_copies":1254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17180,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.652747+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:38:50.652747+0000","last_peered":"2024-09-10T20:38:50.652747+0000","last_clean":"2024-09-10T20:38:50.652747+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:38:50.652747+0000","last_undegraded":"2024-09-10T20:38:50.652747+0000","last_fullsized":"2024-09-10T20:38:50.652747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'19915","reported_seq":20905,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:04.686129+0000","last_change":"2024-09-10T20:38:49.349021+0000","last_active":"2024-09-10T20:39:04.686129+0000","last_peered":"2024-09-10T20:39:04.686129+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:39:04.686129+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2146304,"num_objects":524,"num_object_clones":0,"num_object_copies":1048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17908,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2583}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.654924+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:38:50.654924+0000","last_peered":"2024-09-10T20:38:50.654924+0000","last_clean":"2024-09-10T20:38:50.654924+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:38:50.654924+0000","last_undegraded":"2024-09-10T20:38:50.654924+0000","last_fullsized":"2024-09-10T20:38:50.654924+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20088","reported_seq":20306,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:00.019026+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:00.019026+0000","last_peered":"2024-09-10T20:39:00.019026+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:00.019026+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2630598,"num_objects":644,"num_object_clones":0,"num_object_copies":1288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20086,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2922}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.648348+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:38:50.648348+0000","last_peered":"2024-09-10T20:38:50.648348+0000","last_clean":"2024-09-10T20:38:50.648348+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:38:50.648348+0000","last_undegraded":"2024-09-10T20:38:50.648348+0000","last_fullsized":"2024-09-10T20:38:50.648348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20081","reported_seq":20278,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:04.688603+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:39:04.688603+0000","last_peered":"2024-09-10T20:39:04.688603+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:39:04.688603+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1982464,"num_objects":484,"num_object_clones":0,"num_object_copies":968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17909,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3122}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651037+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:38:50.651037+0000","last_peered":"2024-09-10T20:38:50.651037+0000","last_clean":"2024-09-10T20:38:50.651037+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:38:50.651037+0000","last_undegraded":"2024-09-10T20:38:50.651037+0000","last_fullsized":"2024-09-10T20:38:50.651037+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20147","reported_seq":27522,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:04.683424+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:04.683424+0000","last_peered":"2024-09-10T20:39:04.683424+0000","last_clean":"2024-09-10T20:39:04.683424+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:04.683424+0000","last_undegraded":"2024-09-10T20:39:04.683424+0000","last_fullsized":"2024-09-10T20:39:04.683424+0000","mapping_epoch":48,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2117632,"num_objects":517,"num_object_clones":0,"num_object_copies":1034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17933,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20156","reported_seq":20418,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:59.993363+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:38:59.993363+0000","last_peered":"2024-09-10T20:38:59.993363+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:38:59.993363+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2760704,"num_objects":674,"num_object_clones":0,"num_object_copies":1348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16879,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2918}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651168+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:38:50.651168+0000","last_peered":"2024-09-10T20:38:50.651168+0000","last_clean":"2024-09-10T20:38:50.651168+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:38:50.651168+0000","last_undegraded":"2024-09-10T20:38:50.651168+0000","last_fullsized":"2024-09-10T20:38:50.651168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20009","reported_seq":27243,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:01.992096+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:01.992096+0000","last_peered":"2024-09-10T20:39:01.992096+0000","last_clean":"2024-09-10T20:39:01.992096+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:01.992096+0000","last_undegraded":"2024-09-10T20:39:01.992096+0000","last_fullsized":"2024-09-10T20:39:01.992096+0000","mapping_epoch":50,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2355200,"num_objects":575,"num_object_clones":0,"num_object_copies":1150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16703,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.656849+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:38:50.656849+0000","last_peered":"2024-09-10T20:38:50.656849+0000","last_clean":"2024-09-10T20:38:50.656849+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:38:50.656849+0000","last_undegraded":"2024-09-10T20:38:50.656849+0000","last_fullsized":"2024-09-10T20:38:50.656849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20329","reported_seq":20590,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:59.974359+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:38:59.974359+0000","last_peered":"2024-09-10T20:38:59.974359+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:38:59.974359+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2691072,"num_objects":657,"num_object_clones":0,"num_object_copies":1314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17156,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2998}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651258+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:38:50.651258+0000","last_peered":"2024-09-10T20:38:50.651258+0000","last_clean":"2024-09-10T20:38:50.651258+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:38:50.651258+0000","last_undegraded":"2024-09-10T20:38:50.651258+0000","last_fullsized":"2024-09-10T20:38:50.651258+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651173+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:38:50.651173+0000","last_peered":"2024-09-10T20:38:50.651173+0000","last_clean":"2024-09-10T20:38:50.651173+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:38:50.651173+0000","last_undegraded":"2024-09-10T20:38:50.651173+0000","last_fullsized":"2024-09-10T20:38:50.651173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'19904","reported_seq":27036,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:02.096495+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:02.096495+0000","last_peered":"2024-09-10T20:39:02.096495+0000","last_clean":"2024-09-10T20:39:02.096495+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:02.096495+0000","last_undegraded":"2024-09-10T20:39:02.096495+0000","last_fullsized":"2024-09-10T20:39:02.096495+0000","mapping_epoch":46,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2507235,"num_objects":613,"num_object_clones":0,"num_object_copies":1226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16511,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":38905281,"num_objects":9502,"num_object_clones":0,"num_object_copies":19004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28125,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9502,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":285915,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15024,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":253984768,"data_stored":253926042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160719,"ondisk_log_size":160719,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182525,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":541312,"kb_used_data":90412,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93830528,"statfs":{"total":96636764160,"available":96082460672,"internally_reserved":0,"allocated":92581888,"data_stored":92310235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":461696682},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,4,1,3,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986298,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497488,"kb_used_data":132608,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874352,"statfs":{"total":96636764160,"available":96127336448,"internally_reserved":0,"allocated":135790592,"data_stored":135518944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659244,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506204,"kb_used_data":28868,"kb_used_omap":4,"kb_used_meta":477307,"kb_avail":93865636,"statfs":{"total":96636764160,"available":96118411264,"internally_reserved":0,"allocated":29560832,"data_stored":29245364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":488763054},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,3,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":90583040,"data_stored":90579427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28880896,"data_stored":28833468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134520832,"data_stored":134513147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:39:06.038 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:39:06.605 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:06.606 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:39:06.631 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":373,"stamp":"2024-09-10T20:39:05.045602+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":38905300,"num_objects":9504,"num_object_clones":0,"num_object_copies":19008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28125,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9504,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":285917,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15026,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160722,"ondisk_log_size":160722,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1545004,"kb_used_data":251888,"kb_used_omap":12,"kb_used_meta":1293043,"kb_avail":281570516,"statfs":{"total":289910292480,"available":288328208384,"internally_reserved":0,"allocated":257933312,"data_stored":257074543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1324076034},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,9,5,12,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22855680,"num_objects":-5580,"num_object_clones":0,"num_object_copies":-11160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-387,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5580,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":387,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004440"},"pg_stats":[{"pgid":"2.f","version":"51'20021","reported_seq":20248,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:59.934770+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:38:59.934770+0000","last_peered":"2024-09-10T20:38:59.934770+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:38:59.934770+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2609152,"num_objects":637,"num_object_clones":0,"num_object_copies":1274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20021,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2904}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20437","reported_seq":20562,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:04.427079+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:04.427079+0000","last_peered":"2024-09-10T20:39:04.427079+0000","last_clean":"2024-09-10T20:39:04.427079+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:04.427079+0000","last_undegraded":"2024-09-10T20:39:04.427079+0000","last_fullsized":"2024-09-10T20:39:04.427079+0000","mapping_epoch":35,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2068480,"num_objects":505,"num_object_clones":0,"num_object_copies":1010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18623,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20121","reported_seq":27483,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:02.013269+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:02.013269+0000","last_peered":"2024-09-10T20:39:02.013269+0000","last_clean":"2024-09-10T20:39:02.013269+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:02.013269+0000","last_undegraded":"2024-09-10T20:39:02.013269+0000","last_fullsized":"2024-09-10T20:39:02.013269+0000","mapping_epoch":44,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2433048,"num_objects":595,"num_object_clones":0,"num_object_copies":1190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":595,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18249,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20095","reported_seq":20190,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:04.666142+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:04.666142+0000","last_peered":"2024-09-10T20:39:04.666142+0000","last_clean":"2024-09-10T20:39:04.666142+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:04.666142+0000","last_undegraded":"2024-09-10T20:39:04.666142+0000","last_fullsized":"2024-09-10T20:39:04.666142+0000","mapping_epoch":35,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2084864,"num_objects":509,"num_object_clones":0,"num_object_copies":1018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20095,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20381","reported_seq":20522,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:00.013805+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:00.013805+0000","last_peered":"2024-09-10T20:39:00.013805+0000","last_clean":"2024-09-10T20:39:00.013805+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:00.013805+0000","last_undegraded":"2024-09-10T20:39:00.013805+0000","last_fullsized":"2024-09-10T20:39:00.013805+0000","mapping_epoch":34,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":688,"num_object_clones":0,"num_object_copies":1376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'19867","reported_seq":20122,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:00.012089+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:00.012089+0000","last_peered":"2024-09-10T20:39:00.012089+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:00.012089+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2473984,"num_objects":604,"num_object_clones":0,"num_object_copies":1208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16731,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2957}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19959","reported_seq":20250,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:59.957264+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:38:59.957264+0000","last_peered":"2024-09-10T20:38:59.957264+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:38:59.957264+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2658304,"num_objects":649,"num_object_clones":0,"num_object_copies":1298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16554,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2837}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20109","reported_seq":20248,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:59.899188+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:38:59.899188+0000","last_peered":"2024-09-10T20:38:59.899188+0000","last_clean":"2024-09-10T20:38:59.899188+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:38:59.899188+0000","last_undegraded":"2024-09-10T20:38:59.899188+0000","last_fullsized":"2024-09-10T20:38:59.899188+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2568192,"num_objects":627,"num_object_clones":0,"num_object_copies":1254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17180,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.652747+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:38:50.652747+0000","last_peered":"2024-09-10T20:38:50.652747+0000","last_clean":"2024-09-10T20:38:50.652747+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:38:50.652747+0000","last_undegraded":"2024-09-10T20:38:50.652747+0000","last_fullsized":"2024-09-10T20:38:50.652747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'19915","reported_seq":20905,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:04.686129+0000","last_change":"2024-09-10T20:38:49.349021+0000","last_active":"2024-09-10T20:39:04.686129+0000","last_peered":"2024-09-10T20:39:04.686129+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:39:04.686129+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2146304,"num_objects":524,"num_object_clones":0,"num_object_copies":1048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17908,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2583}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.654924+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:38:50.654924+0000","last_peered":"2024-09-10T20:38:50.654924+0000","last_clean":"2024-09-10T20:38:50.654924+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:38:50.654924+0000","last_undegraded":"2024-09-10T20:38:50.654924+0000","last_fullsized":"2024-09-10T20:38:50.654924+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20088","reported_seq":20306,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:00.019026+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:00.019026+0000","last_peered":"2024-09-10T20:39:00.019026+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:00.019026+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2630598,"num_objects":644,"num_object_clones":0,"num_object_copies":1288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20086,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2922}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.648348+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:38:50.648348+0000","last_peered":"2024-09-10T20:38:50.648348+0000","last_clean":"2024-09-10T20:38:50.648348+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:38:50.648348+0000","last_undegraded":"2024-09-10T20:38:50.648348+0000","last_fullsized":"2024-09-10T20:38:50.648348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20081","reported_seq":20278,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:04.688603+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:39:04.688603+0000","last_peered":"2024-09-10T20:39:04.688603+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:39:04.688603+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1982464,"num_objects":484,"num_object_clones":0,"num_object_copies":968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17909,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3122}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651037+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:38:50.651037+0000","last_peered":"2024-09-10T20:38:50.651037+0000","last_clean":"2024-09-10T20:38:50.651037+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:38:50.651037+0000","last_undegraded":"2024-09-10T20:38:50.651037+0000","last_fullsized":"2024-09-10T20:38:50.651037+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20147","reported_seq":27522,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:04.683424+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:04.683424+0000","last_peered":"2024-09-10T20:39:04.683424+0000","last_clean":"2024-09-10T20:39:04.683424+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:04.683424+0000","last_undegraded":"2024-09-10T20:39:04.683424+0000","last_fullsized":"2024-09-10T20:39:04.683424+0000","mapping_epoch":48,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2117632,"num_objects":517,"num_object_clones":0,"num_object_copies":1034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17933,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20156","reported_seq":20418,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:59.993363+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:38:59.993363+0000","last_peered":"2024-09-10T20:38:59.993363+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:38:59.993363+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2760704,"num_objects":674,"num_object_clones":0,"num_object_copies":1348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16879,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2918}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651168+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:38:50.651168+0000","last_peered":"2024-09-10T20:38:50.651168+0000","last_clean":"2024-09-10T20:38:50.651168+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:38:50.651168+0000","last_undegraded":"2024-09-10T20:38:50.651168+0000","last_fullsized":"2024-09-10T20:38:50.651168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20009","reported_seq":27243,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:01.992096+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:01.992096+0000","last_peered":"2024-09-10T20:39:01.992096+0000","last_clean":"2024-09-10T20:39:01.992096+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:01.992096+0000","last_undegraded":"2024-09-10T20:39:01.992096+0000","last_fullsized":"2024-09-10T20:39:01.992096+0000","mapping_epoch":50,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2355200,"num_objects":575,"num_object_clones":0,"num_object_copies":1150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16703,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.656849+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:38:50.656849+0000","last_peered":"2024-09-10T20:38:50.656849+0000","last_clean":"2024-09-10T20:38:50.656849+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:38:50.656849+0000","last_undegraded":"2024-09-10T20:38:50.656849+0000","last_fullsized":"2024-09-10T20:38:50.656849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20329","reported_seq":20590,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:38:59.974359+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:38:59.974359+0000","last_peered":"2024-09-10T20:38:59.974359+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:38:59.974359+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2691072,"num_objects":657,"num_object_clones":0,"num_object_copies":1314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17156,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2998}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651258+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:38:50.651258+0000","last_peered":"2024-09-10T20:38:50.651258+0000","last_clean":"2024-09-10T20:38:50.651258+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:38:50.651258+0000","last_undegraded":"2024-09-10T20:38:50.651258+0000","last_fullsized":"2024-09-10T20:38:50.651258+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651173+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:38:50.651173+0000","last_peered":"2024-09-10T20:38:50.651173+0000","last_clean":"2024-09-10T20:38:50.651173+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:38:50.651173+0000","last_undegraded":"2024-09-10T20:38:50.651173+0000","last_fullsized":"2024-09-10T20:38:50.651173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'19904","reported_seq":27036,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:02.096495+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:02.096495+0000","last_peered":"2024-09-10T20:39:02.096495+0000","last_clean":"2024-09-10T20:39:02.096495+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:02.096495+0000","last_undegraded":"2024-09-10T20:39:02.096495+0000","last_fullsized":"2024-09-10T20:39:02.096495+0000","mapping_epoch":46,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2507235,"num_objects":613,"num_object_clones":0,"num_object_copies":1226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16511,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":38905281,"num_objects":9502,"num_object_clones":0,"num_object_copies":19004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28125,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9502,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":285915,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15024,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":253984768,"data_stored":253926042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160719,"ondisk_log_size":160719,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182525,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":541312,"kb_used_data":90412,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93830528,"statfs":{"total":96636764160,"available":96082460672,"internally_reserved":0,"allocated":92581888,"data_stored":92310235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":461696682},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,4,1,3,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986298,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497488,"kb_used_data":132608,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874352,"statfs":{"total":96636764160,"available":96127336448,"internally_reserved":0,"allocated":135790592,"data_stored":135518944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659244,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506204,"kb_used_data":28868,"kb_used_omap":4,"kb_used_meta":477307,"kb_avail":93865636,"statfs":{"total":96636764160,"available":96118411264,"internally_reserved":0,"allocated":29560832,"data_stored":29245364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":488763054},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,3,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":90583040,"data_stored":90579427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28880896,"data_stored":28833468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134520832,"data_stored":134513147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:39:06.632 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:39:06.632 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'20021", 'reported_seq': 20248, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:59.934770+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:38:59.934770+0000', 'last_peered': '2024-09-10T20:38:59.934770+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:38:59.934770+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2609152, 'num_objects': 637, 'num_object_clones': 0, 'num_object_copies': 1274, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 637, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20021, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2904}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:06.632 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:39:06.632 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'19867", 'reported_seq': 20122, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:00.012089+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:39:00.012089+0000', 'last_peered': '2024-09-10T20:39:00.012089+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:39:00.012089+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2473984, 'num_objects': 604, 'num_object_clones': 0, 'num_object_copies': 1208, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 604, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16731, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2957}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:06.633 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:39:06.633 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19959", 'reported_seq': 20250, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:59.957264+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:38:59.957264+0000', 'last_peered': '2024-09-10T20:38:59.957264+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:38:59.957264+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2658304, 'num_objects': 649, 'num_object_clones': 0, 'num_object_copies': 1298, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 649, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16554, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2837}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:06.633 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:39:06.633 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'19915", 'reported_seq': 20905, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:04.686129+0000', 'last_change': '2024-09-10T20:38:49.349021+0000', 'last_active': '2024-09-10T20:39:04.686129+0000', 'last_peered': '2024-09-10T20:39:04.686129+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:39:04.686129+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2146304, 'num_objects': 524, 'num_object_clones': 0, 'num_object_copies': 1048, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3107, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 524, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17908, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 421, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2583}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:39:06.633 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:39:06.633 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20088", 'reported_seq': 20306, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:00.019026+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:39:00.019026+0000', 'last_peered': '2024-09-10T20:39:00.019026+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:39:00.019026+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2630598, 'num_objects': 644, 'num_object_clones': 0, 'num_object_copies': 1288, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 644, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20086, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2922}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:06.633 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:39:06.633 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "51'20081", 'reported_seq': 20278, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:04.688603+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:39:04.688603+0000', 'last_peered': '2024-09-10T20:39:04.688603+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:39:04.688603+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1982464, 'num_objects': 484, 'num_object_clones': 0, 'num_object_copies': 968, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 484, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17909, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3122}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:39:06.633 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:39:06.634 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'20156", 'reported_seq': 20418, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:59.993363+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:38:59.993363+0000', 'last_peered': '2024-09-10T20:38:59.993363+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:38:59.993363+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2760704, 'num_objects': 674, 'num_object_clones': 0, 'num_object_copies': 1348, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 674, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16879, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2918}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:06.634 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:39:06.634 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "51'20329", 'reported_seq': 20590, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:38:59.974359+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:38:59.974359+0000', 'last_peered': '2024-09-10T20:38:59.974359+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:38:59.974359+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'10300", 'ondisk_log_start': "45'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2691072, 'num_objects': 657, 'num_object_clones': 0, 'num_object_copies': 1314, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 657, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17156, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2998}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:06.634 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:39:07.332 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:07.332 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 28125/16536 objects degraded (170.083%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":658,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":8268,"data_bytes":33842644,"bytes_used":1578782720,"bytes_avail":288331509760,"bytes_total":289910292480,"degraded_objects":28125,"degraded_total":16536,"degraded_ratio":1.7008345428156746,"recovering_objects_per_sec":19,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":230,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":405},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:39:07.354 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:39:07.355 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:39:07.951 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:07.954 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:39:07.973 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":374,"stamp":"2024-09-10T20:39:07.049927+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":33842644,"num_objects":8268,"num_object_clones":0,"num_object_copies":16536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28125,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8268,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":287153,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15026,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160658,"ondisk_log_size":160658,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1541780,"kb_used_data":244504,"kb_used_omap":12,"kb_used_meta":1297203,"kb_avail":281573740,"statfs":{"total":289910292480,"available":288331509760,"internally_reserved":0,"allocated":250372096,"data_stored":249513327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1328335874},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,7,5,14,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19963904,"num_objects":-4874,"num_object_clones":0,"num_object_copies":-9748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-230,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4874,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007821"},"pg_stats":[{"pgid":"2.f","version":"51'20168","reported_seq":20394,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:05.024928+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:05.024928+0000","last_peered":"2024-09-10T20:39:05.024928+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:05.024928+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2007040,"num_objects":490,"num_object_clones":0,"num_object_copies":980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20168,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3051}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20437","reported_seq":20562,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:04.427079+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:04.427079+0000","last_peered":"2024-09-10T20:39:04.427079+0000","last_clean":"2024-09-10T20:39:04.427079+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:04.427079+0000","last_undegraded":"2024-09-10T20:39:04.427079+0000","last_fullsized":"2024-09-10T20:39:04.427079+0000","mapping_epoch":35,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2068480,"num_objects":505,"num_object_clones":0,"num_object_copies":1010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18623,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20121","reported_seq":27483,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:02.013269+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:02.013269+0000","last_peered":"2024-09-10T20:39:02.013269+0000","last_clean":"2024-09-10T20:39:02.013269+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:02.013269+0000","last_undegraded":"2024-09-10T20:39:02.013269+0000","last_fullsized":"2024-09-10T20:39:02.013269+0000","mapping_epoch":44,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2433048,"num_objects":595,"num_object_clones":0,"num_object_copies":1190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":595,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18249,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20095","reported_seq":20190,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:04.666142+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:04.666142+0000","last_peered":"2024-09-10T20:39:04.666142+0000","last_clean":"2024-09-10T20:39:04.666142+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:04.666142+0000","last_undegraded":"2024-09-10T20:39:04.666142+0000","last_fullsized":"2024-09-10T20:39:04.666142+0000","mapping_epoch":35,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2084864,"num_objects":509,"num_object_clones":0,"num_object_copies":1018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20095,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20546","reported_seq":20687,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:05.035330+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:05.035330+0000","last_peered":"2024-09-10T20:39:05.035330+0000","last_clean":"2024-09-10T20:39:05.035330+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:05.035330+0000","last_undegraded":"2024-09-10T20:39:05.035330+0000","last_fullsized":"2024-09-10T20:39:05.035330+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2142208,"num_objects":523,"num_object_clones":0,"num_object_copies":1046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17542,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'19999","reported_seq":20256,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:04.995126+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:04.995126+0000","last_peered":"2024-09-10T20:39:04.995126+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:04.995126+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1933312,"num_objects":472,"num_object_clones":0,"num_object_copies":944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16863,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3089}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20120","reported_seq":20411,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:05.020806+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:05.020806+0000","last_peered":"2024-09-10T20:39:05.020806+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:05.020806+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1998848,"num_objects":488,"num_object_clones":0,"num_object_copies":976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16715,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2998}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20250","reported_seq":20388,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:05.025764+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:05.025764+0000","last_peered":"2024-09-10T20:39:05.025764+0000","last_clean":"2024-09-10T20:39:05.025764+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:05.025764+0000","last_undegraded":"2024-09-10T20:39:05.025764+0000","last_fullsized":"2024-09-10T20:39:05.025764+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1990656,"num_objects":486,"num_object_clones":0,"num_object_copies":972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17321,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.652747+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:38:50.652747+0000","last_peered":"2024-09-10T20:38:50.652747+0000","last_clean":"2024-09-10T20:38:50.652747+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:38:50.652747+0000","last_undegraded":"2024-09-10T20:38:50.652747+0000","last_fullsized":"2024-09-10T20:38:50.652747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'19915","reported_seq":20905,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:04.686129+0000","last_change":"2024-09-10T20:38:49.349021+0000","last_active":"2024-09-10T20:39:04.686129+0000","last_peered":"2024-09-10T20:39:04.686129+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:39:04.686129+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2146304,"num_objects":524,"num_object_clones":0,"num_object_copies":1048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17908,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2583}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.654924+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:38:50.654924+0000","last_peered":"2024-09-10T20:38:50.654924+0000","last_clean":"2024-09-10T20:38:50.654924+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:38:50.654924+0000","last_undegraded":"2024-09-10T20:38:50.654924+0000","last_fullsized":"2024-09-10T20:38:50.654924+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20242","reported_seq":20463,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:04.995871+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:04.995871+0000","last_peered":"2024-09-10T20:39:04.995871+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:04.995871+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1999814,"num_objects":490,"num_object_clones":0,"num_object_copies":980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20240,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3076}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.648348+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:38:50.648348+0000","last_peered":"2024-09-10T20:38:50.648348+0000","last_clean":"2024-09-10T20:38:50.648348+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:38:50.648348+0000","last_undegraded":"2024-09-10T20:38:50.648348+0000","last_fullsized":"2024-09-10T20:38:50.648348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20081","reported_seq":20278,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:04.688603+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:39:04.688603+0000","last_peered":"2024-09-10T20:39:04.688603+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:39:04.688603+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1982464,"num_objects":484,"num_object_clones":0,"num_object_copies":968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17909,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3122}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651037+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:38:50.651037+0000","last_peered":"2024-09-10T20:38:50.651037+0000","last_clean":"2024-09-10T20:38:50.651037+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:38:50.651037+0000","last_undegraded":"2024-09-10T20:38:50.651037+0000","last_fullsized":"2024-09-10T20:38:50.651037+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20147","reported_seq":27522,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:04.683424+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:04.683424+0000","last_peered":"2024-09-10T20:39:04.683424+0000","last_clean":"2024-09-10T20:39:04.683424+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:04.683424+0000","last_undegraded":"2024-09-10T20:39:04.683424+0000","last_fullsized":"2024-09-10T20:39:04.683424+0000","mapping_epoch":48,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2117632,"num_objects":517,"num_object_clones":0,"num_object_copies":1034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17933,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20319","reported_seq":20581,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:04.976866+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:39:04.976866+0000","last_peered":"2024-09-10T20:39:04.976866+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:39:04.976866+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2093056,"num_objects":511,"num_object_clones":0,"num_object_copies":1022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3081}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651168+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:38:50.651168+0000","last_peered":"2024-09-10T20:38:50.651168+0000","last_clean":"2024-09-10T20:38:50.651168+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:38:50.651168+0000","last_undegraded":"2024-09-10T20:38:50.651168+0000","last_fullsized":"2024-09-10T20:38:50.651168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20009","reported_seq":27243,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:01.992096+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:01.992096+0000","last_peered":"2024-09-10T20:39:01.992096+0000","last_clean":"2024-09-10T20:39:01.992096+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:01.992096+0000","last_undegraded":"2024-09-10T20:39:01.992096+0000","last_fullsized":"2024-09-10T20:39:01.992096+0000","mapping_epoch":50,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2355200,"num_objects":575,"num_object_clones":0,"num_object_copies":1150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16703,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.656849+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:38:50.656849+0000","last_peered":"2024-09-10T20:38:50.656849+0000","last_clean":"2024-09-10T20:38:50.656849+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:38:50.656849+0000","last_undegraded":"2024-09-10T20:38:50.656849+0000","last_fullsized":"2024-09-10T20:38:50.656849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20502","reported_seq":20763,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:04.913584+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:04.913584+0000","last_peered":"2024-09-10T20:39:04.913584+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:04.913584+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1982464,"num_objects":484,"num_object_clones":0,"num_object_copies":968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17329,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3171}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651258+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:38:50.651258+0000","last_peered":"2024-09-10T20:38:50.651258+0000","last_clean":"2024-09-10T20:38:50.651258+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:38:50.651258+0000","last_undegraded":"2024-09-10T20:38:50.651258+0000","last_fullsized":"2024-09-10T20:38:50.651258+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651173+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:38:50.651173+0000","last_peered":"2024-09-10T20:38:50.651173+0000","last_clean":"2024-09-10T20:38:50.651173+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:38:50.651173+0000","last_undegraded":"2024-09-10T20:38:50.651173+0000","last_fullsized":"2024-09-10T20:38:50.651173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'19904","reported_seq":27036,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:02.096495+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:02.096495+0000","last_peered":"2024-09-10T20:39:02.096495+0000","last_clean":"2024-09-10T20:39:02.096495+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:02.096495+0000","last_undegraded":"2024-09-10T20:39:02.096495+0000","last_fullsized":"2024-09-10T20:39:02.096495+0000","mapping_epoch":46,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2507235,"num_objects":613,"num_object_clones":0,"num_object_copies":1226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16511,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":33842625,"num_objects":8266,"num_object_clones":0,"num_object_copies":16532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28125,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8266,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":287151,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15024,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":246448128,"data_stored":246389402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160655,"ondisk_log_size":160655,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182526,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":538088,"kb_used_data":83028,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93833752,"statfs":{"total":96636764160,"available":96085762048,"internally_reserved":0,"allocated":85020672,"data_stored":84749019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":465956522},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,5,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986298,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497488,"kb_used_data":132608,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874352,"statfs":{"total":96636764160,"available":96127336448,"internally_reserved":0,"allocated":135790592,"data_stored":135518944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659244,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506204,"kb_used_data":28868,"kb_used_omap":4,"kb_used_meta":477307,"kb_avail":93865636,"statfs":{"total":96636764160,"available":96118411264,"internally_reserved":0,"allocated":29560832,"data_stored":29245364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":488763054},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,3,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":83046400,"data_stored":83042787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28880896,"data_stored":28833468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134520832,"data_stored":134513147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:39:10.976 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:39:11.569 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:11.570 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:39:11.598 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":376,"stamp":"2024-09-10T20:39:11.051605+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":23573972,"num_objects":5761,"num_object_clones":0,"num_object_copies":11522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5761,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":289660,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15128,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160865,"ondisk_log_size":160865,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1484552,"kb_used_data":224724,"kb_used_omap":12,"kb_used_meta":1259763,"kb_avail":281630968,"statfs":{"total":289910292480,"available":288390111232,"internally_reserved":0,"allocated":230117376,"data_stored":229258607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1289997314},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,2,0,6,4,8,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28270592,"num_objects":-6902,"num_object_clones":0,"num_object_copies":-13804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-332,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6902,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007938"},"pg_stats":[{"pgid":"2.f","version":"51'20335","reported_seq":20562,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:10.064956+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:10.064956+0000","last_peered":"2024-09-10T20:39:10.064956+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:10.064956+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1323008,"num_objects":323,"num_object_clones":0,"num_object_copies":646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20335,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3218}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20610","reported_seq":20734,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:09.731892+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:09.731892+0000","last_peered":"2024-09-10T20:39:09.731892+0000","last_clean":"2024-09-10T20:39:09.731892+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:09.731892+0000","last_undegraded":"2024-09-10T20:39:09.731892+0000","last_fullsized":"2024-09-10T20:39:09.731892+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1359872,"num_objects":332,"num_object_clones":0,"num_object_copies":664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18796,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20265","reported_seq":27627,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:07.103100+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:07.103100+0000","last_peered":"2024-09-10T20:39:07.103100+0000","last_clean":"2024-09-10T20:39:07.103100+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:07.103100+0000","last_undegraded":"2024-09-10T20:39:07.103100+0000","last_fullsized":"2024-09-10T20:39:07.103100+0000","mapping_epoch":44,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1843224,"num_objects":451,"num_object_clones":0,"num_object_copies":902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":451,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18393,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20242","reported_seq":20337,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:09.619977+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:09.619977+0000","last_peered":"2024-09-10T20:39:09.619977+0000","last_clean":"2024-09-10T20:39:09.619977+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:09.619977+0000","last_undegraded":"2024-09-10T20:39:09.619977+0000","last_fullsized":"2024-09-10T20:39:09.619977+0000","mapping_epoch":35,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1482752,"num_objects":362,"num_object_clones":0,"num_object_copies":724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20242,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20705","reported_seq":20845,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:10.099196+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:10.099196+0000","last_peered":"2024-09-10T20:39:10.099196+0000","last_clean":"2024-09-10T20:39:10.099196+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:10.099196+0000","last_undegraded":"2024-09-10T20:39:10.099196+0000","last_fullsized":"2024-09-10T20:39:10.099196+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1490944,"num_objects":364,"num_object_clones":0,"num_object_copies":728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17701,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20160","reported_seq":20417,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:10.050732+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:10.050732+0000","last_peered":"2024-09-10T20:39:10.050732+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:10.050732+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1273856,"num_objects":311,"num_object_clones":0,"num_object_copies":622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17024,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3250}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20278","reported_seq":20569,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:10.061390+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:10.061390+0000","last_peered":"2024-09-10T20:39:10.061390+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:10.061390+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1351680,"num_objects":330,"num_object_clones":0,"num_object_copies":660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3156}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20400","reported_seq":20538,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:10.034799+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:10.034799+0000","last_peered":"2024-09-10T20:39:10.034799+0000","last_clean":"2024-09-10T20:39:10.034799+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:10.034799+0000","last_undegraded":"2024-09-10T20:39:10.034799+0000","last_fullsized":"2024-09-10T20:39:10.034799+0000","mapping_epoch":34,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376256,"num_objects":336,"num_object_clones":0,"num_object_copies":672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17471,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.652747+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:38:50.652747+0000","last_peered":"2024-09-10T20:38:50.652747+0000","last_clean":"2024-09-10T20:38:50.652747+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:38:50.652747+0000","last_undegraded":"2024-09-10T20:38:50.652747+0000","last_fullsized":"2024-09-10T20:38:50.652747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20072","reported_seq":21266,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:09.675342+0000","last_change":"2024-09-10T20:38:49.349021+0000","last_active":"2024-09-10T20:39:09.675342+0000","last_peered":"2024-09-10T20:39:09.675342+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:39:09.675342+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1503232,"num_objects":367,"num_object_clones":0,"num_object_copies":734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3005,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":523,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2638}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.654924+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:38:50.654924+0000","last_peered":"2024-09-10T20:38:50.654924+0000","last_clean":"2024-09-10T20:38:50.654924+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:38:50.654924+0000","last_undegraded":"2024-09-10T20:38:50.654924+0000","last_fullsized":"2024-09-10T20:38:50.654924+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20401","reported_seq":20622,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:10.060111+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:10.060111+0000","last_peered":"2024-09-10T20:39:10.060111+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:10.060111+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1348550,"num_objects":331,"num_object_clones":0,"num_object_copies":662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20399,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3235}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.648348+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:38:50.648348+0000","last_peered":"2024-09-10T20:38:50.648348+0000","last_clean":"2024-09-10T20:38:50.648348+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:38:50.648348+0000","last_undegraded":"2024-09-10T20:38:50.648348+0000","last_fullsized":"2024-09-10T20:38:50.648348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20237","reported_seq":20433,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:09.704240+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:39:09.704240+0000","last_peered":"2024-09-10T20:39:09.704240+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:39:09.704240+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1343488,"num_objects":328,"num_object_clones":0,"num_object_copies":656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3278}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651037+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:38:50.651037+0000","last_peered":"2024-09-10T20:38:50.651037+0000","last_clean":"2024-09-10T20:38:50.651037+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:38:50.651037+0000","last_undegraded":"2024-09-10T20:38:50.651037+0000","last_fullsized":"2024-09-10T20:38:50.651037+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20298","reported_seq":27673,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:09.715212+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:09.715212+0000","last_peered":"2024-09-10T20:39:09.715212+0000","last_clean":"2024-09-10T20:39:09.715212+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:09.715212+0000","last_undegraded":"2024-09-10T20:39:09.715212+0000","last_fullsized":"2024-09-10T20:39:09.715212+0000","mapping_epoch":48,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1499136,"num_objects":366,"num_object_clones":0,"num_object_copies":732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":366,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18084,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20466","reported_seq":20726,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:10.050144+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:39:10.050144+0000","last_peered":"2024-09-10T20:39:10.050144+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:39:10.050144+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1490944,"num_objects":364,"num_object_clones":0,"num_object_copies":728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17189,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3228}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651168+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:38:50.651168+0000","last_peered":"2024-09-10T20:38:50.651168+0000","last_clean":"2024-09-10T20:38:50.651168+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:38:50.651168+0000","last_undegraded":"2024-09-10T20:38:50.651168+0000","last_fullsized":"2024-09-10T20:38:50.651168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20184","reported_seq":27416,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:07.101157+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:07.101157+0000","last_peered":"2024-09-10T20:39:07.101157+0000","last_clean":"2024-09-10T20:39:07.101157+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:07.101157+0000","last_undegraded":"2024-09-10T20:39:07.101157+0000","last_fullsized":"2024-09-10T20:39:07.101157+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638400,"num_objects":400,"num_object_clones":0,"num_object_copies":800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16878,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.656849+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:38:50.656849+0000","last_peered":"2024-09-10T20:38:50.656849+0000","last_clean":"2024-09-10T20:38:50.656849+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:38:50.656849+0000","last_undegraded":"2024-09-10T20:38:50.656849+0000","last_fullsized":"2024-09-10T20:38:50.656849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20645","reported_seq":20905,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:10.095687+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:10.095687+0000","last_peered":"2024-09-10T20:39:10.095687+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:10.095687+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1396736,"num_objects":341,"num_object_clones":0,"num_object_copies":682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17472,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3314}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651258+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:38:50.651258+0000","last_peered":"2024-09-10T20:38:50.651258+0000","last_clean":"2024-09-10T20:38:50.651258+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:38:50.651258+0000","last_undegraded":"2024-09-10T20:38:50.651258+0000","last_fullsized":"2024-09-10T20:38:50.651258+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651173+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:38:50.651173+0000","last_peered":"2024-09-10T20:38:50.651173+0000","last_clean":"2024-09-10T20:38:50.651173+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:38:50.651173+0000","last_undegraded":"2024-09-10T20:38:50.651173+0000","last_fullsized":"2024-09-10T20:38:50.651173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20064","reported_seq":27196,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:07.069832+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:07.069832+0000","last_peered":"2024-09-10T20:39:07.069832+0000","last_clean":"2024-09-10T20:39:07.069832+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:07.069832+0000","last_undegraded":"2024-09-10T20:39:07.069832+0000","last_fullsized":"2024-09-10T20:39:07.069832+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851875,"num_objects":453,"num_object_clones":0,"num_object_copies":906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16671,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":23573953,"num_objects":5759,"num_object_clones":0,"num_object_copies":11518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5759,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":289658,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15126,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":226246656,"data_stored":226187930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160862,"ondisk_log_size":160862,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182527,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518224,"kb_used_data":75644,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93853616,"statfs":{"total":96636764160,"available":96106102784,"internally_reserved":0,"allocated":77459456,"data_stored":77187803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,5,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986299,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480040,"kb_used_data":127640,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93891800,"statfs":{"total":96636764160,"available":96145203200,"internally_reserved":0,"allocated":130703360,"data_stored":130431712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":360836778},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,4,0,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659245,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486288,"kb_used_data":21440,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93885552,"statfs":{"total":96636764160,"available":96138805248,"internally_reserved":0,"allocated":21954560,"data_stored":21639092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":475983534},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,0,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":75493376,"data_stored":75489763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":21299200,"data_stored":21251772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":129454080,"data_stored":129446395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:39:11.599 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:39:12.184 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:12.186 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:39:12.205 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":376,"stamp":"2024-09-10T20:39:11.051605+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":23573972,"num_objects":5761,"num_object_clones":0,"num_object_copies":11522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5761,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":289660,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15128,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160865,"ondisk_log_size":160865,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1484552,"kb_used_data":224724,"kb_used_omap":12,"kb_used_meta":1259763,"kb_avail":281630968,"statfs":{"total":289910292480,"available":288390111232,"internally_reserved":0,"allocated":230117376,"data_stored":229258607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1289997314},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,2,0,6,4,8,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28270592,"num_objects":-6902,"num_object_clones":0,"num_object_copies":-13804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-332,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6902,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007938"},"pg_stats":[{"pgid":"2.f","version":"51'20335","reported_seq":20562,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:10.064956+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:10.064956+0000","last_peered":"2024-09-10T20:39:10.064956+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:10.064956+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1323008,"num_objects":323,"num_object_clones":0,"num_object_copies":646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20335,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3218}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20610","reported_seq":20734,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:09.731892+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:09.731892+0000","last_peered":"2024-09-10T20:39:09.731892+0000","last_clean":"2024-09-10T20:39:09.731892+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:09.731892+0000","last_undegraded":"2024-09-10T20:39:09.731892+0000","last_fullsized":"2024-09-10T20:39:09.731892+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1359872,"num_objects":332,"num_object_clones":0,"num_object_copies":664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18796,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20265","reported_seq":27627,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:07.103100+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:07.103100+0000","last_peered":"2024-09-10T20:39:07.103100+0000","last_clean":"2024-09-10T20:39:07.103100+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:07.103100+0000","last_undegraded":"2024-09-10T20:39:07.103100+0000","last_fullsized":"2024-09-10T20:39:07.103100+0000","mapping_epoch":44,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1843224,"num_objects":451,"num_object_clones":0,"num_object_copies":902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":451,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18393,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20242","reported_seq":20337,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:09.619977+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:09.619977+0000","last_peered":"2024-09-10T20:39:09.619977+0000","last_clean":"2024-09-10T20:39:09.619977+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:09.619977+0000","last_undegraded":"2024-09-10T20:39:09.619977+0000","last_fullsized":"2024-09-10T20:39:09.619977+0000","mapping_epoch":35,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1482752,"num_objects":362,"num_object_clones":0,"num_object_copies":724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20242,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20705","reported_seq":20845,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:10.099196+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:10.099196+0000","last_peered":"2024-09-10T20:39:10.099196+0000","last_clean":"2024-09-10T20:39:10.099196+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:10.099196+0000","last_undegraded":"2024-09-10T20:39:10.099196+0000","last_fullsized":"2024-09-10T20:39:10.099196+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1490944,"num_objects":364,"num_object_clones":0,"num_object_copies":728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17701,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20160","reported_seq":20417,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:10.050732+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:10.050732+0000","last_peered":"2024-09-10T20:39:10.050732+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:10.050732+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1273856,"num_objects":311,"num_object_clones":0,"num_object_copies":622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17024,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3250}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20278","reported_seq":20569,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:10.061390+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:10.061390+0000","last_peered":"2024-09-10T20:39:10.061390+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:10.061390+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1351680,"num_objects":330,"num_object_clones":0,"num_object_copies":660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3156}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20400","reported_seq":20538,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:10.034799+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:10.034799+0000","last_peered":"2024-09-10T20:39:10.034799+0000","last_clean":"2024-09-10T20:39:10.034799+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:10.034799+0000","last_undegraded":"2024-09-10T20:39:10.034799+0000","last_fullsized":"2024-09-10T20:39:10.034799+0000","mapping_epoch":34,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376256,"num_objects":336,"num_object_clones":0,"num_object_copies":672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17471,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.652747+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:38:50.652747+0000","last_peered":"2024-09-10T20:38:50.652747+0000","last_clean":"2024-09-10T20:38:50.652747+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:38:50.652747+0000","last_undegraded":"2024-09-10T20:38:50.652747+0000","last_fullsized":"2024-09-10T20:38:50.652747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20072","reported_seq":21266,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:09.675342+0000","last_change":"2024-09-10T20:38:49.349021+0000","last_active":"2024-09-10T20:39:09.675342+0000","last_peered":"2024-09-10T20:39:09.675342+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:39:09.675342+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1503232,"num_objects":367,"num_object_clones":0,"num_object_copies":734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3005,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":523,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2638}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.654924+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:38:50.654924+0000","last_peered":"2024-09-10T20:38:50.654924+0000","last_clean":"2024-09-10T20:38:50.654924+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:38:50.654924+0000","last_undegraded":"2024-09-10T20:38:50.654924+0000","last_fullsized":"2024-09-10T20:38:50.654924+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20401","reported_seq":20622,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:10.060111+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:10.060111+0000","last_peered":"2024-09-10T20:39:10.060111+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:10.060111+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1348550,"num_objects":331,"num_object_clones":0,"num_object_copies":662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20399,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3235}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.648348+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:38:50.648348+0000","last_peered":"2024-09-10T20:38:50.648348+0000","last_clean":"2024-09-10T20:38:50.648348+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:38:50.648348+0000","last_undegraded":"2024-09-10T20:38:50.648348+0000","last_fullsized":"2024-09-10T20:38:50.648348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20237","reported_seq":20433,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:09.704240+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:39:09.704240+0000","last_peered":"2024-09-10T20:39:09.704240+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:39:09.704240+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1343488,"num_objects":328,"num_object_clones":0,"num_object_copies":656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3278}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651037+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:38:50.651037+0000","last_peered":"2024-09-10T20:38:50.651037+0000","last_clean":"2024-09-10T20:38:50.651037+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:38:50.651037+0000","last_undegraded":"2024-09-10T20:38:50.651037+0000","last_fullsized":"2024-09-10T20:38:50.651037+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20298","reported_seq":27673,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:09.715212+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:09.715212+0000","last_peered":"2024-09-10T20:39:09.715212+0000","last_clean":"2024-09-10T20:39:09.715212+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:09.715212+0000","last_undegraded":"2024-09-10T20:39:09.715212+0000","last_fullsized":"2024-09-10T20:39:09.715212+0000","mapping_epoch":48,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1499136,"num_objects":366,"num_object_clones":0,"num_object_copies":732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":366,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18084,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20466","reported_seq":20726,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:10.050144+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:39:10.050144+0000","last_peered":"2024-09-10T20:39:10.050144+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:39:10.050144+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1490944,"num_objects":364,"num_object_clones":0,"num_object_copies":728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17189,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3228}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651168+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:38:50.651168+0000","last_peered":"2024-09-10T20:38:50.651168+0000","last_clean":"2024-09-10T20:38:50.651168+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:38:50.651168+0000","last_undegraded":"2024-09-10T20:38:50.651168+0000","last_fullsized":"2024-09-10T20:38:50.651168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20184","reported_seq":27416,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:07.101157+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:07.101157+0000","last_peered":"2024-09-10T20:39:07.101157+0000","last_clean":"2024-09-10T20:39:07.101157+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:07.101157+0000","last_undegraded":"2024-09-10T20:39:07.101157+0000","last_fullsized":"2024-09-10T20:39:07.101157+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638400,"num_objects":400,"num_object_clones":0,"num_object_copies":800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16878,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.656849+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:38:50.656849+0000","last_peered":"2024-09-10T20:38:50.656849+0000","last_clean":"2024-09-10T20:38:50.656849+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:38:50.656849+0000","last_undegraded":"2024-09-10T20:38:50.656849+0000","last_fullsized":"2024-09-10T20:38:50.656849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20645","reported_seq":20905,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:10.095687+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:10.095687+0000","last_peered":"2024-09-10T20:39:10.095687+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:10.095687+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1396736,"num_objects":341,"num_object_clones":0,"num_object_copies":682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17472,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3314}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651258+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:38:50.651258+0000","last_peered":"2024-09-10T20:38:50.651258+0000","last_clean":"2024-09-10T20:38:50.651258+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:38:50.651258+0000","last_undegraded":"2024-09-10T20:38:50.651258+0000","last_fullsized":"2024-09-10T20:38:50.651258+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651173+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:38:50.651173+0000","last_peered":"2024-09-10T20:38:50.651173+0000","last_clean":"2024-09-10T20:38:50.651173+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:38:50.651173+0000","last_undegraded":"2024-09-10T20:38:50.651173+0000","last_fullsized":"2024-09-10T20:38:50.651173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20064","reported_seq":27196,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:07.069832+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:07.069832+0000","last_peered":"2024-09-10T20:39:07.069832+0000","last_clean":"2024-09-10T20:39:07.069832+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:07.069832+0000","last_undegraded":"2024-09-10T20:39:07.069832+0000","last_fullsized":"2024-09-10T20:39:07.069832+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851875,"num_objects":453,"num_object_clones":0,"num_object_copies":906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16671,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":23573953,"num_objects":5759,"num_object_clones":0,"num_object_copies":11518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5759,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":289658,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15126,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":226246656,"data_stored":226187930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160862,"ondisk_log_size":160862,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182527,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518224,"kb_used_data":75644,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93853616,"statfs":{"total":96636764160,"available":96106102784,"internally_reserved":0,"allocated":77459456,"data_stored":77187803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,5,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986299,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480040,"kb_used_data":127640,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93891800,"statfs":{"total":96636764160,"available":96145203200,"internally_reserved":0,"allocated":130703360,"data_stored":130431712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":360836778},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,4,0,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659245,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486288,"kb_used_data":21440,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93885552,"statfs":{"total":96636764160,"available":96138805248,"internally_reserved":0,"allocated":21954560,"data_stored":21639092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":475983534},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,0,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":75493376,"data_stored":75489763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":21299200,"data_stored":21251772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":129454080,"data_stored":129446395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:39:12.206 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:39:12.206 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'20335", 'reported_seq': 20562, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:10.064956+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:39:10.064956+0000', 'last_peered': '2024-09-10T20:39:10.064956+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:39:10.064956+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "45'10300", 'ondisk_log_start': "45'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1323008, 'num_objects': 323, 'num_object_clones': 0, 'num_object_copies': 646, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 323, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20335, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3218}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:12.206 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:39:12.206 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20160", 'reported_seq': 20417, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:10.050732+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:39:10.050732+0000', 'last_peered': '2024-09-10T20:39:10.050732+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:39:10.050732+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1273856, 'num_objects': 311, 'num_object_clones': 0, 'num_object_copies': 622, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 311, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17024, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3250}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:12.206 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:39:12.207 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'20278", 'reported_seq': 20569, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:10.061390+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:39:10.061390+0000', 'last_peered': '2024-09-10T20:39:10.061390+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:39:10.061390+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1351680, 'num_objects': 330, 'num_object_clones': 0, 'num_object_copies': 660, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 330, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16873, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3156}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:12.207 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:39:12.207 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20072", 'reported_seq': 21266, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:09.675342+0000', 'last_change': '2024-09-10T20:38:49.349021+0000', 'last_active': '2024-09-10T20:39:09.675342+0000', 'last_peered': '2024-09-10T20:39:09.675342+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:39:09.675342+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1503232, 'num_objects': 367, 'num_object_clones': 0, 'num_object_copies': 734, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3005, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 367, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18065, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 523, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2638}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:39:12.207 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:39:12.207 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20401", 'reported_seq': 20622, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:10.060111+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:39:10.060111+0000', 'last_peered': '2024-09-10T20:39:10.060111+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:39:10.060111+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1348550, 'num_objects': 331, 'num_object_clones': 0, 'num_object_copies': 662, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 331, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20399, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3235}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:12.207 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:39:12.207 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "51'20237", 'reported_seq': 20433, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:09.704240+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:39:09.704240+0000', 'last_peered': '2024-09-10T20:39:09.704240+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:39:09.704240+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1343488, 'num_objects': 328, 'num_object_clones': 0, 'num_object_copies': 656, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 328, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18065, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3278}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:39:12.207 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:39:12.207 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'20466", 'reported_seq': 20726, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:10.050144+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:39:10.050144+0000', 'last_peered': '2024-09-10T20:39:10.050144+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:39:10.050144+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1490944, 'num_objects': 364, 'num_object_clones': 0, 'num_object_copies': 728, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 364, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17189, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3228}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:12.207 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:39:12.208 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "51'20645", 'reported_seq': 20905, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:10.095687+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:39:10.095687+0000', 'last_peered': '2024-09-10T20:39:10.095687+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:39:10.095687+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1396736, 'num_objects': 341, 'num_object_clones': 0, 'num_object_copies': 682, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 341, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17472, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3314}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:12.208 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:39:12.919 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:12.919 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 28023/11522 objects degraded (243.213%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":664,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":5761,"data_bytes":23573972,"bytes_used":1520181248,"bytes_avail":288390111232,"bytes_total":289910292480,"degraded_objects":28023,"degraded_total":11522,"degraded_ratio":2.4321298385696926,"recovering_objects_per_sec":27,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":332,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":574},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:39:12.943 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:39:12.943 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:39:13.531 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:13.534 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:39:13.567 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":377,"stamp":"2024-09-10T20:39:13.052478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":21767636,"num_objects":5320,"num_object_clones":0,"num_object_copies":10640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5320,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":290101,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15128,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160706,"ondisk_log_size":160706,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1485532,"kb_used_data":217372,"kb_used_omap":12,"kb_used_meta":1268083,"kb_avail":281629988,"statfs":{"total":289910292480,"available":288389107712,"internally_reserved":0,"allocated":222588928,"data_stored":221730159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1298516994},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,1,7,2,13,10,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21925888,"num_objects":-5353,"num_object_clones":0,"num_object_copies":-10706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-219,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5353,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":219,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008154"},"pg_stats":[{"pgid":"2.f","version":"51'20335","reported_seq":20562,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:10.064956+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:10.064956+0000","last_peered":"2024-09-10T20:39:10.064956+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:10.064956+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1323008,"num_objects":323,"num_object_clones":0,"num_object_copies":646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20335,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3218}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20610","reported_seq":20734,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:09.731892+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:09.731892+0000","last_peered":"2024-09-10T20:39:09.731892+0000","last_clean":"2024-09-10T20:39:09.731892+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:09.731892+0000","last_undegraded":"2024-09-10T20:39:09.731892+0000","last_fullsized":"2024-09-10T20:39:09.731892+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1359872,"num_objects":332,"num_object_clones":0,"num_object_copies":664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18796,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20417","reported_seq":27780,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:12.082584+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:12.082584+0000","last_peered":"2024-09-10T20:39:12.082584+0000","last_clean":"2024-09-10T20:39:12.082584+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:12.082584+0000","last_undegraded":"2024-09-10T20:39:12.082584+0000","last_fullsized":"2024-09-10T20:39:12.082584+0000","mapping_epoch":44,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1220632,"num_objects":299,"num_object_clones":0,"num_object_copies":598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":299,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18545,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20242","reported_seq":20337,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:09.619977+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:09.619977+0000","last_peered":"2024-09-10T20:39:09.619977+0000","last_clean":"2024-09-10T20:39:09.619977+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:09.619977+0000","last_undegraded":"2024-09-10T20:39:09.619977+0000","last_fullsized":"2024-09-10T20:39:09.619977+0000","mapping_epoch":35,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1482752,"num_objects":362,"num_object_clones":0,"num_object_copies":724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20242,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20705","reported_seq":20845,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:10.099196+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:10.099196+0000","last_peered":"2024-09-10T20:39:10.099196+0000","last_clean":"2024-09-10T20:39:10.099196+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:10.099196+0000","last_undegraded":"2024-09-10T20:39:10.099196+0000","last_fullsized":"2024-09-10T20:39:10.099196+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1490944,"num_objects":364,"num_object_clones":0,"num_object_copies":728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17701,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20160","reported_seq":20417,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:10.050732+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:10.050732+0000","last_peered":"2024-09-10T20:39:10.050732+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:10.050732+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1273856,"num_objects":311,"num_object_clones":0,"num_object_copies":622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17024,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3250}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20278","reported_seq":20569,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:10.061390+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:10.061390+0000","last_peered":"2024-09-10T20:39:10.061390+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:10.061390+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1351680,"num_objects":330,"num_object_clones":0,"num_object_copies":660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3156}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20400","reported_seq":20538,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:10.034799+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:10.034799+0000","last_peered":"2024-09-10T20:39:10.034799+0000","last_clean":"2024-09-10T20:39:10.034799+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:10.034799+0000","last_undegraded":"2024-09-10T20:39:10.034799+0000","last_fullsized":"2024-09-10T20:39:10.034799+0000","mapping_epoch":34,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376256,"num_objects":336,"num_object_clones":0,"num_object_copies":672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17471,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.652747+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:38:50.652747+0000","last_peered":"2024-09-10T20:38:50.652747+0000","last_clean":"2024-09-10T20:38:50.652747+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:38:50.652747+0000","last_undegraded":"2024-09-10T20:38:50.652747+0000","last_fullsized":"2024-09-10T20:38:50.652747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20072","reported_seq":21266,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:09.675342+0000","last_change":"2024-09-10T20:38:49.349021+0000","last_active":"2024-09-10T20:39:09.675342+0000","last_peered":"2024-09-10T20:39:09.675342+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:39:09.675342+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1503232,"num_objects":367,"num_object_clones":0,"num_object_copies":734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3005,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":523,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2638}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.654924+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:38:50.654924+0000","last_peered":"2024-09-10T20:38:50.654924+0000","last_clean":"2024-09-10T20:38:50.654924+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:38:50.654924+0000","last_undegraded":"2024-09-10T20:38:50.654924+0000","last_fullsized":"2024-09-10T20:38:50.654924+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20401","reported_seq":20622,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:10.060111+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:10.060111+0000","last_peered":"2024-09-10T20:39:10.060111+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:10.060111+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1348550,"num_objects":331,"num_object_clones":0,"num_object_copies":662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20399,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3235}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.648348+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:38:50.648348+0000","last_peered":"2024-09-10T20:38:50.648348+0000","last_clean":"2024-09-10T20:38:50.648348+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:38:50.648348+0000","last_undegraded":"2024-09-10T20:38:50.648348+0000","last_fullsized":"2024-09-10T20:38:50.648348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20237","reported_seq":20433,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:09.704240+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:39:09.704240+0000","last_peered":"2024-09-10T20:39:09.704240+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:39:09.704240+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1343488,"num_objects":328,"num_object_clones":0,"num_object_copies":656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3278}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651037+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:38:50.651037+0000","last_peered":"2024-09-10T20:38:50.651037+0000","last_clean":"2024-09-10T20:38:50.651037+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:38:50.651037+0000","last_undegraded":"2024-09-10T20:38:50.651037+0000","last_fullsized":"2024-09-10T20:38:50.651037+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20298","reported_seq":27673,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:09.715212+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:09.715212+0000","last_peered":"2024-09-10T20:39:09.715212+0000","last_clean":"2024-09-10T20:39:09.715212+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:09.715212+0000","last_undegraded":"2024-09-10T20:39:09.715212+0000","last_fullsized":"2024-09-10T20:39:09.715212+0000","mapping_epoch":48,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1499136,"num_objects":366,"num_object_clones":0,"num_object_copies":732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":366,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18084,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20466","reported_seq":20726,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:10.050144+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:39:10.050144+0000","last_peered":"2024-09-10T20:39:10.050144+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:39:10.050144+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1490944,"num_objects":364,"num_object_clones":0,"num_object_copies":728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17189,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3228}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651168+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:38:50.651168+0000","last_peered":"2024-09-10T20:38:50.651168+0000","last_clean":"2024-09-10T20:38:50.651168+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:38:50.651168+0000","last_undegraded":"2024-09-10T20:38:50.651168+0000","last_fullsized":"2024-09-10T20:38:50.651168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20302","reported_seq":27536,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:12.056903+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:12.056903+0000","last_peered":"2024-09-10T20:39:12.056903+0000","last_clean":"2024-09-10T20:39:12.056903+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:12.056903+0000","last_undegraded":"2024-09-10T20:39:12.056903+0000","last_fullsized":"2024-09-10T20:39:12.056903+0000","mapping_epoch":50,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1155072,"num_objects":282,"num_object_clones":0,"num_object_copies":564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16996,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.656849+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:38:50.656849+0000","last_peered":"2024-09-10T20:38:50.656849+0000","last_clean":"2024-09-10T20:38:50.656849+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:38:50.656849+0000","last_undegraded":"2024-09-10T20:38:50.656849+0000","last_fullsized":"2024-09-10T20:38:50.656849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20645","reported_seq":20905,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:10.095687+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:10.095687+0000","last_peered":"2024-09-10T20:39:10.095687+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:10.095687+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1396736,"num_objects":341,"num_object_clones":0,"num_object_copies":682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17472,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3314}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651258+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:38:50.651258+0000","last_peered":"2024-09-10T20:38:50.651258+0000","last_clean":"2024-09-10T20:38:50.651258+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:38:50.651258+0000","last_undegraded":"2024-09-10T20:38:50.651258+0000","last_fullsized":"2024-09-10T20:38:50.651258+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651173+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:38:50.651173+0000","last_peered":"2024-09-10T20:38:50.651173+0000","last_clean":"2024-09-10T20:38:50.651173+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:38:50.651173+0000","last_undegraded":"2024-09-10T20:38:50.651173+0000","last_fullsized":"2024-09-10T20:38:50.651173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20235","reported_seq":27367,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:12.082324+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:12.082324+0000","last_peered":"2024-09-10T20:39:12.082324+0000","last_clean":"2024-09-10T20:39:12.082324+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:12.082324+0000","last_undegraded":"2024-09-10T20:39:12.082324+0000","last_fullsized":"2024-09-10T20:39:12.082324+0000","mapping_epoch":46,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1151459,"num_objects":282,"num_object_clones":0,"num_object_copies":564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16842,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":21767617,"num_objects":5318,"num_object_clones":0,"num_object_copies":10636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5318,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":290099,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15126,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":218632192,"data_stored":218573466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160703,"ondisk_log_size":160703,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182527,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518224,"kb_used_data":75644,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93853616,"statfs":{"total":96636764160,"available":96106102784,"internally_reserved":0,"allocated":77459456,"data_stored":77187803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,5,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986299,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480040,"kb_used_data":127640,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93891800,"statfs":{"total":96636764160,"available":96145203200,"internally_reserved":0,"allocated":130703360,"data_stored":130431712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":360836778},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,4,0,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659246,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487268,"kb_used_data":14088,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93884572,"statfs":{"total":96636764160,"available":96137801728,"internally_reserved":0,"allocated":14426112,"data_stored":14110644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,1,5,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":75493376,"data_stored":75489763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":13684736,"data_stored":13637308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":129454080,"data_stored":129446395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:39:16.570 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:39:17.137 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:17.137 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:39:17.160 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":379,"stamp":"2024-09-10T20:39:17.053875+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13612500,"num_objects":3329,"num_object_clones":0,"num_object_copies":6658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27916,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3329,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":292092,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15235,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160797,"ondisk_log_size":160797,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1485660,"kb_used_data":204988,"kb_used_omap":12,"kb_used_meta":1280563,"kb_avail":281629860,"statfs":{"total":289910292480,"available":288388976640,"internally_reserved":0,"allocated":209907712,"data_stored":209048943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1311296514},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,6,19,10,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25292800,"num_objects":-6175,"num_object_clones":0,"num_object_copies":-12350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-209,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6175,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008272"},"pg_stats":[{"pgid":"2.f","version":"51'20467","reported_seq":20694,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:15.043031+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:15.043031+0000","last_peered":"2024-09-10T20:39:15.043031+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:15.043031+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":782336,"num_objects":191,"num_object_clones":0,"num_object_copies":382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20467,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3350}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20759","reported_seq":20884,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:14.695355+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:14.695355+0000","last_peered":"2024-09-10T20:39:14.695355+0000","last_clean":"2024-09-10T20:39:14.695355+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:14.695355+0000","last_undegraded":"2024-09-10T20:39:14.695355+0000","last_fullsized":"2024-09-10T20:39:14.695355+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":749568,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18945,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20417","reported_seq":27780,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:12.082584+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:12.082584+0000","last_peered":"2024-09-10T20:39:12.082584+0000","last_clean":"2024-09-10T20:39:12.082584+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:12.082584+0000","last_undegraded":"2024-09-10T20:39:12.082584+0000","last_fullsized":"2024-09-10T20:39:12.082584+0000","mapping_epoch":44,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1220632,"num_objects":299,"num_object_clones":0,"num_object_copies":598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":299,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18545,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20391","reported_seq":20486,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:14.730100+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:14.730100+0000","last_peered":"2024-09-10T20:39:14.730100+0000","last_clean":"2024-09-10T20:39:14.730100+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:14.730100+0000","last_undegraded":"2024-09-10T20:39:14.730100+0000","last_fullsized":"2024-09-10T20:39:14.730100+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":872448,"num_objects":213,"num_object_clones":0,"num_object_copies":426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":213,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20391,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20886","reported_seq":21027,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:15.063775+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:15.063775+0000","last_peered":"2024-09-10T20:39:15.063775+0000","last_clean":"2024-09-10T20:39:15.063775+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:15.063775+0000","last_undegraded":"2024-09-10T20:39:15.063775+0000","last_fullsized":"2024-09-10T20:39:15.063775+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":749568,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17882,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20294","reported_seq":20551,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:15.104125+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:15.104125+0000","last_peered":"2024-09-10T20:39:15.104125+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:15.104125+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":724992,"num_objects":177,"num_object_clones":0,"num_object_copies":354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17158,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3384}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20425","reported_seq":20716,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:15.097807+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:15.097807+0000","last_peered":"2024-09-10T20:39:15.097807+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:15.097807+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":749568,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17020,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20552","reported_seq":20691,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:15.100265+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:15.100265+0000","last_peered":"2024-09-10T20:39:15.100265+0000","last_clean":"2024-09-10T20:39:15.100265+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:15.100265+0000","last_undegraded":"2024-09-10T20:39:15.100265+0000","last_fullsized":"2024-09-10T20:39:15.100265+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":753664,"num_objects":184,"num_object_clones":0,"num_object_copies":368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17623,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.652747+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:38:50.652747+0000","last_peered":"2024-09-10T20:38:50.652747+0000","last_clean":"2024-09-10T20:38:50.652747+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:38:50.652747+0000","last_undegraded":"2024-09-10T20:38:50.652747+0000","last_fullsized":"2024-09-10T20:38:50.652747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20221","reported_seq":21629,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:14.730423+0000","last_change":"2024-09-10T20:38:49.349021+0000","last_active":"2024-09-10T20:39:14.730423+0000","last_peered":"2024-09-10T20:39:14.730423+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:39:14.730423+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":892928,"num_objects":218,"num_object_clones":0,"num_object_copies":436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2898,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18214,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":630,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2680}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.654924+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:38:50.654924+0000","last_peered":"2024-09-10T20:38:50.654924+0000","last_clean":"2024-09-10T20:38:50.654924+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:38:50.654924+0000","last_undegraded":"2024-09-10T20:38:50.654924+0000","last_fullsized":"2024-09-10T20:38:50.654924+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20553","reported_seq":20774,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:15.096555+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:15.096555+0000","last_peered":"2024-09-10T20:39:15.096555+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:15.096555+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":725958,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20551,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3387}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.648348+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:38:50.648348+0000","last_peered":"2024-09-10T20:38:50.648348+0000","last_clean":"2024-09-10T20:38:50.648348+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:38:50.648348+0000","last_undegraded":"2024-09-10T20:38:50.648348+0000","last_fullsized":"2024-09-10T20:38:50.648348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20384","reported_seq":20581,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:14.729214+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:39:14.729214+0000","last_peered":"2024-09-10T20:39:14.729214+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:39:14.729214+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":741376,"num_objects":181,"num_object_clones":0,"num_object_copies":362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18212,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3425}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651037+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:38:50.651037+0000","last_peered":"2024-09-10T20:38:50.651037+0000","last_clean":"2024-09-10T20:38:50.651037+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:38:50.651037+0000","last_undegraded":"2024-09-10T20:38:50.651037+0000","last_fullsized":"2024-09-10T20:38:50.651037+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20475","reported_seq":27849,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:14.639917+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:14.639917+0000","last_peered":"2024-09-10T20:39:14.639917+0000","last_clean":"2024-09-10T20:39:14.639917+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:14.639917+0000","last_undegraded":"2024-09-10T20:39:14.639917+0000","last_fullsized":"2024-09-10T20:39:14.639917+0000","mapping_epoch":48,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":774144,"num_objects":189,"num_object_clones":0,"num_object_copies":378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18261,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20628","reported_seq":20888,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:15.139312+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:39:15.139312+0000","last_peered":"2024-09-10T20:39:15.139312+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:39:15.139312+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":827392,"num_objects":202,"num_object_clones":0,"num_object_copies":404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17351,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3390}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651168+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:38:50.651168+0000","last_peered":"2024-09-10T20:38:50.651168+0000","last_clean":"2024-09-10T20:38:50.651168+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:38:50.651168+0000","last_undegraded":"2024-09-10T20:38:50.651168+0000","last_fullsized":"2024-09-10T20:38:50.651168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20302","reported_seq":27536,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:12.056903+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:12.056903+0000","last_peered":"2024-09-10T20:39:12.056903+0000","last_clean":"2024-09-10T20:39:12.056903+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:12.056903+0000","last_undegraded":"2024-09-10T20:39:12.056903+0000","last_fullsized":"2024-09-10T20:39:12.056903+0000","mapping_epoch":50,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1155072,"num_objects":282,"num_object_clones":0,"num_object_copies":564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16996,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.656849+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:38:50.656849+0000","last_peered":"2024-09-10T20:38:50.656849+0000","last_clean":"2024-09-10T20:38:50.656849+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:38:50.656849+0000","last_undegraded":"2024-09-10T20:38:50.656849+0000","last_fullsized":"2024-09-10T20:38:50.656849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20805","reported_seq":21067,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:15.135275+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:15.135275+0000","last_peered":"2024-09-10T20:39:15.135275+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:15.135275+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":741376,"num_objects":181,"num_object_clones":0,"num_object_copies":362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17632,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3474}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651258+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:38:50.651258+0000","last_peered":"2024-09-10T20:38:50.651258+0000","last_clean":"2024-09-10T20:38:50.651258+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:38:50.651258+0000","last_undegraded":"2024-09-10T20:38:50.651258+0000","last_fullsized":"2024-09-10T20:38:50.651258+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651173+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:38:50.651173+0000","last_peered":"2024-09-10T20:38:50.651173+0000","last_clean":"2024-09-10T20:38:50.651173+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:38:50.651173+0000","last_undegraded":"2024-09-10T20:38:50.651173+0000","last_fullsized":"2024-09-10T20:38:50.651173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20235","reported_seq":27367,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:12.082324+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:12.082324+0000","last_peered":"2024-09-10T20:39:12.082324+0000","last_clean":"2024-09-10T20:39:12.082324+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:12.082324+0000","last_undegraded":"2024-09-10T20:39:12.082324+0000","last_fullsized":"2024-09-10T20:39:12.082324+0000","mapping_epoch":46,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1151459,"num_objects":282,"num_object_clones":0,"num_object_copies":564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16842,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":13612481,"num_objects":3327,"num_object_clones":0,"num_object_copies":6654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27916,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3327,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":292090,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15233,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":205893632,"data_stored":205834906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160794,"ondisk_log_size":160794,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182528,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519156,"kb_used_data":68232,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93852684,"statfs":{"total":96636764160,"available":96105148416,"internally_reserved":0,"allocated":69869568,"data_stored":69597915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":461696682},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,11,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986300,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479236,"kb_used_data":122668,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93892604,"statfs":{"total":96636764160,"available":96146026496,"internally_reserved":0,"allocated":125612032,"data_stored":125340384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365096618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659246,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487268,"kb_used_data":14088,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93884572,"statfs":{"total":96636764160,"available":96137801728,"internally_reserved":0,"allocated":14426112,"data_stored":14110644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,1,5,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67903488,"data_stored":67899875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":13684736,"data_stored":13637308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":124305408,"data_stored":124297723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:39:17.162 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:39:17.718 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:17.718 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:39:17.744 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":379,"stamp":"2024-09-10T20:39:17.053875+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13612500,"num_objects":3329,"num_object_clones":0,"num_object_copies":6658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27916,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3329,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":292092,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15235,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160797,"ondisk_log_size":160797,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1485660,"kb_used_data":204988,"kb_used_omap":12,"kb_used_meta":1280563,"kb_avail":281629860,"statfs":{"total":289910292480,"available":288388976640,"internally_reserved":0,"allocated":209907712,"data_stored":209048943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1311296514},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,6,19,10,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25292800,"num_objects":-6175,"num_object_clones":0,"num_object_copies":-12350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-209,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6175,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008272"},"pg_stats":[{"pgid":"2.f","version":"51'20467","reported_seq":20694,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:15.043031+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:15.043031+0000","last_peered":"2024-09-10T20:39:15.043031+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:15.043031+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":782336,"num_objects":191,"num_object_clones":0,"num_object_copies":382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20467,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3350}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20759","reported_seq":20884,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:14.695355+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:14.695355+0000","last_peered":"2024-09-10T20:39:14.695355+0000","last_clean":"2024-09-10T20:39:14.695355+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:14.695355+0000","last_undegraded":"2024-09-10T20:39:14.695355+0000","last_fullsized":"2024-09-10T20:39:14.695355+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":749568,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18945,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20417","reported_seq":27780,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:12.082584+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:12.082584+0000","last_peered":"2024-09-10T20:39:12.082584+0000","last_clean":"2024-09-10T20:39:12.082584+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:12.082584+0000","last_undegraded":"2024-09-10T20:39:12.082584+0000","last_fullsized":"2024-09-10T20:39:12.082584+0000","mapping_epoch":44,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1220632,"num_objects":299,"num_object_clones":0,"num_object_copies":598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":299,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18545,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20391","reported_seq":20486,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:14.730100+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:14.730100+0000","last_peered":"2024-09-10T20:39:14.730100+0000","last_clean":"2024-09-10T20:39:14.730100+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:14.730100+0000","last_undegraded":"2024-09-10T20:39:14.730100+0000","last_fullsized":"2024-09-10T20:39:14.730100+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":872448,"num_objects":213,"num_object_clones":0,"num_object_copies":426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":213,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20391,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20886","reported_seq":21027,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:15.063775+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:15.063775+0000","last_peered":"2024-09-10T20:39:15.063775+0000","last_clean":"2024-09-10T20:39:15.063775+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:15.063775+0000","last_undegraded":"2024-09-10T20:39:15.063775+0000","last_fullsized":"2024-09-10T20:39:15.063775+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":749568,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17882,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20294","reported_seq":20551,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:15.104125+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:15.104125+0000","last_peered":"2024-09-10T20:39:15.104125+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:15.104125+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":724992,"num_objects":177,"num_object_clones":0,"num_object_copies":354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17158,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3384}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20425","reported_seq":20716,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:15.097807+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:15.097807+0000","last_peered":"2024-09-10T20:39:15.097807+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:15.097807+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":749568,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17020,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20552","reported_seq":20691,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:15.100265+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:15.100265+0000","last_peered":"2024-09-10T20:39:15.100265+0000","last_clean":"2024-09-10T20:39:15.100265+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:15.100265+0000","last_undegraded":"2024-09-10T20:39:15.100265+0000","last_fullsized":"2024-09-10T20:39:15.100265+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":753664,"num_objects":184,"num_object_clones":0,"num_object_copies":368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17623,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.652747+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:38:50.652747+0000","last_peered":"2024-09-10T20:38:50.652747+0000","last_clean":"2024-09-10T20:38:50.652747+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:38:50.652747+0000","last_undegraded":"2024-09-10T20:38:50.652747+0000","last_fullsized":"2024-09-10T20:38:50.652747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20221","reported_seq":21629,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:14.730423+0000","last_change":"2024-09-10T20:38:49.349021+0000","last_active":"2024-09-10T20:39:14.730423+0000","last_peered":"2024-09-10T20:39:14.730423+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:39:14.730423+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":892928,"num_objects":218,"num_object_clones":0,"num_object_copies":436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2898,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18214,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":630,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2680}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.654924+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:38:50.654924+0000","last_peered":"2024-09-10T20:38:50.654924+0000","last_clean":"2024-09-10T20:38:50.654924+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:38:50.654924+0000","last_undegraded":"2024-09-10T20:38:50.654924+0000","last_fullsized":"2024-09-10T20:38:50.654924+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20553","reported_seq":20774,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:15.096555+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:15.096555+0000","last_peered":"2024-09-10T20:39:15.096555+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:15.096555+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":725958,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20551,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3387}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.648348+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:38:50.648348+0000","last_peered":"2024-09-10T20:38:50.648348+0000","last_clean":"2024-09-10T20:38:50.648348+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:38:50.648348+0000","last_undegraded":"2024-09-10T20:38:50.648348+0000","last_fullsized":"2024-09-10T20:38:50.648348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20384","reported_seq":20581,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:14.729214+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:39:14.729214+0000","last_peered":"2024-09-10T20:39:14.729214+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:39:14.729214+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":741376,"num_objects":181,"num_object_clones":0,"num_object_copies":362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18212,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3425}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651037+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:38:50.651037+0000","last_peered":"2024-09-10T20:38:50.651037+0000","last_clean":"2024-09-10T20:38:50.651037+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:38:50.651037+0000","last_undegraded":"2024-09-10T20:38:50.651037+0000","last_fullsized":"2024-09-10T20:38:50.651037+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20475","reported_seq":27849,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:14.639917+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:14.639917+0000","last_peered":"2024-09-10T20:39:14.639917+0000","last_clean":"2024-09-10T20:39:14.639917+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:14.639917+0000","last_undegraded":"2024-09-10T20:39:14.639917+0000","last_fullsized":"2024-09-10T20:39:14.639917+0000","mapping_epoch":48,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":774144,"num_objects":189,"num_object_clones":0,"num_object_copies":378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18261,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20628","reported_seq":20888,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:15.139312+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:39:15.139312+0000","last_peered":"2024-09-10T20:39:15.139312+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:39:15.139312+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":827392,"num_objects":202,"num_object_clones":0,"num_object_copies":404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17351,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3390}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651168+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:38:50.651168+0000","last_peered":"2024-09-10T20:38:50.651168+0000","last_clean":"2024-09-10T20:38:50.651168+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:38:50.651168+0000","last_undegraded":"2024-09-10T20:38:50.651168+0000","last_fullsized":"2024-09-10T20:38:50.651168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20302","reported_seq":27536,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:12.056903+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:12.056903+0000","last_peered":"2024-09-10T20:39:12.056903+0000","last_clean":"2024-09-10T20:39:12.056903+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:12.056903+0000","last_undegraded":"2024-09-10T20:39:12.056903+0000","last_fullsized":"2024-09-10T20:39:12.056903+0000","mapping_epoch":50,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1155072,"num_objects":282,"num_object_clones":0,"num_object_copies":564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16996,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.656849+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:38:50.656849+0000","last_peered":"2024-09-10T20:38:50.656849+0000","last_clean":"2024-09-10T20:38:50.656849+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:38:50.656849+0000","last_undegraded":"2024-09-10T20:38:50.656849+0000","last_fullsized":"2024-09-10T20:38:50.656849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20805","reported_seq":21067,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:15.135275+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:15.135275+0000","last_peered":"2024-09-10T20:39:15.135275+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:15.135275+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":741376,"num_objects":181,"num_object_clones":0,"num_object_copies":362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17632,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3474}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651258+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:38:50.651258+0000","last_peered":"2024-09-10T20:38:50.651258+0000","last_clean":"2024-09-10T20:38:50.651258+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:38:50.651258+0000","last_undegraded":"2024-09-10T20:38:50.651258+0000","last_fullsized":"2024-09-10T20:38:50.651258+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651173+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:38:50.651173+0000","last_peered":"2024-09-10T20:38:50.651173+0000","last_clean":"2024-09-10T20:38:50.651173+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:38:50.651173+0000","last_undegraded":"2024-09-10T20:38:50.651173+0000","last_fullsized":"2024-09-10T20:38:50.651173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20235","reported_seq":27367,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:12.082324+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:12.082324+0000","last_peered":"2024-09-10T20:39:12.082324+0000","last_clean":"2024-09-10T20:39:12.082324+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:12.082324+0000","last_undegraded":"2024-09-10T20:39:12.082324+0000","last_fullsized":"2024-09-10T20:39:12.082324+0000","mapping_epoch":46,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1151459,"num_objects":282,"num_object_clones":0,"num_object_copies":564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16842,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":13612481,"num_objects":3327,"num_object_clones":0,"num_object_copies":6654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27916,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3327,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":292090,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15233,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":205893632,"data_stored":205834906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160794,"ondisk_log_size":160794,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182528,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519156,"kb_used_data":68232,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93852684,"statfs":{"total":96636764160,"available":96105148416,"internally_reserved":0,"allocated":69869568,"data_stored":69597915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":461696682},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,11,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986300,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479236,"kb_used_data":122668,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93892604,"statfs":{"total":96636764160,"available":96146026496,"internally_reserved":0,"allocated":125612032,"data_stored":125340384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365096618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659246,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487268,"kb_used_data":14088,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93884572,"statfs":{"total":96636764160,"available":96137801728,"internally_reserved":0,"allocated":14426112,"data_stored":14110644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,1,5,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67903488,"data_stored":67899875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":13684736,"data_stored":13637308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":124305408,"data_stored":124297723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:39:17.745 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:39:17.745 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'20467", 'reported_seq': 20694, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:15.043031+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:39:15.043031+0000', 'last_peered': '2024-09-10T20:39:15.043031+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:39:15.043031+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 782336, 'num_objects': 191, 'num_object_clones': 0, 'num_object_copies': 382, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 191, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20467, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3350}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:17.746 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:39:17.746 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20294", 'reported_seq': 20551, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:15.104125+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:39:15.104125+0000', 'last_peered': '2024-09-10T20:39:15.104125+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:39:15.104125+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 724992, 'num_objects': 177, 'num_object_clones': 0, 'num_object_copies': 354, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 177, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17158, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3384}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:17.746 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:39:17.746 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'20425", 'reported_seq': 20716, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:15.097807+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:39:15.097807+0000', 'last_peered': '2024-09-10T20:39:15.097807+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:39:15.097807+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 749568, 'num_objects': 183, 'num_object_clones': 0, 'num_object_copies': 366, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 183, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17020, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3303}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:17.746 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:39:17.746 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20221", 'reported_seq': 21629, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:14.730423+0000', 'last_change': '2024-09-10T20:38:49.349021+0000', 'last_active': '2024-09-10T20:39:14.730423+0000', 'last_peered': '2024-09-10T20:39:14.730423+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:39:14.730423+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 892928, 'num_objects': 218, 'num_object_clones': 0, 'num_object_copies': 436, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2898, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 218, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18214, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 630, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2680}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:39:17.746 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:39:17.746 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20553", 'reported_seq': 20774, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:15.096555+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:39:15.096555+0000', 'last_peered': '2024-09-10T20:39:15.096555+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:39:15.096555+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 725958, 'num_objects': 179, 'num_object_clones': 0, 'num_object_copies': 358, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 179, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20551, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3387}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:17.746 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:39:17.746 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "51'20384", 'reported_seq': 20581, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:14.729214+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:39:14.729214+0000', 'last_peered': '2024-09-10T20:39:14.729214+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:39:14.729214+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "45'10300", 'ondisk_log_start': "45'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 741376, 'num_objects': 181, 'num_object_clones': 0, 'num_object_copies': 362, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 181, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18212, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3425}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:39:17.746 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:39:17.747 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'20628", 'reported_seq': 20888, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:15.139312+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:39:15.139312+0000', 'last_peered': '2024-09-10T20:39:15.139312+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:39:15.139312+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 827392, 'num_objects': 202, 'num_object_clones': 0, 'num_object_copies': 404, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 202, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17351, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3390}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:17.747 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:39:17.747 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "51'20805", 'reported_seq': 21067, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:15.135275+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:39:15.135275+0000', 'last_peered': '2024-09-10T20:39:15.135275+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:39:15.135275+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 741376, 'num_objects': 181, 'num_object_clones': 0, 'num_object_copies': 362, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 181, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17632, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3474}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:17.747 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:39:18.425 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:18.426 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 27916/6658 objects degraded (419.285%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":670,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3329,"data_bytes":13612500,"bytes_used":1521315840,"bytes_avail":288388976640,"bytes_total":289910292480,"degraded_objects":27916,"degraded_total":6658,"degraded_ratio":4.1928507059176932,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":209,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":514},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:39:18.452 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:39:18.453 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:39:19.043 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:19.044 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:39:19.068 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":379,"stamp":"2024-09-10T20:39:17.053875+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13612500,"num_objects":3329,"num_object_clones":0,"num_object_copies":6658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27916,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3329,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":292092,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15235,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160797,"ondisk_log_size":160797,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1485660,"kb_used_data":204988,"kb_used_omap":12,"kb_used_meta":1280563,"kb_avail":281629860,"statfs":{"total":289910292480,"available":288388976640,"internally_reserved":0,"allocated":209907712,"data_stored":209048943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1311296514},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,6,19,10,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25292800,"num_objects":-6175,"num_object_clones":0,"num_object_copies":-12350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-209,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6175,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008272"},"pg_stats":[{"pgid":"2.f","version":"51'20467","reported_seq":20694,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:15.043031+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:15.043031+0000","last_peered":"2024-09-10T20:39:15.043031+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:15.043031+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":782336,"num_objects":191,"num_object_clones":0,"num_object_copies":382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20467,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3350}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20759","reported_seq":20884,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:14.695355+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:14.695355+0000","last_peered":"2024-09-10T20:39:14.695355+0000","last_clean":"2024-09-10T20:39:14.695355+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:14.695355+0000","last_undegraded":"2024-09-10T20:39:14.695355+0000","last_fullsized":"2024-09-10T20:39:14.695355+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":749568,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18945,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20417","reported_seq":27780,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:12.082584+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:12.082584+0000","last_peered":"2024-09-10T20:39:12.082584+0000","last_clean":"2024-09-10T20:39:12.082584+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:12.082584+0000","last_undegraded":"2024-09-10T20:39:12.082584+0000","last_fullsized":"2024-09-10T20:39:12.082584+0000","mapping_epoch":44,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1220632,"num_objects":299,"num_object_clones":0,"num_object_copies":598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":299,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18545,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20391","reported_seq":20486,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:14.730100+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:14.730100+0000","last_peered":"2024-09-10T20:39:14.730100+0000","last_clean":"2024-09-10T20:39:14.730100+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:14.730100+0000","last_undegraded":"2024-09-10T20:39:14.730100+0000","last_fullsized":"2024-09-10T20:39:14.730100+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":872448,"num_objects":213,"num_object_clones":0,"num_object_copies":426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":213,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20391,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20886","reported_seq":21027,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:15.063775+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:15.063775+0000","last_peered":"2024-09-10T20:39:15.063775+0000","last_clean":"2024-09-10T20:39:15.063775+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:15.063775+0000","last_undegraded":"2024-09-10T20:39:15.063775+0000","last_fullsized":"2024-09-10T20:39:15.063775+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":749568,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17882,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20294","reported_seq":20551,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:15.104125+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:15.104125+0000","last_peered":"2024-09-10T20:39:15.104125+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:15.104125+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":724992,"num_objects":177,"num_object_clones":0,"num_object_copies":354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17158,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3384}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20425","reported_seq":20716,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:15.097807+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:15.097807+0000","last_peered":"2024-09-10T20:39:15.097807+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:15.097807+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":749568,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17020,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20552","reported_seq":20691,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:15.100265+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:15.100265+0000","last_peered":"2024-09-10T20:39:15.100265+0000","last_clean":"2024-09-10T20:39:15.100265+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:15.100265+0000","last_undegraded":"2024-09-10T20:39:15.100265+0000","last_fullsized":"2024-09-10T20:39:15.100265+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":753664,"num_objects":184,"num_object_clones":0,"num_object_copies":368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17623,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.652747+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:38:50.652747+0000","last_peered":"2024-09-10T20:38:50.652747+0000","last_clean":"2024-09-10T20:38:50.652747+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:38:50.652747+0000","last_undegraded":"2024-09-10T20:38:50.652747+0000","last_fullsized":"2024-09-10T20:38:50.652747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20221","reported_seq":21629,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:14.730423+0000","last_change":"2024-09-10T20:38:49.349021+0000","last_active":"2024-09-10T20:39:14.730423+0000","last_peered":"2024-09-10T20:39:14.730423+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:39:14.730423+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":892928,"num_objects":218,"num_object_clones":0,"num_object_copies":436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2898,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18214,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":630,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2680}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.654924+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:38:50.654924+0000","last_peered":"2024-09-10T20:38:50.654924+0000","last_clean":"2024-09-10T20:38:50.654924+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:38:50.654924+0000","last_undegraded":"2024-09-10T20:38:50.654924+0000","last_fullsized":"2024-09-10T20:38:50.654924+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20553","reported_seq":20774,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:15.096555+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:15.096555+0000","last_peered":"2024-09-10T20:39:15.096555+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:15.096555+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":725958,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20551,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3387}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.648348+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:38:50.648348+0000","last_peered":"2024-09-10T20:38:50.648348+0000","last_clean":"2024-09-10T20:38:50.648348+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:38:50.648348+0000","last_undegraded":"2024-09-10T20:38:50.648348+0000","last_fullsized":"2024-09-10T20:38:50.648348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20384","reported_seq":20581,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:14.729214+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:39:14.729214+0000","last_peered":"2024-09-10T20:39:14.729214+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:39:14.729214+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":741376,"num_objects":181,"num_object_clones":0,"num_object_copies":362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18212,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3425}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651037+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:38:50.651037+0000","last_peered":"2024-09-10T20:38:50.651037+0000","last_clean":"2024-09-10T20:38:50.651037+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:38:50.651037+0000","last_undegraded":"2024-09-10T20:38:50.651037+0000","last_fullsized":"2024-09-10T20:38:50.651037+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20475","reported_seq":27849,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:14.639917+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:14.639917+0000","last_peered":"2024-09-10T20:39:14.639917+0000","last_clean":"2024-09-10T20:39:14.639917+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:14.639917+0000","last_undegraded":"2024-09-10T20:39:14.639917+0000","last_fullsized":"2024-09-10T20:39:14.639917+0000","mapping_epoch":48,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":774144,"num_objects":189,"num_object_clones":0,"num_object_copies":378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18261,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20628","reported_seq":20888,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:15.139312+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:39:15.139312+0000","last_peered":"2024-09-10T20:39:15.139312+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:39:15.139312+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":827392,"num_objects":202,"num_object_clones":0,"num_object_copies":404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17351,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3390}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651168+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:38:50.651168+0000","last_peered":"2024-09-10T20:38:50.651168+0000","last_clean":"2024-09-10T20:38:50.651168+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:38:50.651168+0000","last_undegraded":"2024-09-10T20:38:50.651168+0000","last_fullsized":"2024-09-10T20:38:50.651168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20302","reported_seq":27536,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:12.056903+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:12.056903+0000","last_peered":"2024-09-10T20:39:12.056903+0000","last_clean":"2024-09-10T20:39:12.056903+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:12.056903+0000","last_undegraded":"2024-09-10T20:39:12.056903+0000","last_fullsized":"2024-09-10T20:39:12.056903+0000","mapping_epoch":50,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1155072,"num_objects":282,"num_object_clones":0,"num_object_copies":564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16996,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.656849+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:38:50.656849+0000","last_peered":"2024-09-10T20:38:50.656849+0000","last_clean":"2024-09-10T20:38:50.656849+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:38:50.656849+0000","last_undegraded":"2024-09-10T20:38:50.656849+0000","last_fullsized":"2024-09-10T20:38:50.656849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20805","reported_seq":21067,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:15.135275+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:15.135275+0000","last_peered":"2024-09-10T20:39:15.135275+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:15.135275+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":741376,"num_objects":181,"num_object_clones":0,"num_object_copies":362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17632,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3474}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651258+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:38:50.651258+0000","last_peered":"2024-09-10T20:38:50.651258+0000","last_clean":"2024-09-10T20:38:50.651258+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:38:50.651258+0000","last_undegraded":"2024-09-10T20:38:50.651258+0000","last_fullsized":"2024-09-10T20:38:50.651258+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651173+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:38:50.651173+0000","last_peered":"2024-09-10T20:38:50.651173+0000","last_clean":"2024-09-10T20:38:50.651173+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:38:50.651173+0000","last_undegraded":"2024-09-10T20:38:50.651173+0000","last_fullsized":"2024-09-10T20:38:50.651173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20235","reported_seq":27367,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:12.082324+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:12.082324+0000","last_peered":"2024-09-10T20:39:12.082324+0000","last_clean":"2024-09-10T20:39:12.082324+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:12.082324+0000","last_undegraded":"2024-09-10T20:39:12.082324+0000","last_fullsized":"2024-09-10T20:39:12.082324+0000","mapping_epoch":46,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1151459,"num_objects":282,"num_object_clones":0,"num_object_copies":564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16842,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":13612481,"num_objects":3327,"num_object_clones":0,"num_object_copies":6654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27916,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3327,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":292090,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15233,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":205893632,"data_stored":205834906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160794,"ondisk_log_size":160794,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182528,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519156,"kb_used_data":68232,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93852684,"statfs":{"total":96636764160,"available":96105148416,"internally_reserved":0,"allocated":69869568,"data_stored":69597915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":461696682},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,11,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986300,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479236,"kb_used_data":122668,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93892604,"statfs":{"total":96636764160,"available":96146026496,"internally_reserved":0,"allocated":125612032,"data_stored":125340384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365096618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659246,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487268,"kb_used_data":14088,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93884572,"statfs":{"total":96636764160,"available":96137801728,"internally_reserved":0,"allocated":14426112,"data_stored":14110644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,1,5,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67903488,"data_stored":67899875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":13684736,"data_stored":13637308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":124305408,"data_stored":124297723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:39:21.297 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout:Removed 165517 objects 2024-09-10T20:39:21.297 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi131.stdout:Clean up completed and total clean up time :353.21 2024-09-10T20:39:22.070 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:39:22.380 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:22.381 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:39:22.392 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":381,"stamp":"2024-09-10T20:39:21.054887+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3601876,"num_objects":885,"num_object_clones":0,"num_object_copies":1770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27816,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":885,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":294536,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15335,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160741,"ondisk_log_size":160741,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1461720,"kb_used_data":185228,"kb_used_omap":12,"kb_used_meta":1276403,"kb_avail":281653800,"statfs":{"total":289910292480,"available":288413491200,"internally_reserved":0,"allocated":189673472,"data_stored":188814703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1307036674},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,4,10,11,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28278784,"num_objects":-6904,"num_object_clones":0,"num_object_copies":-13808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-309,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6904,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004129"},"pg_stats":[{"pgid":"2.f","version":"51'20620","reported_seq":20847,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:20.171996+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:20.171996+0000","last_peered":"2024-09-10T20:39:20.171996+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:20.171996+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":155648,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20620,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3503}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20904","reported_seq":21029,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:19.747642+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:19.747642+0000","last_peered":"2024-09-10T20:39:19.747642+0000","last_clean":"2024-09-10T20:39:19.747642+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:19.747642+0000","last_undegraded":"2024-09-10T20:39:19.747642+0000","last_fullsized":"2024-09-10T20:39:19.747642+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":155648,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19090,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20597","reported_seq":27960,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:17.149339+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:17.149339+0000","last_peered":"2024-09-10T20:39:17.149339+0000","last_clean":"2024-09-10T20:39:17.149339+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:17.149339+0000","last_undegraded":"2024-09-10T20:39:17.149339+0000","last_fullsized":"2024-09-10T20:39:17.149339+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483352,"num_objects":119,"num_object_clones":0,"num_object_copies":238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":119,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18725,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20545","reported_seq":20640,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:19.727329+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:19.727329+0000","last_peered":"2024-09-10T20:39:19.727329+0000","last_clean":"2024-09-10T20:39:19.727329+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:19.727329+0000","last_undegraded":"2024-09-10T20:39:19.727329+0000","last_fullsized":"2024-09-10T20:39:19.727329+0000","mapping_epoch":35,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":241664,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20545,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21026","reported_seq":21165,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:20.137258+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:20.137258+0000","last_peered":"2024-09-10T20:39:20.137258+0000","last_clean":"2024-09-10T20:39:20.137258+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:20.137258+0000","last_undegraded":"2024-09-10T20:39:20.137258+0000","last_fullsized":"2024-09-10T20:39:20.137258+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":176128,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18022,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20442","reported_seq":20699,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:20.121320+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:20.121320+0000","last_peered":"2024-09-10T20:39:20.121320+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:20.121320+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118784,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20576","reported_seq":20867,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:20.107854+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:20.107854+0000","last_peered":"2024-09-10T20:39:20.107854+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:20.107854+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131072,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17171,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3454}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20701","reported_seq":20840,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:20.148307+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:20.148307+0000","last_peered":"2024-09-10T20:39:20.148307+0000","last_clean":"2024-09-10T20:39:20.148307+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:20.148307+0000","last_undegraded":"2024-09-10T20:39:20.148307+0000","last_fullsized":"2024-09-10T20:39:20.148307+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":143360,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17772,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.652747+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:38:50.652747+0000","last_peered":"2024-09-10T20:38:50.652747+0000","last_clean":"2024-09-10T20:38:50.652747+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:38:50.652747+0000","last_undegraded":"2024-09-10T20:38:50.652747+0000","last_fullsized":"2024-09-10T20:38:50.652747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20382","reported_seq":21989,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:19.786334+0000","last_change":"2024-09-10T20:38:49.349021+0000","last_active":"2024-09-10T20:39:19.786334+0000","last_peered":"2024-09-10T20:39:19.786334+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:39:19.786334+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":233472,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2798,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18375,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":730,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2741}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.654924+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:38:50.654924+0000","last_peered":"2024-09-10T20:38:50.654924+0000","last_clean":"2024-09-10T20:38:50.654924+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:38:50.654924+0000","last_undegraded":"2024-09-10T20:38:50.654924+0000","last_fullsized":"2024-09-10T20:38:50.654924+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20698","reported_seq":20918,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:20.119844+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:20.119844+0000","last_peered":"2024-09-10T20:39:20.119844+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:20.119844+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":132038,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20696,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3532}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.648348+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:38:50.648348+0000","last_peered":"2024-09-10T20:38:50.648348+0000","last_clean":"2024-09-10T20:38:50.648348+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:38:50.648348+0000","last_undegraded":"2024-09-10T20:38:50.648348+0000","last_fullsized":"2024-09-10T20:38:50.648348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20505","reported_seq":20702,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:19.708891+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:39:19.708891+0000","last_peered":"2024-09-10T20:39:19.708891+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:39:19.708891+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":245760,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18333,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3546}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651037+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:38:50.651037+0000","last_peered":"2024-09-10T20:38:50.651037+0000","last_clean":"2024-09-10T20:38:50.651037+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:38:50.651037+0000","last_undegraded":"2024-09-10T20:38:50.651037+0000","last_fullsized":"2024-09-10T20:38:50.651037+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20619","reported_seq":27995,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:19.691907+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:19.691907+0000","last_peered":"2024-09-10T20:39:19.691907+0000","last_clean":"2024-09-10T20:39:19.691907+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:19.691907+0000","last_undegraded":"2024-09-10T20:39:19.691907+0000","last_fullsized":"2024-09-10T20:39:19.691907+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":184320,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18405,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20804","reported_seq":21066,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:20.105862+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:39:20.105862+0000","last_peered":"2024-09-10T20:39:20.105862+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:39:20.105862+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":106496,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17527,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651168+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:38:50.651168+0000","last_peered":"2024-09-10T20:38:50.651168+0000","last_clean":"2024-09-10T20:38:50.651168+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:38:50.651168+0000","last_undegraded":"2024-09-10T20:38:50.651168+0000","last_fullsized":"2024-09-10T20:38:50.651168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20468","reported_seq":27697,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:17.197108+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:17.197108+0000","last_peered":"2024-09-10T20:39:17.197108+0000","last_clean":"2024-09-10T20:39:17.197108+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:17.197108+0000","last_undegraded":"2024-09-10T20:39:17.197108+0000","last_fullsized":"2024-09-10T20:39:17.197108+0000","mapping_epoch":50,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":475136,"num_objects":116,"num_object_clones":0,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17162,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.656849+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:38:50.656849+0000","last_peered":"2024-09-10T20:38:50.656849+0000","last_clean":"2024-09-10T20:38:50.656849+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:38:50.656849+0000","last_undegraded":"2024-09-10T20:38:50.656849+0000","last_fullsized":"2024-09-10T20:38:50.656849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20957","reported_seq":21218,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:20.189578+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:20.189578+0000","last_peered":"2024-09-10T20:39:20.189578+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:20.189578+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118784,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17784,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651258+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:38:50.651258+0000","last_peered":"2024-09-10T20:38:50.651258+0000","last_clean":"2024-09-10T20:38:50.651258+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:38:50.651258+0000","last_undegraded":"2024-09-10T20:38:50.651258+0000","last_fullsized":"2024-09-10T20:38:50.651258+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651173+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:38:50.651173+0000","last_peered":"2024-09-10T20:38:50.651173+0000","last_clean":"2024-09-10T20:38:50.651173+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:38:50.651173+0000","last_undegraded":"2024-09-10T20:38:50.651173+0000","last_fullsized":"2024-09-10T20:38:50.651173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20394","reported_seq":27526,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:17.205124+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:17.205124+0000","last_peered":"2024-09-10T20:39:17.205124+0000","last_clean":"2024-09-10T20:39:17.205124+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:17.205124+0000","last_undegraded":"2024-09-10T20:39:17.205124+0000","last_fullsized":"2024-09-10T20:39:17.205124+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":500195,"num_objects":123,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":3601857,"num_objects":883,"num_object_clones":0,"num_object_copies":1766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27816,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":883,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":294534,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15333,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":185774080,"data_stored":185715354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160738,"ondisk_log_size":160738,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182529,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515908,"kb_used_data":60852,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93855932,"statfs":{"total":96636764160,"available":96108474368,"internally_reserved":0,"allocated":62312448,"data_stored":62040795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":465956522},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986301,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":478444,"kb_used_data":117716,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93893396,"statfs":{"total":96636764160,"available":96146837504,"internally_reserved":0,"allocated":120541184,"data_stored":120269536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369356458},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659247,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":467368,"kb_used_data":6660,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93904472,"statfs":{"total":96636764160,"available":96158179328,"internally_reserved":0,"allocated":6819840,"data_stored":6504372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":471723694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,3,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60428288,"data_stored":60424675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5898240,"data_stored":5850812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":119447552,"data_stored":119439867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:39:22.394 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:39:22.697 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:22.698 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:39:22.711 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":381,"stamp":"2024-09-10T20:39:21.054887+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3601876,"num_objects":885,"num_object_clones":0,"num_object_copies":1770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27816,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":885,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":294536,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15335,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160741,"ondisk_log_size":160741,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1461720,"kb_used_data":185228,"kb_used_omap":12,"kb_used_meta":1276403,"kb_avail":281653800,"statfs":{"total":289910292480,"available":288413491200,"internally_reserved":0,"allocated":189673472,"data_stored":188814703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1307036674},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,4,10,11,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28278784,"num_objects":-6904,"num_object_clones":0,"num_object_copies":-13808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-309,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6904,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004129"},"pg_stats":[{"pgid":"2.f","version":"51'20620","reported_seq":20847,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:20.171996+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:20.171996+0000","last_peered":"2024-09-10T20:39:20.171996+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:20.171996+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":155648,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20620,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3503}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20904","reported_seq":21029,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:19.747642+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:19.747642+0000","last_peered":"2024-09-10T20:39:19.747642+0000","last_clean":"2024-09-10T20:39:19.747642+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:19.747642+0000","last_undegraded":"2024-09-10T20:39:19.747642+0000","last_fullsized":"2024-09-10T20:39:19.747642+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":155648,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19090,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20597","reported_seq":27960,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:17.149339+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:17.149339+0000","last_peered":"2024-09-10T20:39:17.149339+0000","last_clean":"2024-09-10T20:39:17.149339+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:17.149339+0000","last_undegraded":"2024-09-10T20:39:17.149339+0000","last_fullsized":"2024-09-10T20:39:17.149339+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483352,"num_objects":119,"num_object_clones":0,"num_object_copies":238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":119,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18725,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20545","reported_seq":20640,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:19.727329+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:19.727329+0000","last_peered":"2024-09-10T20:39:19.727329+0000","last_clean":"2024-09-10T20:39:19.727329+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:19.727329+0000","last_undegraded":"2024-09-10T20:39:19.727329+0000","last_fullsized":"2024-09-10T20:39:19.727329+0000","mapping_epoch":35,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":241664,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20545,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21026","reported_seq":21165,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:20.137258+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:20.137258+0000","last_peered":"2024-09-10T20:39:20.137258+0000","last_clean":"2024-09-10T20:39:20.137258+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:20.137258+0000","last_undegraded":"2024-09-10T20:39:20.137258+0000","last_fullsized":"2024-09-10T20:39:20.137258+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":176128,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18022,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20442","reported_seq":20699,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:20.121320+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:20.121320+0000","last_peered":"2024-09-10T20:39:20.121320+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:20.121320+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118784,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20576","reported_seq":20867,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:20.107854+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:20.107854+0000","last_peered":"2024-09-10T20:39:20.107854+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:20.107854+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131072,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17171,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3454}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20701","reported_seq":20840,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:20.148307+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:20.148307+0000","last_peered":"2024-09-10T20:39:20.148307+0000","last_clean":"2024-09-10T20:39:20.148307+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:20.148307+0000","last_undegraded":"2024-09-10T20:39:20.148307+0000","last_fullsized":"2024-09-10T20:39:20.148307+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":143360,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17772,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.652747+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:38:50.652747+0000","last_peered":"2024-09-10T20:38:50.652747+0000","last_clean":"2024-09-10T20:38:50.652747+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:38:50.652747+0000","last_undegraded":"2024-09-10T20:38:50.652747+0000","last_fullsized":"2024-09-10T20:38:50.652747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20382","reported_seq":21989,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:19.786334+0000","last_change":"2024-09-10T20:38:49.349021+0000","last_active":"2024-09-10T20:39:19.786334+0000","last_peered":"2024-09-10T20:39:19.786334+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:39:19.786334+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":233472,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2798,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18375,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":730,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2741}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.654924+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:38:50.654924+0000","last_peered":"2024-09-10T20:38:50.654924+0000","last_clean":"2024-09-10T20:38:50.654924+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:38:50.654924+0000","last_undegraded":"2024-09-10T20:38:50.654924+0000","last_fullsized":"2024-09-10T20:38:50.654924+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20698","reported_seq":20918,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:20.119844+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:20.119844+0000","last_peered":"2024-09-10T20:39:20.119844+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:20.119844+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":132038,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20696,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3532}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.648348+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:38:50.648348+0000","last_peered":"2024-09-10T20:38:50.648348+0000","last_clean":"2024-09-10T20:38:50.648348+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:38:50.648348+0000","last_undegraded":"2024-09-10T20:38:50.648348+0000","last_fullsized":"2024-09-10T20:38:50.648348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20505","reported_seq":20702,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:19.708891+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:39:19.708891+0000","last_peered":"2024-09-10T20:39:19.708891+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:39:19.708891+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":245760,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18333,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3546}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651037+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:38:50.651037+0000","last_peered":"2024-09-10T20:38:50.651037+0000","last_clean":"2024-09-10T20:38:50.651037+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:38:50.651037+0000","last_undegraded":"2024-09-10T20:38:50.651037+0000","last_fullsized":"2024-09-10T20:38:50.651037+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20619","reported_seq":27995,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:19.691907+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:19.691907+0000","last_peered":"2024-09-10T20:39:19.691907+0000","last_clean":"2024-09-10T20:39:19.691907+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:19.691907+0000","last_undegraded":"2024-09-10T20:39:19.691907+0000","last_fullsized":"2024-09-10T20:39:19.691907+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":184320,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18405,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20804","reported_seq":21066,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:20.105862+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:39:20.105862+0000","last_peered":"2024-09-10T20:39:20.105862+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:39:20.105862+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":106496,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17527,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651168+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:38:50.651168+0000","last_peered":"2024-09-10T20:38:50.651168+0000","last_clean":"2024-09-10T20:38:50.651168+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:38:50.651168+0000","last_undegraded":"2024-09-10T20:38:50.651168+0000","last_fullsized":"2024-09-10T20:38:50.651168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20468","reported_seq":27697,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:17.197108+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:17.197108+0000","last_peered":"2024-09-10T20:39:17.197108+0000","last_clean":"2024-09-10T20:39:17.197108+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:17.197108+0000","last_undegraded":"2024-09-10T20:39:17.197108+0000","last_fullsized":"2024-09-10T20:39:17.197108+0000","mapping_epoch":50,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":475136,"num_objects":116,"num_object_clones":0,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17162,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.656849+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:38:50.656849+0000","last_peered":"2024-09-10T20:38:50.656849+0000","last_clean":"2024-09-10T20:38:50.656849+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:38:50.656849+0000","last_undegraded":"2024-09-10T20:38:50.656849+0000","last_fullsized":"2024-09-10T20:38:50.656849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20957","reported_seq":21218,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:20.189578+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:20.189578+0000","last_peered":"2024-09-10T20:39:20.189578+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:20.189578+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118784,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17784,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651258+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:38:50.651258+0000","last_peered":"2024-09-10T20:38:50.651258+0000","last_clean":"2024-09-10T20:38:50.651258+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:38:50.651258+0000","last_undegraded":"2024-09-10T20:38:50.651258+0000","last_fullsized":"2024-09-10T20:38:50.651258+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651173+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:38:50.651173+0000","last_peered":"2024-09-10T20:38:50.651173+0000","last_clean":"2024-09-10T20:38:50.651173+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:38:50.651173+0000","last_undegraded":"2024-09-10T20:38:50.651173+0000","last_fullsized":"2024-09-10T20:38:50.651173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20394","reported_seq":27526,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:17.205124+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:17.205124+0000","last_peered":"2024-09-10T20:39:17.205124+0000","last_clean":"2024-09-10T20:39:17.205124+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:17.205124+0000","last_undegraded":"2024-09-10T20:39:17.205124+0000","last_fullsized":"2024-09-10T20:39:17.205124+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":500195,"num_objects":123,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":3601857,"num_objects":883,"num_object_clones":0,"num_object_copies":1766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27816,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":883,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":294534,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15333,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":185774080,"data_stored":185715354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160738,"ondisk_log_size":160738,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182529,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515908,"kb_used_data":60852,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93855932,"statfs":{"total":96636764160,"available":96108474368,"internally_reserved":0,"allocated":62312448,"data_stored":62040795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":465956522},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986301,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":478444,"kb_used_data":117716,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93893396,"statfs":{"total":96636764160,"available":96146837504,"internally_reserved":0,"allocated":120541184,"data_stored":120269536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369356458},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659247,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":467368,"kb_used_data":6660,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93904472,"statfs":{"total":96636764160,"available":96158179328,"internally_reserved":0,"allocated":6819840,"data_stored":6504372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":471723694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,3,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60428288,"data_stored":60424675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5898240,"data_stored":5850812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":119447552,"data_stored":119439867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:39:22.712 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:39:22.712 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'20620", 'reported_seq': 20847, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:20.171996+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:39:20.171996+0000', 'last_peered': '2024-09-10T20:39:20.171996+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:39:20.171996+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 155648, 'num_objects': 38, 'num_object_clones': 0, 'num_object_copies': 76, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 38, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20620, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3503}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:22.712 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:39:22.712 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20442", 'reported_seq': 20699, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:20.121320+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:39:20.121320+0000', 'last_peered': '2024-09-10T20:39:20.121320+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:39:20.121320+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 118784, 'num_objects': 29, 'num_object_clones': 0, 'num_object_copies': 58, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 29, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:22.712 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:39:22.713 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'20576", 'reported_seq': 20867, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:20.107854+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:39:20.107854+0000', 'last_peered': '2024-09-10T20:39:20.107854+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:39:20.107854+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 131072, 'num_objects': 32, 'num_object_clones': 0, 'num_object_copies': 64, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 32, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17171, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3454}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:22.713 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:39:22.713 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20382", 'reported_seq': 21989, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:19.786334+0000', 'last_change': '2024-09-10T20:38:49.349021+0000', 'last_active': '2024-09-10T20:39:19.786334+0000', 'last_peered': '2024-09-10T20:39:19.786334+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:39:19.786334+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "45'10300", 'ondisk_log_start': "45'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 233472, 'num_objects': 57, 'num_object_clones': 0, 'num_object_copies': 114, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2798, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 57, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18375, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 730, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2741}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:39:22.713 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:39:22.713 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20698", 'reported_seq': 20918, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:20.119844+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:39:20.119844+0000', 'last_peered': '2024-09-10T20:39:20.119844+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:39:20.119844+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 132038, 'num_objects': 34, 'num_object_clones': 0, 'num_object_copies': 68, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 34, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20696, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3532}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:22.713 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:39:22.713 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "51'20505", 'reported_seq': 20702, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:19.708891+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:39:19.708891+0000', 'last_peered': '2024-09-10T20:39:19.708891+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:39:19.708891+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 245760, 'num_objects': 60, 'num_object_clones': 0, 'num_object_copies': 120, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 60, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18333, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3546}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:39:22.713 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:39:22.713 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'20804", 'reported_seq': 21066, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:20.105862+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:39:20.105862+0000', 'last_peered': '2024-09-10T20:39:20.105862+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:39:20.105862+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 106496, 'num_objects': 26, 'num_object_clones': 0, 'num_object_copies': 52, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 26, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17527, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:22.713 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:39:22.714 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "51'20957", 'reported_seq': 21218, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:20.189578+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:39:20.189578+0000', 'last_peered': '2024-09-10T20:39:20.189578+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:39:20.189578+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 118784, 'num_objects': 29, 'num_object_clones': 0, 'num_object_copies': 58, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 29, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17784, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3626}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:22.714 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:39:23.087 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:23.088 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 27816/1770 objects degraded (1571.525%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":674,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":885,"data_bytes":3601876,"bytes_used":1496801280,"bytes_avail":288413491200,"bytes_total":289910292480,"degraded_objects":27816,"degraded_total":1770,"degraded_ratio":15.715254237288136,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":575},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:39:23.101 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:39:23.101 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:39:23.434 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:23.434 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:39:23.446 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":382,"stamp":"2024-09-10T20:39:23.055407+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2143676,"num_objects":528,"num_object_clones":0,"num_object_copies":1056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27816,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":528,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":294893,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15335,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160598,"ondisk_log_size":160598,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1463828,"kb_used_data":179036,"kb_used_omap":12,"kb_used_meta":1284723,"kb_avail":281651692,"statfs":{"total":289910292480,"available":288411332608,"internally_reserved":0,"allocated":183332864,"data_stored":182478167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1315556354},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,7,7,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21430296,"num_objects":-5233,"num_object_clones":0,"num_object_copies":-10466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-207,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5233,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003802"},"pg_stats":[{"pgid":"2.f","version":"51'20620","reported_seq":20847,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:20.171996+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:20.171996+0000","last_peered":"2024-09-10T20:39:20.171996+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:20.171996+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":155648,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20620,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3503}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20904","reported_seq":21029,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:19.747642+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:19.747642+0000","last_peered":"2024-09-10T20:39:19.747642+0000","last_clean":"2024-09-10T20:39:19.747642+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:19.747642+0000","last_undegraded":"2024-09-10T20:39:19.747642+0000","last_fullsized":"2024-09-10T20:39:19.747642+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":155648,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19090,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28079,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.285316+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:21.285316+0000","last_peered":"2024-09-10T20:39:21.285316+0000","last_clean":"2024-09-10T20:39:21.285316+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:21.285316+0000","last_undegraded":"2024-09-10T20:39:21.285316+0000","last_fullsized":"2024-09-10T20:39:21.285316+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20545","reported_seq":20640,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:19.727329+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:19.727329+0000","last_peered":"2024-09-10T20:39:19.727329+0000","last_clean":"2024-09-10T20:39:19.727329+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:19.727329+0000","last_undegraded":"2024-09-10T20:39:19.727329+0000","last_fullsized":"2024-09-10T20:39:19.727329+0000","mapping_epoch":35,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":241664,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20545,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21026","reported_seq":21165,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:20.137258+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:20.137258+0000","last_peered":"2024-09-10T20:39:20.137258+0000","last_clean":"2024-09-10T20:39:20.137258+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:20.137258+0000","last_undegraded":"2024-09-10T20:39:20.137258+0000","last_fullsized":"2024-09-10T20:39:20.137258+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":176128,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18022,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20442","reported_seq":20699,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:20.121320+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:20.121320+0000","last_peered":"2024-09-10T20:39:20.121320+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:20.121320+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118784,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20576","reported_seq":20867,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:20.107854+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:20.107854+0000","last_peered":"2024-09-10T20:39:20.107854+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:20.107854+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131072,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17171,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3454}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20701","reported_seq":20840,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:20.148307+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:20.148307+0000","last_peered":"2024-09-10T20:39:20.148307+0000","last_clean":"2024-09-10T20:39:20.148307+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:20.148307+0000","last_undegraded":"2024-09-10T20:39:20.148307+0000","last_fullsized":"2024-09-10T20:39:20.148307+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":143360,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17772,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.652747+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:38:50.652747+0000","last_peered":"2024-09-10T20:38:50.652747+0000","last_clean":"2024-09-10T20:38:50.652747+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:38:50.652747+0000","last_undegraded":"2024-09-10T20:38:50.652747+0000","last_fullsized":"2024-09-10T20:38:50.652747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20382","reported_seq":21989,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:19.786334+0000","last_change":"2024-09-10T20:38:49.349021+0000","last_active":"2024-09-10T20:39:19.786334+0000","last_peered":"2024-09-10T20:39:19.786334+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:39:19.786334+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":233472,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2798,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18375,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":730,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2741}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.654924+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:38:50.654924+0000","last_peered":"2024-09-10T20:38:50.654924+0000","last_clean":"2024-09-10T20:38:50.654924+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:38:50.654924+0000","last_undegraded":"2024-09-10T20:38:50.654924+0000","last_fullsized":"2024-09-10T20:38:50.654924+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20698","reported_seq":20918,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:20.119844+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:20.119844+0000","last_peered":"2024-09-10T20:39:20.119844+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:20.119844+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":132038,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20696,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3532}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.648348+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:38:50.648348+0000","last_peered":"2024-09-10T20:38:50.648348+0000","last_clean":"2024-09-10T20:38:50.648348+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:38:50.648348+0000","last_undegraded":"2024-09-10T20:38:50.648348+0000","last_fullsized":"2024-09-10T20:38:50.648348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20505","reported_seq":20702,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:19.708891+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:39:19.708891+0000","last_peered":"2024-09-10T20:39:19.708891+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:39:19.708891+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":245760,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18333,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3546}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651037+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:38:50.651037+0000","last_peered":"2024-09-10T20:38:50.651037+0000","last_clean":"2024-09-10T20:38:50.651037+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:38:50.651037+0000","last_undegraded":"2024-09-10T20:38:50.651037+0000","last_fullsized":"2024-09-10T20:38:50.651037+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20619","reported_seq":27995,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:19.691907+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:19.691907+0000","last_peered":"2024-09-10T20:39:19.691907+0000","last_clean":"2024-09-10T20:39:19.691907+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:19.691907+0000","last_undegraded":"2024-09-10T20:39:19.691907+0000","last_fullsized":"2024-09-10T20:39:19.691907+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":184320,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18405,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20804","reported_seq":21066,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:20.105862+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:39:20.105862+0000","last_peered":"2024-09-10T20:39:20.105862+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:39:20.105862+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":106496,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17527,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651168+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:38:50.651168+0000","last_peered":"2024-09-10T20:38:50.651168+0000","last_clean":"2024-09-10T20:38:50.651168+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:38:50.651168+0000","last_undegraded":"2024-09-10T20:38:50.651168+0000","last_fullsized":"2024-09-10T20:38:50.651168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27818,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.284289+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:21.284289+0000","last_peered":"2024-09-10T20:39:21.284289+0000","last_clean":"2024-09-10T20:39:21.284289+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:21.284289+0000","last_undegraded":"2024-09-10T20:39:21.284289+0000","last_fullsized":"2024-09-10T20:39:21.284289+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.656849+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:38:50.656849+0000","last_peered":"2024-09-10T20:38:50.656849+0000","last_clean":"2024-09-10T20:38:50.656849+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:38:50.656849+0000","last_undegraded":"2024-09-10T20:38:50.656849+0000","last_fullsized":"2024-09-10T20:38:50.656849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20957","reported_seq":21218,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:20.189578+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:20.189578+0000","last_peered":"2024-09-10T20:39:20.189578+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:20.189578+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118784,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17784,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651258+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:38:50.651258+0000","last_peered":"2024-09-10T20:38:50.651258+0000","last_clean":"2024-09-10T20:38:50.651258+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:38:50.651258+0000","last_undegraded":"2024-09-10T20:38:50.651258+0000","last_fullsized":"2024-09-10T20:38:50.651258+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651173+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:38:50.651173+0000","last_peered":"2024-09-10T20:38:50.651173+0000","last_clean":"2024-09-10T20:38:50.651173+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:38:50.651173+0000","last_undegraded":"2024-09-10T20:38:50.651173+0000","last_fullsized":"2024-09-10T20:38:50.651173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27648,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.233073+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:21.233073+0000","last_peered":"2024-09-10T20:39:21.233073+0000","last_clean":"2024-09-10T20:39:21.233073+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:21.233073+0000","last_undegraded":"2024-09-10T20:39:21.233073+0000","last_fullsized":"2024-09-10T20:39:21.233073+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":2143657,"num_objects":526,"num_object_clones":0,"num_object_copies":1052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27816,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":526,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":294891,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15333,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":179924992,"data_stored":179870338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160595,"ondisk_log_size":160595,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182529,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515908,"kb_used_data":60852,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93855932,"statfs":{"total":96636764160,"available":96108474368,"internally_reserved":0,"allocated":62312448,"data_stored":62040795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":465956522},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986301,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":478444,"kb_used_data":117716,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93893396,"statfs":{"total":96636764160,"available":96146837504,"internally_reserved":0,"allocated":120541184,"data_stored":120269536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369356458},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659248,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469476,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902364,"statfs":{"total":96636764160,"available":96156020736,"internally_reserved":0,"allocated":479232,"data_stored":167836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480243374},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60428288,"data_stored":60424675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":119447552,"data_stored":119439867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:39:26.449 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:39:26.788 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:26.788 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:39:26.802 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":383,"stamp":"2024-09-10T20:39:25.055774+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1082812,"num_objects":269,"num_object_clones":0,"num_object_copies":538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27314,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":269,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295152,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15837,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160657,"ondisk_log_size":160657,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1465692,"kb_used_data":176748,"kb_used_omap":12,"kb_used_meta":1288883,"kb_avail":281649828,"statfs":{"total":289910292480,"available":288409423872,"internally_reserved":0,"allocated":180989952,"data_stored":180139327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1319816194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20684824,"num_objects":-5051,"num_object_clones":0,"num_object_copies":-10102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-709,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5051,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":709,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003295"},"pg_stats":[{"pgid":"2.f","version":"51'20620","reported_seq":20847,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:20.171996+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:20.171996+0000","last_peered":"2024-09-10T20:39:20.171996+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:20.171996+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":155648,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20620,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3503}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21067,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.267807+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:21.267807+0000","last_peered":"2024-09-10T20:39:21.267807+0000","last_clean":"2024-09-10T20:39:21.267807+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:21.267807+0000","last_undegraded":"2024-09-10T20:39:21.267807+0000","last_fullsized":"2024-09-10T20:39:21.267807+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28079,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.285316+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:21.285316+0000","last_peered":"2024-09-10T20:39:21.285316+0000","last_clean":"2024-09-10T20:39:21.285316+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:21.285316+0000","last_undegraded":"2024-09-10T20:39:21.285316+0000","last_fullsized":"2024-09-10T20:39:21.285316+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20699,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.098186+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:21.098186+0000","last_peered":"2024-09-10T20:39:21.098186+0000","last_clean":"2024-09-10T20:39:21.098186+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:21.098186+0000","last_undegraded":"2024-09-10T20:39:21.098186+0000","last_fullsized":"2024-09-10T20:39:21.098186+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21026","reported_seq":21165,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:20.137258+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:20.137258+0000","last_peered":"2024-09-10T20:39:20.137258+0000","last_clean":"2024-09-10T20:39:20.137258+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:20.137258+0000","last_undegraded":"2024-09-10T20:39:20.137258+0000","last_fullsized":"2024-09-10T20:39:20.137258+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":176128,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18022,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20442","reported_seq":20699,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:20.121320+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:20.121320+0000","last_peered":"2024-09-10T20:39:20.121320+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:20.121320+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118784,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20576","reported_seq":20867,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:20.107854+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:20.107854+0000","last_peered":"2024-09-10T20:39:20.107854+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:20.107854+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131072,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17171,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3454}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20701","reported_seq":20840,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:20.148307+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:20.148307+0000","last_peered":"2024-09-10T20:39:20.148307+0000","last_clean":"2024-09-10T20:39:20.148307+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:20.148307+0000","last_undegraded":"2024-09-10T20:39:20.148307+0000","last_fullsized":"2024-09-10T20:39:20.148307+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":143360,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17772,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.652747+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:38:50.652747+0000","last_peered":"2024-09-10T20:38:50.652747+0000","last_clean":"2024-09-10T20:38:50.652747+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:38:50.652747+0000","last_undegraded":"2024-09-10T20:38:50.652747+0000","last_fullsized":"2024-09-10T20:38:50.652747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":23051,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:24.791807+0000","last_change":"2024-09-10T20:38:49.349021+0000","last_active":"2024-09-10T20:39:24.791807+0000","last_peered":"2024-09-10T20:39:24.791807+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:39:24.791807+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2296,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1232,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2296}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.654924+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:38:50.654924+0000","last_peered":"2024-09-10T20:38:50.654924+0000","last_clean":"2024-09-10T20:38:50.654924+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:38:50.654924+0000","last_undegraded":"2024-09-10T20:38:50.654924+0000","last_fullsized":"2024-09-10T20:38:50.654924+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20698","reported_seq":20918,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:20.119844+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:20.119844+0000","last_peered":"2024-09-10T20:39:20.119844+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:20.119844+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":132038,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20696,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3532}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.648348+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:38:50.648348+0000","last_peered":"2024-09-10T20:38:50.648348+0000","last_clean":"2024-09-10T20:38:50.648348+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:38:50.648348+0000","last_undegraded":"2024-09-10T20:38:50.648348+0000","last_fullsized":"2024-09-10T20:38:50.648348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":20762,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.279026+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:39:21.279026+0000","last_peered":"2024-09-10T20:39:21.279026+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:39:21.279026+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3606}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651037+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:38:50.651037+0000","last_peered":"2024-09-10T20:38:50.651037+0000","last_clean":"2024-09-10T20:38:50.651037+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:38:50.651037+0000","last_undegraded":"2024-09-10T20:38:50.651037+0000","last_fullsized":"2024-09-10T20:38:50.651037+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28040,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.184258+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:21.184258+0000","last_peered":"2024-09-10T20:39:21.184258+0000","last_clean":"2024-09-10T20:39:21.184258+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:21.184258+0000","last_undegraded":"2024-09-10T20:39:21.184258+0000","last_fullsized":"2024-09-10T20:39:21.184258+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20804","reported_seq":21066,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:20.105862+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:39:20.105862+0000","last_peered":"2024-09-10T20:39:20.105862+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:39:20.105862+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":106496,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17527,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651168+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:38:50.651168+0000","last_peered":"2024-09-10T20:38:50.651168+0000","last_clean":"2024-09-10T20:38:50.651168+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:38:50.651168+0000","last_undegraded":"2024-09-10T20:38:50.651168+0000","last_fullsized":"2024-09-10T20:38:50.651168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27818,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.284289+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:21.284289+0000","last_peered":"2024-09-10T20:39:21.284289+0000","last_clean":"2024-09-10T20:39:21.284289+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:21.284289+0000","last_undegraded":"2024-09-10T20:39:21.284289+0000","last_fullsized":"2024-09-10T20:39:21.284289+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.656849+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:38:50.656849+0000","last_peered":"2024-09-10T20:38:50.656849+0000","last_clean":"2024-09-10T20:38:50.656849+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:38:50.656849+0000","last_undegraded":"2024-09-10T20:38:50.656849+0000","last_fullsized":"2024-09-10T20:38:50.656849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20957","reported_seq":21218,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:20.189578+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:20.189578+0000","last_peered":"2024-09-10T20:39:20.189578+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:20.189578+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118784,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17784,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651258+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:38:50.651258+0000","last_peered":"2024-09-10T20:38:50.651258+0000","last_clean":"2024-09-10T20:38:50.651258+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:38:50.651258+0000","last_undegraded":"2024-09-10T20:38:50.651258+0000","last_fullsized":"2024-09-10T20:38:50.651258+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651173+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:38:50.651173+0000","last_peered":"2024-09-10T20:38:50.651173+0000","last_clean":"2024-09-10T20:38:50.651173+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:38:50.651173+0000","last_undegraded":"2024-09-10T20:38:50.651173+0000","last_fullsized":"2024-09-10T20:38:50.651173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27648,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.233073+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:21.233073+0000","last_peered":"2024-09-10T20:39:21.233073+0000","last_clean":"2024-09-10T20:39:21.233073+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:21.233073+0000","last_undegraded":"2024-09-10T20:39:21.233073+0000","last_fullsized":"2024-09-10T20:39:21.233073+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1082793,"num_objects":267,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27314,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":267,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295150,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15835,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":178249728,"data_stored":178199146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160654,"ondisk_log_size":160654,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182529,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515908,"kb_used_data":60852,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93855932,"statfs":{"total":96636764160,"available":96108474368,"internally_reserved":0,"allocated":62312448,"data_stored":62040795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":465956522},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986302,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480308,"kb_used_data":115428,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93891532,"statfs":{"total":96636764160,"available":96144928768,"internally_reserved":0,"allocated":118198272,"data_stored":117930696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659248,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469476,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902364,"statfs":{"total":96636764160,"available":96156020736,"internally_reserved":0,"allocated":479232,"data_stored":167836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480243374},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60428288,"data_stored":60424675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117772288,"data_stored":117768675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:39:26.803 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:39:27.110 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:27.110 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:39:27.123 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2024-09-10T20:39:27.056109+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27314,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15837,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1466908,"kb_used_data":173804,"kb_used_omap":12,"kb_used_meta":1293043,"kb_avail":281648612,"statfs":{"total":289910292480,"available":288408178688,"internally_reserved":0,"allocated":177975296,"data_stored":177124671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1324076034},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18608152,"num_objects":-4544,"num_object_clones":0,"num_object_copies":-9088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4544,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":602,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002902"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20885,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.291632+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:21.291632+0000","last_peered":"2024-09-10T20:39:21.291632+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:21.291632+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21067,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.267807+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:21.267807+0000","last_peered":"2024-09-10T20:39:21.267807+0000","last_clean":"2024-09-10T20:39:21.267807+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:21.267807+0000","last_undegraded":"2024-09-10T20:39:21.267807+0000","last_fullsized":"2024-09-10T20:39:21.267807+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28079,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.285316+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:21.285316+0000","last_peered":"2024-09-10T20:39:21.285316+0000","last_clean":"2024-09-10T20:39:21.285316+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:21.285316+0000","last_undegraded":"2024-09-10T20:39:21.285316+0000","last_fullsized":"2024-09-10T20:39:21.285316+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20699,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.098186+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:21.098186+0000","last_peered":"2024-09-10T20:39:21.098186+0000","last_clean":"2024-09-10T20:39:21.098186+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:21.098186+0000","last_undegraded":"2024-09-10T20:39:21.098186+0000","last_fullsized":"2024-09-10T20:39:21.098186+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21210,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.249795+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:21.249795+0000","last_peered":"2024-09-10T20:39:21.249795+0000","last_clean":"2024-09-10T20:39:21.249795+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:21.249795+0000","last_undegraded":"2024-09-10T20:39:21.249795+0000","last_fullsized":"2024-09-10T20:39:21.249795+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20728,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.174133+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:21.174133+0000","last_peered":"2024-09-10T20:39:21.174133+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:21.174133+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20899,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.261812+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:21.261812+0000","last_peered":"2024-09-10T20:39:21.261812+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:21.261812+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20875,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.259383+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:21.259383+0000","last_peered":"2024-09-10T20:39:21.259383+0000","last_clean":"2024-09-10T20:39:21.259383+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:21.259383+0000","last_undegraded":"2024-09-10T20:39:21.259383+0000","last_fullsized":"2024-09-10T20:39:21.259383+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.652747+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:38:50.652747+0000","last_peered":"2024-09-10T20:38:50.652747+0000","last_clean":"2024-09-10T20:38:50.652747+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:38:50.652747+0000","last_undegraded":"2024-09-10T20:38:50.652747+0000","last_fullsized":"2024-09-10T20:38:50.652747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":23051,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:24.791807+0000","last_change":"2024-09-10T20:38:49.349021+0000","last_active":"2024-09-10T20:39:24.791807+0000","last_peered":"2024-09-10T20:39:24.791807+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:39:24.791807+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2296,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1232,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2296}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.654924+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:38:50.654924+0000","last_peered":"2024-09-10T20:38:50.654924+0000","last_clean":"2024-09-10T20:38:50.654924+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:38:50.654924+0000","last_undegraded":"2024-09-10T20:38:50.654924+0000","last_fullsized":"2024-09-10T20:38:50.654924+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":20952,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.281085+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:21.281085+0000","last_peered":"2024-09-10T20:39:21.281085+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:21.281085+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3564}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.648401+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:38:50.648401+0000","last_peered":"2024-09-10T20:38:50.648401+0000","last_clean":"2024-09-10T20:38:50.648401+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:38:50.648401+0000","last_undegraded":"2024-09-10T20:38:50.648401+0000","last_fullsized":"2024-09-10T20:38:50.648401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":20762,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.279026+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:39:21.279026+0000","last_peered":"2024-09-10T20:39:21.279026+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:39:21.279026+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3606}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651037+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:38:50.651037+0000","last_peered":"2024-09-10T20:38:50.651037+0000","last_clean":"2024-09-10T20:38:50.651037+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:38:50.651037+0000","last_undegraded":"2024-09-10T20:38:50.651037+0000","last_fullsized":"2024-09-10T20:38:50.651037+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28040,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.184258+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:21.184258+0000","last_peered":"2024-09-10T20:39:21.184258+0000","last_clean":"2024-09-10T20:39:21.184258+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:21.184258+0000","last_undegraded":"2024-09-10T20:39:21.184258+0000","last_fullsized":"2024-09-10T20:39:21.184258+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21092,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.283321+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:39:21.283321+0000","last_peered":"2024-09-10T20:39:21.283321+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:39:21.283321+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3592}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651168+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:38:50.651168+0000","last_peered":"2024-09-10T20:38:50.651168+0000","last_clean":"2024-09-10T20:38:50.651168+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:38:50.651168+0000","last_undegraded":"2024-09-10T20:38:50.651168+0000","last_fullsized":"2024-09-10T20:38:50.651168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27818,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.284289+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:21.284289+0000","last_peered":"2024-09-10T20:39:21.284289+0000","last_clean":"2024-09-10T20:39:21.284289+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:21.284289+0000","last_undegraded":"2024-09-10T20:39:21.284289+0000","last_fullsized":"2024-09-10T20:39:21.284289+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.656849+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:38:50.656849+0000","last_peered":"2024-09-10T20:38:50.656849+0000","last_clean":"2024-09-10T20:38:50.656849+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:38:50.656849+0000","last_undegraded":"2024-09-10T20:38:50.656849+0000","last_fullsized":"2024-09-10T20:38:50.656849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21248,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.248938+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:21.248938+0000","last_peered":"2024-09-10T20:39:21.248938+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:21.248938+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651258+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:38:50.651258+0000","last_peered":"2024-09-10T20:38:50.651258+0000","last_clean":"2024-09-10T20:38:50.651258+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:38:50.651258+0000","last_undegraded":"2024-09-10T20:38:50.651258+0000","last_fullsized":"2024-09-10T20:38:50.651258+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651173+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:38:50.651173+0000","last_peered":"2024-09-10T20:38:50.651173+0000","last_clean":"2024-09-10T20:38:50.651173+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:38:50.651173+0000","last_undegraded":"2024-09-10T20:38:50.651173+0000","last_fullsized":"2024-09-10T20:38:50.651173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27648,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.233073+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:21.233073+0000","last_peered":"2024-09-10T20:39:21.233073+0000","last_clean":"2024-09-10T20:39:21.233073+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:21.233073+0000","last_undegraded":"2024-09-10T20:39:21.233073+0000","last_fullsized":"2024-09-10T20:39:21.233073+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27314,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15835,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176689152,"data_stored":176638570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182530,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517124,"kb_used_data":57908,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854716,"statfs":{"total":96636764160,"available":96107229184,"internally_reserved":0,"allocated":59297792,"data_stored":59026139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986302,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480308,"kb_used_data":115428,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93891532,"statfs":{"total":96636764160,"available":96144928768,"internally_reserved":0,"allocated":118198272,"data_stored":117930696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659248,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469476,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902364,"statfs":{"total":96636764160,"available":96156020736,"internally_reserved":0,"allocated":479232,"data_stored":167836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480243374},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117772288,"data_stored":117768675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:39:27.124 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:39:27.124 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'20658", 'reported_seq': 20885, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:21.291632+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:39:21.291632+0000', 'last_peered': '2024-09-10T20:39:21.291632+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:39:21.291632+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20658, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3541}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:27.124 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:39:27.124 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20471", 'reported_seq': 20728, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:21.174133+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:39:21.174133+0000', 'last_peered': '2024-09-10T20:39:21.174133+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:39:21.174133+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17335, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3561}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:27.125 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:39:27.125 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'20608", 'reported_seq': 20899, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:21.261812+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:39:21.261812+0000', 'last_peered': '2024-09-10T20:39:21.261812+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:39:21.261812+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17203, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3486}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:27.125 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:39:27.125 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20439", 'reported_seq': 23051, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:24.791807+0000', 'last_change': '2024-09-10T20:38:49.349021+0000', 'last_active': '2024-09-10T20:39:24.791807+0000', 'last_peered': '2024-09-10T20:39:24.791807+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:39:24.791807+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2296, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18432, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1232, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2296}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:39:27.125 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:39:27.125 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20730", 'reported_seq': 20952, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:21.281085+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:39:21.281085+0000', 'last_peered': '2024-09-10T20:39:21.281085+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:39:21.281085+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20728, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3564}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:27.125 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:39:27.125 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "51'20565", 'reported_seq': 20762, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:21.279026+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:39:21.279026+0000', 'last_peered': '2024-09-10T20:39:21.279026+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:39:21.279026+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18393, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3606}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:39:27.125 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:39:27.125 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'20830", 'reported_seq': 21092, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:21.283321+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:39:21.283321+0000', 'last_peered': '2024-09-10T20:39:21.283321+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:39:21.283321+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17553, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3592}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:27.126 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:39:27.126 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "51'20986", 'reported_seq': 21248, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:21.248938+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:39:21.248938+0000', 'last_peered': '2024-09-10T20:39:21.248938+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:39:21.248938+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17813, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3655}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:27.126 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:39:27.501 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:27.501 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 27314/10 objects degraded (273140.000%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":679,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":5,"data_bytes":1468,"bytes_used":1502113792,"bytes_avail":288408178688,"bytes_total":289910292480,"degraded_objects":27314,"degraded_total":10,"degraded_ratio":2731.4000000000001,"recovering_objects_per_sec":50,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":602,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":378},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:39:27.515 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:39:27.516 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:39:27.819 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:27.820 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:39:27.832 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2024-09-10T20:39:27.056109+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27314,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15837,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1466908,"kb_used_data":173804,"kb_used_omap":12,"kb_used_meta":1293043,"kb_avail":281648612,"statfs":{"total":289910292480,"available":288408178688,"internally_reserved":0,"allocated":177975296,"data_stored":177124671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1324076034},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18608152,"num_objects":-4544,"num_object_clones":0,"num_object_copies":-9088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4544,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":602,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002902"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20885,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.291632+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:21.291632+0000","last_peered":"2024-09-10T20:39:21.291632+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:21.291632+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21067,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.267807+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:21.267807+0000","last_peered":"2024-09-10T20:39:21.267807+0000","last_clean":"2024-09-10T20:39:21.267807+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:21.267807+0000","last_undegraded":"2024-09-10T20:39:21.267807+0000","last_fullsized":"2024-09-10T20:39:21.267807+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28079,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.285316+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:21.285316+0000","last_peered":"2024-09-10T20:39:21.285316+0000","last_clean":"2024-09-10T20:39:21.285316+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:21.285316+0000","last_undegraded":"2024-09-10T20:39:21.285316+0000","last_fullsized":"2024-09-10T20:39:21.285316+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20699,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.098186+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:21.098186+0000","last_peered":"2024-09-10T20:39:21.098186+0000","last_clean":"2024-09-10T20:39:21.098186+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:21.098186+0000","last_undegraded":"2024-09-10T20:39:21.098186+0000","last_fullsized":"2024-09-10T20:39:21.098186+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21210,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.249795+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:21.249795+0000","last_peered":"2024-09-10T20:39:21.249795+0000","last_clean":"2024-09-10T20:39:21.249795+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:21.249795+0000","last_undegraded":"2024-09-10T20:39:21.249795+0000","last_fullsized":"2024-09-10T20:39:21.249795+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20728,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.174133+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:21.174133+0000","last_peered":"2024-09-10T20:39:21.174133+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:21.174133+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20899,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.261812+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:21.261812+0000","last_peered":"2024-09-10T20:39:21.261812+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:21.261812+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20875,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.259383+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:21.259383+0000","last_peered":"2024-09-10T20:39:21.259383+0000","last_clean":"2024-09-10T20:39:21.259383+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:21.259383+0000","last_undegraded":"2024-09-10T20:39:21.259383+0000","last_fullsized":"2024-09-10T20:39:21.259383+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.652747+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:38:50.652747+0000","last_peered":"2024-09-10T20:38:50.652747+0000","last_clean":"2024-09-10T20:38:50.652747+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:38:50.652747+0000","last_undegraded":"2024-09-10T20:38:50.652747+0000","last_fullsized":"2024-09-10T20:38:50.652747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":23051,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:24.791807+0000","last_change":"2024-09-10T20:38:49.349021+0000","last_active":"2024-09-10T20:39:24.791807+0000","last_peered":"2024-09-10T20:39:24.791807+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:39:24.791807+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2296,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1232,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2296}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.654924+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:38:50.654924+0000","last_peered":"2024-09-10T20:38:50.654924+0000","last_clean":"2024-09-10T20:38:50.654924+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:38:50.654924+0000","last_undegraded":"2024-09-10T20:38:50.654924+0000","last_fullsized":"2024-09-10T20:38:50.654924+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":20952,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.281085+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:21.281085+0000","last_peered":"2024-09-10T20:39:21.281085+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:21.281085+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3564}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.648401+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:38:50.648401+0000","last_peered":"2024-09-10T20:38:50.648401+0000","last_clean":"2024-09-10T20:38:50.648401+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:38:50.648401+0000","last_undegraded":"2024-09-10T20:38:50.648401+0000","last_fullsized":"2024-09-10T20:38:50.648401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":20762,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.279026+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:39:21.279026+0000","last_peered":"2024-09-10T20:39:21.279026+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:39:21.279026+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3606}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651037+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:38:50.651037+0000","last_peered":"2024-09-10T20:38:50.651037+0000","last_clean":"2024-09-10T20:38:50.651037+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:38:50.651037+0000","last_undegraded":"2024-09-10T20:38:50.651037+0000","last_fullsized":"2024-09-10T20:38:50.651037+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28040,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.184258+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:21.184258+0000","last_peered":"2024-09-10T20:39:21.184258+0000","last_clean":"2024-09-10T20:39:21.184258+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:21.184258+0000","last_undegraded":"2024-09-10T20:39:21.184258+0000","last_fullsized":"2024-09-10T20:39:21.184258+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21092,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.283321+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:39:21.283321+0000","last_peered":"2024-09-10T20:39:21.283321+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:39:21.283321+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3592}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651168+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:38:50.651168+0000","last_peered":"2024-09-10T20:38:50.651168+0000","last_clean":"2024-09-10T20:38:50.651168+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:38:50.651168+0000","last_undegraded":"2024-09-10T20:38:50.651168+0000","last_fullsized":"2024-09-10T20:38:50.651168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27818,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.284289+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:21.284289+0000","last_peered":"2024-09-10T20:39:21.284289+0000","last_clean":"2024-09-10T20:39:21.284289+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:21.284289+0000","last_undegraded":"2024-09-10T20:39:21.284289+0000","last_fullsized":"2024-09-10T20:39:21.284289+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.656849+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:38:50.656849+0000","last_peered":"2024-09-10T20:38:50.656849+0000","last_clean":"2024-09-10T20:38:50.656849+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:38:50.656849+0000","last_undegraded":"2024-09-10T20:38:50.656849+0000","last_fullsized":"2024-09-10T20:38:50.656849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21248,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.248938+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:21.248938+0000","last_peered":"2024-09-10T20:39:21.248938+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:21.248938+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651258+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:38:50.651258+0000","last_peered":"2024-09-10T20:38:50.651258+0000","last_clean":"2024-09-10T20:38:50.651258+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:38:50.651258+0000","last_undegraded":"2024-09-10T20:38:50.651258+0000","last_fullsized":"2024-09-10T20:38:50.651258+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651173+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:38:50.651173+0000","last_peered":"2024-09-10T20:38:50.651173+0000","last_clean":"2024-09-10T20:38:50.651173+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:38:50.651173+0000","last_undegraded":"2024-09-10T20:38:50.651173+0000","last_fullsized":"2024-09-10T20:38:50.651173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27648,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.233073+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:21.233073+0000","last_peered":"2024-09-10T20:39:21.233073+0000","last_clean":"2024-09-10T20:39:21.233073+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:21.233073+0000","last_undegraded":"2024-09-10T20:39:21.233073+0000","last_fullsized":"2024-09-10T20:39:21.233073+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27314,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15835,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176689152,"data_stored":176638570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182530,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517124,"kb_used_data":57908,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854716,"statfs":{"total":96636764160,"available":96107229184,"internally_reserved":0,"allocated":59297792,"data_stored":59026139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986302,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480308,"kb_used_data":115428,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93891532,"statfs":{"total":96636764160,"available":96144928768,"internally_reserved":0,"allocated":118198272,"data_stored":117930696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659248,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469476,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902364,"statfs":{"total":96636764160,"available":96156020736,"internally_reserved":0,"allocated":479232,"data_stored":167836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480243374},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117772288,"data_stored":117768675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:39:30.833 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:39:31.140 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:31.141 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:39:31.153 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2024-09-10T20:39:31.056807+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26515,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16636,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1471068,"kb_used_data":173804,"kb_used_omap":12,"kb_used_meta":1297203,"kb_avail":281644452,"statfs":{"total":289910292480,"available":288403918848,"internally_reserved":0,"allocated":177975296,"data_stored":177124671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1328335874},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11542552,"num_objects":-2819,"num_object_clones":0,"num_object_copies":-5638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2819,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002484"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20885,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.291632+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:21.291632+0000","last_peered":"2024-09-10T20:39:21.291632+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:21.291632+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21067,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.267807+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:21.267807+0000","last_peered":"2024-09-10T20:39:21.267807+0000","last_clean":"2024-09-10T20:39:21.267807+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:21.267807+0000","last_undegraded":"2024-09-10T20:39:21.267807+0000","last_fullsized":"2024-09-10T20:39:21.267807+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28079,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.285316+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:21.285316+0000","last_peered":"2024-09-10T20:39:21.285316+0000","last_clean":"2024-09-10T20:39:21.285316+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:21.285316+0000","last_undegraded":"2024-09-10T20:39:21.285316+0000","last_fullsized":"2024-09-10T20:39:21.285316+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20699,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.098186+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:21.098186+0000","last_peered":"2024-09-10T20:39:21.098186+0000","last_clean":"2024-09-10T20:39:21.098186+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:21.098186+0000","last_undegraded":"2024-09-10T20:39:21.098186+0000","last_fullsized":"2024-09-10T20:39:21.098186+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21210,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.249795+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:21.249795+0000","last_peered":"2024-09-10T20:39:21.249795+0000","last_clean":"2024-09-10T20:39:21.249795+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:21.249795+0000","last_undegraded":"2024-09-10T20:39:21.249795+0000","last_fullsized":"2024-09-10T20:39:21.249795+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20728,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.174133+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:21.174133+0000","last_peered":"2024-09-10T20:39:21.174133+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:21.174133+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20899,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.261812+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:21.261812+0000","last_peered":"2024-09-10T20:39:21.261812+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:21.261812+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20875,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.259383+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:21.259383+0000","last_peered":"2024-09-10T20:39:21.259383+0000","last_clean":"2024-09-10T20:39:21.259383+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:21.259383+0000","last_undegraded":"2024-09-10T20:39:21.259383+0000","last_fullsized":"2024-09-10T20:39:21.259383+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.652747+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:38:50.652747+0000","last_peered":"2024-09-10T20:38:50.652747+0000","last_clean":"2024-09-10T20:38:50.652747+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:38:50.652747+0000","last_undegraded":"2024-09-10T20:38:50.652747+0000","last_fullsized":"2024-09-10T20:38:50.652747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":24649,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:29.791800+0000","last_change":"2024-09-10T20:38:49.349021+0000","last_active":"2024-09-10T20:39:29.791800+0000","last_peered":"2024-09-10T20:39:29.791800+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:39:29.791800+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1497,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2031,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1497}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.654924+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:38:50.654924+0000","last_peered":"2024-09-10T20:38:50.654924+0000","last_clean":"2024-09-10T20:38:50.654924+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:38:50.654924+0000","last_undegraded":"2024-09-10T20:38:50.654924+0000","last_fullsized":"2024-09-10T20:38:50.654924+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":20952,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.281085+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:21.281085+0000","last_peered":"2024-09-10T20:39:21.281085+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:21.281085+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3564}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.648401+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:38:50.648401+0000","last_peered":"2024-09-10T20:38:50.648401+0000","last_clean":"2024-09-10T20:38:50.648401+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:38:50.648401+0000","last_undegraded":"2024-09-10T20:38:50.648401+0000","last_fullsized":"2024-09-10T20:38:50.648401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":20762,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.279026+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:39:21.279026+0000","last_peered":"2024-09-10T20:39:21.279026+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:39:21.279026+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3606}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651037+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:38:50.651037+0000","last_peered":"2024-09-10T20:38:50.651037+0000","last_clean":"2024-09-10T20:38:50.651037+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:38:50.651037+0000","last_undegraded":"2024-09-10T20:38:50.651037+0000","last_fullsized":"2024-09-10T20:38:50.651037+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28040,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.184258+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:21.184258+0000","last_peered":"2024-09-10T20:39:21.184258+0000","last_clean":"2024-09-10T20:39:21.184258+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:21.184258+0000","last_undegraded":"2024-09-10T20:39:21.184258+0000","last_fullsized":"2024-09-10T20:39:21.184258+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21092,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.283321+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:39:21.283321+0000","last_peered":"2024-09-10T20:39:21.283321+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:39:21.283321+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3592}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651168+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:38:50.651168+0000","last_peered":"2024-09-10T20:38:50.651168+0000","last_clean":"2024-09-10T20:38:50.651168+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:38:50.651168+0000","last_undegraded":"2024-09-10T20:38:50.651168+0000","last_fullsized":"2024-09-10T20:38:50.651168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27818,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.284289+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:21.284289+0000","last_peered":"2024-09-10T20:39:21.284289+0000","last_clean":"2024-09-10T20:39:21.284289+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:21.284289+0000","last_undegraded":"2024-09-10T20:39:21.284289+0000","last_fullsized":"2024-09-10T20:39:21.284289+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.656849+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:38:50.656849+0000","last_peered":"2024-09-10T20:38:50.656849+0000","last_clean":"2024-09-10T20:38:50.656849+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:38:50.656849+0000","last_undegraded":"2024-09-10T20:38:50.656849+0000","last_fullsized":"2024-09-10T20:38:50.656849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21248,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.248938+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:21.248938+0000","last_peered":"2024-09-10T20:39:21.248938+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:21.248938+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651258+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:38:50.651258+0000","last_peered":"2024-09-10T20:38:50.651258+0000","last_clean":"2024-09-10T20:38:50.651258+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:38:50.651258+0000","last_undegraded":"2024-09-10T20:38:50.651258+0000","last_fullsized":"2024-09-10T20:38:50.651258+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651173+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:38:50.651173+0000","last_peered":"2024-09-10T20:38:50.651173+0000","last_clean":"2024-09-10T20:38:50.651173+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:38:50.651173+0000","last_undegraded":"2024-09-10T20:38:50.651173+0000","last_fullsized":"2024-09-10T20:38:50.651173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27648,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.233073+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:21.233073+0000","last_peered":"2024-09-10T20:39:21.233073+0000","last_clean":"2024-09-10T20:39:21.233073+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:21.233073+0000","last_undegraded":"2024-09-10T20:39:21.233073+0000","last_fullsized":"2024-09-10T20:39:21.233073+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26515,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16634,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176689152,"data_stored":176638570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182531,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517124,"kb_used_data":57908,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854716,"statfs":{"total":96636764160,"available":96107229184,"internally_reserved":0,"allocated":59297792,"data_stored":59026139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986303,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480308,"kb_used_data":115428,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93891532,"statfs":{"total":96636764160,"available":96144928768,"internally_reserved":0,"allocated":118198272,"data_stored":117930696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659249,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473636,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898204,"statfs":{"total":96636764160,"available":96151760896,"internally_reserved":0,"allocated":479232,"data_stored":167836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117772288,"data_stored":117768675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:39:31.154 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:39:31.453 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:31.453 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:39:31.468 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2024-09-10T20:39:31.056807+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26515,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16636,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1471068,"kb_used_data":173804,"kb_used_omap":12,"kb_used_meta":1297203,"kb_avail":281644452,"statfs":{"total":289910292480,"available":288403918848,"internally_reserved":0,"allocated":177975296,"data_stored":177124671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1328335874},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11542552,"num_objects":-2819,"num_object_clones":0,"num_object_copies":-5638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2819,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002484"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20885,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.291632+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:21.291632+0000","last_peered":"2024-09-10T20:39:21.291632+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:21.291632+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21067,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.267807+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:21.267807+0000","last_peered":"2024-09-10T20:39:21.267807+0000","last_clean":"2024-09-10T20:39:21.267807+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:21.267807+0000","last_undegraded":"2024-09-10T20:39:21.267807+0000","last_fullsized":"2024-09-10T20:39:21.267807+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28079,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.285316+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:21.285316+0000","last_peered":"2024-09-10T20:39:21.285316+0000","last_clean":"2024-09-10T20:39:21.285316+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:21.285316+0000","last_undegraded":"2024-09-10T20:39:21.285316+0000","last_fullsized":"2024-09-10T20:39:21.285316+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20699,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.098186+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:21.098186+0000","last_peered":"2024-09-10T20:39:21.098186+0000","last_clean":"2024-09-10T20:39:21.098186+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:21.098186+0000","last_undegraded":"2024-09-10T20:39:21.098186+0000","last_fullsized":"2024-09-10T20:39:21.098186+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21210,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.249795+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:21.249795+0000","last_peered":"2024-09-10T20:39:21.249795+0000","last_clean":"2024-09-10T20:39:21.249795+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:21.249795+0000","last_undegraded":"2024-09-10T20:39:21.249795+0000","last_fullsized":"2024-09-10T20:39:21.249795+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20728,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.174133+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:21.174133+0000","last_peered":"2024-09-10T20:39:21.174133+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:21.174133+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20899,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.261812+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:21.261812+0000","last_peered":"2024-09-10T20:39:21.261812+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:21.261812+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20875,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.259383+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:21.259383+0000","last_peered":"2024-09-10T20:39:21.259383+0000","last_clean":"2024-09-10T20:39:21.259383+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:21.259383+0000","last_undegraded":"2024-09-10T20:39:21.259383+0000","last_fullsized":"2024-09-10T20:39:21.259383+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.652747+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:38:50.652747+0000","last_peered":"2024-09-10T20:38:50.652747+0000","last_clean":"2024-09-10T20:38:50.652747+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:38:50.652747+0000","last_undegraded":"2024-09-10T20:38:50.652747+0000","last_fullsized":"2024-09-10T20:38:50.652747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":24649,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:29.791800+0000","last_change":"2024-09-10T20:38:49.349021+0000","last_active":"2024-09-10T20:39:29.791800+0000","last_peered":"2024-09-10T20:39:29.791800+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:39:29.791800+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1497,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2031,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1497}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.654924+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:38:50.654924+0000","last_peered":"2024-09-10T20:38:50.654924+0000","last_clean":"2024-09-10T20:38:50.654924+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:38:50.654924+0000","last_undegraded":"2024-09-10T20:38:50.654924+0000","last_fullsized":"2024-09-10T20:38:50.654924+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":20952,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.281085+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:21.281085+0000","last_peered":"2024-09-10T20:39:21.281085+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:21.281085+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3564}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.648401+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:38:50.648401+0000","last_peered":"2024-09-10T20:38:50.648401+0000","last_clean":"2024-09-10T20:38:50.648401+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:38:50.648401+0000","last_undegraded":"2024-09-10T20:38:50.648401+0000","last_fullsized":"2024-09-10T20:38:50.648401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":20762,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.279026+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:39:21.279026+0000","last_peered":"2024-09-10T20:39:21.279026+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:39:21.279026+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3606}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651037+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:38:50.651037+0000","last_peered":"2024-09-10T20:38:50.651037+0000","last_clean":"2024-09-10T20:38:50.651037+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:38:50.651037+0000","last_undegraded":"2024-09-10T20:38:50.651037+0000","last_fullsized":"2024-09-10T20:38:50.651037+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28040,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.184258+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:21.184258+0000","last_peered":"2024-09-10T20:39:21.184258+0000","last_clean":"2024-09-10T20:39:21.184258+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:21.184258+0000","last_undegraded":"2024-09-10T20:39:21.184258+0000","last_fullsized":"2024-09-10T20:39:21.184258+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21092,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.283321+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:39:21.283321+0000","last_peered":"2024-09-10T20:39:21.283321+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:39:21.283321+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3592}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651168+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:38:50.651168+0000","last_peered":"2024-09-10T20:38:50.651168+0000","last_clean":"2024-09-10T20:38:50.651168+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:38:50.651168+0000","last_undegraded":"2024-09-10T20:38:50.651168+0000","last_fullsized":"2024-09-10T20:38:50.651168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27818,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.284289+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:21.284289+0000","last_peered":"2024-09-10T20:39:21.284289+0000","last_clean":"2024-09-10T20:39:21.284289+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:21.284289+0000","last_undegraded":"2024-09-10T20:39:21.284289+0000","last_fullsized":"2024-09-10T20:39:21.284289+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.656849+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:38:50.656849+0000","last_peered":"2024-09-10T20:38:50.656849+0000","last_clean":"2024-09-10T20:38:50.656849+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:38:50.656849+0000","last_undegraded":"2024-09-10T20:38:50.656849+0000","last_fullsized":"2024-09-10T20:38:50.656849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21248,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.248938+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:21.248938+0000","last_peered":"2024-09-10T20:39:21.248938+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:21.248938+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651258+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:38:50.651258+0000","last_peered":"2024-09-10T20:38:50.651258+0000","last_clean":"2024-09-10T20:38:50.651258+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:38:50.651258+0000","last_undegraded":"2024-09-10T20:38:50.651258+0000","last_fullsized":"2024-09-10T20:38:50.651258+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651173+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:38:50.651173+0000","last_peered":"2024-09-10T20:38:50.651173+0000","last_clean":"2024-09-10T20:38:50.651173+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:38:50.651173+0000","last_undegraded":"2024-09-10T20:38:50.651173+0000","last_fullsized":"2024-09-10T20:38:50.651173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27648,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.233073+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:21.233073+0000","last_peered":"2024-09-10T20:39:21.233073+0000","last_clean":"2024-09-10T20:39:21.233073+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:21.233073+0000","last_undegraded":"2024-09-10T20:39:21.233073+0000","last_fullsized":"2024-09-10T20:39:21.233073+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26515,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16634,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176689152,"data_stored":176638570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182531,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517124,"kb_used_data":57908,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854716,"statfs":{"total":96636764160,"available":96107229184,"internally_reserved":0,"allocated":59297792,"data_stored":59026139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986303,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480308,"kb_used_data":115428,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93891532,"statfs":{"total":96636764160,"available":96144928768,"internally_reserved":0,"allocated":118198272,"data_stored":117930696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659249,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473636,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898204,"statfs":{"total":96636764160,"available":96151760896,"internally_reserved":0,"allocated":479232,"data_stored":167836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117772288,"data_stored":117768675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:39:31.469 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:39:31.469 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'20658", 'reported_seq': 20885, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:21.291632+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:39:21.291632+0000', 'last_peered': '2024-09-10T20:39:21.291632+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:39:21.291632+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20658, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3541}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:31.470 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:39:31.470 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20471", 'reported_seq': 20728, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:21.174133+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:39:21.174133+0000', 'last_peered': '2024-09-10T20:39:21.174133+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:39:21.174133+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17335, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3561}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:31.470 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:39:31.470 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'20608", 'reported_seq': 20899, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:21.261812+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:39:21.261812+0000', 'last_peered': '2024-09-10T20:39:21.261812+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:39:21.261812+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17203, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3486}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:31.470 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:39:31.470 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20439", 'reported_seq': 24649, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:29.791800+0000', 'last_change': '2024-09-10T20:38:49.349021+0000', 'last_active': '2024-09-10T20:39:29.791800+0000', 'last_peered': '2024-09-10T20:39:29.791800+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:39:29.791800+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1497, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18432, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2031, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1497}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:39:31.470 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:39:31.470 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20730", 'reported_seq': 20952, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:21.281085+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:39:21.281085+0000', 'last_peered': '2024-09-10T20:39:21.281085+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:39:21.281085+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20728, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3564}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:31.470 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:39:31.470 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "51'20565", 'reported_seq': 20762, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:21.279026+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:39:21.279026+0000', 'last_peered': '2024-09-10T20:39:21.279026+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:39:21.279026+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18393, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3606}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:39:31.471 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:39:31.471 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'20830", 'reported_seq': 21092, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:21.283321+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:39:21.283321+0000', 'last_peered': '2024-09-10T20:39:21.283321+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:39:21.283321+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17553, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3592}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:31.471 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:39:31.471 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "51'20986", 'reported_seq': 21248, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:21.248938+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:39:21.248938+0000', 'last_peered': '2024-09-10T20:39:21.248938+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:39:21.248938+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17813, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3655}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:31.471 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:39:31.844 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:31.845 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 26515/10 objects degraded (265150.000%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":683,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":5,"data_bytes":1468,"bytes_used":1506373632,"bytes_avail":288403918848,"bytes_total":289910292480,"degraded_objects":26515,"degraded_total":10,"degraded_ratio":2651.5,"recovering_objects_per_sec":116,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1401,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":234},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:39:31.858 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:39:31.858 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:39:32.169 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:32.170 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:39:32.182 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2024-09-10T20:39:31.056807+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26515,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16636,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1471068,"kb_used_data":173804,"kb_used_omap":12,"kb_used_meta":1297203,"kb_avail":281644452,"statfs":{"total":289910292480,"available":288403918848,"internally_reserved":0,"allocated":177975296,"data_stored":177124671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1328335874},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11542552,"num_objects":-2819,"num_object_clones":0,"num_object_copies":-5638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2819,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002484"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20885,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.291632+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:21.291632+0000","last_peered":"2024-09-10T20:39:21.291632+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:21.291632+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21067,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.267807+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:21.267807+0000","last_peered":"2024-09-10T20:39:21.267807+0000","last_clean":"2024-09-10T20:39:21.267807+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:21.267807+0000","last_undegraded":"2024-09-10T20:39:21.267807+0000","last_fullsized":"2024-09-10T20:39:21.267807+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28079,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.285316+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:21.285316+0000","last_peered":"2024-09-10T20:39:21.285316+0000","last_clean":"2024-09-10T20:39:21.285316+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:21.285316+0000","last_undegraded":"2024-09-10T20:39:21.285316+0000","last_fullsized":"2024-09-10T20:39:21.285316+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20699,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.098186+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:21.098186+0000","last_peered":"2024-09-10T20:39:21.098186+0000","last_clean":"2024-09-10T20:39:21.098186+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:21.098186+0000","last_undegraded":"2024-09-10T20:39:21.098186+0000","last_fullsized":"2024-09-10T20:39:21.098186+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21210,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.249795+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:21.249795+0000","last_peered":"2024-09-10T20:39:21.249795+0000","last_clean":"2024-09-10T20:39:21.249795+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:21.249795+0000","last_undegraded":"2024-09-10T20:39:21.249795+0000","last_fullsized":"2024-09-10T20:39:21.249795+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20728,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.174133+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:21.174133+0000","last_peered":"2024-09-10T20:39:21.174133+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:21.174133+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20899,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.261812+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:21.261812+0000","last_peered":"2024-09-10T20:39:21.261812+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:21.261812+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20875,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.259383+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:21.259383+0000","last_peered":"2024-09-10T20:39:21.259383+0000","last_clean":"2024-09-10T20:39:21.259383+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:21.259383+0000","last_undegraded":"2024-09-10T20:39:21.259383+0000","last_fullsized":"2024-09-10T20:39:21.259383+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.652747+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:38:50.652747+0000","last_peered":"2024-09-10T20:38:50.652747+0000","last_clean":"2024-09-10T20:38:50.652747+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:38:50.652747+0000","last_undegraded":"2024-09-10T20:38:50.652747+0000","last_fullsized":"2024-09-10T20:38:50.652747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":24649,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:29.791800+0000","last_change":"2024-09-10T20:38:49.349021+0000","last_active":"2024-09-10T20:39:29.791800+0000","last_peered":"2024-09-10T20:39:29.791800+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:39:29.791800+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1497,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2031,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1497}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.654924+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:38:50.654924+0000","last_peered":"2024-09-10T20:38:50.654924+0000","last_clean":"2024-09-10T20:38:50.654924+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:38:50.654924+0000","last_undegraded":"2024-09-10T20:38:50.654924+0000","last_fullsized":"2024-09-10T20:38:50.654924+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":20952,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.281085+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:21.281085+0000","last_peered":"2024-09-10T20:39:21.281085+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:21.281085+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3564}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.648401+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:38:50.648401+0000","last_peered":"2024-09-10T20:38:50.648401+0000","last_clean":"2024-09-10T20:38:50.648401+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:38:50.648401+0000","last_undegraded":"2024-09-10T20:38:50.648401+0000","last_fullsized":"2024-09-10T20:38:50.648401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":20762,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.279026+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:39:21.279026+0000","last_peered":"2024-09-10T20:39:21.279026+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:39:21.279026+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3606}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651037+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:38:50.651037+0000","last_peered":"2024-09-10T20:38:50.651037+0000","last_clean":"2024-09-10T20:38:50.651037+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:38:50.651037+0000","last_undegraded":"2024-09-10T20:38:50.651037+0000","last_fullsized":"2024-09-10T20:38:50.651037+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28040,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.184258+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:21.184258+0000","last_peered":"2024-09-10T20:39:21.184258+0000","last_clean":"2024-09-10T20:39:21.184258+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:21.184258+0000","last_undegraded":"2024-09-10T20:39:21.184258+0000","last_fullsized":"2024-09-10T20:39:21.184258+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21092,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.283321+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:39:21.283321+0000","last_peered":"2024-09-10T20:39:21.283321+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:39:21.283321+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3592}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651168+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:38:50.651168+0000","last_peered":"2024-09-10T20:38:50.651168+0000","last_clean":"2024-09-10T20:38:50.651168+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:38:50.651168+0000","last_undegraded":"2024-09-10T20:38:50.651168+0000","last_fullsized":"2024-09-10T20:38:50.651168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27818,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.284289+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:21.284289+0000","last_peered":"2024-09-10T20:39:21.284289+0000","last_clean":"2024-09-10T20:39:21.284289+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:21.284289+0000","last_undegraded":"2024-09-10T20:39:21.284289+0000","last_fullsized":"2024-09-10T20:39:21.284289+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.656849+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:38:50.656849+0000","last_peered":"2024-09-10T20:38:50.656849+0000","last_clean":"2024-09-10T20:38:50.656849+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:38:50.656849+0000","last_undegraded":"2024-09-10T20:38:50.656849+0000","last_fullsized":"2024-09-10T20:38:50.656849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21248,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.248938+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:21.248938+0000","last_peered":"2024-09-10T20:39:21.248938+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:21.248938+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651258+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:38:50.651258+0000","last_peered":"2024-09-10T20:38:50.651258+0000","last_clean":"2024-09-10T20:38:50.651258+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:38:50.651258+0000","last_undegraded":"2024-09-10T20:38:50.651258+0000","last_fullsized":"2024-09-10T20:38:50.651258+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651173+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:38:50.651173+0000","last_peered":"2024-09-10T20:38:50.651173+0000","last_clean":"2024-09-10T20:38:50.651173+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:38:50.651173+0000","last_undegraded":"2024-09-10T20:38:50.651173+0000","last_fullsized":"2024-09-10T20:38:50.651173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27648,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.233073+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:21.233073+0000","last_peered":"2024-09-10T20:39:21.233073+0000","last_clean":"2024-09-10T20:39:21.233073+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:21.233073+0000","last_undegraded":"2024-09-10T20:39:21.233073+0000","last_fullsized":"2024-09-10T20:39:21.233073+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26515,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16634,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176689152,"data_stored":176638570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182531,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517124,"kb_used_data":57908,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854716,"statfs":{"total":96636764160,"available":96107229184,"internally_reserved":0,"allocated":59297792,"data_stored":59026139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986303,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480308,"kb_used_data":115428,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93891532,"statfs":{"total":96636764160,"available":96144928768,"internally_reserved":0,"allocated":118198272,"data_stored":117930696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659249,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473636,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898204,"statfs":{"total":96636764160,"available":96151760896,"internally_reserved":0,"allocated":479232,"data_stored":167836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117772288,"data_stored":117768675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:39:35.184 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:39:35.490 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:35.490 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:39:35.503 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":388,"stamp":"2024-09-10T20:39:35.057643+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25469,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17682,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1462748,"kb_used_data":173804,"kb_used_omap":12,"kb_used_meta":1288883,"kb_avail":281652772,"statfs":{"total":289910292480,"available":288412438528,"internally_reserved":0,"allocated":177975296,"data_stored":177124671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1319816194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2142208,"num_objects":-523,"num_object_clones":0,"num_object_copies":-1046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":523,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2347,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002236"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20885,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.291632+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:21.291632+0000","last_peered":"2024-09-10T20:39:21.291632+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:21.291632+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21067,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.267807+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:21.267807+0000","last_peered":"2024-09-10T20:39:21.267807+0000","last_clean":"2024-09-10T20:39:21.267807+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:21.267807+0000","last_undegraded":"2024-09-10T20:39:21.267807+0000","last_fullsized":"2024-09-10T20:39:21.267807+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28079,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.285316+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:21.285316+0000","last_peered":"2024-09-10T20:39:21.285316+0000","last_clean":"2024-09-10T20:39:21.285316+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:21.285316+0000","last_undegraded":"2024-09-10T20:39:21.285316+0000","last_fullsized":"2024-09-10T20:39:21.285316+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20699,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.098186+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:21.098186+0000","last_peered":"2024-09-10T20:39:21.098186+0000","last_clean":"2024-09-10T20:39:21.098186+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:21.098186+0000","last_undegraded":"2024-09-10T20:39:21.098186+0000","last_fullsized":"2024-09-10T20:39:21.098186+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21210,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.249795+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:21.249795+0000","last_peered":"2024-09-10T20:39:21.249795+0000","last_clean":"2024-09-10T20:39:21.249795+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:21.249795+0000","last_undegraded":"2024-09-10T20:39:21.249795+0000","last_fullsized":"2024-09-10T20:39:21.249795+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20728,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.174133+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:21.174133+0000","last_peered":"2024-09-10T20:39:21.174133+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:21.174133+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20899,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.261812+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:21.261812+0000","last_peered":"2024-09-10T20:39:21.261812+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:21.261812+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20875,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.259383+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:21.259383+0000","last_peered":"2024-09-10T20:39:21.259383+0000","last_clean":"2024-09-10T20:39:21.259383+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:21.259383+0000","last_undegraded":"2024-09-10T20:39:21.259383+0000","last_fullsized":"2024-09-10T20:39:21.259383+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.652747+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:38:50.652747+0000","last_peered":"2024-09-10T20:38:50.652747+0000","last_clean":"2024-09-10T20:38:50.652747+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:38:50.652747+0000","last_undegraded":"2024-09-10T20:38:50.652747+0000","last_fullsized":"2024-09-10T20:38:50.652747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":26741,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:34.793335+0000","last_change":"2024-09-10T20:38:49.349021+0000","last_active":"2024-09-10T20:39:34.793335+0000","last_peered":"2024-09-10T20:39:34.793335+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:39:34.793335+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":451,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3077,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-451}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.654924+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:38:50.654924+0000","last_peered":"2024-09-10T20:38:50.654924+0000","last_clean":"2024-09-10T20:38:50.654924+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:38:50.654924+0000","last_undegraded":"2024-09-10T20:38:50.654924+0000","last_fullsized":"2024-09-10T20:38:50.654924+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":20952,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.281085+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:21.281085+0000","last_peered":"2024-09-10T20:39:21.281085+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:21.281085+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3564}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.648401+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:38:50.648401+0000","last_peered":"2024-09-10T20:38:50.648401+0000","last_clean":"2024-09-10T20:38:50.648401+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:38:50.648401+0000","last_undegraded":"2024-09-10T20:38:50.648401+0000","last_fullsized":"2024-09-10T20:38:50.648401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":20762,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.279026+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:39:21.279026+0000","last_peered":"2024-09-10T20:39:21.279026+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:39:21.279026+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3606}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651037+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:38:50.651037+0000","last_peered":"2024-09-10T20:38:50.651037+0000","last_clean":"2024-09-10T20:38:50.651037+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:38:50.651037+0000","last_undegraded":"2024-09-10T20:38:50.651037+0000","last_fullsized":"2024-09-10T20:38:50.651037+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28040,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.184258+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:21.184258+0000","last_peered":"2024-09-10T20:39:21.184258+0000","last_clean":"2024-09-10T20:39:21.184258+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:21.184258+0000","last_undegraded":"2024-09-10T20:39:21.184258+0000","last_fullsized":"2024-09-10T20:39:21.184258+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21092,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.283321+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:39:21.283321+0000","last_peered":"2024-09-10T20:39:21.283321+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:39:21.283321+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3592}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651168+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:38:50.651168+0000","last_peered":"2024-09-10T20:38:50.651168+0000","last_clean":"2024-09-10T20:38:50.651168+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:38:50.651168+0000","last_undegraded":"2024-09-10T20:38:50.651168+0000","last_fullsized":"2024-09-10T20:38:50.651168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27818,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.284289+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:21.284289+0000","last_peered":"2024-09-10T20:39:21.284289+0000","last_clean":"2024-09-10T20:39:21.284289+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:21.284289+0000","last_undegraded":"2024-09-10T20:39:21.284289+0000","last_fullsized":"2024-09-10T20:39:21.284289+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.656849+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:38:50.656849+0000","last_peered":"2024-09-10T20:38:50.656849+0000","last_clean":"2024-09-10T20:38:50.656849+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:38:50.656849+0000","last_undegraded":"2024-09-10T20:38:50.656849+0000","last_fullsized":"2024-09-10T20:38:50.656849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21248,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.248938+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:21.248938+0000","last_peered":"2024-09-10T20:39:21.248938+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:21.248938+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651258+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:38:50.651258+0000","last_peered":"2024-09-10T20:38:50.651258+0000","last_clean":"2024-09-10T20:38:50.651258+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:38:50.651258+0000","last_undegraded":"2024-09-10T20:38:50.651258+0000","last_fullsized":"2024-09-10T20:38:50.651258+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651173+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:38:50.651173+0000","last_peered":"2024-09-10T20:38:50.651173+0000","last_clean":"2024-09-10T20:38:50.651173+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:38:50.651173+0000","last_undegraded":"2024-09-10T20:38:50.651173+0000","last_fullsized":"2024-09-10T20:38:50.651173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27648,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.233073+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:21.233073+0000","last_peered":"2024-09-10T20:39:21.233073+0000","last_clean":"2024-09-10T20:39:21.233073+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:21.233073+0000","last_undegraded":"2024-09-10T20:39:21.233073+0000","last_fullsized":"2024-09-10T20:39:21.233073+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25469,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17680,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176689152,"data_stored":176638570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182531,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517124,"kb_used_data":57908,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854716,"statfs":{"total":96636764160,"available":96107229184,"internally_reserved":0,"allocated":59297792,"data_stored":59026139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986304,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480308,"kb_used_data":115428,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93891532,"statfs":{"total":96636764160,"available":96144928768,"internally_reserved":0,"allocated":118198272,"data_stored":117930696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659250,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465316,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906524,"statfs":{"total":96636764160,"available":96160280576,"internally_reserved":0,"allocated":479232,"data_stored":167836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":475983534},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117772288,"data_stored":117768675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:39:35.504 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:39:35.829 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:35.830 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:39:35.843 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":388,"stamp":"2024-09-10T20:39:35.057643+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25469,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17682,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1462748,"kb_used_data":173804,"kb_used_omap":12,"kb_used_meta":1288883,"kb_avail":281652772,"statfs":{"total":289910292480,"available":288412438528,"internally_reserved":0,"allocated":177975296,"data_stored":177124671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1319816194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2142208,"num_objects":-523,"num_object_clones":0,"num_object_copies":-1046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":523,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2347,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002236"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20885,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.291632+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:21.291632+0000","last_peered":"2024-09-10T20:39:21.291632+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:21.291632+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21067,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.267807+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:21.267807+0000","last_peered":"2024-09-10T20:39:21.267807+0000","last_clean":"2024-09-10T20:39:21.267807+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:21.267807+0000","last_undegraded":"2024-09-10T20:39:21.267807+0000","last_fullsized":"2024-09-10T20:39:21.267807+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28079,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.285316+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:21.285316+0000","last_peered":"2024-09-10T20:39:21.285316+0000","last_clean":"2024-09-10T20:39:21.285316+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:21.285316+0000","last_undegraded":"2024-09-10T20:39:21.285316+0000","last_fullsized":"2024-09-10T20:39:21.285316+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20699,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.098186+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:21.098186+0000","last_peered":"2024-09-10T20:39:21.098186+0000","last_clean":"2024-09-10T20:39:21.098186+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:21.098186+0000","last_undegraded":"2024-09-10T20:39:21.098186+0000","last_fullsized":"2024-09-10T20:39:21.098186+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21210,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.249795+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:21.249795+0000","last_peered":"2024-09-10T20:39:21.249795+0000","last_clean":"2024-09-10T20:39:21.249795+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:21.249795+0000","last_undegraded":"2024-09-10T20:39:21.249795+0000","last_fullsized":"2024-09-10T20:39:21.249795+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20728,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.174133+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:21.174133+0000","last_peered":"2024-09-10T20:39:21.174133+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:21.174133+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20899,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.261812+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:21.261812+0000","last_peered":"2024-09-10T20:39:21.261812+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:21.261812+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20875,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.259383+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:21.259383+0000","last_peered":"2024-09-10T20:39:21.259383+0000","last_clean":"2024-09-10T20:39:21.259383+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:21.259383+0000","last_undegraded":"2024-09-10T20:39:21.259383+0000","last_fullsized":"2024-09-10T20:39:21.259383+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.652747+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:38:50.652747+0000","last_peered":"2024-09-10T20:38:50.652747+0000","last_clean":"2024-09-10T20:38:50.652747+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:38:50.652747+0000","last_undegraded":"2024-09-10T20:38:50.652747+0000","last_fullsized":"2024-09-10T20:38:50.652747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":26741,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:34.793335+0000","last_change":"2024-09-10T20:38:49.349021+0000","last_active":"2024-09-10T20:39:34.793335+0000","last_peered":"2024-09-10T20:39:34.793335+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:39:34.793335+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":451,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3077,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-451}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.654924+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:38:50.654924+0000","last_peered":"2024-09-10T20:38:50.654924+0000","last_clean":"2024-09-10T20:38:50.654924+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:38:50.654924+0000","last_undegraded":"2024-09-10T20:38:50.654924+0000","last_fullsized":"2024-09-10T20:38:50.654924+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":20952,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.281085+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:21.281085+0000","last_peered":"2024-09-10T20:39:21.281085+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:21.281085+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3564}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.648401+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:38:50.648401+0000","last_peered":"2024-09-10T20:38:50.648401+0000","last_clean":"2024-09-10T20:38:50.648401+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:38:50.648401+0000","last_undegraded":"2024-09-10T20:38:50.648401+0000","last_fullsized":"2024-09-10T20:38:50.648401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":20762,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.279026+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:39:21.279026+0000","last_peered":"2024-09-10T20:39:21.279026+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:39:21.279026+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3606}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651037+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:38:50.651037+0000","last_peered":"2024-09-10T20:38:50.651037+0000","last_clean":"2024-09-10T20:38:50.651037+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:38:50.651037+0000","last_undegraded":"2024-09-10T20:38:50.651037+0000","last_fullsized":"2024-09-10T20:38:50.651037+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28040,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.184258+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:21.184258+0000","last_peered":"2024-09-10T20:39:21.184258+0000","last_clean":"2024-09-10T20:39:21.184258+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:21.184258+0000","last_undegraded":"2024-09-10T20:39:21.184258+0000","last_fullsized":"2024-09-10T20:39:21.184258+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21092,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.283321+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:39:21.283321+0000","last_peered":"2024-09-10T20:39:21.283321+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:39:21.283321+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3592}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651168+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:38:50.651168+0000","last_peered":"2024-09-10T20:38:50.651168+0000","last_clean":"2024-09-10T20:38:50.651168+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:38:50.651168+0000","last_undegraded":"2024-09-10T20:38:50.651168+0000","last_fullsized":"2024-09-10T20:38:50.651168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27818,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.284289+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:21.284289+0000","last_peered":"2024-09-10T20:39:21.284289+0000","last_clean":"2024-09-10T20:39:21.284289+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:21.284289+0000","last_undegraded":"2024-09-10T20:39:21.284289+0000","last_fullsized":"2024-09-10T20:39:21.284289+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.656849+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:38:50.656849+0000","last_peered":"2024-09-10T20:38:50.656849+0000","last_clean":"2024-09-10T20:38:50.656849+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:38:50.656849+0000","last_undegraded":"2024-09-10T20:38:50.656849+0000","last_fullsized":"2024-09-10T20:38:50.656849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21248,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.248938+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:21.248938+0000","last_peered":"2024-09-10T20:39:21.248938+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:21.248938+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651258+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:38:50.651258+0000","last_peered":"2024-09-10T20:38:50.651258+0000","last_clean":"2024-09-10T20:38:50.651258+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:38:50.651258+0000","last_undegraded":"2024-09-10T20:38:50.651258+0000","last_fullsized":"2024-09-10T20:38:50.651258+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651173+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:38:50.651173+0000","last_peered":"2024-09-10T20:38:50.651173+0000","last_clean":"2024-09-10T20:38:50.651173+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:38:50.651173+0000","last_undegraded":"2024-09-10T20:38:50.651173+0000","last_fullsized":"2024-09-10T20:38:50.651173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27648,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.233073+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:21.233073+0000","last_peered":"2024-09-10T20:39:21.233073+0000","last_clean":"2024-09-10T20:39:21.233073+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:21.233073+0000","last_undegraded":"2024-09-10T20:39:21.233073+0000","last_fullsized":"2024-09-10T20:39:21.233073+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25469,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17680,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176689152,"data_stored":176638570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182531,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517124,"kb_used_data":57908,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854716,"statfs":{"total":96636764160,"available":96107229184,"internally_reserved":0,"allocated":59297792,"data_stored":59026139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986304,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480308,"kb_used_data":115428,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93891532,"statfs":{"total":96636764160,"available":96144928768,"internally_reserved":0,"allocated":118198272,"data_stored":117930696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659250,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465316,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906524,"statfs":{"total":96636764160,"available":96160280576,"internally_reserved":0,"allocated":479232,"data_stored":167836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":475983534},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117772288,"data_stored":117768675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:39:35.844 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:39:35.844 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'20658", 'reported_seq': 20885, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:21.291632+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:39:21.291632+0000', 'last_peered': '2024-09-10T20:39:21.291632+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:39:21.291632+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20658, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3541}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:35.844 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:39:35.844 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20471", 'reported_seq': 20728, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:21.174133+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:39:21.174133+0000', 'last_peered': '2024-09-10T20:39:21.174133+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:39:21.174133+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17335, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3561}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:35.844 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:39:35.845 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'20608", 'reported_seq': 20899, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:21.261812+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:39:21.261812+0000', 'last_peered': '2024-09-10T20:39:21.261812+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:39:21.261812+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17203, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3486}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:35.845 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:39:35.845 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20439", 'reported_seq': 26741, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:34.793335+0000', 'last_change': '2024-09-10T20:38:49.349021+0000', 'last_active': '2024-09-10T20:39:34.793335+0000', 'last_peered': '2024-09-10T20:39:34.793335+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:39:34.793335+0000', 'last_undegraded': '2024-09-10T20:30:05.352777+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 451, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18432, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3077, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:39:35.845 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:39:35.845 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20730", 'reported_seq': 20952, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:21.281085+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:39:21.281085+0000', 'last_peered': '2024-09-10T20:39:21.281085+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:39:21.281085+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20728, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3564}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:35.845 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:39:35.845 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "51'20565", 'reported_seq': 20762, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:21.279026+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:39:21.279026+0000', 'last_peered': '2024-09-10T20:39:21.279026+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:39:21.279026+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18393, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3606}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:39:35.845 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:39:35.845 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'20830", 'reported_seq': 21092, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:21.283321+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:39:21.283321+0000', 'last_peered': '2024-09-10T20:39:21.283321+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:39:21.283321+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17553, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3592}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:35.845 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:39:35.845 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "51'20986", 'reported_seq': 21248, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:21.248938+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:39:21.248938+0000', 'last_peered': '2024-09-10T20:39:21.248938+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:39:21.248938+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17813, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3655}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:35.846 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:39:36.259 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:36.259 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 25469/10 objects degraded (254690.000%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":687,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":5,"data_bytes":1468,"bytes_used":1497853952,"bytes_avail":288412438528,"bytes_total":289910292480,"degraded_objects":25469,"degraded_total":10,"degraded_ratio":2546.9000000000001,"recovering_objects_per_sec":195,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2347,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":43},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:39:36.273 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:39:36.273 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:39:36.578 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:36.578 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:39:36.591 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":388,"stamp":"2024-09-10T20:39:35.057643+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25469,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17682,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1462748,"kb_used_data":173804,"kb_used_omap":12,"kb_used_meta":1288883,"kb_avail":281652772,"statfs":{"total":289910292480,"available":288412438528,"internally_reserved":0,"allocated":177975296,"data_stored":177124671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1319816194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2142208,"num_objects":-523,"num_object_clones":0,"num_object_copies":-1046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":523,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2347,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002236"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20885,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.291632+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:21.291632+0000","last_peered":"2024-09-10T20:39:21.291632+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:21.291632+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21067,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.267807+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:21.267807+0000","last_peered":"2024-09-10T20:39:21.267807+0000","last_clean":"2024-09-10T20:39:21.267807+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:21.267807+0000","last_undegraded":"2024-09-10T20:39:21.267807+0000","last_fullsized":"2024-09-10T20:39:21.267807+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28079,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.285316+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:21.285316+0000","last_peered":"2024-09-10T20:39:21.285316+0000","last_clean":"2024-09-10T20:39:21.285316+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:21.285316+0000","last_undegraded":"2024-09-10T20:39:21.285316+0000","last_fullsized":"2024-09-10T20:39:21.285316+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20699,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.098186+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:21.098186+0000","last_peered":"2024-09-10T20:39:21.098186+0000","last_clean":"2024-09-10T20:39:21.098186+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:21.098186+0000","last_undegraded":"2024-09-10T20:39:21.098186+0000","last_fullsized":"2024-09-10T20:39:21.098186+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21210,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.249795+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:21.249795+0000","last_peered":"2024-09-10T20:39:21.249795+0000","last_clean":"2024-09-10T20:39:21.249795+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:21.249795+0000","last_undegraded":"2024-09-10T20:39:21.249795+0000","last_fullsized":"2024-09-10T20:39:21.249795+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20728,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.174133+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:21.174133+0000","last_peered":"2024-09-10T20:39:21.174133+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:21.174133+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20899,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.261812+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:21.261812+0000","last_peered":"2024-09-10T20:39:21.261812+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:21.261812+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20875,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.259383+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:21.259383+0000","last_peered":"2024-09-10T20:39:21.259383+0000","last_clean":"2024-09-10T20:39:21.259383+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:21.259383+0000","last_undegraded":"2024-09-10T20:39:21.259383+0000","last_fullsized":"2024-09-10T20:39:21.259383+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.652747+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:38:50.652747+0000","last_peered":"2024-09-10T20:38:50.652747+0000","last_clean":"2024-09-10T20:38:50.652747+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:38:50.652747+0000","last_undegraded":"2024-09-10T20:38:50.652747+0000","last_fullsized":"2024-09-10T20:38:50.652747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":26741,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:34.793335+0000","last_change":"2024-09-10T20:38:49.349021+0000","last_active":"2024-09-10T20:39:34.793335+0000","last_peered":"2024-09-10T20:39:34.793335+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:39:34.793335+0000","last_undegraded":"2024-09-10T20:30:05.352777+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":451,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3077,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-451}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.654924+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:38:50.654924+0000","last_peered":"2024-09-10T20:38:50.654924+0000","last_clean":"2024-09-10T20:38:50.654924+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:38:50.654924+0000","last_undegraded":"2024-09-10T20:38:50.654924+0000","last_fullsized":"2024-09-10T20:38:50.654924+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":20952,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.281085+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:21.281085+0000","last_peered":"2024-09-10T20:39:21.281085+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:21.281085+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3564}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.648401+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:38:50.648401+0000","last_peered":"2024-09-10T20:38:50.648401+0000","last_clean":"2024-09-10T20:38:50.648401+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:38:50.648401+0000","last_undegraded":"2024-09-10T20:38:50.648401+0000","last_fullsized":"2024-09-10T20:38:50.648401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":20762,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.279026+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:39:21.279026+0000","last_peered":"2024-09-10T20:39:21.279026+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:39:21.279026+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3606}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651037+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:38:50.651037+0000","last_peered":"2024-09-10T20:38:50.651037+0000","last_clean":"2024-09-10T20:38:50.651037+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:38:50.651037+0000","last_undegraded":"2024-09-10T20:38:50.651037+0000","last_fullsized":"2024-09-10T20:38:50.651037+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28040,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.184258+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:21.184258+0000","last_peered":"2024-09-10T20:39:21.184258+0000","last_clean":"2024-09-10T20:39:21.184258+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:21.184258+0000","last_undegraded":"2024-09-10T20:39:21.184258+0000","last_fullsized":"2024-09-10T20:39:21.184258+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21092,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.283321+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:39:21.283321+0000","last_peered":"2024-09-10T20:39:21.283321+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:39:21.283321+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3592}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651168+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:38:50.651168+0000","last_peered":"2024-09-10T20:38:50.651168+0000","last_clean":"2024-09-10T20:38:50.651168+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:38:50.651168+0000","last_undegraded":"2024-09-10T20:38:50.651168+0000","last_fullsized":"2024-09-10T20:38:50.651168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27818,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.284289+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:21.284289+0000","last_peered":"2024-09-10T20:39:21.284289+0000","last_clean":"2024-09-10T20:39:21.284289+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:21.284289+0000","last_undegraded":"2024-09-10T20:39:21.284289+0000","last_fullsized":"2024-09-10T20:39:21.284289+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.656849+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:38:50.656849+0000","last_peered":"2024-09-10T20:38:50.656849+0000","last_clean":"2024-09-10T20:38:50.656849+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:38:50.656849+0000","last_undegraded":"2024-09-10T20:38:50.656849+0000","last_fullsized":"2024-09-10T20:38:50.656849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21248,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.248938+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:21.248938+0000","last_peered":"2024-09-10T20:39:21.248938+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:21.248938+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651258+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:38:50.651258+0000","last_peered":"2024-09-10T20:38:50.651258+0000","last_clean":"2024-09-10T20:38:50.651258+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:38:50.651258+0000","last_undegraded":"2024-09-10T20:38:50.651258+0000","last_fullsized":"2024-09-10T20:38:50.651258+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.651173+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:38:50.651173+0000","last_peered":"2024-09-10T20:38:50.651173+0000","last_clean":"2024-09-10T20:38:50.651173+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:38:50.651173+0000","last_undegraded":"2024-09-10T20:38:50.651173+0000","last_fullsized":"2024-09-10T20:38:50.651173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27648,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.233073+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:21.233073+0000","last_peered":"2024-09-10T20:39:21.233073+0000","last_clean":"2024-09-10T20:39:21.233073+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:21.233073+0000","last_undegraded":"2024-09-10T20:39:21.233073+0000","last_fullsized":"2024-09-10T20:39:21.233073+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25469,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17680,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176689152,"data_stored":176638570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182531,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517124,"kb_used_data":57908,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854716,"statfs":{"total":96636764160,"available":96107229184,"internally_reserved":0,"allocated":59297792,"data_stored":59026139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986304,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480308,"kb_used_data":115428,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93891532,"statfs":{"total":96636764160,"available":96144928768,"internally_reserved":0,"allocated":118198272,"data_stored":117930696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659250,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465316,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906524,"statfs":{"total":96636764160,"available":96160280576,"internally_reserved":0,"allocated":479232,"data_stored":167836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":475983534},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117772288,"data_stored":117768675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:39:39.593 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:39:39.934 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:39.935 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:39:39.949 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":392,"stamp":"2024-09-10T20:39:39.058349+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25018,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18133,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1471064,"kb_used_data":173800,"kb_used_omap":12,"kb_used_meta":1297203,"kb_avail":281644456,"statfs":{"total":289910292480,"available":288403922944,"internally_reserved":0,"allocated":177971200,"data_stored":177124188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1328335874},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1497,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1497,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001541"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20885,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.291632+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:21.291632+0000","last_peered":"2024-09-10T20:39:21.291632+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:21.291632+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21067,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.267807+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:21.267807+0000","last_peered":"2024-09-10T20:39:21.267807+0000","last_clean":"2024-09-10T20:39:21.267807+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:21.267807+0000","last_undegraded":"2024-09-10T20:39:21.267807+0000","last_fullsized":"2024-09-10T20:39:21.267807+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28081,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T20:39:37.233504+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:37.233504+0000","last_peered":"2024-09-10T20:39:37.233504+0000","last_clean":"2024-09-10T20:39:37.233504+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:37.233504+0000","last_undegraded":"2024-09-10T20:39:37.233504+0000","last_fullsized":"2024-09-10T20:39:37.233504+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20699,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.098186+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:21.098186+0000","last_peered":"2024-09-10T20:39:21.098186+0000","last_clean":"2024-09-10T20:39:21.098186+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:21.098186+0000","last_undegraded":"2024-09-10T20:39:21.098186+0000","last_fullsized":"2024-09-10T20:39:21.098186+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21210,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.249795+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:21.249795+0000","last_peered":"2024-09-10T20:39:21.249795+0000","last_clean":"2024-09-10T20:39:21.249795+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:21.249795+0000","last_undegraded":"2024-09-10T20:39:21.249795+0000","last_fullsized":"2024-09-10T20:39:21.249795+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20728,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.174133+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:21.174133+0000","last_peered":"2024-09-10T20:39:21.174133+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:21.174133+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20899,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.261812+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:21.261812+0000","last_peered":"2024-09-10T20:39:21.261812+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:21.261812+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20875,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.259383+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:21.259383+0000","last_peered":"2024-09-10T20:39:21.259383+0000","last_clean":"2024-09-10T20:39:21.259383+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:21.259383+0000","last_undegraded":"2024-09-10T20:39:21.259383+0000","last_fullsized":"2024-09-10T20:39:21.259383+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":86,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T20:39:37.233616+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:39:37.233616+0000","last_peered":"2024-09-10T20:39:37.233616+0000","last_clean":"2024-09-10T20:39:37.233616+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:39:37.233616+0000","last_undegraded":"2024-09-10T20:39:37.233616+0000","last_fullsized":"2024-09-10T20:39:37.233616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22113,"reported_epoch":52,"state":"peering","last_fresh":"2024-09-10T20:39:36.810979+0000","last_change":"2024-09-10T20:39:37.253595+0000","last_active":"2024-09-10T20:39:37.253595+0000","last_peered":"2024-09-10T20:39:36.810979+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:39:36.810979+0000","last_undegraded":"2024-09-10T20:39:36.810979+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.654924+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:38:50.654924+0000","last_peered":"2024-09-10T20:38:50.654924+0000","last_clean":"2024-09-10T20:38:50.654924+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:38:50.654924+0000","last_undegraded":"2024-09-10T20:38:50.654924+0000","last_fullsized":"2024-09-10T20:38:50.654924+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":20952,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.281085+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:21.281085+0000","last_peered":"2024-09-10T20:39:21.281085+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:21.281085+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3564}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.648401+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:38:50.648401+0000","last_peered":"2024-09-10T20:38:50.648401+0000","last_clean":"2024-09-10T20:38:50.648401+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:38:50.648401+0000","last_undegraded":"2024-09-10T20:38:50.648401+0000","last_fullsized":"2024-09-10T20:38:50.648401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":20762,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.279026+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:39:21.279026+0000","last_peered":"2024-09-10T20:39:21.279026+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:39:21.279026+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3606}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":96,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T20:39:37.232946+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:39:37.232946+0000","last_peered":"2024-09-10T20:39:37.232946+0000","last_clean":"2024-09-10T20:39:37.232946+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:39:37.232946+0000","last_undegraded":"2024-09-10T20:39:37.232946+0000","last_fullsized":"2024-09-10T20:39:37.232946+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28040,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.184258+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:21.184258+0000","last_peered":"2024-09-10T20:39:21.184258+0000","last_clean":"2024-09-10T20:39:21.184258+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:21.184258+0000","last_undegraded":"2024-09-10T20:39:21.184258+0000","last_fullsized":"2024-09-10T20:39:21.184258+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21092,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.283321+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:39:21.283321+0000","last_peered":"2024-09-10T20:39:21.283321+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:39:21.283321+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3592}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":86,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T20:39:37.233402+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:39:37.233402+0000","last_peered":"2024-09-10T20:39:37.233402+0000","last_clean":"2024-09-10T20:39:37.233402+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:39:37.233402+0000","last_undegraded":"2024-09-10T20:39:37.233402+0000","last_fullsized":"2024-09-10T20:39:37.233402+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27820,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T20:39:37.234117+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:37.234117+0000","last_peered":"2024-09-10T20:39:37.234117+0000","last_clean":"2024-09-10T20:39:37.234117+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:37.234117+0000","last_undegraded":"2024-09-10T20:39:37.234117+0000","last_fullsized":"2024-09-10T20:39:37.234117+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.656849+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:38:50.656849+0000","last_peered":"2024-09-10T20:38:50.656849+0000","last_clean":"2024-09-10T20:38:50.656849+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:38:50.656849+0000","last_undegraded":"2024-09-10T20:38:50.656849+0000","last_fullsized":"2024-09-10T20:38:50.656849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21249,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.267810+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:21.267810+0000","last_peered":"2024-09-10T20:39:21.267810+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:21.267810+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":86,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T20:39:37.234041+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:39:37.234041+0000","last_peered":"2024-09-10T20:39:37.234041+0000","last_clean":"2024-09-10T20:39:37.234041+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:39:37.234041+0000","last_undegraded":"2024-09-10T20:39:37.234041+0000","last_fullsized":"2024-09-10T20:39:37.234041+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":91,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T20:39:37.232935+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:39:37.232935+0000","last_peered":"2024-09-10T20:39:37.232935+0000","last_clean":"2024-09-10T20:39:37.232935+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:39:37.232935+0000","last_undegraded":"2024-09-10T20:39:37.232935+0000","last_fullsized":"2024-09-10T20:39:37.232935+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27650,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T20:39:37.233314+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:37.233314+0000","last_peered":"2024-09-10T20:39:37.233314+0000","last_clean":"2024-09-10T20:39:37.233314+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:37.233314+0000","last_undegraded":"2024-09-10T20:39:37.233314+0000","last_fullsized":"2024-09-10T20:39:37.233314+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25018,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18131,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176685056,"data_stored":176638087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182532,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517124,"kb_used_data":57908,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854716,"statfs":{"total":96636764160,"available":96107229184,"internally_reserved":0,"allocated":59297792,"data_stored":59026139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986304,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480308,"kb_used_data":115428,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93891532,"statfs":{"total":96636764160,"available":96144928768,"internally_reserved":0,"allocated":118198272,"data_stored":117930696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659251,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473632,"kb_used_data":464,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898208,"statfs":{"total":96636764160,"available":96151764992,"internally_reserved":0,"allocated":475136,"data_stored":167353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117772288,"data_stored":117768675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:39:39.950 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:39:40.303 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:40.304 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:39:40.319 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":392,"stamp":"2024-09-10T20:39:39.058349+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25018,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18133,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1471064,"kb_used_data":173800,"kb_used_omap":12,"kb_used_meta":1297203,"kb_avail":281644456,"statfs":{"total":289910292480,"available":288403922944,"internally_reserved":0,"allocated":177971200,"data_stored":177124188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1328335874},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1497,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1497,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001541"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20885,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.291632+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:21.291632+0000","last_peered":"2024-09-10T20:39:21.291632+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:21.291632+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21067,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.267807+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:21.267807+0000","last_peered":"2024-09-10T20:39:21.267807+0000","last_clean":"2024-09-10T20:39:21.267807+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:21.267807+0000","last_undegraded":"2024-09-10T20:39:21.267807+0000","last_fullsized":"2024-09-10T20:39:21.267807+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28081,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T20:39:37.233504+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:37.233504+0000","last_peered":"2024-09-10T20:39:37.233504+0000","last_clean":"2024-09-10T20:39:37.233504+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:37.233504+0000","last_undegraded":"2024-09-10T20:39:37.233504+0000","last_fullsized":"2024-09-10T20:39:37.233504+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20699,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.098186+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:21.098186+0000","last_peered":"2024-09-10T20:39:21.098186+0000","last_clean":"2024-09-10T20:39:21.098186+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:21.098186+0000","last_undegraded":"2024-09-10T20:39:21.098186+0000","last_fullsized":"2024-09-10T20:39:21.098186+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21210,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.249795+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:21.249795+0000","last_peered":"2024-09-10T20:39:21.249795+0000","last_clean":"2024-09-10T20:39:21.249795+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:21.249795+0000","last_undegraded":"2024-09-10T20:39:21.249795+0000","last_fullsized":"2024-09-10T20:39:21.249795+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20728,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.174133+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:21.174133+0000","last_peered":"2024-09-10T20:39:21.174133+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:21.174133+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20899,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.261812+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:21.261812+0000","last_peered":"2024-09-10T20:39:21.261812+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:21.261812+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20875,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.259383+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:21.259383+0000","last_peered":"2024-09-10T20:39:21.259383+0000","last_clean":"2024-09-10T20:39:21.259383+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:21.259383+0000","last_undegraded":"2024-09-10T20:39:21.259383+0000","last_fullsized":"2024-09-10T20:39:21.259383+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":86,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T20:39:37.233616+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:39:37.233616+0000","last_peered":"2024-09-10T20:39:37.233616+0000","last_clean":"2024-09-10T20:39:37.233616+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:39:37.233616+0000","last_undegraded":"2024-09-10T20:39:37.233616+0000","last_fullsized":"2024-09-10T20:39:37.233616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22113,"reported_epoch":52,"state":"peering","last_fresh":"2024-09-10T20:39:36.810979+0000","last_change":"2024-09-10T20:39:37.253595+0000","last_active":"2024-09-10T20:39:37.253595+0000","last_peered":"2024-09-10T20:39:36.810979+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:39:36.810979+0000","last_undegraded":"2024-09-10T20:39:36.810979+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.654924+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:38:50.654924+0000","last_peered":"2024-09-10T20:38:50.654924+0000","last_clean":"2024-09-10T20:38:50.654924+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:38:50.654924+0000","last_undegraded":"2024-09-10T20:38:50.654924+0000","last_fullsized":"2024-09-10T20:38:50.654924+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":20952,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.281085+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:21.281085+0000","last_peered":"2024-09-10T20:39:21.281085+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:21.281085+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3564}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.648401+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:38:50.648401+0000","last_peered":"2024-09-10T20:38:50.648401+0000","last_clean":"2024-09-10T20:38:50.648401+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:38:50.648401+0000","last_undegraded":"2024-09-10T20:38:50.648401+0000","last_fullsized":"2024-09-10T20:38:50.648401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":20762,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.279026+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:39:21.279026+0000","last_peered":"2024-09-10T20:39:21.279026+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:39:21.279026+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3606}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":96,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T20:39:37.232946+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:39:37.232946+0000","last_peered":"2024-09-10T20:39:37.232946+0000","last_clean":"2024-09-10T20:39:37.232946+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:39:37.232946+0000","last_undegraded":"2024-09-10T20:39:37.232946+0000","last_fullsized":"2024-09-10T20:39:37.232946+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28040,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:39:21.184258+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:21.184258+0000","last_peered":"2024-09-10T20:39:21.184258+0000","last_clean":"2024-09-10T20:39:21.184258+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:21.184258+0000","last_undegraded":"2024-09-10T20:39:21.184258+0000","last_fullsized":"2024-09-10T20:39:21.184258+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21092,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.283321+0000","last_change":"2024-09-10T20:30:05.741072+0000","last_active":"2024-09-10T20:39:21.283321+0000","last_peered":"2024-09-10T20:39:21.283321+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:39:21.283321+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3592}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":86,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T20:39:37.233402+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:39:37.233402+0000","last_peered":"2024-09-10T20:39:37.233402+0000","last_clean":"2024-09-10T20:39:37.233402+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:39:37.233402+0000","last_undegraded":"2024-09-10T20:39:37.233402+0000","last_fullsized":"2024-09-10T20:39:37.233402+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27820,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T20:39:37.234117+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:37.234117+0000","last_peered":"2024-09-10T20:39:37.234117+0000","last_clean":"2024-09-10T20:39:37.234117+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:37.234117+0000","last_undegraded":"2024-09-10T20:39:37.234117+0000","last_fullsized":"2024-09-10T20:39:37.234117+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T20:38:50.656849+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:38:50.656849+0000","last_peered":"2024-09-10T20:38:50.656849+0000","last_clean":"2024-09-10T20:38:50.656849+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:38:50.656849+0000","last_undegraded":"2024-09-10T20:38:50.656849+0000","last_fullsized":"2024-09-10T20:38:50.656849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21249,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:21.267810+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:21.267810+0000","last_peered":"2024-09-10T20:39:21.267810+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:21.267810+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":86,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T20:39:37.234041+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:39:37.234041+0000","last_peered":"2024-09-10T20:39:37.234041+0000","last_clean":"2024-09-10T20:39:37.234041+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:39:37.234041+0000","last_undegraded":"2024-09-10T20:39:37.234041+0000","last_fullsized":"2024-09-10T20:39:37.234041+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":91,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T20:39:37.232935+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:39:37.232935+0000","last_peered":"2024-09-10T20:39:37.232935+0000","last_clean":"2024-09-10T20:39:37.232935+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:39:37.232935+0000","last_undegraded":"2024-09-10T20:39:37.232935+0000","last_fullsized":"2024-09-10T20:39:37.232935+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27650,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T20:39:37.233314+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:37.233314+0000","last_peered":"2024-09-10T20:39:37.233314+0000","last_clean":"2024-09-10T20:39:37.233314+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:37.233314+0000","last_undegraded":"2024-09-10T20:39:37.233314+0000","last_fullsized":"2024-09-10T20:39:37.233314+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25018,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18131,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176685056,"data_stored":176638087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182532,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517124,"kb_used_data":57908,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854716,"statfs":{"total":96636764160,"available":96107229184,"internally_reserved":0,"allocated":59297792,"data_stored":59026139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986304,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480308,"kb_used_data":115428,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93891532,"statfs":{"total":96636764160,"available":96144928768,"internally_reserved":0,"allocated":118198272,"data_stored":117930696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659251,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473632,"kb_used_data":464,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898208,"statfs":{"total":96636764160,"available":96151764992,"internally_reserved":0,"allocated":475136,"data_stored":167353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117772288,"data_stored":117768675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:39:40.321 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:39:40.321 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'20658", 'reported_seq': 20885, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:21.291632+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:39:21.291632+0000', 'last_peered': '2024-09-10T20:39:21.291632+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:39:21.291632+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20658, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3541}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:40.321 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:39:40.321 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20471", 'reported_seq': 20728, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:21.174133+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:39:21.174133+0000', 'last_peered': '2024-09-10T20:39:21.174133+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:39:21.174133+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17335, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3561}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:40.321 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:39:40.321 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'20608", 'reported_seq': 20899, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:21.261812+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:39:21.261812+0000', 'last_peered': '2024-09-10T20:39:21.261812+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:39:21.261812+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17203, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3486}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:40.321 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T20:39:40.321 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20439", 'reported_seq': 22113, 'reported_epoch': 52, 'state': 'peering', 'last_fresh': '2024-09-10T20:39:36.810979+0000', 'last_change': '2024-09-10T20:39:37.253595+0000', 'last_active': '2024-09-10T20:39:37.253595+0000', 'last_peered': '2024-09-10T20:39:36.810979+0000', 'last_clean': '2024-09-10T20:30:03.319921+0000', 'last_became_active': '2024-09-10T20:30:05.715814+0000', 'last_became_peered': '2024-09-10T20:30:05.715814+0000', 'last_unstale': '2024-09-10T20:39:36.810979+0000', 'last_undegraded': '2024-09-10T20:39:36.810979+0000', 'last_fullsized': '2024-09-10T20:30:05.312060+0000', 'mapping_epoch': 52, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:43.225780+0000', 'objects_scrubbed': 2000, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 607, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18432, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3528, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T20:39:40.322 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:39:40.322 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20730", 'reported_seq': 20952, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:21.281085+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:39:21.281085+0000', 'last_peered': '2024-09-10T20:39:21.281085+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:39:21.281085+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20728, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3564}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:40.322 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:39:40.322 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "51'20565", 'reported_seq': 20762, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:21.279026+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:39:21.279026+0000', 'last_peered': '2024-09-10T20:39:21.279026+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:39:21.279026+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18393, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3606}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:39:40.322 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:39:40.322 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'20830", 'reported_seq': 21092, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:21.283321+0000', 'last_change': '2024-09-10T20:30:05.741072+0000', 'last_active': '2024-09-10T20:39:21.283321+0000', 'last_peered': '2024-09-10T20:39:21.283321+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:39:21.283321+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17553, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3592}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:40.322 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:39:40.322 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "51'20986", 'reported_seq': 21249, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:21.267810+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:39:21.267810+0000', 'last_peered': '2024-09-10T20:39:21.267810+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:39:21.267810+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17813, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3655}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:40.322 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:39:40.789 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:40.789 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 25018/10 objects degraded (250180.000%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":692,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"peering","count":1}],"num_pgs":24,"num_pools":2,"num_objects":5,"data_bytes":1468,"bytes_used":1506369536,"bytes_avail":288403922944,"bytes_total":289910292480,"inactive_pgs_ratio":0.041666667908430099,"degraded_objects":25018,"degraded_total":10,"degraded_ratio":2501.8000000000002,"recovering_objects_per_sec":187,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1497,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:39:40.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:39:40.804 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:39:41.166 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:41.167 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:39:41.180 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-09-10T20:39:41.058680+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24406,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18745,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1471112,"kb_used_data":173832,"kb_used_omap":12,"kb_used_meta":1297203,"kb_avail":281644408,"statfs":{"total":289910292480,"available":288403873792,"internally_reserved":0,"allocated":178003968,"data_stored":177137446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1328335874},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2109,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2109,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001458"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20889,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.239347+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:38.239347+0000","last_peered":"2024-09-10T20:39:38.239347+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:38.239347+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21071,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.241208+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:38.241208+0000","last_peered":"2024-09-10T20:39:38.241208+0000","last_clean":"2024-09-10T20:39:38.241208+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:38.241208+0000","last_undegraded":"2024-09-10T20:39:38.241208+0000","last_fullsized":"2024-09-10T20:39:38.241208+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28081,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T20:39:37.233504+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:37.233504+0000","last_peered":"2024-09-10T20:39:37.233504+0000","last_clean":"2024-09-10T20:39:37.233504+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:37.233504+0000","last_undegraded":"2024-09-10T20:39:37.233504+0000","last_fullsized":"2024-09-10T20:39:37.233504+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20703,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.240860+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:38.240860+0000","last_peered":"2024-09-10T20:39:38.240860+0000","last_clean":"2024-09-10T20:39:38.240860+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:38.240860+0000","last_undegraded":"2024-09-10T20:39:38.240860+0000","last_fullsized":"2024-09-10T20:39:38.240860+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21214,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.236997+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:38.236997+0000","last_peered":"2024-09-10T20:39:38.236997+0000","last_clean":"2024-09-10T20:39:38.236997+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:38.236997+0000","last_undegraded":"2024-09-10T20:39:38.236997+0000","last_fullsized":"2024-09-10T20:39:38.236997+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20732,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.236538+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:38.236538+0000","last_peered":"2024-09-10T20:39:38.236538+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:38.236538+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20903,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.236686+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:38.236686+0000","last_peered":"2024-09-10T20:39:38.236686+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:38.236686+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20879,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238057+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:38.238057+0000","last_peered":"2024-09-10T20:39:38.238057+0000","last_clean":"2024-09-10T20:39:38.238057+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:38.238057+0000","last_undegraded":"2024-09-10T20:39:38.238057+0000","last_fullsized":"2024-09-10T20:39:38.238057+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":86,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T20:39:37.233616+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:39:37.233616+0000","last_peered":"2024-09-10T20:39:37.233616+0000","last_clean":"2024-09-10T20:39:37.233616+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:39:37.233616+0000","last_undegraded":"2024-09-10T20:39:37.233616+0000","last_fullsized":"2024-09-10T20:39:37.233616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22113,"reported_epoch":52,"state":"peering","last_fresh":"2024-09-10T20:39:36.810979+0000","last_change":"2024-09-10T20:39:37.253595+0000","last_active":"2024-09-10T20:39:37.253595+0000","last_peered":"2024-09-10T20:39:36.810979+0000","last_clean":"2024-09-10T20:30:03.319921+0000","last_became_active":"2024-09-10T20:30:05.715814+0000","last_became_peered":"2024-09-10T20:30:05.715814+0000","last_unstale":"2024-09-10T20:39:36.810979+0000","last_undegraded":"2024-09-10T20:39:36.810979+0000","last_fullsized":"2024-09-10T20:30:05.312060+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.240786+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:39:38.240786+0000","last_peered":"2024-09-10T20:39:38.240786+0000","last_clean":"2024-09-10T20:39:38.240786+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:39:38.240786+0000","last_undegraded":"2024-09-10T20:39:38.240786+0000","last_fullsized":"2024-09-10T20:39:38.240786+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":20956,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.237580+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:38.237580+0000","last_peered":"2024-09-10T20:39:38.237580+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:38.237580+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3564}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.236424+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:39:38.236424+0000","last_peered":"2024-09-10T20:39:38.236424+0000","last_clean":"2024-09-10T20:39:38.236424+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:39:38.236424+0000","last_undegraded":"2024-09-10T20:39:38.236424+0000","last_fullsized":"2024-09-10T20:39:38.236424+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":20767,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.241791+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:39:38.241791+0000","last_peered":"2024-09-10T20:39:38.241791+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:39:38.241791+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3606}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":96,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T20:39:37.232946+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:39:37.232946+0000","last_peered":"2024-09-10T20:39:37.232946+0000","last_clean":"2024-09-10T20:39:37.232946+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:39:37.232946+0000","last_undegraded":"2024-09-10T20:39:37.232946+0000","last_fullsized":"2024-09-10T20:39:37.232946+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28044,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.240573+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:38.240573+0000","last_peered":"2024-09-10T20:39:38.240573+0000","last_clean":"2024-09-10T20:39:38.240573+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:38.240573+0000","last_undegraded":"2024-09-10T20:39:38.240573+0000","last_fullsized":"2024-09-10T20:39:38.240573+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":22321,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:40.211503+0000","last_change":"2024-09-10T20:39:36.811261+0000","last_active":"2024-09-10T20:39:40.211503+0000","last_peered":"2024-09-10T20:39:40.211503+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:39:40.211503+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2981,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":668,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2981}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":86,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T20:39:37.233402+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:39:37.233402+0000","last_peered":"2024-09-10T20:39:37.233402+0000","last_clean":"2024-09-10T20:39:37.233402+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:39:37.233402+0000","last_undegraded":"2024-09-10T20:39:37.233402+0000","last_fullsized":"2024-09-10T20:39:37.233402+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27820,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T20:39:37.234117+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:37.234117+0000","last_peered":"2024-09-10T20:39:37.234117+0000","last_clean":"2024-09-10T20:39:37.234117+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:37.234117+0000","last_undegraded":"2024-09-10T20:39:37.234117+0000","last_fullsized":"2024-09-10T20:39:37.234117+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.241067+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:39:38.241067+0000","last_peered":"2024-09-10T20:39:38.241067+0000","last_clean":"2024-09-10T20:39:38.241067+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:39:38.241067+0000","last_undegraded":"2024-09-10T20:39:38.241067+0000","last_fullsized":"2024-09-10T20:39:38.241067+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21253,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.237167+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:38.237167+0000","last_peered":"2024-09-10T20:39:38.237167+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:38.237167+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":86,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T20:39:37.234041+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:39:37.234041+0000","last_peered":"2024-09-10T20:39:37.234041+0000","last_clean":"2024-09-10T20:39:37.234041+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:39:37.234041+0000","last_undegraded":"2024-09-10T20:39:37.234041+0000","last_fullsized":"2024-09-10T20:39:37.234041+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":91,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T20:39:37.232935+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:39:37.232935+0000","last_peered":"2024-09-10T20:39:37.232935+0000","last_clean":"2024-09-10T20:39:37.232935+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:39:37.232935+0000","last_undegraded":"2024-09-10T20:39:37.232935+0000","last_fullsized":"2024-09-10T20:39:37.232935+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27650,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T20:39:37.233314+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:37.233314+0000","last_peered":"2024-09-10T20:39:37.233314+0000","last_clean":"2024-09-10T20:39:37.233314+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:37.233314+0000","last_undegraded":"2024-09-10T20:39:37.233314+0000","last_fullsized":"2024-09-10T20:39:37.233314+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24406,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18743,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176685056,"data_stored":176638087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182533,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517148,"kb_used_data":57924,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854692,"statfs":{"total":96636764160,"available":96107204608,"internally_reserved":0,"allocated":59314176,"data_stored":59032768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986305,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480332,"kb_used_data":115444,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93891508,"statfs":{"total":96636764160,"available":96144904192,"internally_reserved":0,"allocated":118214656,"data_stored":117937325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659251,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473632,"kb_used_data":464,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898208,"statfs":{"total":96636764160,"available":96151764992,"internally_reserved":0,"allocated":475136,"data_stored":167353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117772288,"data_stored":117768675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:39:44.183 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:39:44.494 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:44.494 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:39:44.509 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":394,"stamp":"2024-09-10T20:39:43.059043+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24406,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18745,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1462808,"kb_used_data":173848,"kb_used_omap":12,"kb_used_meta":1288883,"kb_avail":281652712,"statfs":{"total":289910292480,"available":288412377088,"internally_reserved":0,"allocated":178020352,"data_stored":177144075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1319816194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1063,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1063,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001400"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20889,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.239347+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:38.239347+0000","last_peered":"2024-09-10T20:39:38.239347+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:38.239347+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21071,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.241208+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:38.241208+0000","last_peered":"2024-09-10T20:39:38.241208+0000","last_clean":"2024-09-10T20:39:38.241208+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:38.241208+0000","last_undegraded":"2024-09-10T20:39:38.241208+0000","last_fullsized":"2024-09-10T20:39:38.241208+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28083,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238999+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:38.238999+0000","last_peered":"2024-09-10T20:39:38.238999+0000","last_clean":"2024-09-10T20:39:38.238999+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:38.238999+0000","last_undegraded":"2024-09-10T20:39:38.238999+0000","last_fullsized":"2024-09-10T20:39:38.238999+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20703,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.240860+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:38.240860+0000","last_peered":"2024-09-10T20:39:38.240860+0000","last_clean":"2024-09-10T20:39:38.240860+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:38.240860+0000","last_undegraded":"2024-09-10T20:39:38.240860+0000","last_fullsized":"2024-09-10T20:39:38.240860+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21214,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.236997+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:38.236997+0000","last_peered":"2024-09-10T20:39:38.236997+0000","last_clean":"2024-09-10T20:39:38.236997+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:38.236997+0000","last_undegraded":"2024-09-10T20:39:38.236997+0000","last_fullsized":"2024-09-10T20:39:38.236997+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20732,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.236538+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:38.236538+0000","last_peered":"2024-09-10T20:39:38.236538+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:38.236538+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20903,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.236686+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:38.236686+0000","last_peered":"2024-09-10T20:39:38.236686+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:38.236686+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20879,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238057+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:38.238057+0000","last_peered":"2024-09-10T20:39:38.238057+0000","last_clean":"2024-09-10T20:39:38.238057+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:38.238057+0000","last_undegraded":"2024-09-10T20:39:38.238057+0000","last_fullsized":"2024-09-10T20:39:38.238057+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.237911+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:39:38.237911+0000","last_peered":"2024-09-10T20:39:38.237911+0000","last_clean":"2024-09-10T20:39:38.237911+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:39:38.237911+0000","last_undegraded":"2024-09-10T20:39:38.237911+0000","last_fullsized":"2024-09-10T20:39:38.237911+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.340635+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:39:38.340635+0000","last_peered":"2024-09-10T20:39:38.340635+0000","last_clean":"2024-09-10T20:39:38.340635+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:39:38.340635+0000","last_undegraded":"2024-09-10T20:39:38.340635+0000","last_fullsized":"2024-09-10T20:39:38.340635+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.240786+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:39:38.240786+0000","last_peered":"2024-09-10T20:39:38.240786+0000","last_clean":"2024-09-10T20:39:38.240786+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:39:38.240786+0000","last_undegraded":"2024-09-10T20:39:38.240786+0000","last_fullsized":"2024-09-10T20:39:38.240786+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":20956,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.237580+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:38.237580+0000","last_peered":"2024-09-10T20:39:38.237580+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:38.237580+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3564}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.236424+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:39:38.236424+0000","last_peered":"2024-09-10T20:39:38.236424+0000","last_clean":"2024-09-10T20:39:38.236424+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:39:38.236424+0000","last_undegraded":"2024-09-10T20:39:38.236424+0000","last_fullsized":"2024-09-10T20:39:38.236424+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":20767,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.241791+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:39:38.241791+0000","last_peered":"2024-09-10T20:39:38.241791+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:39:38.241791+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3606}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238378+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:39:38.238378+0000","last_peered":"2024-09-10T20:39:38.238378+0000","last_clean":"2024-09-10T20:39:38.238378+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:39:38.238378+0000","last_undegraded":"2024-09-10T20:39:38.238378+0000","last_fullsized":"2024-09-10T20:39:38.238378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28044,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.240573+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:38.240573+0000","last_peered":"2024-09-10T20:39:38.240573+0000","last_clean":"2024-09-10T20:39:38.240573+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:38.240573+0000","last_undegraded":"2024-09-10T20:39:38.240573+0000","last_fullsized":"2024-09-10T20:39:38.240573+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":22321,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:40.211503+0000","last_change":"2024-09-10T20:39:36.811261+0000","last_active":"2024-09-10T20:39:40.211503+0000","last_peered":"2024-09-10T20:39:40.211503+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:39:40.211503+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2981,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":668,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2981}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238954+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:39:38.238954+0000","last_peered":"2024-09-10T20:39:38.238954+0000","last_clean":"2024-09-10T20:39:38.238954+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:39:38.238954+0000","last_undegraded":"2024-09-10T20:39:38.238954+0000","last_fullsized":"2024-09-10T20:39:38.238954+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27822,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.239210+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:38.239210+0000","last_peered":"2024-09-10T20:39:38.239210+0000","last_clean":"2024-09-10T20:39:38.239210+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:38.239210+0000","last_undegraded":"2024-09-10T20:39:38.239210+0000","last_fullsized":"2024-09-10T20:39:38.239210+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.241067+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:39:38.241067+0000","last_peered":"2024-09-10T20:39:38.241067+0000","last_clean":"2024-09-10T20:39:38.241067+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:39:38.241067+0000","last_undegraded":"2024-09-10T20:39:38.241067+0000","last_fullsized":"2024-09-10T20:39:38.241067+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21253,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.237167+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:38.237167+0000","last_peered":"2024-09-10T20:39:38.237167+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:38.237167+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.239124+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:39:38.239124+0000","last_peered":"2024-09-10T20:39:38.239124+0000","last_clean":"2024-09-10T20:39:38.239124+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:39:38.239124+0000","last_undegraded":"2024-09-10T20:39:38.239124+0000","last_fullsized":"2024-09-10T20:39:38.239124+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.237911+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:39:38.237911+0000","last_peered":"2024-09-10T20:39:38.237911+0000","last_clean":"2024-09-10T20:39:38.237911+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:39:38.237911+0000","last_undegraded":"2024-09-10T20:39:38.237911+0000","last_fullsized":"2024-09-10T20:39:38.237911+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27652,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238867+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:38.238867+0000","last_peered":"2024-09-10T20:39:38.238867+0000","last_clean":"2024-09-10T20:39:38.238867+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:38.238867+0000","last_undegraded":"2024-09-10T20:39:38.238867+0000","last_fullsized":"2024-09-10T20:39:38.238867+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24406,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18743,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176685056,"data_stored":176638087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182533,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517148,"kb_used_data":57924,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854692,"statfs":{"total":96636764160,"available":96107204608,"internally_reserved":0,"allocated":59314176,"data_stored":59032768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986305,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480332,"kb_used_data":115444,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93891508,"statfs":{"total":96636764160,"available":96144904192,"internally_reserved":0,"allocated":118214656,"data_stored":117937325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659252,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465328,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906512,"statfs":{"total":96636764160,"available":96160268288,"internally_reserved":0,"allocated":491520,"data_stored":173982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":475983534},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117772288,"data_stored":117768675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:39:44.510 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:39:44.873 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:44.874 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:39:44.885 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":394,"stamp":"2024-09-10T20:39:43.059043+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24406,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18745,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1462808,"kb_used_data":173848,"kb_used_omap":12,"kb_used_meta":1288883,"kb_avail":281652712,"statfs":{"total":289910292480,"available":288412377088,"internally_reserved":0,"allocated":178020352,"data_stored":177144075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1319816194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1063,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1063,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001400"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20889,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.239347+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:38.239347+0000","last_peered":"2024-09-10T20:39:38.239347+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:38.239347+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21071,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.241208+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:38.241208+0000","last_peered":"2024-09-10T20:39:38.241208+0000","last_clean":"2024-09-10T20:39:38.241208+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:38.241208+0000","last_undegraded":"2024-09-10T20:39:38.241208+0000","last_fullsized":"2024-09-10T20:39:38.241208+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28083,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238999+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:38.238999+0000","last_peered":"2024-09-10T20:39:38.238999+0000","last_clean":"2024-09-10T20:39:38.238999+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:38.238999+0000","last_undegraded":"2024-09-10T20:39:38.238999+0000","last_fullsized":"2024-09-10T20:39:38.238999+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20703,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.240860+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:38.240860+0000","last_peered":"2024-09-10T20:39:38.240860+0000","last_clean":"2024-09-10T20:39:38.240860+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:38.240860+0000","last_undegraded":"2024-09-10T20:39:38.240860+0000","last_fullsized":"2024-09-10T20:39:38.240860+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21214,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.236997+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:38.236997+0000","last_peered":"2024-09-10T20:39:38.236997+0000","last_clean":"2024-09-10T20:39:38.236997+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:38.236997+0000","last_undegraded":"2024-09-10T20:39:38.236997+0000","last_fullsized":"2024-09-10T20:39:38.236997+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20732,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.236538+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:38.236538+0000","last_peered":"2024-09-10T20:39:38.236538+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:38.236538+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20903,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.236686+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:38.236686+0000","last_peered":"2024-09-10T20:39:38.236686+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:38.236686+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20879,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238057+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:38.238057+0000","last_peered":"2024-09-10T20:39:38.238057+0000","last_clean":"2024-09-10T20:39:38.238057+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:38.238057+0000","last_undegraded":"2024-09-10T20:39:38.238057+0000","last_fullsized":"2024-09-10T20:39:38.238057+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.237911+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:39:38.237911+0000","last_peered":"2024-09-10T20:39:38.237911+0000","last_clean":"2024-09-10T20:39:38.237911+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:39:38.237911+0000","last_undegraded":"2024-09-10T20:39:38.237911+0000","last_fullsized":"2024-09-10T20:39:38.237911+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.340635+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:39:38.340635+0000","last_peered":"2024-09-10T20:39:38.340635+0000","last_clean":"2024-09-10T20:39:38.340635+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:39:38.340635+0000","last_undegraded":"2024-09-10T20:39:38.340635+0000","last_fullsized":"2024-09-10T20:39:38.340635+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.240786+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:39:38.240786+0000","last_peered":"2024-09-10T20:39:38.240786+0000","last_clean":"2024-09-10T20:39:38.240786+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:39:38.240786+0000","last_undegraded":"2024-09-10T20:39:38.240786+0000","last_fullsized":"2024-09-10T20:39:38.240786+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":20956,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.237580+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:38.237580+0000","last_peered":"2024-09-10T20:39:38.237580+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:38.237580+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3564}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.236424+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:39:38.236424+0000","last_peered":"2024-09-10T20:39:38.236424+0000","last_clean":"2024-09-10T20:39:38.236424+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:39:38.236424+0000","last_undegraded":"2024-09-10T20:39:38.236424+0000","last_fullsized":"2024-09-10T20:39:38.236424+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":20767,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.241791+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:39:38.241791+0000","last_peered":"2024-09-10T20:39:38.241791+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:39:38.241791+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3606}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238378+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:39:38.238378+0000","last_peered":"2024-09-10T20:39:38.238378+0000","last_clean":"2024-09-10T20:39:38.238378+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:39:38.238378+0000","last_undegraded":"2024-09-10T20:39:38.238378+0000","last_fullsized":"2024-09-10T20:39:38.238378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28044,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.240573+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:38.240573+0000","last_peered":"2024-09-10T20:39:38.240573+0000","last_clean":"2024-09-10T20:39:38.240573+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:38.240573+0000","last_undegraded":"2024-09-10T20:39:38.240573+0000","last_fullsized":"2024-09-10T20:39:38.240573+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":22321,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:40.211503+0000","last_change":"2024-09-10T20:39:36.811261+0000","last_active":"2024-09-10T20:39:40.211503+0000","last_peered":"2024-09-10T20:39:40.211503+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:39:40.211503+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2981,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":668,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2981}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238954+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:39:38.238954+0000","last_peered":"2024-09-10T20:39:38.238954+0000","last_clean":"2024-09-10T20:39:38.238954+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:39:38.238954+0000","last_undegraded":"2024-09-10T20:39:38.238954+0000","last_fullsized":"2024-09-10T20:39:38.238954+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27822,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.239210+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:38.239210+0000","last_peered":"2024-09-10T20:39:38.239210+0000","last_clean":"2024-09-10T20:39:38.239210+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:38.239210+0000","last_undegraded":"2024-09-10T20:39:38.239210+0000","last_fullsized":"2024-09-10T20:39:38.239210+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.241067+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:39:38.241067+0000","last_peered":"2024-09-10T20:39:38.241067+0000","last_clean":"2024-09-10T20:39:38.241067+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:39:38.241067+0000","last_undegraded":"2024-09-10T20:39:38.241067+0000","last_fullsized":"2024-09-10T20:39:38.241067+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21253,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.237167+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:38.237167+0000","last_peered":"2024-09-10T20:39:38.237167+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:38.237167+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.239124+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:39:38.239124+0000","last_peered":"2024-09-10T20:39:38.239124+0000","last_clean":"2024-09-10T20:39:38.239124+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:39:38.239124+0000","last_undegraded":"2024-09-10T20:39:38.239124+0000","last_fullsized":"2024-09-10T20:39:38.239124+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.237911+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:39:38.237911+0000","last_peered":"2024-09-10T20:39:38.237911+0000","last_clean":"2024-09-10T20:39:38.237911+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:39:38.237911+0000","last_undegraded":"2024-09-10T20:39:38.237911+0000","last_fullsized":"2024-09-10T20:39:38.237911+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27652,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238867+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:38.238867+0000","last_peered":"2024-09-10T20:39:38.238867+0000","last_clean":"2024-09-10T20:39:38.238867+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:38.238867+0000","last_undegraded":"2024-09-10T20:39:38.238867+0000","last_fullsized":"2024-09-10T20:39:38.238867+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24406,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18743,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176685056,"data_stored":176638087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182533,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517148,"kb_used_data":57924,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854692,"statfs":{"total":96636764160,"available":96107204608,"internally_reserved":0,"allocated":59314176,"data_stored":59032768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986305,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480332,"kb_used_data":115444,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93891508,"statfs":{"total":96636764160,"available":96144904192,"internally_reserved":0,"allocated":118214656,"data_stored":117937325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659252,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465328,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906512,"statfs":{"total":96636764160,"available":96160268288,"internally_reserved":0,"allocated":491520,"data_stored":173982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":475983534},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117772288,"data_stored":117768675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:39:44.886 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:39:44.887 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'20658", 'reported_seq': 20889, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:38.239347+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:39:38.239347+0000', 'last_peered': '2024-09-10T20:39:38.239347+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:39:38.239347+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20658, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3541}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:44.887 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:39:44.887 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20471", 'reported_seq': 20732, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:38.236538+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:39:38.236538+0000', 'last_peered': '2024-09-10T20:39:38.236538+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:39:38.236538+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17335, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3561}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:44.887 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:39:44.887 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'20608", 'reported_seq': 20903, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:38.236686+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:39:38.236686+0000', 'last_peered': '2024-09-10T20:39:38.236686+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:39:38.236686+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17203, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3486}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:44.887 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:39:44.887 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20730", 'reported_seq': 20956, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:38.237580+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:39:38.237580+0000', 'last_peered': '2024-09-10T20:39:38.237580+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:39:38.237580+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20728, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3564}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:44.887 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:39:44.887 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "51'20565", 'reported_seq': 20767, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:38.241791+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:39:38.241791+0000', 'last_peered': '2024-09-10T20:39:38.241791+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:39:38.241791+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18393, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3606}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:39:44.888 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:39:44.888 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'20830", 'reported_seq': 22321, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:40.211503+0000', 'last_change': '2024-09-10T20:39:36.811261+0000', 'last_active': '2024-09-10T20:39:40.211503+0000', 'last_peered': '2024-09-10T20:39:40.211503+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:39:40.211503+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2981, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17553, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 668, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2981}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:44.888 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:39:44.888 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "51'20986", 'reported_seq': 21253, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:38.237167+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:39:38.237167+0000', 'last_peered': '2024-09-10T20:39:38.237167+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:39:38.237167+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17813, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3655}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:44.888 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:39:45.280 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:45.281 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 24406/10 objects degraded (244060.000%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":696,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":5,"data_bytes":1468,"bytes_used":1497915392,"bytes_avail":288412377088,"bytes_total":289910292480,"degraded_objects":24406,"degraded_total":10,"degraded_ratio":2440.5999999999999,"recovering_objects_per_sec":132,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1063,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:39:45.294 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:39:45.294 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:39:45.653 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:45.654 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:39:45.669 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":395,"stamp":"2024-09-10T20:39:45.059373+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24406,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18745,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1462808,"kb_used_data":173848,"kb_used_omap":12,"kb_used_meta":1288883,"kb_avail":281652712,"statfs":{"total":289910292480,"available":288412377088,"internally_reserved":0,"allocated":178020352,"data_stored":177144075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1319816194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1063,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1063,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001394"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20889,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.239347+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:38.239347+0000","last_peered":"2024-09-10T20:39:38.239347+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:38.239347+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21071,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.241208+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:38.241208+0000","last_peered":"2024-09-10T20:39:38.241208+0000","last_clean":"2024-09-10T20:39:38.241208+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:38.241208+0000","last_undegraded":"2024-09-10T20:39:38.241208+0000","last_fullsized":"2024-09-10T20:39:38.241208+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28083,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238999+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:38.238999+0000","last_peered":"2024-09-10T20:39:38.238999+0000","last_clean":"2024-09-10T20:39:38.238999+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:38.238999+0000","last_undegraded":"2024-09-10T20:39:38.238999+0000","last_fullsized":"2024-09-10T20:39:38.238999+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20703,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.240860+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:38.240860+0000","last_peered":"2024-09-10T20:39:38.240860+0000","last_clean":"2024-09-10T20:39:38.240860+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:38.240860+0000","last_undegraded":"2024-09-10T20:39:38.240860+0000","last_fullsized":"2024-09-10T20:39:38.240860+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21214,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.236997+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:38.236997+0000","last_peered":"2024-09-10T20:39:38.236997+0000","last_clean":"2024-09-10T20:39:38.236997+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:38.236997+0000","last_undegraded":"2024-09-10T20:39:38.236997+0000","last_fullsized":"2024-09-10T20:39:38.236997+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20732,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.236538+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:38.236538+0000","last_peered":"2024-09-10T20:39:38.236538+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:38.236538+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20903,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.236686+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:38.236686+0000","last_peered":"2024-09-10T20:39:38.236686+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:38.236686+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20879,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238057+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:38.238057+0000","last_peered":"2024-09-10T20:39:38.238057+0000","last_clean":"2024-09-10T20:39:38.238057+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:38.238057+0000","last_undegraded":"2024-09-10T20:39:38.238057+0000","last_fullsized":"2024-09-10T20:39:38.238057+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.237911+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:39:38.237911+0000","last_peered":"2024-09-10T20:39:38.237911+0000","last_clean":"2024-09-10T20:39:38.237911+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:39:38.237911+0000","last_undegraded":"2024-09-10T20:39:38.237911+0000","last_fullsized":"2024-09-10T20:39:38.237911+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.340635+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:39:38.340635+0000","last_peered":"2024-09-10T20:39:38.340635+0000","last_clean":"2024-09-10T20:39:38.340635+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:39:38.340635+0000","last_undegraded":"2024-09-10T20:39:38.340635+0000","last_fullsized":"2024-09-10T20:39:38.340635+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.240786+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:39:38.240786+0000","last_peered":"2024-09-10T20:39:38.240786+0000","last_clean":"2024-09-10T20:39:38.240786+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:39:38.240786+0000","last_undegraded":"2024-09-10T20:39:38.240786+0000","last_fullsized":"2024-09-10T20:39:38.240786+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":20956,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.237580+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:38.237580+0000","last_peered":"2024-09-10T20:39:38.237580+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:38.237580+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3564}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.236424+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:39:38.236424+0000","last_peered":"2024-09-10T20:39:38.236424+0000","last_clean":"2024-09-10T20:39:38.236424+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:39:38.236424+0000","last_undegraded":"2024-09-10T20:39:38.236424+0000","last_fullsized":"2024-09-10T20:39:38.236424+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":20767,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.241791+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:39:38.241791+0000","last_peered":"2024-09-10T20:39:38.241791+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:39:38.241791+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3606}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238378+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:39:38.238378+0000","last_peered":"2024-09-10T20:39:38.238378+0000","last_clean":"2024-09-10T20:39:38.238378+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:39:38.238378+0000","last_undegraded":"2024-09-10T20:39:38.238378+0000","last_fullsized":"2024-09-10T20:39:38.238378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28044,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.240573+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:38.240573+0000","last_peered":"2024-09-10T20:39:38.240573+0000","last_clean":"2024-09-10T20:39:38.240573+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:38.240573+0000","last_undegraded":"2024-09-10T20:39:38.240573+0000","last_fullsized":"2024-09-10T20:39:38.240573+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":22321,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:40.211503+0000","last_change":"2024-09-10T20:39:36.811261+0000","last_active":"2024-09-10T20:39:40.211503+0000","last_peered":"2024-09-10T20:39:40.211503+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:39:40.211503+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2981,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":668,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2981}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238954+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:39:38.238954+0000","last_peered":"2024-09-10T20:39:38.238954+0000","last_clean":"2024-09-10T20:39:38.238954+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:39:38.238954+0000","last_undegraded":"2024-09-10T20:39:38.238954+0000","last_fullsized":"2024-09-10T20:39:38.238954+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27822,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.239210+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:38.239210+0000","last_peered":"2024-09-10T20:39:38.239210+0000","last_clean":"2024-09-10T20:39:38.239210+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:38.239210+0000","last_undegraded":"2024-09-10T20:39:38.239210+0000","last_fullsized":"2024-09-10T20:39:38.239210+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.241067+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:39:38.241067+0000","last_peered":"2024-09-10T20:39:38.241067+0000","last_clean":"2024-09-10T20:39:38.241067+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:39:38.241067+0000","last_undegraded":"2024-09-10T20:39:38.241067+0000","last_fullsized":"2024-09-10T20:39:38.241067+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21253,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.237167+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:38.237167+0000","last_peered":"2024-09-10T20:39:38.237167+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:38.237167+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.239124+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:39:38.239124+0000","last_peered":"2024-09-10T20:39:38.239124+0000","last_clean":"2024-09-10T20:39:38.239124+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:39:38.239124+0000","last_undegraded":"2024-09-10T20:39:38.239124+0000","last_fullsized":"2024-09-10T20:39:38.239124+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.237911+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:39:38.237911+0000","last_peered":"2024-09-10T20:39:38.237911+0000","last_clean":"2024-09-10T20:39:38.237911+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:39:38.237911+0000","last_undegraded":"2024-09-10T20:39:38.237911+0000","last_fullsized":"2024-09-10T20:39:38.237911+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27652,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238867+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:38.238867+0000","last_peered":"2024-09-10T20:39:38.238867+0000","last_clean":"2024-09-10T20:39:38.238867+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:38.238867+0000","last_undegraded":"2024-09-10T20:39:38.238867+0000","last_fullsized":"2024-09-10T20:39:38.238867+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24406,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18743,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176685056,"data_stored":176638087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182533,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517148,"kb_used_data":57924,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854692,"statfs":{"total":96636764160,"available":96107204608,"internally_reserved":0,"allocated":59314176,"data_stored":59032768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986306,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480332,"kb_used_data":115444,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93891508,"statfs":{"total":96636764160,"available":96144904192,"internally_reserved":0,"allocated":118214656,"data_stored":117937325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659252,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465328,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906512,"statfs":{"total":96636764160,"available":96160268288,"internally_reserved":0,"allocated":491520,"data_stored":173982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":475983534},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117772288,"data_stored":117768675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:39:48.672 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:39:48.980 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:48.981 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:39:48.995 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":396,"stamp":"2024-09-10T20:39:47.059823+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23592,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19559,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1462808,"kb_used_data":173848,"kb_used_omap":12,"kb_used_meta":1288883,"kb_avail":281652712,"statfs":{"total":289910292480,"available":288412377088,"internally_reserved":0,"allocated":178020352,"data_stored":177144075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1319816194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1877,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1877,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.828206"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20889,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.239347+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:38.239347+0000","last_peered":"2024-09-10T20:39:38.239347+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:38.239347+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21071,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.241208+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:38.241208+0000","last_peered":"2024-09-10T20:39:38.241208+0000","last_clean":"2024-09-10T20:39:38.241208+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:38.241208+0000","last_undegraded":"2024-09-10T20:39:38.241208+0000","last_fullsized":"2024-09-10T20:39:38.241208+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28083,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238999+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:38.238999+0000","last_peered":"2024-09-10T20:39:38.238999+0000","last_clean":"2024-09-10T20:39:38.238999+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:38.238999+0000","last_undegraded":"2024-09-10T20:39:38.238999+0000","last_fullsized":"2024-09-10T20:39:38.238999+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20703,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.240860+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:38.240860+0000","last_peered":"2024-09-10T20:39:38.240860+0000","last_clean":"2024-09-10T20:39:38.240860+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:38.240860+0000","last_undegraded":"2024-09-10T20:39:38.240860+0000","last_fullsized":"2024-09-10T20:39:38.240860+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21214,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.236997+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:38.236997+0000","last_peered":"2024-09-10T20:39:38.236997+0000","last_clean":"2024-09-10T20:39:38.236997+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:38.236997+0000","last_undegraded":"2024-09-10T20:39:38.236997+0000","last_fullsized":"2024-09-10T20:39:38.236997+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20732,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.236538+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:38.236538+0000","last_peered":"2024-09-10T20:39:38.236538+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:38.236538+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20903,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.236686+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:38.236686+0000","last_peered":"2024-09-10T20:39:38.236686+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:38.236686+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20879,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238057+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:38.238057+0000","last_peered":"2024-09-10T20:39:38.238057+0000","last_clean":"2024-09-10T20:39:38.238057+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:38.238057+0000","last_undegraded":"2024-09-10T20:39:38.238057+0000","last_fullsized":"2024-09-10T20:39:38.238057+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.237911+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:39:38.237911+0000","last_peered":"2024-09-10T20:39:38.237911+0000","last_clean":"2024-09-10T20:39:38.237911+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:39:38.237911+0000","last_undegraded":"2024-09-10T20:39:38.237911+0000","last_fullsized":"2024-09-10T20:39:38.237911+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.340635+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:39:38.340635+0000","last_peered":"2024-09-10T20:39:38.340635+0000","last_clean":"2024-09-10T20:39:38.340635+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:39:38.340635+0000","last_undegraded":"2024-09-10T20:39:38.340635+0000","last_fullsized":"2024-09-10T20:39:38.340635+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.240786+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:39:38.240786+0000","last_peered":"2024-09-10T20:39:38.240786+0000","last_clean":"2024-09-10T20:39:38.240786+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:39:38.240786+0000","last_undegraded":"2024-09-10T20:39:38.240786+0000","last_fullsized":"2024-09-10T20:39:38.240786+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":20956,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.237580+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:38.237580+0000","last_peered":"2024-09-10T20:39:38.237580+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:38.237580+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3564}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.236424+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:39:38.236424+0000","last_peered":"2024-09-10T20:39:38.236424+0000","last_clean":"2024-09-10T20:39:38.236424+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:39:38.236424+0000","last_undegraded":"2024-09-10T20:39:38.236424+0000","last_fullsized":"2024-09-10T20:39:38.236424+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":20767,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.241791+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:39:38.241791+0000","last_peered":"2024-09-10T20:39:38.241791+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:39:38.241791+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3606}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238378+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:39:38.238378+0000","last_peered":"2024-09-10T20:39:38.238378+0000","last_clean":"2024-09-10T20:39:38.238378+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:39:38.238378+0000","last_undegraded":"2024-09-10T20:39:38.238378+0000","last_fullsized":"2024-09-10T20:39:38.238378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28044,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.240573+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:38.240573+0000","last_peered":"2024-09-10T20:39:38.240573+0000","last_clean":"2024-09-10T20:39:38.240573+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:38.240573+0000","last_undegraded":"2024-09-10T20:39:38.240573+0000","last_fullsized":"2024-09-10T20:39:38.240573+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":23949,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:45.211671+0000","last_change":"2024-09-10T20:39:36.811261+0000","last_active":"2024-09-10T20:39:45.211671+0000","last_peered":"2024-09-10T20:39:45.211671+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:39:45.211671+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2167,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1482,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2167}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238954+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:39:38.238954+0000","last_peered":"2024-09-10T20:39:38.238954+0000","last_clean":"2024-09-10T20:39:38.238954+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:39:38.238954+0000","last_undegraded":"2024-09-10T20:39:38.238954+0000","last_fullsized":"2024-09-10T20:39:38.238954+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27822,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.239210+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:38.239210+0000","last_peered":"2024-09-10T20:39:38.239210+0000","last_clean":"2024-09-10T20:39:38.239210+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:38.239210+0000","last_undegraded":"2024-09-10T20:39:38.239210+0000","last_fullsized":"2024-09-10T20:39:38.239210+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.241067+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:39:38.241067+0000","last_peered":"2024-09-10T20:39:38.241067+0000","last_clean":"2024-09-10T20:39:38.241067+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:39:38.241067+0000","last_undegraded":"2024-09-10T20:39:38.241067+0000","last_fullsized":"2024-09-10T20:39:38.241067+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21253,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.237167+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:38.237167+0000","last_peered":"2024-09-10T20:39:38.237167+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:38.237167+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.239124+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:39:38.239124+0000","last_peered":"2024-09-10T20:39:38.239124+0000","last_clean":"2024-09-10T20:39:38.239124+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:39:38.239124+0000","last_undegraded":"2024-09-10T20:39:38.239124+0000","last_fullsized":"2024-09-10T20:39:38.239124+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.237911+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:39:38.237911+0000","last_peered":"2024-09-10T20:39:38.237911+0000","last_clean":"2024-09-10T20:39:38.237911+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:39:38.237911+0000","last_undegraded":"2024-09-10T20:39:38.237911+0000","last_fullsized":"2024-09-10T20:39:38.237911+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27652,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238867+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:38.238867+0000","last_peered":"2024-09-10T20:39:38.238867+0000","last_clean":"2024-09-10T20:39:38.238867+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:38.238867+0000","last_undegraded":"2024-09-10T20:39:38.238867+0000","last_fullsized":"2024-09-10T20:39:38.238867+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23592,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19557,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176685056,"data_stored":176638087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182534,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517148,"kb_used_data":57924,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854692,"statfs":{"total":96636764160,"available":96107204608,"internally_reserved":0,"allocated":59314176,"data_stored":59032768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986306,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480332,"kb_used_data":115444,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93891508,"statfs":{"total":96636764160,"available":96144904192,"internally_reserved":0,"allocated":118214656,"data_stored":117937325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659252,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465328,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906512,"statfs":{"total":96636764160,"available":96160268288,"internally_reserved":0,"allocated":491520,"data_stored":173982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":475983534},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117772288,"data_stored":117768675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:39:48.996 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:39:49.326 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:49.326 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:39:49.338 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":397,"stamp":"2024-09-10T20:39:49.060194+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23592,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19559,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1466968,"kb_used_data":173848,"kb_used_omap":12,"kb_used_meta":1293043,"kb_avail":281648552,"statfs":{"total":289910292480,"available":288408117248,"internally_reserved":0,"allocated":178020352,"data_stored":177144075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1324076034},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1426,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1426,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.824895"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20889,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.239347+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:38.239347+0000","last_peered":"2024-09-10T20:39:38.239347+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:38.239347+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21071,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.241208+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:38.241208+0000","last_peered":"2024-09-10T20:39:38.241208+0000","last_clean":"2024-09-10T20:39:38.241208+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:38.241208+0000","last_undegraded":"2024-09-10T20:39:38.241208+0000","last_fullsized":"2024-09-10T20:39:38.241208+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28083,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238999+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:38.238999+0000","last_peered":"2024-09-10T20:39:38.238999+0000","last_clean":"2024-09-10T20:39:38.238999+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:38.238999+0000","last_undegraded":"2024-09-10T20:39:38.238999+0000","last_fullsized":"2024-09-10T20:39:38.238999+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20703,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.240860+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:38.240860+0000","last_peered":"2024-09-10T20:39:38.240860+0000","last_clean":"2024-09-10T20:39:38.240860+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:38.240860+0000","last_undegraded":"2024-09-10T20:39:38.240860+0000","last_fullsized":"2024-09-10T20:39:38.240860+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21214,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.236997+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:38.236997+0000","last_peered":"2024-09-10T20:39:38.236997+0000","last_clean":"2024-09-10T20:39:38.236997+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:38.236997+0000","last_undegraded":"2024-09-10T20:39:38.236997+0000","last_fullsized":"2024-09-10T20:39:38.236997+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20732,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.236538+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:38.236538+0000","last_peered":"2024-09-10T20:39:38.236538+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:38.236538+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20903,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.236686+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:38.236686+0000","last_peered":"2024-09-10T20:39:38.236686+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:38.236686+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20879,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238057+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:38.238057+0000","last_peered":"2024-09-10T20:39:38.238057+0000","last_clean":"2024-09-10T20:39:38.238057+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:38.238057+0000","last_undegraded":"2024-09-10T20:39:38.238057+0000","last_fullsized":"2024-09-10T20:39:38.238057+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.237911+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:39:38.237911+0000","last_peered":"2024-09-10T20:39:38.237911+0000","last_clean":"2024-09-10T20:39:38.237911+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:39:38.237911+0000","last_undegraded":"2024-09-10T20:39:38.237911+0000","last_fullsized":"2024-09-10T20:39:38.237911+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.340635+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:39:38.340635+0000","last_peered":"2024-09-10T20:39:38.340635+0000","last_clean":"2024-09-10T20:39:38.340635+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:39:38.340635+0000","last_undegraded":"2024-09-10T20:39:38.340635+0000","last_fullsized":"2024-09-10T20:39:38.340635+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.240786+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:39:38.240786+0000","last_peered":"2024-09-10T20:39:38.240786+0000","last_clean":"2024-09-10T20:39:38.240786+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:39:38.240786+0000","last_undegraded":"2024-09-10T20:39:38.240786+0000","last_fullsized":"2024-09-10T20:39:38.240786+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":20956,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.237580+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:38.237580+0000","last_peered":"2024-09-10T20:39:38.237580+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:38.237580+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3564}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.236424+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:39:38.236424+0000","last_peered":"2024-09-10T20:39:38.236424+0000","last_clean":"2024-09-10T20:39:38.236424+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:39:38.236424+0000","last_undegraded":"2024-09-10T20:39:38.236424+0000","last_fullsized":"2024-09-10T20:39:38.236424+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":20767,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.241791+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:39:38.241791+0000","last_peered":"2024-09-10T20:39:38.241791+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:39:38.241791+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3606}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238378+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:39:38.238378+0000","last_peered":"2024-09-10T20:39:38.238378+0000","last_clean":"2024-09-10T20:39:38.238378+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:39:38.238378+0000","last_undegraded":"2024-09-10T20:39:38.238378+0000","last_fullsized":"2024-09-10T20:39:38.238378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28044,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.240573+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:38.240573+0000","last_peered":"2024-09-10T20:39:38.240573+0000","last_clean":"2024-09-10T20:39:38.240573+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:38.240573+0000","last_undegraded":"2024-09-10T20:39:38.240573+0000","last_fullsized":"2024-09-10T20:39:38.240573+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":23949,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:45.211671+0000","last_change":"2024-09-10T20:39:36.811261+0000","last_active":"2024-09-10T20:39:45.211671+0000","last_peered":"2024-09-10T20:39:45.211671+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:39:45.211671+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2167,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1482,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2167}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238954+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:39:38.238954+0000","last_peered":"2024-09-10T20:39:38.238954+0000","last_clean":"2024-09-10T20:39:38.238954+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:39:38.238954+0000","last_undegraded":"2024-09-10T20:39:38.238954+0000","last_fullsized":"2024-09-10T20:39:38.238954+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27822,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.239210+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:38.239210+0000","last_peered":"2024-09-10T20:39:38.239210+0000","last_clean":"2024-09-10T20:39:38.239210+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:38.239210+0000","last_undegraded":"2024-09-10T20:39:38.239210+0000","last_fullsized":"2024-09-10T20:39:38.239210+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.241067+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:39:38.241067+0000","last_peered":"2024-09-10T20:39:38.241067+0000","last_clean":"2024-09-10T20:39:38.241067+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:39:38.241067+0000","last_undegraded":"2024-09-10T20:39:38.241067+0000","last_fullsized":"2024-09-10T20:39:38.241067+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21253,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.237167+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:38.237167+0000","last_peered":"2024-09-10T20:39:38.237167+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:38.237167+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.239124+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:39:38.239124+0000","last_peered":"2024-09-10T20:39:38.239124+0000","last_clean":"2024-09-10T20:39:38.239124+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:39:38.239124+0000","last_undegraded":"2024-09-10T20:39:38.239124+0000","last_fullsized":"2024-09-10T20:39:38.239124+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.237911+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:39:38.237911+0000","last_peered":"2024-09-10T20:39:38.237911+0000","last_clean":"2024-09-10T20:39:38.237911+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:39:38.237911+0000","last_undegraded":"2024-09-10T20:39:38.237911+0000","last_fullsized":"2024-09-10T20:39:38.237911+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27652,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238867+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:38.238867+0000","last_peered":"2024-09-10T20:39:38.238867+0000","last_clean":"2024-09-10T20:39:38.238867+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:38.238867+0000","last_undegraded":"2024-09-10T20:39:38.238867+0000","last_fullsized":"2024-09-10T20:39:38.238867+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23592,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19557,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176685056,"data_stored":176638087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182534,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517148,"kb_used_data":57924,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854692,"statfs":{"total":96636764160,"available":96107204608,"internally_reserved":0,"allocated":59314176,"data_stored":59032768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986306,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480332,"kb_used_data":115444,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93891508,"statfs":{"total":96636764160,"available":96144904192,"internally_reserved":0,"allocated":118214656,"data_stored":117937325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659253,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469488,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902352,"statfs":{"total":96636764160,"available":96156008448,"internally_reserved":0,"allocated":491520,"data_stored":173982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480243374},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117772288,"data_stored":117768675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:39:49.340 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:39:49.340 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'20658", 'reported_seq': 20889, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:38.239347+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:39:38.239347+0000', 'last_peered': '2024-09-10T20:39:38.239347+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:39:38.239347+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20658, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3541}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:49.340 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:39:49.340 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20471", 'reported_seq': 20732, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:38.236538+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:39:38.236538+0000', 'last_peered': '2024-09-10T20:39:38.236538+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:39:38.236538+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17335, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3561}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:49.340 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:39:49.340 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'20608", 'reported_seq': 20903, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:38.236686+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:39:38.236686+0000', 'last_peered': '2024-09-10T20:39:38.236686+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:39:38.236686+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17203, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3486}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:49.340 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:39:49.341 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20730", 'reported_seq': 20956, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:38.237580+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:39:38.237580+0000', 'last_peered': '2024-09-10T20:39:38.237580+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:39:38.237580+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20728, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3564}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:49.341 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:39:49.341 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "51'20565", 'reported_seq': 20767, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:38.241791+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:39:38.241791+0000', 'last_peered': '2024-09-10T20:39:38.241791+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:39:38.241791+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18393, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3606}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:39:49.341 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:39:49.341 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'20830", 'reported_seq': 23949, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:45.211671+0000', 'last_change': '2024-09-10T20:39:36.811261+0000', 'last_active': '2024-09-10T20:39:45.211671+0000', 'last_peered': '2024-09-10T20:39:45.211671+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:39:45.211671+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2167, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17553, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1482, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2167}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:49.341 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:39:49.341 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "51'20986", 'reported_seq': 21253, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:38.237167+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:39:38.237167+0000', 'last_peered': '2024-09-10T20:39:38.237167+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:39:38.237167+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17813, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3655}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:49.342 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:39:49.775 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:49.776 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 23592/10 objects degraded (235920.000%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":701,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":5,"data_bytes":1468,"bytes_used":1502175232,"bytes_avail":288408117248,"bytes_total":289910292480,"degraded_objects":23592,"degraded_total":10,"degraded_ratio":2359.1999999999998,"recovering_objects_per_sec":131,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1426,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:39:49.789 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:39:49.790 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:39:50.089 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:50.090 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:39:50.102 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":397,"stamp":"2024-09-10T20:39:49.060194+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23592,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19559,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1466968,"kb_used_data":173848,"kb_used_omap":12,"kb_used_meta":1293043,"kb_avail":281648552,"statfs":{"total":289910292480,"available":288408117248,"internally_reserved":0,"allocated":178020352,"data_stored":177144075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1324076034},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1426,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1426,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.824895"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20889,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.239347+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:38.239347+0000","last_peered":"2024-09-10T20:39:38.239347+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:38.239347+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21071,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.241208+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:38.241208+0000","last_peered":"2024-09-10T20:39:38.241208+0000","last_clean":"2024-09-10T20:39:38.241208+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:38.241208+0000","last_undegraded":"2024-09-10T20:39:38.241208+0000","last_fullsized":"2024-09-10T20:39:38.241208+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28083,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238999+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:38.238999+0000","last_peered":"2024-09-10T20:39:38.238999+0000","last_clean":"2024-09-10T20:39:38.238999+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:38.238999+0000","last_undegraded":"2024-09-10T20:39:38.238999+0000","last_fullsized":"2024-09-10T20:39:38.238999+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20703,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.240860+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:38.240860+0000","last_peered":"2024-09-10T20:39:38.240860+0000","last_clean":"2024-09-10T20:39:38.240860+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:38.240860+0000","last_undegraded":"2024-09-10T20:39:38.240860+0000","last_fullsized":"2024-09-10T20:39:38.240860+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21214,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.236997+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:38.236997+0000","last_peered":"2024-09-10T20:39:38.236997+0000","last_clean":"2024-09-10T20:39:38.236997+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:38.236997+0000","last_undegraded":"2024-09-10T20:39:38.236997+0000","last_fullsized":"2024-09-10T20:39:38.236997+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20732,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.236538+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:38.236538+0000","last_peered":"2024-09-10T20:39:38.236538+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:38.236538+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20903,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.236686+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:38.236686+0000","last_peered":"2024-09-10T20:39:38.236686+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:38.236686+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20879,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238057+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:38.238057+0000","last_peered":"2024-09-10T20:39:38.238057+0000","last_clean":"2024-09-10T20:39:38.238057+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:38.238057+0000","last_undegraded":"2024-09-10T20:39:38.238057+0000","last_fullsized":"2024-09-10T20:39:38.238057+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.237911+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:39:38.237911+0000","last_peered":"2024-09-10T20:39:38.237911+0000","last_clean":"2024-09-10T20:39:38.237911+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:39:38.237911+0000","last_undegraded":"2024-09-10T20:39:38.237911+0000","last_fullsized":"2024-09-10T20:39:38.237911+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.340635+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:39:38.340635+0000","last_peered":"2024-09-10T20:39:38.340635+0000","last_clean":"2024-09-10T20:39:38.340635+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:39:38.340635+0000","last_undegraded":"2024-09-10T20:39:38.340635+0000","last_fullsized":"2024-09-10T20:39:38.340635+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.240786+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:39:38.240786+0000","last_peered":"2024-09-10T20:39:38.240786+0000","last_clean":"2024-09-10T20:39:38.240786+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:39:38.240786+0000","last_undegraded":"2024-09-10T20:39:38.240786+0000","last_fullsized":"2024-09-10T20:39:38.240786+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":20956,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.237580+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:38.237580+0000","last_peered":"2024-09-10T20:39:38.237580+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:38.237580+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3564}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.236424+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:39:38.236424+0000","last_peered":"2024-09-10T20:39:38.236424+0000","last_clean":"2024-09-10T20:39:38.236424+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:39:38.236424+0000","last_undegraded":"2024-09-10T20:39:38.236424+0000","last_fullsized":"2024-09-10T20:39:38.236424+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":20767,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.241791+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:39:38.241791+0000","last_peered":"2024-09-10T20:39:38.241791+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:39:38.241791+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3606}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238378+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:39:38.238378+0000","last_peered":"2024-09-10T20:39:38.238378+0000","last_clean":"2024-09-10T20:39:38.238378+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:39:38.238378+0000","last_undegraded":"2024-09-10T20:39:38.238378+0000","last_fullsized":"2024-09-10T20:39:38.238378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28044,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.240573+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:38.240573+0000","last_peered":"2024-09-10T20:39:38.240573+0000","last_clean":"2024-09-10T20:39:38.240573+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:38.240573+0000","last_undegraded":"2024-09-10T20:39:38.240573+0000","last_fullsized":"2024-09-10T20:39:38.240573+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":23949,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:45.211671+0000","last_change":"2024-09-10T20:39:36.811261+0000","last_active":"2024-09-10T20:39:45.211671+0000","last_peered":"2024-09-10T20:39:45.211671+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:39:45.211671+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2167,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1482,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2167}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238954+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:39:38.238954+0000","last_peered":"2024-09-10T20:39:38.238954+0000","last_clean":"2024-09-10T20:39:38.238954+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:39:38.238954+0000","last_undegraded":"2024-09-10T20:39:38.238954+0000","last_fullsized":"2024-09-10T20:39:38.238954+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27822,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.239210+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:38.239210+0000","last_peered":"2024-09-10T20:39:38.239210+0000","last_clean":"2024-09-10T20:39:38.239210+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:38.239210+0000","last_undegraded":"2024-09-10T20:39:38.239210+0000","last_fullsized":"2024-09-10T20:39:38.239210+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.241067+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:39:38.241067+0000","last_peered":"2024-09-10T20:39:38.241067+0000","last_clean":"2024-09-10T20:39:38.241067+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:39:38.241067+0000","last_undegraded":"2024-09-10T20:39:38.241067+0000","last_fullsized":"2024-09-10T20:39:38.241067+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21253,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.237167+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:38.237167+0000","last_peered":"2024-09-10T20:39:38.237167+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:38.237167+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.239124+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:39:38.239124+0000","last_peered":"2024-09-10T20:39:38.239124+0000","last_clean":"2024-09-10T20:39:38.239124+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:39:38.239124+0000","last_undegraded":"2024-09-10T20:39:38.239124+0000","last_fullsized":"2024-09-10T20:39:38.239124+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.237911+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:39:38.237911+0000","last_peered":"2024-09-10T20:39:38.237911+0000","last_clean":"2024-09-10T20:39:38.237911+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:39:38.237911+0000","last_undegraded":"2024-09-10T20:39:38.237911+0000","last_fullsized":"2024-09-10T20:39:38.237911+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27652,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238867+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:38.238867+0000","last_peered":"2024-09-10T20:39:38.238867+0000","last_clean":"2024-09-10T20:39:38.238867+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:38.238867+0000","last_undegraded":"2024-09-10T20:39:38.238867+0000","last_fullsized":"2024-09-10T20:39:38.238867+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23592,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19557,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176685056,"data_stored":176638087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182534,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517148,"kb_used_data":57924,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854692,"statfs":{"total":96636764160,"available":96107204608,"internally_reserved":0,"allocated":59314176,"data_stored":59032768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986306,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480332,"kb_used_data":115444,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93891508,"statfs":{"total":96636764160,"available":96144904192,"internally_reserved":0,"allocated":118214656,"data_stored":117937325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659253,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469488,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902352,"statfs":{"total":96636764160,"available":96156008448,"internally_reserved":0,"allocated":491520,"data_stored":173982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480243374},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117772288,"data_stored":117768675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:39:53.104 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:39:53.424 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:53.424 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:39:53.436 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":399,"stamp":"2024-09-10T20:39:53.060963+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22807,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20344,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1476696,"kb_used_data":173848,"kb_used_omap":12,"kb_used_meta":1302771,"kb_avail":281638824,"statfs":{"total":289910292480,"available":288398155776,"internally_reserved":0,"allocated":178020352,"data_stored":177144075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1334037506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1599,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1599,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002283"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20889,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.239347+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:38.239347+0000","last_peered":"2024-09-10T20:39:38.239347+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:38.239347+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21071,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.241208+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:38.241208+0000","last_peered":"2024-09-10T20:39:38.241208+0000","last_clean":"2024-09-10T20:39:38.241208+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:38.241208+0000","last_undegraded":"2024-09-10T20:39:38.241208+0000","last_fullsized":"2024-09-10T20:39:38.241208+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28083,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238999+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:38.238999+0000","last_peered":"2024-09-10T20:39:38.238999+0000","last_clean":"2024-09-10T20:39:38.238999+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:38.238999+0000","last_undegraded":"2024-09-10T20:39:38.238999+0000","last_fullsized":"2024-09-10T20:39:38.238999+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20703,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.240860+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:38.240860+0000","last_peered":"2024-09-10T20:39:38.240860+0000","last_clean":"2024-09-10T20:39:38.240860+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:38.240860+0000","last_undegraded":"2024-09-10T20:39:38.240860+0000","last_fullsized":"2024-09-10T20:39:38.240860+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21214,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.236997+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:38.236997+0000","last_peered":"2024-09-10T20:39:38.236997+0000","last_clean":"2024-09-10T20:39:38.236997+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:38.236997+0000","last_undegraded":"2024-09-10T20:39:38.236997+0000","last_fullsized":"2024-09-10T20:39:38.236997+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20732,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.236538+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:38.236538+0000","last_peered":"2024-09-10T20:39:38.236538+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:38.236538+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20903,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.236686+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:38.236686+0000","last_peered":"2024-09-10T20:39:38.236686+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:38.236686+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20880,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238170+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:38.238170+0000","last_peered":"2024-09-10T20:39:38.238170+0000","last_clean":"2024-09-10T20:39:38.238170+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:38.238170+0000","last_undegraded":"2024-09-10T20:39:38.238170+0000","last_fullsized":"2024-09-10T20:39:38.238170+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.237911+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:39:38.237911+0000","last_peered":"2024-09-10T20:39:38.237911+0000","last_clean":"2024-09-10T20:39:38.237911+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:39:38.237911+0000","last_undegraded":"2024-09-10T20:39:38.237911+0000","last_fullsized":"2024-09-10T20:39:38.237911+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.340635+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:39:38.340635+0000","last_peered":"2024-09-10T20:39:38.340635+0000","last_clean":"2024-09-10T20:39:38.340635+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:39:38.340635+0000","last_undegraded":"2024-09-10T20:39:38.340635+0000","last_fullsized":"2024-09-10T20:39:38.340635+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.240786+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:39:38.240786+0000","last_peered":"2024-09-10T20:39:38.240786+0000","last_clean":"2024-09-10T20:39:38.240786+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:39:38.240786+0000","last_undegraded":"2024-09-10T20:39:38.240786+0000","last_fullsized":"2024-09-10T20:39:38.240786+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":20956,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.237580+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:38.237580+0000","last_peered":"2024-09-10T20:39:38.237580+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:38.237580+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3564}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.236424+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:39:38.236424+0000","last_peered":"2024-09-10T20:39:38.236424+0000","last_clean":"2024-09-10T20:39:38.236424+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:39:38.236424+0000","last_undegraded":"2024-09-10T20:39:38.236424+0000","last_fullsized":"2024-09-10T20:39:38.236424+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":20767,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.241791+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:39:38.241791+0000","last_peered":"2024-09-10T20:39:38.241791+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:39:38.241791+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3606}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238378+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:39:38.238378+0000","last_peered":"2024-09-10T20:39:38.238378+0000","last_clean":"2024-09-10T20:39:38.238378+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:39:38.238378+0000","last_undegraded":"2024-09-10T20:39:38.238378+0000","last_fullsized":"2024-09-10T20:39:38.238378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28044,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.240573+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:38.240573+0000","last_peered":"2024-09-10T20:39:38.240573+0000","last_clean":"2024-09-10T20:39:38.240573+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:38.240573+0000","last_undegraded":"2024-09-10T20:39:38.240573+0000","last_fullsized":"2024-09-10T20:39:38.240573+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":25519,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:50.214661+0000","last_change":"2024-09-10T20:39:36.811261+0000","last_active":"2024-09-10T20:39:50.214661+0000","last_peered":"2024-09-10T20:39:50.214661+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:39:50.214661+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2267,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1382}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238954+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:39:38.238954+0000","last_peered":"2024-09-10T20:39:38.238954+0000","last_clean":"2024-09-10T20:39:38.238954+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:39:38.238954+0000","last_undegraded":"2024-09-10T20:39:38.238954+0000","last_fullsized":"2024-09-10T20:39:38.238954+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27822,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.239210+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:38.239210+0000","last_peered":"2024-09-10T20:39:38.239210+0000","last_clean":"2024-09-10T20:39:38.239210+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:38.239210+0000","last_undegraded":"2024-09-10T20:39:38.239210+0000","last_fullsized":"2024-09-10T20:39:38.239210+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.241067+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:39:38.241067+0000","last_peered":"2024-09-10T20:39:38.241067+0000","last_clean":"2024-09-10T20:39:38.241067+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:39:38.241067+0000","last_undegraded":"2024-09-10T20:39:38.241067+0000","last_fullsized":"2024-09-10T20:39:38.241067+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21253,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.237167+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:38.237167+0000","last_peered":"2024-09-10T20:39:38.237167+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:38.237167+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.239124+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:39:38.239124+0000","last_peered":"2024-09-10T20:39:38.239124+0000","last_clean":"2024-09-10T20:39:38.239124+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:39:38.239124+0000","last_undegraded":"2024-09-10T20:39:38.239124+0000","last_fullsized":"2024-09-10T20:39:38.239124+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.237911+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:39:38.237911+0000","last_peered":"2024-09-10T20:39:38.237911+0000","last_clean":"2024-09-10T20:39:38.237911+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:39:38.237911+0000","last_undegraded":"2024-09-10T20:39:38.237911+0000","last_fullsized":"2024-09-10T20:39:38.237911+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27652,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238867+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:38.238867+0000","last_peered":"2024-09-10T20:39:38.238867+0000","last_clean":"2024-09-10T20:39:38.238867+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:38.238867+0000","last_undegraded":"2024-09-10T20:39:38.238867+0000","last_fullsized":"2024-09-10T20:39:38.238867+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22807,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20342,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176685056,"data_stored":176638087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182535,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517148,"kb_used_data":57924,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854692,"statfs":{"total":96636764160,"available":96107204608,"internally_reserved":0,"allocated":59314176,"data_stored":59032768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986307,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480332,"kb_used_data":115444,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93891508,"statfs":{"total":96636764160,"available":96144904192,"internally_reserved":0,"allocated":118214656,"data_stored":117937325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659254,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479216,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892624,"statfs":{"total":96636764160,"available":96146046976,"internally_reserved":0,"allocated":491520,"data_stored":173982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":490204846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117772288,"data_stored":117768675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:39:53.438 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:39:53.742 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:53.742 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:39:53.755 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":399,"stamp":"2024-09-10T20:39:53.060963+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22807,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20344,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1476696,"kb_used_data":173848,"kb_used_omap":12,"kb_used_meta":1302771,"kb_avail":281638824,"statfs":{"total":289910292480,"available":288398155776,"internally_reserved":0,"allocated":178020352,"data_stored":177144075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1334037506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1599,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1599,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002283"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20889,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.239347+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:38.239347+0000","last_peered":"2024-09-10T20:39:38.239347+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:38.239347+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21071,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.241208+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:38.241208+0000","last_peered":"2024-09-10T20:39:38.241208+0000","last_clean":"2024-09-10T20:39:38.241208+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:38.241208+0000","last_undegraded":"2024-09-10T20:39:38.241208+0000","last_fullsized":"2024-09-10T20:39:38.241208+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28083,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238999+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:38.238999+0000","last_peered":"2024-09-10T20:39:38.238999+0000","last_clean":"2024-09-10T20:39:38.238999+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:38.238999+0000","last_undegraded":"2024-09-10T20:39:38.238999+0000","last_fullsized":"2024-09-10T20:39:38.238999+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20703,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.240860+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:38.240860+0000","last_peered":"2024-09-10T20:39:38.240860+0000","last_clean":"2024-09-10T20:39:38.240860+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:38.240860+0000","last_undegraded":"2024-09-10T20:39:38.240860+0000","last_fullsized":"2024-09-10T20:39:38.240860+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21214,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.236997+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:38.236997+0000","last_peered":"2024-09-10T20:39:38.236997+0000","last_clean":"2024-09-10T20:39:38.236997+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:38.236997+0000","last_undegraded":"2024-09-10T20:39:38.236997+0000","last_fullsized":"2024-09-10T20:39:38.236997+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20732,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.236538+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:38.236538+0000","last_peered":"2024-09-10T20:39:38.236538+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:38.236538+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20903,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.236686+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:38.236686+0000","last_peered":"2024-09-10T20:39:38.236686+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:38.236686+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20880,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238170+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:38.238170+0000","last_peered":"2024-09-10T20:39:38.238170+0000","last_clean":"2024-09-10T20:39:38.238170+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:38.238170+0000","last_undegraded":"2024-09-10T20:39:38.238170+0000","last_fullsized":"2024-09-10T20:39:38.238170+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.237911+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:39:38.237911+0000","last_peered":"2024-09-10T20:39:38.237911+0000","last_clean":"2024-09-10T20:39:38.237911+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:39:38.237911+0000","last_undegraded":"2024-09-10T20:39:38.237911+0000","last_fullsized":"2024-09-10T20:39:38.237911+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.340635+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:39:38.340635+0000","last_peered":"2024-09-10T20:39:38.340635+0000","last_clean":"2024-09-10T20:39:38.340635+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:39:38.340635+0000","last_undegraded":"2024-09-10T20:39:38.340635+0000","last_fullsized":"2024-09-10T20:39:38.340635+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.240786+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:39:38.240786+0000","last_peered":"2024-09-10T20:39:38.240786+0000","last_clean":"2024-09-10T20:39:38.240786+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:39:38.240786+0000","last_undegraded":"2024-09-10T20:39:38.240786+0000","last_fullsized":"2024-09-10T20:39:38.240786+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":20956,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.237580+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:38.237580+0000","last_peered":"2024-09-10T20:39:38.237580+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:38.237580+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3564}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.236424+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:39:38.236424+0000","last_peered":"2024-09-10T20:39:38.236424+0000","last_clean":"2024-09-10T20:39:38.236424+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:39:38.236424+0000","last_undegraded":"2024-09-10T20:39:38.236424+0000","last_fullsized":"2024-09-10T20:39:38.236424+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":20767,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.241791+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:39:38.241791+0000","last_peered":"2024-09-10T20:39:38.241791+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:39:38.241791+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3606}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238378+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:39:38.238378+0000","last_peered":"2024-09-10T20:39:38.238378+0000","last_clean":"2024-09-10T20:39:38.238378+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:39:38.238378+0000","last_undegraded":"2024-09-10T20:39:38.238378+0000","last_fullsized":"2024-09-10T20:39:38.238378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28044,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.240573+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:38.240573+0000","last_peered":"2024-09-10T20:39:38.240573+0000","last_clean":"2024-09-10T20:39:38.240573+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:38.240573+0000","last_undegraded":"2024-09-10T20:39:38.240573+0000","last_fullsized":"2024-09-10T20:39:38.240573+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":25519,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:50.214661+0000","last_change":"2024-09-10T20:39:36.811261+0000","last_active":"2024-09-10T20:39:50.214661+0000","last_peered":"2024-09-10T20:39:50.214661+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:39:50.214661+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2267,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1382}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238954+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:39:38.238954+0000","last_peered":"2024-09-10T20:39:38.238954+0000","last_clean":"2024-09-10T20:39:38.238954+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:39:38.238954+0000","last_undegraded":"2024-09-10T20:39:38.238954+0000","last_fullsized":"2024-09-10T20:39:38.238954+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27822,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.239210+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:38.239210+0000","last_peered":"2024-09-10T20:39:38.239210+0000","last_clean":"2024-09-10T20:39:38.239210+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:38.239210+0000","last_undegraded":"2024-09-10T20:39:38.239210+0000","last_fullsized":"2024-09-10T20:39:38.239210+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.241067+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:39:38.241067+0000","last_peered":"2024-09-10T20:39:38.241067+0000","last_clean":"2024-09-10T20:39:38.241067+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:39:38.241067+0000","last_undegraded":"2024-09-10T20:39:38.241067+0000","last_fullsized":"2024-09-10T20:39:38.241067+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21253,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.237167+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:38.237167+0000","last_peered":"2024-09-10T20:39:38.237167+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:38.237167+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.239124+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:39:38.239124+0000","last_peered":"2024-09-10T20:39:38.239124+0000","last_clean":"2024-09-10T20:39:38.239124+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:39:38.239124+0000","last_undegraded":"2024-09-10T20:39:38.239124+0000","last_fullsized":"2024-09-10T20:39:38.239124+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.237911+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:39:38.237911+0000","last_peered":"2024-09-10T20:39:38.237911+0000","last_clean":"2024-09-10T20:39:38.237911+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:39:38.237911+0000","last_undegraded":"2024-09-10T20:39:38.237911+0000","last_fullsized":"2024-09-10T20:39:38.237911+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27652,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238867+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:38.238867+0000","last_peered":"2024-09-10T20:39:38.238867+0000","last_clean":"2024-09-10T20:39:38.238867+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:38.238867+0000","last_undegraded":"2024-09-10T20:39:38.238867+0000","last_fullsized":"2024-09-10T20:39:38.238867+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22807,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20342,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176685056,"data_stored":176638087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182535,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517148,"kb_used_data":57924,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854692,"statfs":{"total":96636764160,"available":96107204608,"internally_reserved":0,"allocated":59314176,"data_stored":59032768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986307,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480332,"kb_used_data":115444,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93891508,"statfs":{"total":96636764160,"available":96144904192,"internally_reserved":0,"allocated":118214656,"data_stored":117937325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659254,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479216,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892624,"statfs":{"total":96636764160,"available":96146046976,"internally_reserved":0,"allocated":491520,"data_stored":173982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":490204846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117772288,"data_stored":117768675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:39:53.756 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:39:53.757 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'20658", 'reported_seq': 20889, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:38.239347+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:39:38.239347+0000', 'last_peered': '2024-09-10T20:39:38.239347+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:39:38.239347+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20658, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3541}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:53.757 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:39:53.757 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20471", 'reported_seq': 20732, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:38.236538+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:39:38.236538+0000', 'last_peered': '2024-09-10T20:39:38.236538+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:39:38.236538+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17335, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3561}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:53.757 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:39:53.757 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'20608", 'reported_seq': 20903, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:38.236686+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:39:38.236686+0000', 'last_peered': '2024-09-10T20:39:38.236686+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:39:38.236686+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17203, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3486}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:53.757 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:39:53.757 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20730", 'reported_seq': 20956, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:38.237580+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:39:38.237580+0000', 'last_peered': '2024-09-10T20:39:38.237580+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:39:38.237580+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20728, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3564}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:53.757 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:39:53.757 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "51'20565", 'reported_seq': 20767, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:38.241791+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:39:38.241791+0000', 'last_peered': '2024-09-10T20:39:38.241791+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:39:38.241791+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18393, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3606}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:39:53.757 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:39:53.758 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'20830", 'reported_seq': 25519, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:50.214661+0000', 'last_change': '2024-09-10T20:39:36.811261+0000', 'last_active': '2024-09-10T20:39:50.214661+0000', 'last_peered': '2024-09-10T20:39:50.214661+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:39:50.214661+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1382, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17553, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2267, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1382}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:53.758 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:39:53.758 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "51'20986", 'reported_seq': 21253, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:38.237167+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:39:38.237167+0000', 'last_peered': '2024-09-10T20:39:38.237167+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:39:38.237167+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17813, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3655}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:53.758 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:39:54.154 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:54.154 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 22807/10 objects degraded (228070.000%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":705,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":5,"data_bytes":1468,"bytes_used":1512136704,"bytes_avail":288398155776,"bytes_total":289910292480,"degraded_objects":22807,"degraded_total":10,"degraded_ratio":2280.6999999999998,"recovering_objects_per_sec":133,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1599,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:39:54.168 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:39:54.168 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:39:54.468 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:54.468 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:39:54.481 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":399,"stamp":"2024-09-10T20:39:53.060963+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22807,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20344,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1476696,"kb_used_data":173848,"kb_used_omap":12,"kb_used_meta":1302771,"kb_avail":281638824,"statfs":{"total":289910292480,"available":288398155776,"internally_reserved":0,"allocated":178020352,"data_stored":177144075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1334037506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1599,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1599,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002283"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20889,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.239347+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:38.239347+0000","last_peered":"2024-09-10T20:39:38.239347+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:38.239347+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21071,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.241208+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:38.241208+0000","last_peered":"2024-09-10T20:39:38.241208+0000","last_clean":"2024-09-10T20:39:38.241208+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:38.241208+0000","last_undegraded":"2024-09-10T20:39:38.241208+0000","last_fullsized":"2024-09-10T20:39:38.241208+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28083,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238999+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:38.238999+0000","last_peered":"2024-09-10T20:39:38.238999+0000","last_clean":"2024-09-10T20:39:38.238999+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:38.238999+0000","last_undegraded":"2024-09-10T20:39:38.238999+0000","last_fullsized":"2024-09-10T20:39:38.238999+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20703,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.240860+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:38.240860+0000","last_peered":"2024-09-10T20:39:38.240860+0000","last_clean":"2024-09-10T20:39:38.240860+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:38.240860+0000","last_undegraded":"2024-09-10T20:39:38.240860+0000","last_fullsized":"2024-09-10T20:39:38.240860+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21214,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.236997+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:38.236997+0000","last_peered":"2024-09-10T20:39:38.236997+0000","last_clean":"2024-09-10T20:39:38.236997+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:38.236997+0000","last_undegraded":"2024-09-10T20:39:38.236997+0000","last_fullsized":"2024-09-10T20:39:38.236997+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20732,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.236538+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:38.236538+0000","last_peered":"2024-09-10T20:39:38.236538+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:38.236538+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20903,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.236686+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:38.236686+0000","last_peered":"2024-09-10T20:39:38.236686+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:38.236686+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20880,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238170+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:38.238170+0000","last_peered":"2024-09-10T20:39:38.238170+0000","last_clean":"2024-09-10T20:39:38.238170+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:38.238170+0000","last_undegraded":"2024-09-10T20:39:38.238170+0000","last_fullsized":"2024-09-10T20:39:38.238170+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.237911+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:39:38.237911+0000","last_peered":"2024-09-10T20:39:38.237911+0000","last_clean":"2024-09-10T20:39:38.237911+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:39:38.237911+0000","last_undegraded":"2024-09-10T20:39:38.237911+0000","last_fullsized":"2024-09-10T20:39:38.237911+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.340635+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:39:38.340635+0000","last_peered":"2024-09-10T20:39:38.340635+0000","last_clean":"2024-09-10T20:39:38.340635+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:39:38.340635+0000","last_undegraded":"2024-09-10T20:39:38.340635+0000","last_fullsized":"2024-09-10T20:39:38.340635+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.240786+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:39:38.240786+0000","last_peered":"2024-09-10T20:39:38.240786+0000","last_clean":"2024-09-10T20:39:38.240786+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:39:38.240786+0000","last_undegraded":"2024-09-10T20:39:38.240786+0000","last_fullsized":"2024-09-10T20:39:38.240786+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":20956,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.237580+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:38.237580+0000","last_peered":"2024-09-10T20:39:38.237580+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:38.237580+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3564}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.236424+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:39:38.236424+0000","last_peered":"2024-09-10T20:39:38.236424+0000","last_clean":"2024-09-10T20:39:38.236424+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:39:38.236424+0000","last_undegraded":"2024-09-10T20:39:38.236424+0000","last_fullsized":"2024-09-10T20:39:38.236424+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":20767,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.241791+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:39:38.241791+0000","last_peered":"2024-09-10T20:39:38.241791+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:39:38.241791+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3606}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238378+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:39:38.238378+0000","last_peered":"2024-09-10T20:39:38.238378+0000","last_clean":"2024-09-10T20:39:38.238378+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:39:38.238378+0000","last_undegraded":"2024-09-10T20:39:38.238378+0000","last_fullsized":"2024-09-10T20:39:38.238378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28044,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.240573+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:38.240573+0000","last_peered":"2024-09-10T20:39:38.240573+0000","last_clean":"2024-09-10T20:39:38.240573+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:38.240573+0000","last_undegraded":"2024-09-10T20:39:38.240573+0000","last_fullsized":"2024-09-10T20:39:38.240573+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":25519,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:50.214661+0000","last_change":"2024-09-10T20:39:36.811261+0000","last_active":"2024-09-10T20:39:50.214661+0000","last_peered":"2024-09-10T20:39:50.214661+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:39:50.214661+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2267,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1382}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238954+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:39:38.238954+0000","last_peered":"2024-09-10T20:39:38.238954+0000","last_clean":"2024-09-10T20:39:38.238954+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:39:38.238954+0000","last_undegraded":"2024-09-10T20:39:38.238954+0000","last_fullsized":"2024-09-10T20:39:38.238954+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27822,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.239210+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:38.239210+0000","last_peered":"2024-09-10T20:39:38.239210+0000","last_clean":"2024-09-10T20:39:38.239210+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:38.239210+0000","last_undegraded":"2024-09-10T20:39:38.239210+0000","last_fullsized":"2024-09-10T20:39:38.239210+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.241067+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:39:38.241067+0000","last_peered":"2024-09-10T20:39:38.241067+0000","last_clean":"2024-09-10T20:39:38.241067+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:39:38.241067+0000","last_undegraded":"2024-09-10T20:39:38.241067+0000","last_fullsized":"2024-09-10T20:39:38.241067+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21253,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.237167+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:38.237167+0000","last_peered":"2024-09-10T20:39:38.237167+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:38.237167+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.239124+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:39:38.239124+0000","last_peered":"2024-09-10T20:39:38.239124+0000","last_clean":"2024-09-10T20:39:38.239124+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:39:38.239124+0000","last_undegraded":"2024-09-10T20:39:38.239124+0000","last_fullsized":"2024-09-10T20:39:38.239124+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.237911+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:39:38.237911+0000","last_peered":"2024-09-10T20:39:38.237911+0000","last_clean":"2024-09-10T20:39:38.237911+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:39:38.237911+0000","last_undegraded":"2024-09-10T20:39:38.237911+0000","last_fullsized":"2024-09-10T20:39:38.237911+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27652,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238867+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:38.238867+0000","last_peered":"2024-09-10T20:39:38.238867+0000","last_clean":"2024-09-10T20:39:38.238867+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:38.238867+0000","last_undegraded":"2024-09-10T20:39:38.238867+0000","last_fullsized":"2024-09-10T20:39:38.238867+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22807,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20342,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176685056,"data_stored":176638087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182535,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517148,"kb_used_data":57924,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854692,"statfs":{"total":96636764160,"available":96107204608,"internally_reserved":0,"allocated":59314176,"data_stored":59032768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986307,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480332,"kb_used_data":115444,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93891508,"statfs":{"total":96636764160,"available":96144904192,"internally_reserved":0,"allocated":118214656,"data_stored":117937325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659254,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479216,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892624,"statfs":{"total":96636764160,"available":96146046976,"internally_reserved":0,"allocated":491520,"data_stored":173982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":490204846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117772288,"data_stored":117768675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:39:57.484 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:39:57.795 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:57.795 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:39:57.808 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":402,"stamp":"2024-09-10T20:39:57.255673+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21764,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21387,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1476696,"kb_used_data":173848,"kb_used_omap":12,"kb_used_meta":1302771,"kb_avail":281638824,"statfs":{"total":289910292480,"available":288398155776,"internally_reserved":0,"allocated":178020352,"data_stored":177144075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1334037506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1828,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1828,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.195849"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20889,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.239347+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:38.239347+0000","last_peered":"2024-09-10T20:39:38.239347+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:38.239347+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21071,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.241208+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:38.241208+0000","last_peered":"2024-09-10T20:39:38.241208+0000","last_clean":"2024-09-10T20:39:38.241208+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:38.241208+0000","last_undegraded":"2024-09-10T20:39:38.241208+0000","last_fullsized":"2024-09-10T20:39:38.241208+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28083,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238999+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:38.238999+0000","last_peered":"2024-09-10T20:39:38.238999+0000","last_clean":"2024-09-10T20:39:38.238999+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:38.238999+0000","last_undegraded":"2024-09-10T20:39:38.238999+0000","last_fullsized":"2024-09-10T20:39:38.238999+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20703,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.240860+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:38.240860+0000","last_peered":"2024-09-10T20:39:38.240860+0000","last_clean":"2024-09-10T20:39:38.240860+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:38.240860+0000","last_undegraded":"2024-09-10T20:39:38.240860+0000","last_fullsized":"2024-09-10T20:39:38.240860+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21214,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.236997+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:38.236997+0000","last_peered":"2024-09-10T20:39:38.236997+0000","last_clean":"2024-09-10T20:39:38.236997+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:38.236997+0000","last_undegraded":"2024-09-10T20:39:38.236997+0000","last_fullsized":"2024-09-10T20:39:38.236997+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20732,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.236538+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:38.236538+0000","last_peered":"2024-09-10T20:39:38.236538+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:38.236538+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20903,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.236686+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:38.236686+0000","last_peered":"2024-09-10T20:39:38.236686+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:38.236686+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20880,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238170+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:38.238170+0000","last_peered":"2024-09-10T20:39:38.238170+0000","last_clean":"2024-09-10T20:39:38.238170+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:38.238170+0000","last_undegraded":"2024-09-10T20:39:38.238170+0000","last_fullsized":"2024-09-10T20:39:38.238170+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.237911+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:39:38.237911+0000","last_peered":"2024-09-10T20:39:38.237911+0000","last_clean":"2024-09-10T20:39:38.237911+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:39:38.237911+0000","last_undegraded":"2024-09-10T20:39:38.237911+0000","last_fullsized":"2024-09-10T20:39:38.237911+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.340635+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:39:38.340635+0000","last_peered":"2024-09-10T20:39:38.340635+0000","last_clean":"2024-09-10T20:39:38.340635+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:39:38.340635+0000","last_undegraded":"2024-09-10T20:39:38.340635+0000","last_fullsized":"2024-09-10T20:39:38.340635+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.240786+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:39:38.240786+0000","last_peered":"2024-09-10T20:39:38.240786+0000","last_clean":"2024-09-10T20:39:38.240786+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:39:38.240786+0000","last_undegraded":"2024-09-10T20:39:38.240786+0000","last_fullsized":"2024-09-10T20:39:38.240786+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":20956,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.237580+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:38.237580+0000","last_peered":"2024-09-10T20:39:38.237580+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:38.237580+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3564}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.236424+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:39:38.236424+0000","last_peered":"2024-09-10T20:39:38.236424+0000","last_clean":"2024-09-10T20:39:38.236424+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:39:38.236424+0000","last_undegraded":"2024-09-10T20:39:38.236424+0000","last_fullsized":"2024-09-10T20:39:38.236424+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":20767,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.241791+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:39:38.241791+0000","last_peered":"2024-09-10T20:39:38.241791+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:39:38.241791+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3606}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238378+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:39:38.238378+0000","last_peered":"2024-09-10T20:39:38.238378+0000","last_clean":"2024-09-10T20:39:38.238378+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:39:38.238378+0000","last_undegraded":"2024-09-10T20:39:38.238378+0000","last_fullsized":"2024-09-10T20:39:38.238378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28044,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.240573+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:38.240573+0000","last_peered":"2024-09-10T20:39:38.240573+0000","last_clean":"2024-09-10T20:39:38.240573+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:38.240573+0000","last_undegraded":"2024-09-10T20:39:38.240573+0000","last_fullsized":"2024-09-10T20:39:38.240573+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":27605,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:55.211737+0000","last_change":"2024-09-10T20:39:36.811261+0000","last_active":"2024-09-10T20:39:55.211737+0000","last_peered":"2024-09-10T20:39:55.211737+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:39:55.211737+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":339,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3310,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-339}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238954+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:39:38.238954+0000","last_peered":"2024-09-10T20:39:38.238954+0000","last_clean":"2024-09-10T20:39:38.238954+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:39:38.238954+0000","last_undegraded":"2024-09-10T20:39:38.238954+0000","last_fullsized":"2024-09-10T20:39:38.238954+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27822,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.239210+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:38.239210+0000","last_peered":"2024-09-10T20:39:38.239210+0000","last_clean":"2024-09-10T20:39:38.239210+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:38.239210+0000","last_undegraded":"2024-09-10T20:39:38.239210+0000","last_fullsized":"2024-09-10T20:39:38.239210+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.241067+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:39:38.241067+0000","last_peered":"2024-09-10T20:39:38.241067+0000","last_clean":"2024-09-10T20:39:38.241067+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:39:38.241067+0000","last_undegraded":"2024-09-10T20:39:38.241067+0000","last_fullsized":"2024-09-10T20:39:38.241067+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21253,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.237167+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:38.237167+0000","last_peered":"2024-09-10T20:39:38.237167+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:38.237167+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.239124+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:39:38.239124+0000","last_peered":"2024-09-10T20:39:38.239124+0000","last_clean":"2024-09-10T20:39:38.239124+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:39:38.239124+0000","last_undegraded":"2024-09-10T20:39:38.239124+0000","last_fullsized":"2024-09-10T20:39:38.239124+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.237911+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:39:38.237911+0000","last_peered":"2024-09-10T20:39:38.237911+0000","last_clean":"2024-09-10T20:39:38.237911+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:39:38.237911+0000","last_undegraded":"2024-09-10T20:39:38.237911+0000","last_fullsized":"2024-09-10T20:39:38.237911+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27652,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238867+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:38.238867+0000","last_peered":"2024-09-10T20:39:38.238867+0000","last_clean":"2024-09-10T20:39:38.238867+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:38.238867+0000","last_undegraded":"2024-09-10T20:39:38.238867+0000","last_fullsized":"2024-09-10T20:39:38.238867+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21764,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21385,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176685056,"data_stored":176638087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182536,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517148,"kb_used_data":57924,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854692,"statfs":{"total":96636764160,"available":96107204608,"internally_reserved":0,"allocated":59314176,"data_stored":59032768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986308,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480332,"kb_used_data":115444,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93891508,"statfs":{"total":96636764160,"available":96144904192,"internally_reserved":0,"allocated":118214656,"data_stored":117937325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659254,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479216,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892624,"statfs":{"total":96636764160,"available":96146046976,"internally_reserved":0,"allocated":491520,"data_stored":173982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":490204846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117772288,"data_stored":117768675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:39:57.809 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:39:58.129 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:58.130 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:39:58.144 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":402,"stamp":"2024-09-10T20:39:57.255673+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21764,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21387,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1476696,"kb_used_data":173848,"kb_used_omap":12,"kb_used_meta":1302771,"kb_avail":281638824,"statfs":{"total":289910292480,"available":288398155776,"internally_reserved":0,"allocated":178020352,"data_stored":177144075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1334037506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1828,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1828,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.195849"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20889,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.239347+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:38.239347+0000","last_peered":"2024-09-10T20:39:38.239347+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:38.239347+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21071,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.241208+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:38.241208+0000","last_peered":"2024-09-10T20:39:38.241208+0000","last_clean":"2024-09-10T20:39:38.241208+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:38.241208+0000","last_undegraded":"2024-09-10T20:39:38.241208+0000","last_fullsized":"2024-09-10T20:39:38.241208+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28083,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238999+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:38.238999+0000","last_peered":"2024-09-10T20:39:38.238999+0000","last_clean":"2024-09-10T20:39:38.238999+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:38.238999+0000","last_undegraded":"2024-09-10T20:39:38.238999+0000","last_fullsized":"2024-09-10T20:39:38.238999+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20703,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.240860+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:38.240860+0000","last_peered":"2024-09-10T20:39:38.240860+0000","last_clean":"2024-09-10T20:39:38.240860+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:38.240860+0000","last_undegraded":"2024-09-10T20:39:38.240860+0000","last_fullsized":"2024-09-10T20:39:38.240860+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21214,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.236997+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:38.236997+0000","last_peered":"2024-09-10T20:39:38.236997+0000","last_clean":"2024-09-10T20:39:38.236997+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:38.236997+0000","last_undegraded":"2024-09-10T20:39:38.236997+0000","last_fullsized":"2024-09-10T20:39:38.236997+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20732,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.236538+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:38.236538+0000","last_peered":"2024-09-10T20:39:38.236538+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:38.236538+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20903,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.236686+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:38.236686+0000","last_peered":"2024-09-10T20:39:38.236686+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:38.236686+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20880,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238170+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:38.238170+0000","last_peered":"2024-09-10T20:39:38.238170+0000","last_clean":"2024-09-10T20:39:38.238170+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:38.238170+0000","last_undegraded":"2024-09-10T20:39:38.238170+0000","last_fullsized":"2024-09-10T20:39:38.238170+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.237911+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:39:38.237911+0000","last_peered":"2024-09-10T20:39:38.237911+0000","last_clean":"2024-09-10T20:39:38.237911+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:39:38.237911+0000","last_undegraded":"2024-09-10T20:39:38.237911+0000","last_fullsized":"2024-09-10T20:39:38.237911+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.340635+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:39:38.340635+0000","last_peered":"2024-09-10T20:39:38.340635+0000","last_clean":"2024-09-10T20:39:38.340635+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:39:38.340635+0000","last_undegraded":"2024-09-10T20:39:38.340635+0000","last_fullsized":"2024-09-10T20:39:38.340635+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.240786+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:39:38.240786+0000","last_peered":"2024-09-10T20:39:38.240786+0000","last_clean":"2024-09-10T20:39:38.240786+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:39:38.240786+0000","last_undegraded":"2024-09-10T20:39:38.240786+0000","last_fullsized":"2024-09-10T20:39:38.240786+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":20956,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.237580+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:38.237580+0000","last_peered":"2024-09-10T20:39:38.237580+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:38.237580+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3564}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.236424+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:39:38.236424+0000","last_peered":"2024-09-10T20:39:38.236424+0000","last_clean":"2024-09-10T20:39:38.236424+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:39:38.236424+0000","last_undegraded":"2024-09-10T20:39:38.236424+0000","last_fullsized":"2024-09-10T20:39:38.236424+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":20767,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.241791+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:39:38.241791+0000","last_peered":"2024-09-10T20:39:38.241791+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:39:38.241791+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3606}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238378+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:39:38.238378+0000","last_peered":"2024-09-10T20:39:38.238378+0000","last_clean":"2024-09-10T20:39:38.238378+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:39:38.238378+0000","last_undegraded":"2024-09-10T20:39:38.238378+0000","last_fullsized":"2024-09-10T20:39:38.238378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28044,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.240573+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:38.240573+0000","last_peered":"2024-09-10T20:39:38.240573+0000","last_clean":"2024-09-10T20:39:38.240573+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:38.240573+0000","last_undegraded":"2024-09-10T20:39:38.240573+0000","last_fullsized":"2024-09-10T20:39:38.240573+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":27605,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:55.211737+0000","last_change":"2024-09-10T20:39:36.811261+0000","last_active":"2024-09-10T20:39:55.211737+0000","last_peered":"2024-09-10T20:39:55.211737+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:39:55.211737+0000","last_undegraded":"2024-09-10T20:30:05.393181+0000","last_fullsized":"2024-09-10T20:30:05.305784+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":339,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3310,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-339}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238954+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:39:38.238954+0000","last_peered":"2024-09-10T20:39:38.238954+0000","last_clean":"2024-09-10T20:39:38.238954+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:39:38.238954+0000","last_undegraded":"2024-09-10T20:39:38.238954+0000","last_fullsized":"2024-09-10T20:39:38.238954+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27822,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.239210+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:38.239210+0000","last_peered":"2024-09-10T20:39:38.239210+0000","last_clean":"2024-09-10T20:39:38.239210+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:38.239210+0000","last_undegraded":"2024-09-10T20:39:38.239210+0000","last_fullsized":"2024-09-10T20:39:38.239210+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.241067+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:39:38.241067+0000","last_peered":"2024-09-10T20:39:38.241067+0000","last_clean":"2024-09-10T20:39:38.241067+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:39:38.241067+0000","last_undegraded":"2024-09-10T20:39:38.241067+0000","last_fullsized":"2024-09-10T20:39:38.241067+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21253,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.237167+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:38.237167+0000","last_peered":"2024-09-10T20:39:38.237167+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:38.237167+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.239124+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:39:38.239124+0000","last_peered":"2024-09-10T20:39:38.239124+0000","last_clean":"2024-09-10T20:39:38.239124+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:39:38.239124+0000","last_undegraded":"2024-09-10T20:39:38.239124+0000","last_fullsized":"2024-09-10T20:39:38.239124+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.237911+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:39:38.237911+0000","last_peered":"2024-09-10T20:39:38.237911+0000","last_clean":"2024-09-10T20:39:38.237911+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:39:38.237911+0000","last_undegraded":"2024-09-10T20:39:38.237911+0000","last_fullsized":"2024-09-10T20:39:38.237911+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27652,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238867+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:38.238867+0000","last_peered":"2024-09-10T20:39:38.238867+0000","last_clean":"2024-09-10T20:39:38.238867+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:38.238867+0000","last_undegraded":"2024-09-10T20:39:38.238867+0000","last_fullsized":"2024-09-10T20:39:38.238867+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21764,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21385,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176685056,"data_stored":176638087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182536,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517148,"kb_used_data":57924,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854692,"statfs":{"total":96636764160,"available":96107204608,"internally_reserved":0,"allocated":59314176,"data_stored":59032768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986308,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480332,"kb_used_data":115444,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93891508,"statfs":{"total":96636764160,"available":96144904192,"internally_reserved":0,"allocated":118214656,"data_stored":117937325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659254,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479216,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892624,"statfs":{"total":96636764160,"available":96146046976,"internally_reserved":0,"allocated":491520,"data_stored":173982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":490204846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117772288,"data_stored":117768675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:39:58.145 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:39:58.145 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'20658", 'reported_seq': 20889, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:38.239347+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:39:38.239347+0000', 'last_peered': '2024-09-10T20:39:38.239347+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:39:38.239347+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20658, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3541}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:58.145 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:39:58.145 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20471", 'reported_seq': 20732, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:38.236538+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:39:38.236538+0000', 'last_peered': '2024-09-10T20:39:38.236538+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:39:38.236538+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17335, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3561}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:58.145 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:39:58.145 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'20608", 'reported_seq': 20903, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:38.236686+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:39:38.236686+0000', 'last_peered': '2024-09-10T20:39:38.236686+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:39:38.236686+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17203, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3486}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:58.146 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:39:58.146 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20730", 'reported_seq': 20956, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:38.237580+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:39:38.237580+0000', 'last_peered': '2024-09-10T20:39:38.237580+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:39:38.237580+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20728, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3564}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:58.146 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:39:58.146 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "51'20565", 'reported_seq': 20767, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:38.241791+0000', 'last_change': '2024-09-10T20:30:05.790299+0000', 'last_active': '2024-09-10T20:39:38.241791+0000', 'last_peered': '2024-09-10T20:39:38.241791+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:39:38.241791+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18393, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3606}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:39:58.146 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:39:58.146 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'20830", 'reported_seq': 27605, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:55.211737+0000', 'last_change': '2024-09-10T20:39:36.811261+0000', 'last_active': '2024-09-10T20:39:55.211737+0000', 'last_peered': '2024-09-10T20:39:55.211737+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:39:55.211737+0000', 'last_undegraded': '2024-09-10T20:30:05.393181+0000', 'last_fullsized': '2024-09-10T20:30:05.305784+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 339, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17553, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3310, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -339}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:58.146 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:39:58.146 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "51'20986", 'reported_seq': 21253, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:38.237167+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:39:38.237167+0000', 'last_peered': '2024-09-10T20:39:38.237167+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:39:38.237167+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17813, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3655}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:39:58.146 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:39:58.577 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:58.577 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 21764/10 objects degraded (217640.000%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":710,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":5,"data_bytes":1468,"bytes_used":1512136704,"bytes_avail":288398155776,"bytes_total":289910292480,"degraded_objects":21764,"degraded_total":10,"degraded_ratio":2176.4000000000001,"recovering_objects_per_sec":220,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2642,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:39:58.591 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:39:58.591 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:39:58.957 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:39:58.958 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:39:58.972 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":403,"stamp":"2024-09-10T20:39:58.261295+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21425,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18133,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1485008,"kb_used_data":173840,"kb_used_omap":12,"kb_used_meta":1311091,"kb_avail":281630512,"statfs":{"total":289910292480,"available":288389644288,"internally_reserved":0,"allocated":178012160,"data_stored":177143109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1342557186},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2167,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1426,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.201101"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20889,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.239347+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:38.239347+0000","last_peered":"2024-09-10T20:39:38.239347+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:38.239347+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21071,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.241208+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:38.241208+0000","last_peered":"2024-09-10T20:39:38.241208+0000","last_clean":"2024-09-10T20:39:38.241208+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:38.241208+0000","last_undegraded":"2024-09-10T20:39:38.241208+0000","last_fullsized":"2024-09-10T20:39:38.241208+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28085,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T20:39:57.258005+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:57.258005+0000","last_peered":"2024-09-10T20:39:57.258005+0000","last_clean":"2024-09-10T20:39:57.258005+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:57.258005+0000","last_undegraded":"2024-09-10T20:39:57.258005+0000","last_fullsized":"2024-09-10T20:39:57.258005+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20703,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.240860+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:38.240860+0000","last_peered":"2024-09-10T20:39:38.240860+0000","last_clean":"2024-09-10T20:39:38.240860+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:38.240860+0000","last_undegraded":"2024-09-10T20:39:38.240860+0000","last_fullsized":"2024-09-10T20:39:38.240860+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21214,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.236997+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:38.236997+0000","last_peered":"2024-09-10T20:39:38.236997+0000","last_clean":"2024-09-10T20:39:38.236997+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:38.236997+0000","last_undegraded":"2024-09-10T20:39:38.236997+0000","last_fullsized":"2024-09-10T20:39:38.236997+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20732,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.236538+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:38.236538+0000","last_peered":"2024-09-10T20:39:38.236538+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:38.236538+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20903,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.236686+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:38.236686+0000","last_peered":"2024-09-10T20:39:38.236686+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:38.236686+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20880,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.238170+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:38.238170+0000","last_peered":"2024-09-10T20:39:38.238170+0000","last_clean":"2024-09-10T20:39:38.238170+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:38.238170+0000","last_undegraded":"2024-09-10T20:39:38.238170+0000","last_fullsized":"2024-09-10T20:39:38.238170+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":90,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T20:39:57.257812+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:39:57.257812+0000","last_peered":"2024-09-10T20:39:57.257812+0000","last_clean":"2024-09-10T20:39:57.257812+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:39:57.257812+0000","last_undegraded":"2024-09-10T20:39:57.257812+0000","last_fullsized":"2024-09-10T20:39:57.257812+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22124,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T20:39:57.257974+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:39:57.257974+0000","last_peered":"2024-09-10T20:39:57.257974+0000","last_clean":"2024-09-10T20:39:57.257974+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:39:57.257974+0000","last_undegraded":"2024-09-10T20:39:57.257974+0000","last_fullsized":"2024-09-10T20:39:57.257974+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.240786+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:39:38.240786+0000","last_peered":"2024-09-10T20:39:38.240786+0000","last_clean":"2024-09-10T20:39:38.240786+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:39:38.240786+0000","last_undegraded":"2024-09-10T20:39:38.240786+0000","last_fullsized":"2024-09-10T20:39:38.240786+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":20956,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.237580+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:38.237580+0000","last_peered":"2024-09-10T20:39:38.237580+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:38.237580+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3564}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.236424+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:39:38.236424+0000","last_peered":"2024-09-10T20:39:38.236424+0000","last_clean":"2024-09-10T20:39:38.236424+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:39:38.236424+0000","last_undegraded":"2024-09-10T20:39:38.236424+0000","last_fullsized":"2024-09-10T20:39:38.236424+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":20767,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.241791+0000","last_change":"2024-09-10T20:30:05.790299+0000","last_active":"2024-09-10T20:39:38.241791+0000","last_peered":"2024-09-10T20:39:38.241791+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:39:38.241791+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3606}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":100,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T20:39:57.257752+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:39:57.257752+0000","last_peered":"2024-09-10T20:39:57.257752+0000","last_clean":"2024-09-10T20:39:57.257752+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:39:57.257752+0000","last_undegraded":"2024-09-10T20:39:57.257752+0000","last_fullsized":"2024-09-10T20:39:57.257752+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28044,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.240573+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:38.240573+0000","last_peered":"2024-09-10T20:39:38.240573+0000","last_clean":"2024-09-10T20:39:38.240573+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:38.240573+0000","last_undegraded":"2024-09-10T20:39:38.240573+0000","last_fullsized":"2024-09-10T20:39:38.240573+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21093,"reported_epoch":54,"state":"peering","last_fresh":"2024-09-10T20:39:57.258623+0000","last_change":"2024-09-10T20:39:57.258623+0000","last_active":"2024-09-10T20:39:57.258623+0000","last_peered":"2024-09-10T20:39:21.283321+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:39:57.258623+0000","last_undegraded":"2024-09-10T20:39:57.258623+0000","last_fullsized":"2024-09-10T20:39:57.258623+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":90,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T20:39:57.257912+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:39:57.257912+0000","last_peered":"2024-09-10T20:39:57.257912+0000","last_clean":"2024-09-10T20:39:57.257912+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:39:57.257912+0000","last_undegraded":"2024-09-10T20:39:57.257912+0000","last_fullsized":"2024-09-10T20:39:57.257912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27824,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T20:39:57.258238+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:57.258238+0000","last_peered":"2024-09-10T20:39:57.258238+0000","last_clean":"2024-09-10T20:39:57.258238+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:57.258238+0000","last_undegraded":"2024-09-10T20:39:57.258238+0000","last_fullsized":"2024-09-10T20:39:57.258238+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T20:39:38.241067+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:39:38.241067+0000","last_peered":"2024-09-10T20:39:38.241067+0000","last_clean":"2024-09-10T20:39:38.241067+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:39:38.241067+0000","last_undegraded":"2024-09-10T20:39:38.241067+0000","last_fullsized":"2024-09-10T20:39:38.241067+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21253,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:38.237167+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:38.237167+0000","last_peered":"2024-09-10T20:39:38.237167+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:38.237167+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":90,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T20:39:57.258154+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:39:57.258154+0000","last_peered":"2024-09-10T20:39:57.258154+0000","last_clean":"2024-09-10T20:39:57.258154+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:39:57.258154+0000","last_undegraded":"2024-09-10T20:39:57.258154+0000","last_fullsized":"2024-09-10T20:39:57.258154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":95,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T20:39:57.258274+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:39:57.258274+0000","last_peered":"2024-09-10T20:39:57.258274+0000","last_clean":"2024-09-10T20:39:57.258274+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:39:57.258274+0000","last_undegraded":"2024-09-10T20:39:57.258274+0000","last_fullsized":"2024-09-10T20:39:57.258274+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27654,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T20:39:57.257808+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:57.257808+0000","last_peered":"2024-09-10T20:39:57.257808+0000","last_clean":"2024-09-10T20:39:57.257808+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:57.257808+0000","last_undegraded":"2024-09-10T20:39:57.257808+0000","last_fullsized":"2024-09-10T20:39:57.257808+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21425,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18131,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176676864,"data_stored":176637121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182536,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517148,"kb_used_data":57924,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854692,"statfs":{"total":96636764160,"available":96107204608,"internally_reserved":0,"allocated":59314176,"data_stored":59032768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986308,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480332,"kb_used_data":115444,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93891508,"statfs":{"total":96636764160,"available":96144904192,"internally_reserved":0,"allocated":118214656,"data_stored":117937325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659255,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487528,"kb_used_data":472,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884312,"statfs":{"total":96636764160,"available":96137535488,"internally_reserved":0,"allocated":483328,"data_stored":173016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117772288,"data_stored":117768675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:40:01.975 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:40:02.360 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:02.361 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:40:02.374 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":405,"stamp":"2024-09-10T20:40:01.062404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20959,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18599,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1485036,"kb_used_data":173872,"kb_used_omap":12,"kb_used_meta":1311091,"kb_avail":281630484,"statfs":{"total":289910292480,"available":288389615616,"internally_reserved":0,"allocated":178044928,"data_stored":177156267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1342557186},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1848,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1745,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001440"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20893,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.263246+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:58.263246+0000","last_peered":"2024-09-10T20:39:58.263246+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:58.263246+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21075,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277759+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:58.277759+0000","last_peered":"2024-09-10T20:39:58.277759+0000","last_clean":"2024-09-10T20:39:58.277759+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:58.277759+0000","last_undegraded":"2024-09-10T20:39:58.277759+0000","last_fullsized":"2024-09-10T20:39:58.277759+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28085,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T20:39:57.258005+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:57.258005+0000","last_peered":"2024-09-10T20:39:57.258005+0000","last_clean":"2024-09-10T20:39:57.258005+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:57.258005+0000","last_undegraded":"2024-09-10T20:39:57.258005+0000","last_fullsized":"2024-09-10T20:39:57.258005+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20707,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277394+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:58.277394+0000","last_peered":"2024-09-10T20:39:58.277394+0000","last_clean":"2024-09-10T20:39:58.277394+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:58.277394+0000","last_undegraded":"2024-09-10T20:39:58.277394+0000","last_fullsized":"2024-09-10T20:39:58.277394+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21218,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.265097+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:58.265097+0000","last_peered":"2024-09-10T20:39:58.265097+0000","last_clean":"2024-09-10T20:39:58.265097+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:58.265097+0000","last_undegraded":"2024-09-10T20:39:58.265097+0000","last_fullsized":"2024-09-10T20:39:58.265097+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20737,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.265623+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:58.265623+0000","last_peered":"2024-09-10T20:39:58.265623+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:58.265623+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20907,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.263985+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:58.263985+0000","last_peered":"2024-09-10T20:39:58.263985+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:58.263985+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20884,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.262813+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:58.262813+0000","last_peered":"2024-09-10T20:39:58.262813+0000","last_clean":"2024-09-10T20:39:58.262813+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:58.262813+0000","last_undegraded":"2024-09-10T20:39:58.262813+0000","last_fullsized":"2024-09-10T20:39:58.262813+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":90,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T20:39:57.257812+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:39:57.257812+0000","last_peered":"2024-09-10T20:39:57.257812+0000","last_clean":"2024-09-10T20:39:57.257812+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:39:57.257812+0000","last_undegraded":"2024-09-10T20:39:57.257812+0000","last_fullsized":"2024-09-10T20:39:57.257812+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22124,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T20:39:57.257974+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:39:57.257974+0000","last_peered":"2024-09-10T20:39:57.257974+0000","last_clean":"2024-09-10T20:39:57.257974+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:39:57.257974+0000","last_undegraded":"2024-09-10T20:39:57.257974+0000","last_fullsized":"2024-09-10T20:39:57.257974+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277094+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:39:58.277094+0000","last_peered":"2024-09-10T20:39:58.277094+0000","last_clean":"2024-09-10T20:39:58.277094+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:39:58.277094+0000","last_undegraded":"2024-09-10T20:39:58.277094+0000","last_fullsized":"2024-09-10T20:39:58.277094+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":20960,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.265211+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:58.265211+0000","last_peered":"2024-09-10T20:39:58.265211+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:58.265211+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3564}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.264754+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:39:58.264754+0000","last_peered":"2024-09-10T20:39:58.264754+0000","last_clean":"2024-09-10T20:39:58.264754+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:39:58.264754+0000","last_undegraded":"2024-09-10T20:39:58.264754+0000","last_fullsized":"2024-09-10T20:39:58.264754+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":21704,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:59.794969+0000","last_change":"2024-09-10T20:39:56.460609+0000","last_active":"2024-09-10T20:39:59.794969+0000","last_peered":"2024-09-10T20:39:59.794969+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:39:59.794969+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3142}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":100,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T20:39:57.257752+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:39:57.257752+0000","last_peered":"2024-09-10T20:39:57.257752+0000","last_clean":"2024-09-10T20:39:57.257752+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:39:57.257752+0000","last_undegraded":"2024-09-10T20:39:57.257752+0000","last_fullsized":"2024-09-10T20:39:57.257752+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28048,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.276958+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:58.276958+0000","last_peered":"2024-09-10T20:39:58.276958+0000","last_clean":"2024-09-10T20:39:58.276958+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:58.276958+0000","last_undegraded":"2024-09-10T20:39:58.276958+0000","last_fullsized":"2024-09-10T20:39:58.276958+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21093,"reported_epoch":54,"state":"peering","last_fresh":"2024-09-10T20:39:57.258623+0000","last_change":"2024-09-10T20:39:57.258623+0000","last_active":"2024-09-10T20:39:57.258623+0000","last_peered":"2024-09-10T20:39:21.283321+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:39:57.258623+0000","last_undegraded":"2024-09-10T20:39:57.258623+0000","last_fullsized":"2024-09-10T20:39:57.258623+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":90,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T20:39:57.257912+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:39:57.257912+0000","last_peered":"2024-09-10T20:39:57.257912+0000","last_clean":"2024-09-10T20:39:57.257912+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:39:57.257912+0000","last_undegraded":"2024-09-10T20:39:57.257912+0000","last_fullsized":"2024-09-10T20:39:57.257912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27824,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T20:39:57.258238+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:57.258238+0000","last_peered":"2024-09-10T20:39:57.258238+0000","last_clean":"2024-09-10T20:39:57.258238+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:57.258238+0000","last_undegraded":"2024-09-10T20:39:57.258238+0000","last_fullsized":"2024-09-10T20:39:57.258238+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277571+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:39:58.277571+0000","last_peered":"2024-09-10T20:39:58.277571+0000","last_clean":"2024-09-10T20:39:58.277571+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:39:58.277571+0000","last_undegraded":"2024-09-10T20:39:58.277571+0000","last_fullsized":"2024-09-10T20:39:58.277571+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21257,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.263453+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:58.263453+0000","last_peered":"2024-09-10T20:39:58.263453+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:58.263453+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":90,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T20:39:57.258154+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:39:57.258154+0000","last_peered":"2024-09-10T20:39:57.258154+0000","last_clean":"2024-09-10T20:39:57.258154+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:39:57.258154+0000","last_undegraded":"2024-09-10T20:39:57.258154+0000","last_fullsized":"2024-09-10T20:39:57.258154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":95,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T20:39:57.258274+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:39:57.258274+0000","last_peered":"2024-09-10T20:39:57.258274+0000","last_clean":"2024-09-10T20:39:57.258274+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:39:57.258274+0000","last_undegraded":"2024-09-10T20:39:57.258274+0000","last_fullsized":"2024-09-10T20:39:57.258274+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27654,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T20:39:57.257808+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:57.257808+0000","last_peered":"2024-09-10T20:39:57.257808+0000","last_clean":"2024-09-10T20:39:57.257808+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:57.257808+0000","last_undegraded":"2024-09-10T20:39:57.257808+0000","last_fullsized":"2024-09-10T20:39:57.257808+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20959,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18597,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176676864,"data_stored":176637121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182537,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517164,"kb_used_data":57940,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854676,"statfs":{"total":96636764160,"available":96107188224,"internally_reserved":0,"allocated":59330560,"data_stored":59039347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986309,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480344,"kb_used_data":115460,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93891496,"statfs":{"total":96636764160,"available":96144891904,"internally_reserved":0,"allocated":118231040,"data_stored":117943904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659255,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487528,"kb_used_data":472,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884312,"statfs":{"total":96636764160,"available":96137535488,"internally_reserved":0,"allocated":483328,"data_stored":173016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117772288,"data_stored":117768675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:40:02.375 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:40:02.718 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:02.719 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:40:02.731 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":405,"stamp":"2024-09-10T20:40:01.062404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20959,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18599,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1485036,"kb_used_data":173872,"kb_used_omap":12,"kb_used_meta":1311091,"kb_avail":281630484,"statfs":{"total":289910292480,"available":288389615616,"internally_reserved":0,"allocated":178044928,"data_stored":177156267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1342557186},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1848,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1745,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001440"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20893,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.263246+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:58.263246+0000","last_peered":"2024-09-10T20:39:58.263246+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:58.263246+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21075,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277759+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:58.277759+0000","last_peered":"2024-09-10T20:39:58.277759+0000","last_clean":"2024-09-10T20:39:58.277759+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:58.277759+0000","last_undegraded":"2024-09-10T20:39:58.277759+0000","last_fullsized":"2024-09-10T20:39:58.277759+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28085,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T20:39:57.258005+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:57.258005+0000","last_peered":"2024-09-10T20:39:57.258005+0000","last_clean":"2024-09-10T20:39:57.258005+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:57.258005+0000","last_undegraded":"2024-09-10T20:39:57.258005+0000","last_fullsized":"2024-09-10T20:39:57.258005+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20707,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277394+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:58.277394+0000","last_peered":"2024-09-10T20:39:58.277394+0000","last_clean":"2024-09-10T20:39:58.277394+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:58.277394+0000","last_undegraded":"2024-09-10T20:39:58.277394+0000","last_fullsized":"2024-09-10T20:39:58.277394+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21218,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.265097+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:58.265097+0000","last_peered":"2024-09-10T20:39:58.265097+0000","last_clean":"2024-09-10T20:39:58.265097+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:58.265097+0000","last_undegraded":"2024-09-10T20:39:58.265097+0000","last_fullsized":"2024-09-10T20:39:58.265097+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20737,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.265623+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:58.265623+0000","last_peered":"2024-09-10T20:39:58.265623+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:58.265623+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20907,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.263985+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:58.263985+0000","last_peered":"2024-09-10T20:39:58.263985+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:58.263985+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20884,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.262813+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:58.262813+0000","last_peered":"2024-09-10T20:39:58.262813+0000","last_clean":"2024-09-10T20:39:58.262813+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:58.262813+0000","last_undegraded":"2024-09-10T20:39:58.262813+0000","last_fullsized":"2024-09-10T20:39:58.262813+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":90,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T20:39:57.257812+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:39:57.257812+0000","last_peered":"2024-09-10T20:39:57.257812+0000","last_clean":"2024-09-10T20:39:57.257812+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:39:57.257812+0000","last_undegraded":"2024-09-10T20:39:57.257812+0000","last_fullsized":"2024-09-10T20:39:57.257812+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22124,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T20:39:57.257974+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:39:57.257974+0000","last_peered":"2024-09-10T20:39:57.257974+0000","last_clean":"2024-09-10T20:39:57.257974+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:39:57.257974+0000","last_undegraded":"2024-09-10T20:39:57.257974+0000","last_fullsized":"2024-09-10T20:39:57.257974+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277094+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:39:58.277094+0000","last_peered":"2024-09-10T20:39:58.277094+0000","last_clean":"2024-09-10T20:39:58.277094+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:39:58.277094+0000","last_undegraded":"2024-09-10T20:39:58.277094+0000","last_fullsized":"2024-09-10T20:39:58.277094+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":20960,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.265211+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:58.265211+0000","last_peered":"2024-09-10T20:39:58.265211+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:58.265211+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3564}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.264754+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:39:58.264754+0000","last_peered":"2024-09-10T20:39:58.264754+0000","last_clean":"2024-09-10T20:39:58.264754+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:39:58.264754+0000","last_undegraded":"2024-09-10T20:39:58.264754+0000","last_fullsized":"2024-09-10T20:39:58.264754+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":21704,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:59.794969+0000","last_change":"2024-09-10T20:39:56.460609+0000","last_active":"2024-09-10T20:39:59.794969+0000","last_peered":"2024-09-10T20:39:59.794969+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:39:59.794969+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3142}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":100,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T20:39:57.257752+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:39:57.257752+0000","last_peered":"2024-09-10T20:39:57.257752+0000","last_clean":"2024-09-10T20:39:57.257752+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:39:57.257752+0000","last_undegraded":"2024-09-10T20:39:57.257752+0000","last_fullsized":"2024-09-10T20:39:57.257752+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28048,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.276958+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:58.276958+0000","last_peered":"2024-09-10T20:39:58.276958+0000","last_clean":"2024-09-10T20:39:58.276958+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:58.276958+0000","last_undegraded":"2024-09-10T20:39:58.276958+0000","last_fullsized":"2024-09-10T20:39:58.276958+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21093,"reported_epoch":54,"state":"peering","last_fresh":"2024-09-10T20:39:57.258623+0000","last_change":"2024-09-10T20:39:57.258623+0000","last_active":"2024-09-10T20:39:57.258623+0000","last_peered":"2024-09-10T20:39:21.283321+0000","last_clean":"2024-09-10T20:30:03.279229+0000","last_became_active":"2024-09-10T20:30:05.731060+0000","last_became_peered":"2024-09-10T20:30:05.731060+0000","last_unstale":"2024-09-10T20:39:57.258623+0000","last_undegraded":"2024-09-10T20:39:57.258623+0000","last_fullsized":"2024-09-10T20:39:57.258623+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":90,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T20:39:57.257912+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:39:57.257912+0000","last_peered":"2024-09-10T20:39:57.257912+0000","last_clean":"2024-09-10T20:39:57.257912+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:39:57.257912+0000","last_undegraded":"2024-09-10T20:39:57.257912+0000","last_fullsized":"2024-09-10T20:39:57.257912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27824,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T20:39:57.258238+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:57.258238+0000","last_peered":"2024-09-10T20:39:57.258238+0000","last_clean":"2024-09-10T20:39:57.258238+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:57.258238+0000","last_undegraded":"2024-09-10T20:39:57.258238+0000","last_fullsized":"2024-09-10T20:39:57.258238+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277571+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:39:58.277571+0000","last_peered":"2024-09-10T20:39:58.277571+0000","last_clean":"2024-09-10T20:39:58.277571+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:39:58.277571+0000","last_undegraded":"2024-09-10T20:39:58.277571+0000","last_fullsized":"2024-09-10T20:39:58.277571+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21257,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.263453+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:58.263453+0000","last_peered":"2024-09-10T20:39:58.263453+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:58.263453+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":90,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T20:39:57.258154+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:39:57.258154+0000","last_peered":"2024-09-10T20:39:57.258154+0000","last_clean":"2024-09-10T20:39:57.258154+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:39:57.258154+0000","last_undegraded":"2024-09-10T20:39:57.258154+0000","last_fullsized":"2024-09-10T20:39:57.258154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":95,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T20:39:57.258274+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:39:57.258274+0000","last_peered":"2024-09-10T20:39:57.258274+0000","last_clean":"2024-09-10T20:39:57.258274+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:39:57.258274+0000","last_undegraded":"2024-09-10T20:39:57.258274+0000","last_fullsized":"2024-09-10T20:39:57.258274+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27654,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T20:39:57.257808+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:57.257808+0000","last_peered":"2024-09-10T20:39:57.257808+0000","last_clean":"2024-09-10T20:39:57.257808+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:57.257808+0000","last_undegraded":"2024-09-10T20:39:57.257808+0000","last_fullsized":"2024-09-10T20:39:57.257808+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20959,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18597,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176676864,"data_stored":176637121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182537,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517164,"kb_used_data":57940,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854676,"statfs":{"total":96636764160,"available":96107188224,"internally_reserved":0,"allocated":59330560,"data_stored":59039347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986309,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480344,"kb_used_data":115460,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93891496,"statfs":{"total":96636764160,"available":96144891904,"internally_reserved":0,"allocated":118231040,"data_stored":117943904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659255,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487528,"kb_used_data":472,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884312,"statfs":{"total":96636764160,"available":96137535488,"internally_reserved":0,"allocated":483328,"data_stored":173016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117772288,"data_stored":117768675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:40:02.733 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:40:02.733 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'20658", 'reported_seq': 20893, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:58.263246+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:39:58.263246+0000', 'last_peered': '2024-09-10T20:39:58.263246+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:39:58.263246+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20658, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3541}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:02.733 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:40:02.733 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20471", 'reported_seq': 20737, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:58.265623+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:39:58.265623+0000', 'last_peered': '2024-09-10T20:39:58.265623+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:39:58.265623+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17335, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3561}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:02.733 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:40:02.733 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'20608", 'reported_seq': 20907, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:58.263985+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:39:58.263985+0000', 'last_peered': '2024-09-10T20:39:58.263985+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:39:58.263985+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17203, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3486}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:02.733 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:40:02.733 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20730", 'reported_seq': 20960, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:58.265211+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:39:58.265211+0000', 'last_peered': '2024-09-10T20:39:58.265211+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:39:58.265211+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20728, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3564}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:02.734 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:40:02.734 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "51'20565", 'reported_seq': 21704, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:59.794969+0000', 'last_change': '2024-09-10T20:39:56.460609+0000', 'last_active': '2024-09-10T20:39:59.794969+0000', 'last_peered': '2024-09-10T20:39:59.794969+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:39:59.794969+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18393, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 493, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3142}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:40:02.734 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T20:40:02.734 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'20830", 'reported_seq': 21093, 'reported_epoch': 54, 'state': 'peering', 'last_fresh': '2024-09-10T20:39:57.258623+0000', 'last_change': '2024-09-10T20:39:57.258623+0000', 'last_active': '2024-09-10T20:39:57.258623+0000', 'last_peered': '2024-09-10T20:39:21.283321+0000', 'last_clean': '2024-09-10T20:30:03.279229+0000', 'last_became_active': '2024-09-10T20:30:05.731060+0000', 'last_became_peered': '2024-09-10T20:30:05.731060+0000', 'last_unstale': '2024-09-10T20:39:57.258623+0000', 'last_undegraded': '2024-09-10T20:39:57.258623+0000', 'last_fullsized': '2024-09-10T20:39:57.258623+0000', 'mapping_epoch': 54, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3319", 'last_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:56.793691+0000', 'objects_scrubbed': 3275, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1221, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17553, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0, 2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T20:40:02.734 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:40:02.734 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "51'20986", 'reported_seq': 21257, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:58.263453+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:39:58.263453+0000', 'last_peered': '2024-09-10T20:39:58.263453+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:39:58.263453+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17813, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3655}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:02.734 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:40:03.183 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:03.183 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 20959/10 objects degraded (209590.000%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":714,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"peering","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":5,"data_bytes":1468,"bytes_used":1520676864,"bytes_avail":288389615616,"bytes_total":289910292480,"inactive_pgs_ratio":0.041666667908430099,"degraded_objects":20959,"degraded_total":10,"degraded_ratio":2095.9000000000001},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:40:03.199 INFO:tasks.rep_lost_unfound_delete.ceph_manager:no progress seen, keeping timeout for now 2024-09-10T20:40:03.199 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:40:03.521 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:03.521 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:40:03.534 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":406,"stamp":"2024-09-10T20:40:03.062751+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20959,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22192,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1493372,"kb_used_data":173888,"kb_used_omap":12,"kb_used_meta":1319411,"kb_avail":281622148,"statfs":{"total":289910292480,"available":288381079552,"internally_reserved":0,"allocated":178061312,"data_stored":177162846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1351076866},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1848,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1848,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001449"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20893,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.263246+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:58.263246+0000","last_peered":"2024-09-10T20:39:58.263246+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:58.263246+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21075,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277759+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:58.277759+0000","last_peered":"2024-09-10T20:39:58.277759+0000","last_clean":"2024-09-10T20:39:58.277759+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:58.277759+0000","last_undegraded":"2024-09-10T20:39:58.277759+0000","last_fullsized":"2024-09-10T20:39:58.277759+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28087,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263157+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:58.263157+0000","last_peered":"2024-09-10T20:39:58.263157+0000","last_clean":"2024-09-10T20:39:58.263157+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:58.263157+0000","last_undegraded":"2024-09-10T20:39:58.263157+0000","last_fullsized":"2024-09-10T20:39:58.263157+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20707,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277394+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:58.277394+0000","last_peered":"2024-09-10T20:39:58.277394+0000","last_clean":"2024-09-10T20:39:58.277394+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:58.277394+0000","last_undegraded":"2024-09-10T20:39:58.277394+0000","last_fullsized":"2024-09-10T20:39:58.277394+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21218,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.265097+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:58.265097+0000","last_peered":"2024-09-10T20:39:58.265097+0000","last_clean":"2024-09-10T20:39:58.265097+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:58.265097+0000","last_undegraded":"2024-09-10T20:39:58.265097+0000","last_fullsized":"2024-09-10T20:39:58.265097+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20737,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.265623+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:58.265623+0000","last_peered":"2024-09-10T20:39:58.265623+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:58.265623+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20907,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.263985+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:58.263985+0000","last_peered":"2024-09-10T20:39:58.263985+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:58.263985+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20884,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.262813+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:58.262813+0000","last_peered":"2024-09-10T20:39:58.262813+0000","last_clean":"2024-09-10T20:39:58.262813+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:58.262813+0000","last_undegraded":"2024-09-10T20:39:58.262813+0000","last_fullsized":"2024-09-10T20:39:58.262813+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.262961+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:39:58.262961+0000","last_peered":"2024-09-10T20:39:58.262961+0000","last_clean":"2024-09-10T20:39:58.262961+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:39:58.262961+0000","last_undegraded":"2024-09-10T20:39:58.262961+0000","last_fullsized":"2024-09-10T20:39:58.262961+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22126,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.262973+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:39:58.262973+0000","last_peered":"2024-09-10T20:39:58.262973+0000","last_clean":"2024-09-10T20:39:58.262973+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:39:58.262973+0000","last_undegraded":"2024-09-10T20:39:58.262973+0000","last_fullsized":"2024-09-10T20:39:58.262973+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277094+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:39:58.277094+0000","last_peered":"2024-09-10T20:39:58.277094+0000","last_clean":"2024-09-10T20:39:58.277094+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:39:58.277094+0000","last_undegraded":"2024-09-10T20:39:58.277094+0000","last_fullsized":"2024-09-10T20:39:58.277094+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":20960,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.265211+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:58.265211+0000","last_peered":"2024-09-10T20:39:58.265211+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:58.265211+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3564}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.264754+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:39:58.264754+0000","last_peered":"2024-09-10T20:39:58.264754+0000","last_clean":"2024-09-10T20:39:58.264754+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:39:58.264754+0000","last_undegraded":"2024-09-10T20:39:58.264754+0000","last_fullsized":"2024-09-10T20:39:58.264754+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":21704,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:59.794969+0000","last_change":"2024-09-10T20:39:56.460609+0000","last_active":"2024-09-10T20:39:59.794969+0000","last_peered":"2024-09-10T20:39:59.794969+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:39:59.794969+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3142}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":102,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.262885+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:39:58.262885+0000","last_peered":"2024-09-10T20:39:58.262885+0000","last_clean":"2024-09-10T20:39:58.262885+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:39:58.262885+0000","last_undegraded":"2024-09-10T20:39:58.262885+0000","last_fullsized":"2024-09-10T20:39:58.262885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28048,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.276958+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:58.276958+0000","last_peered":"2024-09-10T20:39:58.276958+0000","last_clean":"2024-09-10T20:39:58.276958+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:58.276958+0000","last_undegraded":"2024-09-10T20:39:58.276958+0000","last_fullsized":"2024-09-10T20:39:58.276958+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21105,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.338466+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:39:58.338466+0000","last_peered":"2024-09-10T20:39:58.338466+0000","last_clean":"2024-09-10T20:39:58.338466+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:39:58.338466+0000","last_undegraded":"2024-09-10T20:39:58.338466+0000","last_fullsized":"2024-09-10T20:39:58.338466+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263152+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:39:58.263152+0000","last_peered":"2024-09-10T20:39:58.263152+0000","last_clean":"2024-09-10T20:39:58.263152+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:39:58.263152+0000","last_undegraded":"2024-09-10T20:39:58.263152+0000","last_fullsized":"2024-09-10T20:39:58.263152+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27826,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263403+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:58.263403+0000","last_peered":"2024-09-10T20:39:58.263403+0000","last_clean":"2024-09-10T20:39:58.263403+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:58.263403+0000","last_undegraded":"2024-09-10T20:39:58.263403+0000","last_fullsized":"2024-09-10T20:39:58.263403+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277571+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:39:58.277571+0000","last_peered":"2024-09-10T20:39:58.277571+0000","last_clean":"2024-09-10T20:39:58.277571+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:39:58.277571+0000","last_undegraded":"2024-09-10T20:39:58.277571+0000","last_fullsized":"2024-09-10T20:39:58.277571+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21257,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.263453+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:58.263453+0000","last_peered":"2024-09-10T20:39:58.263453+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:58.263453+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":92,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263317+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:39:58.263317+0000","last_peered":"2024-09-10T20:39:58.263317+0000","last_clean":"2024-09-10T20:39:58.263317+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:39:58.263317+0000","last_undegraded":"2024-09-10T20:39:58.263317+0000","last_fullsized":"2024-09-10T20:39:58.263317+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263276+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:39:58.263276+0000","last_peered":"2024-09-10T20:39:58.263276+0000","last_clean":"2024-09-10T20:39:58.263276+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:39:58.263276+0000","last_undegraded":"2024-09-10T20:39:58.263276+0000","last_fullsized":"2024-09-10T20:39:58.263276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27656,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263068+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:58.263068+0000","last_peered":"2024-09-10T20:39:58.263068+0000","last_clean":"2024-09-10T20:39:58.263068+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:58.263068+0000","last_undegraded":"2024-09-10T20:39:58.263068+0000","last_fullsized":"2024-09-10T20:39:58.263068+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20959,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22190,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176676864,"data_stored":176637121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182537,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517164,"kb_used_data":57940,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854676,"statfs":{"total":96636764160,"available":96107188224,"internally_reserved":0,"allocated":59330560,"data_stored":59039347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986309,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480344,"kb_used_data":115460,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93891496,"statfs":{"total":96636764160,"available":96144891904,"internally_reserved":0,"allocated":118231040,"data_stored":117943904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659256,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":495864,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":495355,"kb_avail":93875976,"statfs":{"total":96636764160,"available":96128999424,"internally_reserved":0,"allocated":499712,"data_stored":179595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":507244206},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117772288,"data_stored":117768675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:40:06.536 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:40:06.985 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:06.985 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:40:06.999 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":407,"stamp":"2024-09-10T20:40:05.063116+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19996,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23155,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1493372,"kb_used_data":173888,"kb_used_omap":12,"kb_used_meta":1319411,"kb_avail":281622148,"statfs":{"total":289910292480,"available":288381079552,"internally_reserved":0,"allocated":178061312,"data_stored":177162846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1351076866},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1768,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1768,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001441"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20893,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.263246+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:58.263246+0000","last_peered":"2024-09-10T20:39:58.263246+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:58.263246+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21075,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277759+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:58.277759+0000","last_peered":"2024-09-10T20:39:58.277759+0000","last_clean":"2024-09-10T20:39:58.277759+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:58.277759+0000","last_undegraded":"2024-09-10T20:39:58.277759+0000","last_fullsized":"2024-09-10T20:39:58.277759+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28087,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263157+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:58.263157+0000","last_peered":"2024-09-10T20:39:58.263157+0000","last_clean":"2024-09-10T20:39:58.263157+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:58.263157+0000","last_undegraded":"2024-09-10T20:39:58.263157+0000","last_fullsized":"2024-09-10T20:39:58.263157+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20707,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277394+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:58.277394+0000","last_peered":"2024-09-10T20:39:58.277394+0000","last_clean":"2024-09-10T20:39:58.277394+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:58.277394+0000","last_undegraded":"2024-09-10T20:39:58.277394+0000","last_fullsized":"2024-09-10T20:39:58.277394+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21218,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.265097+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:58.265097+0000","last_peered":"2024-09-10T20:39:58.265097+0000","last_clean":"2024-09-10T20:39:58.265097+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:58.265097+0000","last_undegraded":"2024-09-10T20:39:58.265097+0000","last_fullsized":"2024-09-10T20:39:58.265097+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20737,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.265623+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:58.265623+0000","last_peered":"2024-09-10T20:39:58.265623+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:58.265623+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20907,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.263985+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:58.263985+0000","last_peered":"2024-09-10T20:39:58.263985+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:58.263985+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20884,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.262813+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:58.262813+0000","last_peered":"2024-09-10T20:39:58.262813+0000","last_clean":"2024-09-10T20:39:58.262813+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:58.262813+0000","last_undegraded":"2024-09-10T20:39:58.262813+0000","last_fullsized":"2024-09-10T20:39:58.262813+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.262961+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:39:58.262961+0000","last_peered":"2024-09-10T20:39:58.262961+0000","last_clean":"2024-09-10T20:39:58.262961+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:39:58.262961+0000","last_undegraded":"2024-09-10T20:39:58.262961+0000","last_fullsized":"2024-09-10T20:39:58.262961+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22126,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.262973+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:39:58.262973+0000","last_peered":"2024-09-10T20:39:58.262973+0000","last_clean":"2024-09-10T20:39:58.262973+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:39:58.262973+0000","last_undegraded":"2024-09-10T20:39:58.262973+0000","last_fullsized":"2024-09-10T20:39:58.262973+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277094+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:39:58.277094+0000","last_peered":"2024-09-10T20:39:58.277094+0000","last_clean":"2024-09-10T20:39:58.277094+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:39:58.277094+0000","last_undegraded":"2024-09-10T20:39:58.277094+0000","last_fullsized":"2024-09-10T20:39:58.277094+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":20960,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.265211+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:58.265211+0000","last_peered":"2024-09-10T20:39:58.265211+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:58.265211+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3564}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.264754+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:39:58.264754+0000","last_peered":"2024-09-10T20:39:58.264754+0000","last_clean":"2024-09-10T20:39:58.264754+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:39:58.264754+0000","last_undegraded":"2024-09-10T20:39:58.264754+0000","last_fullsized":"2024-09-10T20:39:58.264754+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":23630,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:04.796118+0000","last_change":"2024-09-10T20:39:56.460609+0000","last_active":"2024-09-10T20:40:04.796118+0000","last_peered":"2024-09-10T20:40:04.796118+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:40:04.796118+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1456,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2179}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":102,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.262885+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:39:58.262885+0000","last_peered":"2024-09-10T20:39:58.262885+0000","last_clean":"2024-09-10T20:39:58.262885+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:39:58.262885+0000","last_undegraded":"2024-09-10T20:39:58.262885+0000","last_fullsized":"2024-09-10T20:39:58.262885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28048,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.276958+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:58.276958+0000","last_peered":"2024-09-10T20:39:58.276958+0000","last_clean":"2024-09-10T20:39:58.276958+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:58.276958+0000","last_undegraded":"2024-09-10T20:39:58.276958+0000","last_fullsized":"2024-09-10T20:39:58.276958+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21105,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.338466+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:39:58.338466+0000","last_peered":"2024-09-10T20:39:58.338466+0000","last_clean":"2024-09-10T20:39:58.338466+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:39:58.338466+0000","last_undegraded":"2024-09-10T20:39:58.338466+0000","last_fullsized":"2024-09-10T20:39:58.338466+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263152+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:39:58.263152+0000","last_peered":"2024-09-10T20:39:58.263152+0000","last_clean":"2024-09-10T20:39:58.263152+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:39:58.263152+0000","last_undegraded":"2024-09-10T20:39:58.263152+0000","last_fullsized":"2024-09-10T20:39:58.263152+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27826,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263403+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:58.263403+0000","last_peered":"2024-09-10T20:39:58.263403+0000","last_clean":"2024-09-10T20:39:58.263403+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:58.263403+0000","last_undegraded":"2024-09-10T20:39:58.263403+0000","last_fullsized":"2024-09-10T20:39:58.263403+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277571+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:39:58.277571+0000","last_peered":"2024-09-10T20:39:58.277571+0000","last_clean":"2024-09-10T20:39:58.277571+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:39:58.277571+0000","last_undegraded":"2024-09-10T20:39:58.277571+0000","last_fullsized":"2024-09-10T20:39:58.277571+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21257,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.263453+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:58.263453+0000","last_peered":"2024-09-10T20:39:58.263453+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:58.263453+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":92,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263317+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:39:58.263317+0000","last_peered":"2024-09-10T20:39:58.263317+0000","last_clean":"2024-09-10T20:39:58.263317+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:39:58.263317+0000","last_undegraded":"2024-09-10T20:39:58.263317+0000","last_fullsized":"2024-09-10T20:39:58.263317+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263276+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:39:58.263276+0000","last_peered":"2024-09-10T20:39:58.263276+0000","last_clean":"2024-09-10T20:39:58.263276+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:39:58.263276+0000","last_undegraded":"2024-09-10T20:39:58.263276+0000","last_fullsized":"2024-09-10T20:39:58.263276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27656,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263068+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:58.263068+0000","last_peered":"2024-09-10T20:39:58.263068+0000","last_clean":"2024-09-10T20:39:58.263068+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:58.263068+0000","last_undegraded":"2024-09-10T20:39:58.263068+0000","last_fullsized":"2024-09-10T20:39:58.263068+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19996,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23153,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176676864,"data_stored":176637121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182537,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517164,"kb_used_data":57940,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854676,"statfs":{"total":96636764160,"available":96107188224,"internally_reserved":0,"allocated":59330560,"data_stored":59039347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986310,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480344,"kb_used_data":115460,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93891496,"statfs":{"total":96636764160,"available":96144891904,"internally_reserved":0,"allocated":118231040,"data_stored":117943904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659256,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":495864,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":495355,"kb_avail":93875976,"statfs":{"total":96636764160,"available":96128999424,"internally_reserved":0,"allocated":499712,"data_stored":179595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":507244206},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117772288,"data_stored":117768675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:40:07.000 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:40:07.303 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:07.303 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:40:07.316 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":408,"stamp":"2024-09-10T20:40:07.063525+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19996,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23155,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1493372,"kb_used_data":173888,"kb_used_omap":12,"kb_used_meta":1319411,"kb_avail":281622148,"statfs":{"total":289910292480,"available":288381079552,"internally_reserved":0,"allocated":178061312,"data_stored":177162846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1351076866},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1768,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1768,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.807851"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20893,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.263246+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:58.263246+0000","last_peered":"2024-09-10T20:39:58.263246+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:58.263246+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21075,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277759+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:58.277759+0000","last_peered":"2024-09-10T20:39:58.277759+0000","last_clean":"2024-09-10T20:39:58.277759+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:58.277759+0000","last_undegraded":"2024-09-10T20:39:58.277759+0000","last_fullsized":"2024-09-10T20:39:58.277759+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28087,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263157+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:58.263157+0000","last_peered":"2024-09-10T20:39:58.263157+0000","last_clean":"2024-09-10T20:39:58.263157+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:58.263157+0000","last_undegraded":"2024-09-10T20:39:58.263157+0000","last_fullsized":"2024-09-10T20:39:58.263157+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20707,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277394+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:58.277394+0000","last_peered":"2024-09-10T20:39:58.277394+0000","last_clean":"2024-09-10T20:39:58.277394+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:58.277394+0000","last_undegraded":"2024-09-10T20:39:58.277394+0000","last_fullsized":"2024-09-10T20:39:58.277394+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21218,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.265097+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:58.265097+0000","last_peered":"2024-09-10T20:39:58.265097+0000","last_clean":"2024-09-10T20:39:58.265097+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:58.265097+0000","last_undegraded":"2024-09-10T20:39:58.265097+0000","last_fullsized":"2024-09-10T20:39:58.265097+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20737,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.265623+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:58.265623+0000","last_peered":"2024-09-10T20:39:58.265623+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:58.265623+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20907,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.263985+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:58.263985+0000","last_peered":"2024-09-10T20:39:58.263985+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:58.263985+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20884,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.262813+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:58.262813+0000","last_peered":"2024-09-10T20:39:58.262813+0000","last_clean":"2024-09-10T20:39:58.262813+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:58.262813+0000","last_undegraded":"2024-09-10T20:39:58.262813+0000","last_fullsized":"2024-09-10T20:39:58.262813+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.262961+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:39:58.262961+0000","last_peered":"2024-09-10T20:39:58.262961+0000","last_clean":"2024-09-10T20:39:58.262961+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:39:58.262961+0000","last_undegraded":"2024-09-10T20:39:58.262961+0000","last_fullsized":"2024-09-10T20:39:58.262961+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22126,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.262973+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:39:58.262973+0000","last_peered":"2024-09-10T20:39:58.262973+0000","last_clean":"2024-09-10T20:39:58.262973+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:39:58.262973+0000","last_undegraded":"2024-09-10T20:39:58.262973+0000","last_fullsized":"2024-09-10T20:39:58.262973+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277094+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:39:58.277094+0000","last_peered":"2024-09-10T20:39:58.277094+0000","last_clean":"2024-09-10T20:39:58.277094+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:39:58.277094+0000","last_undegraded":"2024-09-10T20:39:58.277094+0000","last_fullsized":"2024-09-10T20:39:58.277094+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":20960,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.265211+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:58.265211+0000","last_peered":"2024-09-10T20:39:58.265211+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:58.265211+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3564}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.264754+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:39:58.264754+0000","last_peered":"2024-09-10T20:39:58.264754+0000","last_clean":"2024-09-10T20:39:58.264754+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:39:58.264754+0000","last_undegraded":"2024-09-10T20:39:58.264754+0000","last_fullsized":"2024-09-10T20:39:58.264754+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":23630,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:04.796118+0000","last_change":"2024-09-10T20:39:56.460609+0000","last_active":"2024-09-10T20:40:04.796118+0000","last_peered":"2024-09-10T20:40:04.796118+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:40:04.796118+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1456,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2179}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":102,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.262885+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:39:58.262885+0000","last_peered":"2024-09-10T20:39:58.262885+0000","last_clean":"2024-09-10T20:39:58.262885+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:39:58.262885+0000","last_undegraded":"2024-09-10T20:39:58.262885+0000","last_fullsized":"2024-09-10T20:39:58.262885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28048,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.276958+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:58.276958+0000","last_peered":"2024-09-10T20:39:58.276958+0000","last_clean":"2024-09-10T20:39:58.276958+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:58.276958+0000","last_undegraded":"2024-09-10T20:39:58.276958+0000","last_fullsized":"2024-09-10T20:39:58.276958+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21105,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.338466+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:39:58.338466+0000","last_peered":"2024-09-10T20:39:58.338466+0000","last_clean":"2024-09-10T20:39:58.338466+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:39:58.338466+0000","last_undegraded":"2024-09-10T20:39:58.338466+0000","last_fullsized":"2024-09-10T20:39:58.338466+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263152+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:39:58.263152+0000","last_peered":"2024-09-10T20:39:58.263152+0000","last_clean":"2024-09-10T20:39:58.263152+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:39:58.263152+0000","last_undegraded":"2024-09-10T20:39:58.263152+0000","last_fullsized":"2024-09-10T20:39:58.263152+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27826,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263403+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:58.263403+0000","last_peered":"2024-09-10T20:39:58.263403+0000","last_clean":"2024-09-10T20:39:58.263403+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:58.263403+0000","last_undegraded":"2024-09-10T20:39:58.263403+0000","last_fullsized":"2024-09-10T20:39:58.263403+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277571+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:39:58.277571+0000","last_peered":"2024-09-10T20:39:58.277571+0000","last_clean":"2024-09-10T20:39:58.277571+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:39:58.277571+0000","last_undegraded":"2024-09-10T20:39:58.277571+0000","last_fullsized":"2024-09-10T20:39:58.277571+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21257,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.263453+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:58.263453+0000","last_peered":"2024-09-10T20:39:58.263453+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:58.263453+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":92,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263317+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:39:58.263317+0000","last_peered":"2024-09-10T20:39:58.263317+0000","last_clean":"2024-09-10T20:39:58.263317+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:39:58.263317+0000","last_undegraded":"2024-09-10T20:39:58.263317+0000","last_fullsized":"2024-09-10T20:39:58.263317+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263276+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:39:58.263276+0000","last_peered":"2024-09-10T20:39:58.263276+0000","last_clean":"2024-09-10T20:39:58.263276+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:39:58.263276+0000","last_undegraded":"2024-09-10T20:39:58.263276+0000","last_fullsized":"2024-09-10T20:39:58.263276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27656,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263068+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:58.263068+0000","last_peered":"2024-09-10T20:39:58.263068+0000","last_clean":"2024-09-10T20:39:58.263068+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:58.263068+0000","last_undegraded":"2024-09-10T20:39:58.263068+0000","last_fullsized":"2024-09-10T20:39:58.263068+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19996,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23153,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176676864,"data_stored":176637121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182538,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517164,"kb_used_data":57940,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854676,"statfs":{"total":96636764160,"available":96107188224,"internally_reserved":0,"allocated":59330560,"data_stored":59039347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986310,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480344,"kb_used_data":115460,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93891496,"statfs":{"total":96636764160,"available":96144891904,"internally_reserved":0,"allocated":118231040,"data_stored":117943904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659256,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":495864,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":495355,"kb_avail":93875976,"statfs":{"total":96636764160,"available":96128999424,"internally_reserved":0,"allocated":499712,"data_stored":179595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":507244206},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117772288,"data_stored":117768675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:40:07.317 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:40:07.318 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'20658", 'reported_seq': 20893, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:58.263246+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:39:58.263246+0000', 'last_peered': '2024-09-10T20:39:58.263246+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:39:58.263246+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20658, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3541}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:07.318 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:40:07.318 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20471", 'reported_seq': 20737, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:58.265623+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:39:58.265623+0000', 'last_peered': '2024-09-10T20:39:58.265623+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:39:58.265623+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17335, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3561}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:07.318 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:40:07.318 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'20608", 'reported_seq': 20907, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:58.263985+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:39:58.263985+0000', 'last_peered': '2024-09-10T20:39:58.263985+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:39:58.263985+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17203, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3486}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:07.318 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:40:07.318 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20730", 'reported_seq': 20960, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:58.265211+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:39:58.265211+0000', 'last_peered': '2024-09-10T20:39:58.265211+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:39:58.265211+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20728, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3564}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:07.319 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:40:07.319 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "51'20565", 'reported_seq': 23630, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:04.796118+0000', 'last_change': '2024-09-10T20:39:56.460609+0000', 'last_active': '2024-09-10T20:40:04.796118+0000', 'last_peered': '2024-09-10T20:40:04.796118+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:40:04.796118+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2179, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18393, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1456, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2179}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:40:07.319 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:40:07.319 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "51'20986", 'reported_seq': 21257, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:58.263453+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:39:58.263453+0000', 'last_peered': '2024-09-10T20:39:58.263453+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:39:58.263453+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17813, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3655}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:07.319 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:40:07.707 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:07.707 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 19996/10 objects degraded (199960.000%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":719,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":5,"data_bytes":1468,"bytes_used":1529212928,"bytes_avail":288381079552,"bytes_total":289910292480,"degraded_objects":19996,"degraded_total":10,"degraded_ratio":1999.5999999999999,"recovering_objects_per_sec":180,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1768,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:40:07.721 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:40:07.721 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:40:08.026 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:08.026 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:40:08.040 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":408,"stamp":"2024-09-10T20:40:07.063525+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19996,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23155,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1493372,"kb_used_data":173888,"kb_used_omap":12,"kb_used_meta":1319411,"kb_avail":281622148,"statfs":{"total":289910292480,"available":288381079552,"internally_reserved":0,"allocated":178061312,"data_stored":177162846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1351076866},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1768,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1768,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.807851"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20893,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.263246+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:58.263246+0000","last_peered":"2024-09-10T20:39:58.263246+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:58.263246+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21075,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277759+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:58.277759+0000","last_peered":"2024-09-10T20:39:58.277759+0000","last_clean":"2024-09-10T20:39:58.277759+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:58.277759+0000","last_undegraded":"2024-09-10T20:39:58.277759+0000","last_fullsized":"2024-09-10T20:39:58.277759+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28087,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263157+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:58.263157+0000","last_peered":"2024-09-10T20:39:58.263157+0000","last_clean":"2024-09-10T20:39:58.263157+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:58.263157+0000","last_undegraded":"2024-09-10T20:39:58.263157+0000","last_fullsized":"2024-09-10T20:39:58.263157+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20707,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277394+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:58.277394+0000","last_peered":"2024-09-10T20:39:58.277394+0000","last_clean":"2024-09-10T20:39:58.277394+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:58.277394+0000","last_undegraded":"2024-09-10T20:39:58.277394+0000","last_fullsized":"2024-09-10T20:39:58.277394+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21218,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.265097+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:58.265097+0000","last_peered":"2024-09-10T20:39:58.265097+0000","last_clean":"2024-09-10T20:39:58.265097+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:58.265097+0000","last_undegraded":"2024-09-10T20:39:58.265097+0000","last_fullsized":"2024-09-10T20:39:58.265097+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20737,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.265623+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:58.265623+0000","last_peered":"2024-09-10T20:39:58.265623+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:58.265623+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20907,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.263985+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:58.263985+0000","last_peered":"2024-09-10T20:39:58.263985+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:58.263985+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20884,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.262813+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:58.262813+0000","last_peered":"2024-09-10T20:39:58.262813+0000","last_clean":"2024-09-10T20:39:58.262813+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:58.262813+0000","last_undegraded":"2024-09-10T20:39:58.262813+0000","last_fullsized":"2024-09-10T20:39:58.262813+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.262961+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:39:58.262961+0000","last_peered":"2024-09-10T20:39:58.262961+0000","last_clean":"2024-09-10T20:39:58.262961+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:39:58.262961+0000","last_undegraded":"2024-09-10T20:39:58.262961+0000","last_fullsized":"2024-09-10T20:39:58.262961+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22126,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.262973+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:39:58.262973+0000","last_peered":"2024-09-10T20:39:58.262973+0000","last_clean":"2024-09-10T20:39:58.262973+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:39:58.262973+0000","last_undegraded":"2024-09-10T20:39:58.262973+0000","last_fullsized":"2024-09-10T20:39:58.262973+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277094+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:39:58.277094+0000","last_peered":"2024-09-10T20:39:58.277094+0000","last_clean":"2024-09-10T20:39:58.277094+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:39:58.277094+0000","last_undegraded":"2024-09-10T20:39:58.277094+0000","last_fullsized":"2024-09-10T20:39:58.277094+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":20960,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.265211+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:58.265211+0000","last_peered":"2024-09-10T20:39:58.265211+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:58.265211+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3564}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.264754+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:39:58.264754+0000","last_peered":"2024-09-10T20:39:58.264754+0000","last_clean":"2024-09-10T20:39:58.264754+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:39:58.264754+0000","last_undegraded":"2024-09-10T20:39:58.264754+0000","last_fullsized":"2024-09-10T20:39:58.264754+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":23630,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:04.796118+0000","last_change":"2024-09-10T20:39:56.460609+0000","last_active":"2024-09-10T20:40:04.796118+0000","last_peered":"2024-09-10T20:40:04.796118+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:40:04.796118+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1456,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2179}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":102,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.262885+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:39:58.262885+0000","last_peered":"2024-09-10T20:39:58.262885+0000","last_clean":"2024-09-10T20:39:58.262885+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:39:58.262885+0000","last_undegraded":"2024-09-10T20:39:58.262885+0000","last_fullsized":"2024-09-10T20:39:58.262885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28048,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.276958+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:58.276958+0000","last_peered":"2024-09-10T20:39:58.276958+0000","last_clean":"2024-09-10T20:39:58.276958+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:58.276958+0000","last_undegraded":"2024-09-10T20:39:58.276958+0000","last_fullsized":"2024-09-10T20:39:58.276958+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21105,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.338466+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:39:58.338466+0000","last_peered":"2024-09-10T20:39:58.338466+0000","last_clean":"2024-09-10T20:39:58.338466+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:39:58.338466+0000","last_undegraded":"2024-09-10T20:39:58.338466+0000","last_fullsized":"2024-09-10T20:39:58.338466+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263152+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:39:58.263152+0000","last_peered":"2024-09-10T20:39:58.263152+0000","last_clean":"2024-09-10T20:39:58.263152+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:39:58.263152+0000","last_undegraded":"2024-09-10T20:39:58.263152+0000","last_fullsized":"2024-09-10T20:39:58.263152+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27826,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263403+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:58.263403+0000","last_peered":"2024-09-10T20:39:58.263403+0000","last_clean":"2024-09-10T20:39:58.263403+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:58.263403+0000","last_undegraded":"2024-09-10T20:39:58.263403+0000","last_fullsized":"2024-09-10T20:39:58.263403+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277571+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:39:58.277571+0000","last_peered":"2024-09-10T20:39:58.277571+0000","last_clean":"2024-09-10T20:39:58.277571+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:39:58.277571+0000","last_undegraded":"2024-09-10T20:39:58.277571+0000","last_fullsized":"2024-09-10T20:39:58.277571+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21257,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.263453+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:58.263453+0000","last_peered":"2024-09-10T20:39:58.263453+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:58.263453+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":92,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263317+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:39:58.263317+0000","last_peered":"2024-09-10T20:39:58.263317+0000","last_clean":"2024-09-10T20:39:58.263317+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:39:58.263317+0000","last_undegraded":"2024-09-10T20:39:58.263317+0000","last_fullsized":"2024-09-10T20:39:58.263317+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263276+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:39:58.263276+0000","last_peered":"2024-09-10T20:39:58.263276+0000","last_clean":"2024-09-10T20:39:58.263276+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:39:58.263276+0000","last_undegraded":"2024-09-10T20:39:58.263276+0000","last_fullsized":"2024-09-10T20:39:58.263276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27656,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263068+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:58.263068+0000","last_peered":"2024-09-10T20:39:58.263068+0000","last_clean":"2024-09-10T20:39:58.263068+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:58.263068+0000","last_undegraded":"2024-09-10T20:39:58.263068+0000","last_fullsized":"2024-09-10T20:39:58.263068+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19996,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23153,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176676864,"data_stored":176637121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182538,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517164,"kb_used_data":57940,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854676,"statfs":{"total":96636764160,"available":96107188224,"internally_reserved":0,"allocated":59330560,"data_stored":59039347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986310,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480344,"kb_used_data":115460,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93891496,"statfs":{"total":96636764160,"available":96144891904,"internally_reserved":0,"allocated":118231040,"data_stored":117943904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659256,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":495864,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":495355,"kb_avail":93875976,"statfs":{"total":96636764160,"available":96128999424,"internally_reserved":0,"allocated":499712,"data_stored":179595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":507244206},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117772288,"data_stored":117768675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:40:11.042 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:40:11.362 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:11.362 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:40:11.376 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":410,"stamp":"2024-09-10T20:40:11.064320+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19186,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23965,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1480892,"kb_used_data":173888,"kb_used_omap":12,"kb_used_meta":1306931,"kb_avail":281634628,"statfs":{"total":289910292480,"available":288393859072,"internally_reserved":0,"allocated":178061312,"data_stored":177162846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1338297346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2239,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5832,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002223"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20893,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.263246+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:58.263246+0000","last_peered":"2024-09-10T20:39:58.263246+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:58.263246+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21075,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277759+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:58.277759+0000","last_peered":"2024-09-10T20:39:58.277759+0000","last_clean":"2024-09-10T20:39:58.277759+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:58.277759+0000","last_undegraded":"2024-09-10T20:39:58.277759+0000","last_fullsized":"2024-09-10T20:39:58.277759+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28087,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263157+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:58.263157+0000","last_peered":"2024-09-10T20:39:58.263157+0000","last_clean":"2024-09-10T20:39:58.263157+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:58.263157+0000","last_undegraded":"2024-09-10T20:39:58.263157+0000","last_fullsized":"2024-09-10T20:39:58.263157+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20707,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277394+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:58.277394+0000","last_peered":"2024-09-10T20:39:58.277394+0000","last_clean":"2024-09-10T20:39:58.277394+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:58.277394+0000","last_undegraded":"2024-09-10T20:39:58.277394+0000","last_fullsized":"2024-09-10T20:39:58.277394+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21219,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.265142+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:58.265142+0000","last_peered":"2024-09-10T20:39:58.265142+0000","last_clean":"2024-09-10T20:39:58.265142+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:58.265142+0000","last_undegraded":"2024-09-10T20:39:58.265142+0000","last_fullsized":"2024-09-10T20:39:58.265142+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20737,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.265623+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:58.265623+0000","last_peered":"2024-09-10T20:39:58.265623+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:58.265623+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20907,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.263985+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:58.263985+0000","last_peered":"2024-09-10T20:39:58.263985+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:58.263985+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20884,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.262813+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:58.262813+0000","last_peered":"2024-09-10T20:39:58.262813+0000","last_clean":"2024-09-10T20:39:58.262813+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:58.262813+0000","last_undegraded":"2024-09-10T20:39:58.262813+0000","last_fullsized":"2024-09-10T20:39:58.262813+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263001+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:39:58.263001+0000","last_peered":"2024-09-10T20:39:58.263001+0000","last_clean":"2024-09-10T20:39:58.263001+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:39:58.263001+0000","last_undegraded":"2024-09-10T20:39:58.263001+0000","last_fullsized":"2024-09-10T20:39:58.263001+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22126,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.262973+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:39:58.262973+0000","last_peered":"2024-09-10T20:39:58.262973+0000","last_clean":"2024-09-10T20:39:58.262973+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:39:58.262973+0000","last_undegraded":"2024-09-10T20:39:58.262973+0000","last_fullsized":"2024-09-10T20:39:58.262973+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277094+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:39:58.277094+0000","last_peered":"2024-09-10T20:39:58.277094+0000","last_clean":"2024-09-10T20:39:58.277094+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:39:58.277094+0000","last_undegraded":"2024-09-10T20:39:58.277094+0000","last_fullsized":"2024-09-10T20:39:58.277094+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":20960,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.265211+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:58.265211+0000","last_peered":"2024-09-10T20:39:58.265211+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:58.265211+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3564}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.264754+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:39:58.264754+0000","last_peered":"2024-09-10T20:39:58.264754+0000","last_clean":"2024-09-10T20:39:58.264754+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:39:58.264754+0000","last_undegraded":"2024-09-10T20:39:58.264754+0000","last_fullsized":"2024-09-10T20:39:58.264754+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":25250,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:09.796741+0000","last_change":"2024-09-10T20:39:56.460609+0000","last_active":"2024-09-10T20:40:09.796741+0000","last_peered":"2024-09-10T20:40:09.796741+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:40:09.796741+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1369,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2266,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1369}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":102,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.262885+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:39:58.262885+0000","last_peered":"2024-09-10T20:39:58.262885+0000","last_clean":"2024-09-10T20:39:58.262885+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:39:58.262885+0000","last_undegraded":"2024-09-10T20:39:58.262885+0000","last_fullsized":"2024-09-10T20:39:58.262885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28048,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.276958+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:58.276958+0000","last_peered":"2024-09-10T20:39:58.276958+0000","last_clean":"2024-09-10T20:39:58.276958+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:58.276958+0000","last_undegraded":"2024-09-10T20:39:58.276958+0000","last_fullsized":"2024-09-10T20:39:58.276958+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21105,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.338466+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:39:58.338466+0000","last_peered":"2024-09-10T20:39:58.338466+0000","last_clean":"2024-09-10T20:39:58.338466+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:39:58.338466+0000","last_undegraded":"2024-09-10T20:39:58.338466+0000","last_fullsized":"2024-09-10T20:39:58.338466+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263152+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:39:58.263152+0000","last_peered":"2024-09-10T20:39:58.263152+0000","last_clean":"2024-09-10T20:39:58.263152+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:39:58.263152+0000","last_undegraded":"2024-09-10T20:39:58.263152+0000","last_fullsized":"2024-09-10T20:39:58.263152+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27826,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263403+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:58.263403+0000","last_peered":"2024-09-10T20:39:58.263403+0000","last_clean":"2024-09-10T20:39:58.263403+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:58.263403+0000","last_undegraded":"2024-09-10T20:39:58.263403+0000","last_fullsized":"2024-09-10T20:39:58.263403+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277571+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:39:58.277571+0000","last_peered":"2024-09-10T20:39:58.277571+0000","last_clean":"2024-09-10T20:39:58.277571+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:39:58.277571+0000","last_undegraded":"2024-09-10T20:39:58.277571+0000","last_fullsized":"2024-09-10T20:39:58.277571+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21257,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.263453+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:58.263453+0000","last_peered":"2024-09-10T20:39:58.263453+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:58.263453+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":92,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263317+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:39:58.263317+0000","last_peered":"2024-09-10T20:39:58.263317+0000","last_clean":"2024-09-10T20:39:58.263317+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:39:58.263317+0000","last_undegraded":"2024-09-10T20:39:58.263317+0000","last_fullsized":"2024-09-10T20:39:58.263317+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263276+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:39:58.263276+0000","last_peered":"2024-09-10T20:39:58.263276+0000","last_clean":"2024-09-10T20:39:58.263276+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:39:58.263276+0000","last_undegraded":"2024-09-10T20:39:58.263276+0000","last_fullsized":"2024-09-10T20:39:58.263276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27656,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263068+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:58.263068+0000","last_peered":"2024-09-10T20:39:58.263068+0000","last_clean":"2024-09-10T20:39:58.263068+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:58.263068+0000","last_undegraded":"2024-09-10T20:39:58.263068+0000","last_fullsized":"2024-09-10T20:39:58.263068+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19186,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23963,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176676864,"data_stored":176637121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182539,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517164,"kb_used_data":57940,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854676,"statfs":{"total":96636764160,"available":96107188224,"internally_reserved":0,"allocated":59330560,"data_stored":59039347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986311,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480344,"kb_used_data":115460,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93891496,"statfs":{"total":96636764160,"available":96144891904,"internally_reserved":0,"allocated":118231040,"data_stored":117943904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659257,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483384,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888456,"statfs":{"total":96636764160,"available":96141778944,"internally_reserved":0,"allocated":499712,"data_stored":179595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117772288,"data_stored":117768675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:40:11.377 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:40:11.705 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:11.705 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:40:11.721 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":410,"stamp":"2024-09-10T20:40:11.064320+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19186,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23965,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1480892,"kb_used_data":173888,"kb_used_omap":12,"kb_used_meta":1306931,"kb_avail":281634628,"statfs":{"total":289910292480,"available":288393859072,"internally_reserved":0,"allocated":178061312,"data_stored":177162846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1338297346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2239,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5832,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002223"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20893,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.263246+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:58.263246+0000","last_peered":"2024-09-10T20:39:58.263246+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:58.263246+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21075,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277759+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:58.277759+0000","last_peered":"2024-09-10T20:39:58.277759+0000","last_clean":"2024-09-10T20:39:58.277759+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:58.277759+0000","last_undegraded":"2024-09-10T20:39:58.277759+0000","last_fullsized":"2024-09-10T20:39:58.277759+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28087,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263157+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:58.263157+0000","last_peered":"2024-09-10T20:39:58.263157+0000","last_clean":"2024-09-10T20:39:58.263157+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:58.263157+0000","last_undegraded":"2024-09-10T20:39:58.263157+0000","last_fullsized":"2024-09-10T20:39:58.263157+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20707,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277394+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:58.277394+0000","last_peered":"2024-09-10T20:39:58.277394+0000","last_clean":"2024-09-10T20:39:58.277394+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:58.277394+0000","last_undegraded":"2024-09-10T20:39:58.277394+0000","last_fullsized":"2024-09-10T20:39:58.277394+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21219,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.265142+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:58.265142+0000","last_peered":"2024-09-10T20:39:58.265142+0000","last_clean":"2024-09-10T20:39:58.265142+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:58.265142+0000","last_undegraded":"2024-09-10T20:39:58.265142+0000","last_fullsized":"2024-09-10T20:39:58.265142+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20737,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.265623+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:58.265623+0000","last_peered":"2024-09-10T20:39:58.265623+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:58.265623+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20907,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.263985+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:58.263985+0000","last_peered":"2024-09-10T20:39:58.263985+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:58.263985+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20884,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.262813+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:58.262813+0000","last_peered":"2024-09-10T20:39:58.262813+0000","last_clean":"2024-09-10T20:39:58.262813+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:58.262813+0000","last_undegraded":"2024-09-10T20:39:58.262813+0000","last_fullsized":"2024-09-10T20:39:58.262813+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263001+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:39:58.263001+0000","last_peered":"2024-09-10T20:39:58.263001+0000","last_clean":"2024-09-10T20:39:58.263001+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:39:58.263001+0000","last_undegraded":"2024-09-10T20:39:58.263001+0000","last_fullsized":"2024-09-10T20:39:58.263001+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22126,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.262973+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:39:58.262973+0000","last_peered":"2024-09-10T20:39:58.262973+0000","last_clean":"2024-09-10T20:39:58.262973+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:39:58.262973+0000","last_undegraded":"2024-09-10T20:39:58.262973+0000","last_fullsized":"2024-09-10T20:39:58.262973+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277094+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:39:58.277094+0000","last_peered":"2024-09-10T20:39:58.277094+0000","last_clean":"2024-09-10T20:39:58.277094+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:39:58.277094+0000","last_undegraded":"2024-09-10T20:39:58.277094+0000","last_fullsized":"2024-09-10T20:39:58.277094+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":20960,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.265211+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:58.265211+0000","last_peered":"2024-09-10T20:39:58.265211+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:58.265211+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3564}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.264754+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:39:58.264754+0000","last_peered":"2024-09-10T20:39:58.264754+0000","last_clean":"2024-09-10T20:39:58.264754+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:39:58.264754+0000","last_undegraded":"2024-09-10T20:39:58.264754+0000","last_fullsized":"2024-09-10T20:39:58.264754+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":25250,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:09.796741+0000","last_change":"2024-09-10T20:39:56.460609+0000","last_active":"2024-09-10T20:40:09.796741+0000","last_peered":"2024-09-10T20:40:09.796741+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:40:09.796741+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1369,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2266,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1369}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":102,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.262885+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:39:58.262885+0000","last_peered":"2024-09-10T20:39:58.262885+0000","last_clean":"2024-09-10T20:39:58.262885+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:39:58.262885+0000","last_undegraded":"2024-09-10T20:39:58.262885+0000","last_fullsized":"2024-09-10T20:39:58.262885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28048,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.276958+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:58.276958+0000","last_peered":"2024-09-10T20:39:58.276958+0000","last_clean":"2024-09-10T20:39:58.276958+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:58.276958+0000","last_undegraded":"2024-09-10T20:39:58.276958+0000","last_fullsized":"2024-09-10T20:39:58.276958+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21105,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.338466+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:39:58.338466+0000","last_peered":"2024-09-10T20:39:58.338466+0000","last_clean":"2024-09-10T20:39:58.338466+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:39:58.338466+0000","last_undegraded":"2024-09-10T20:39:58.338466+0000","last_fullsized":"2024-09-10T20:39:58.338466+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263152+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:39:58.263152+0000","last_peered":"2024-09-10T20:39:58.263152+0000","last_clean":"2024-09-10T20:39:58.263152+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:39:58.263152+0000","last_undegraded":"2024-09-10T20:39:58.263152+0000","last_fullsized":"2024-09-10T20:39:58.263152+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27826,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263403+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:58.263403+0000","last_peered":"2024-09-10T20:39:58.263403+0000","last_clean":"2024-09-10T20:39:58.263403+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:58.263403+0000","last_undegraded":"2024-09-10T20:39:58.263403+0000","last_fullsized":"2024-09-10T20:39:58.263403+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277571+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:39:58.277571+0000","last_peered":"2024-09-10T20:39:58.277571+0000","last_clean":"2024-09-10T20:39:58.277571+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:39:58.277571+0000","last_undegraded":"2024-09-10T20:39:58.277571+0000","last_fullsized":"2024-09-10T20:39:58.277571+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21257,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.263453+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:58.263453+0000","last_peered":"2024-09-10T20:39:58.263453+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:58.263453+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":92,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263317+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:39:58.263317+0000","last_peered":"2024-09-10T20:39:58.263317+0000","last_clean":"2024-09-10T20:39:58.263317+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:39:58.263317+0000","last_undegraded":"2024-09-10T20:39:58.263317+0000","last_fullsized":"2024-09-10T20:39:58.263317+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263276+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:39:58.263276+0000","last_peered":"2024-09-10T20:39:58.263276+0000","last_clean":"2024-09-10T20:39:58.263276+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:39:58.263276+0000","last_undegraded":"2024-09-10T20:39:58.263276+0000","last_fullsized":"2024-09-10T20:39:58.263276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27656,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263068+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:58.263068+0000","last_peered":"2024-09-10T20:39:58.263068+0000","last_clean":"2024-09-10T20:39:58.263068+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:58.263068+0000","last_undegraded":"2024-09-10T20:39:58.263068+0000","last_fullsized":"2024-09-10T20:39:58.263068+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19186,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23963,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176676864,"data_stored":176637121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182539,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517164,"kb_used_data":57940,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854676,"statfs":{"total":96636764160,"available":96107188224,"internally_reserved":0,"allocated":59330560,"data_stored":59039347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986311,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480344,"kb_used_data":115460,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93891496,"statfs":{"total":96636764160,"available":96144891904,"internally_reserved":0,"allocated":118231040,"data_stored":117943904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659257,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483384,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888456,"statfs":{"total":96636764160,"available":96141778944,"internally_reserved":0,"allocated":499712,"data_stored":179595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117772288,"data_stored":117768675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:40:11.722 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:40:11.722 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'20658", 'reported_seq': 20893, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:58.263246+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:39:58.263246+0000', 'last_peered': '2024-09-10T20:39:58.263246+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:39:58.263246+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20658, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3541}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:11.722 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:40:11.722 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20471", 'reported_seq': 20737, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:58.265623+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:39:58.265623+0000', 'last_peered': '2024-09-10T20:39:58.265623+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:39:58.265623+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17335, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3561}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:11.722 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:40:11.723 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'20608", 'reported_seq': 20907, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:58.263985+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:39:58.263985+0000', 'last_peered': '2024-09-10T20:39:58.263985+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:39:58.263985+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17203, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3486}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:11.723 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:40:11.723 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20730", 'reported_seq': 20960, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:58.265211+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:39:58.265211+0000', 'last_peered': '2024-09-10T20:39:58.265211+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:39:58.265211+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20728, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3564}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:11.723 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:40:11.723 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "51'20565", 'reported_seq': 25250, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:09.796741+0000', 'last_change': '2024-09-10T20:39:56.460609+0000', 'last_active': '2024-09-10T20:40:09.796741+0000', 'last_peered': '2024-09-10T20:40:09.796741+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:40:09.796741+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1369, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18393, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2266, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1369}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:40:11.723 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:40:11.723 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "51'20986", 'reported_seq': 21257, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:58.263453+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:39:58.263453+0000', 'last_peered': '2024-09-10T20:39:58.263453+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:39:58.263453+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17813, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3655}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:11.723 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:40:12.103 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:12.103 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 19186/10 objects degraded (191860.000%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":723,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":5,"data_bytes":1468,"bytes_used":1516433408,"bytes_avail":288393859072,"bytes_total":289910292480,"degraded_objects":19186,"degraded_total":10,"degraded_ratio":1918.5999999999999,"recovering_objects_per_sec":485,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":5832,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:40:12.118 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:40:12.118 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:40:12.430 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:12.430 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:40:12.443 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":410,"stamp":"2024-09-10T20:40:11.064320+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19186,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23965,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1480892,"kb_used_data":173888,"kb_used_omap":12,"kb_used_meta":1306931,"kb_avail":281634628,"statfs":{"total":289910292480,"available":288393859072,"internally_reserved":0,"allocated":178061312,"data_stored":177162846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1338297346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2239,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5832,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002223"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20893,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.263246+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:58.263246+0000","last_peered":"2024-09-10T20:39:58.263246+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:58.263246+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21075,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277759+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:58.277759+0000","last_peered":"2024-09-10T20:39:58.277759+0000","last_clean":"2024-09-10T20:39:58.277759+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:58.277759+0000","last_undegraded":"2024-09-10T20:39:58.277759+0000","last_fullsized":"2024-09-10T20:39:58.277759+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28087,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263157+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:58.263157+0000","last_peered":"2024-09-10T20:39:58.263157+0000","last_clean":"2024-09-10T20:39:58.263157+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:58.263157+0000","last_undegraded":"2024-09-10T20:39:58.263157+0000","last_fullsized":"2024-09-10T20:39:58.263157+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20707,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277394+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:58.277394+0000","last_peered":"2024-09-10T20:39:58.277394+0000","last_clean":"2024-09-10T20:39:58.277394+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:58.277394+0000","last_undegraded":"2024-09-10T20:39:58.277394+0000","last_fullsized":"2024-09-10T20:39:58.277394+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21219,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.265142+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:58.265142+0000","last_peered":"2024-09-10T20:39:58.265142+0000","last_clean":"2024-09-10T20:39:58.265142+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:58.265142+0000","last_undegraded":"2024-09-10T20:39:58.265142+0000","last_fullsized":"2024-09-10T20:39:58.265142+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20737,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.265623+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:58.265623+0000","last_peered":"2024-09-10T20:39:58.265623+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:58.265623+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20907,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.263985+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:58.263985+0000","last_peered":"2024-09-10T20:39:58.263985+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:58.263985+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20884,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.262813+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:58.262813+0000","last_peered":"2024-09-10T20:39:58.262813+0000","last_clean":"2024-09-10T20:39:58.262813+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:58.262813+0000","last_undegraded":"2024-09-10T20:39:58.262813+0000","last_fullsized":"2024-09-10T20:39:58.262813+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263001+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:39:58.263001+0000","last_peered":"2024-09-10T20:39:58.263001+0000","last_clean":"2024-09-10T20:39:58.263001+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:39:58.263001+0000","last_undegraded":"2024-09-10T20:39:58.263001+0000","last_fullsized":"2024-09-10T20:39:58.263001+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22126,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.262973+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:39:58.262973+0000","last_peered":"2024-09-10T20:39:58.262973+0000","last_clean":"2024-09-10T20:39:58.262973+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:39:58.262973+0000","last_undegraded":"2024-09-10T20:39:58.262973+0000","last_fullsized":"2024-09-10T20:39:58.262973+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277094+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:39:58.277094+0000","last_peered":"2024-09-10T20:39:58.277094+0000","last_clean":"2024-09-10T20:39:58.277094+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:39:58.277094+0000","last_undegraded":"2024-09-10T20:39:58.277094+0000","last_fullsized":"2024-09-10T20:39:58.277094+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":20960,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.265211+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:58.265211+0000","last_peered":"2024-09-10T20:39:58.265211+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:58.265211+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3564}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.264754+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:39:58.264754+0000","last_peered":"2024-09-10T20:39:58.264754+0000","last_clean":"2024-09-10T20:39:58.264754+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:39:58.264754+0000","last_undegraded":"2024-09-10T20:39:58.264754+0000","last_fullsized":"2024-09-10T20:39:58.264754+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":25250,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:09.796741+0000","last_change":"2024-09-10T20:39:56.460609+0000","last_active":"2024-09-10T20:40:09.796741+0000","last_peered":"2024-09-10T20:40:09.796741+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:40:09.796741+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1369,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2266,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1369}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":102,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.262885+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:39:58.262885+0000","last_peered":"2024-09-10T20:39:58.262885+0000","last_clean":"2024-09-10T20:39:58.262885+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:39:58.262885+0000","last_undegraded":"2024-09-10T20:39:58.262885+0000","last_fullsized":"2024-09-10T20:39:58.262885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28048,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.276958+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:58.276958+0000","last_peered":"2024-09-10T20:39:58.276958+0000","last_clean":"2024-09-10T20:39:58.276958+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:58.276958+0000","last_undegraded":"2024-09-10T20:39:58.276958+0000","last_fullsized":"2024-09-10T20:39:58.276958+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21105,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.338466+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:39:58.338466+0000","last_peered":"2024-09-10T20:39:58.338466+0000","last_clean":"2024-09-10T20:39:58.338466+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:39:58.338466+0000","last_undegraded":"2024-09-10T20:39:58.338466+0000","last_fullsized":"2024-09-10T20:39:58.338466+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263152+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:39:58.263152+0000","last_peered":"2024-09-10T20:39:58.263152+0000","last_clean":"2024-09-10T20:39:58.263152+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:39:58.263152+0000","last_undegraded":"2024-09-10T20:39:58.263152+0000","last_fullsized":"2024-09-10T20:39:58.263152+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27826,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263403+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:58.263403+0000","last_peered":"2024-09-10T20:39:58.263403+0000","last_clean":"2024-09-10T20:39:58.263403+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:58.263403+0000","last_undegraded":"2024-09-10T20:39:58.263403+0000","last_fullsized":"2024-09-10T20:39:58.263403+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277571+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:39:58.277571+0000","last_peered":"2024-09-10T20:39:58.277571+0000","last_clean":"2024-09-10T20:39:58.277571+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:39:58.277571+0000","last_undegraded":"2024-09-10T20:39:58.277571+0000","last_fullsized":"2024-09-10T20:39:58.277571+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21257,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.263453+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:58.263453+0000","last_peered":"2024-09-10T20:39:58.263453+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:58.263453+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":92,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263317+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:39:58.263317+0000","last_peered":"2024-09-10T20:39:58.263317+0000","last_clean":"2024-09-10T20:39:58.263317+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:39:58.263317+0000","last_undegraded":"2024-09-10T20:39:58.263317+0000","last_fullsized":"2024-09-10T20:39:58.263317+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263276+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:39:58.263276+0000","last_peered":"2024-09-10T20:39:58.263276+0000","last_clean":"2024-09-10T20:39:58.263276+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:39:58.263276+0000","last_undegraded":"2024-09-10T20:39:58.263276+0000","last_fullsized":"2024-09-10T20:39:58.263276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27656,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263068+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:58.263068+0000","last_peered":"2024-09-10T20:39:58.263068+0000","last_clean":"2024-09-10T20:39:58.263068+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:58.263068+0000","last_undegraded":"2024-09-10T20:39:58.263068+0000","last_fullsized":"2024-09-10T20:39:58.263068+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19186,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23963,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176676864,"data_stored":176637121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182539,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517164,"kb_used_data":57940,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854676,"statfs":{"total":96636764160,"available":96107188224,"internally_reserved":0,"allocated":59330560,"data_stored":59039347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986311,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480344,"kb_used_data":115460,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93891496,"statfs":{"total":96636764160,"available":96144891904,"internally_reserved":0,"allocated":118231040,"data_stored":117943904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659257,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483384,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888456,"statfs":{"total":96636764160,"available":96141778944,"internally_reserved":0,"allocated":499712,"data_stored":179595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117772288,"data_stored":117768675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:40:15.446 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:40:15.744 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:15.744 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:40:15.756 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":412,"stamp":"2024-09-10T20:40:15.065257+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18140,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25011,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1489212,"kb_used_data":173888,"kb_used_omap":12,"kb_used_meta":1315251,"kb_avail":281626308,"statfs":{"total":289910292480,"available":288385339392,"internally_reserved":0,"allocated":178061312,"data_stored":177162846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1346817026},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2819,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2819,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002506"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20893,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.263246+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:58.263246+0000","last_peered":"2024-09-10T20:39:58.263246+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:58.263246+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21075,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277759+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:58.277759+0000","last_peered":"2024-09-10T20:39:58.277759+0000","last_clean":"2024-09-10T20:39:58.277759+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:58.277759+0000","last_undegraded":"2024-09-10T20:39:58.277759+0000","last_fullsized":"2024-09-10T20:39:58.277759+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28087,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263157+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:58.263157+0000","last_peered":"2024-09-10T20:39:58.263157+0000","last_clean":"2024-09-10T20:39:58.263157+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:58.263157+0000","last_undegraded":"2024-09-10T20:39:58.263157+0000","last_fullsized":"2024-09-10T20:39:58.263157+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20707,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277394+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:58.277394+0000","last_peered":"2024-09-10T20:39:58.277394+0000","last_clean":"2024-09-10T20:39:58.277394+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:58.277394+0000","last_undegraded":"2024-09-10T20:39:58.277394+0000","last_fullsized":"2024-09-10T20:39:58.277394+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21219,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.265142+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:58.265142+0000","last_peered":"2024-09-10T20:39:58.265142+0000","last_clean":"2024-09-10T20:39:58.265142+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:58.265142+0000","last_undegraded":"2024-09-10T20:39:58.265142+0000","last_fullsized":"2024-09-10T20:39:58.265142+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20737,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.265623+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:58.265623+0000","last_peered":"2024-09-10T20:39:58.265623+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:58.265623+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20907,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.263985+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:58.263985+0000","last_peered":"2024-09-10T20:39:58.263985+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:58.263985+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20884,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.262813+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:58.262813+0000","last_peered":"2024-09-10T20:39:58.262813+0000","last_clean":"2024-09-10T20:39:58.262813+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:58.262813+0000","last_undegraded":"2024-09-10T20:39:58.262813+0000","last_fullsized":"2024-09-10T20:39:58.262813+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263001+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:39:58.263001+0000","last_peered":"2024-09-10T20:39:58.263001+0000","last_clean":"2024-09-10T20:39:58.263001+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:39:58.263001+0000","last_undegraded":"2024-09-10T20:39:58.263001+0000","last_fullsized":"2024-09-10T20:39:58.263001+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22126,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.262973+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:39:58.262973+0000","last_peered":"2024-09-10T20:39:58.262973+0000","last_clean":"2024-09-10T20:39:58.262973+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:39:58.262973+0000","last_undegraded":"2024-09-10T20:39:58.262973+0000","last_fullsized":"2024-09-10T20:39:58.262973+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277094+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:39:58.277094+0000","last_peered":"2024-09-10T20:39:58.277094+0000","last_clean":"2024-09-10T20:39:58.277094+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:39:58.277094+0000","last_undegraded":"2024-09-10T20:39:58.277094+0000","last_fullsized":"2024-09-10T20:39:58.277094+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":20960,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.265211+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:58.265211+0000","last_peered":"2024-09-10T20:39:58.265211+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:58.265211+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3564}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.264754+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:39:58.264754+0000","last_peered":"2024-09-10T20:39:58.264754+0000","last_clean":"2024-09-10T20:39:58.264754+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:39:58.264754+0000","last_undegraded":"2024-09-10T20:39:58.264754+0000","last_fullsized":"2024-09-10T20:39:58.264754+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":27342,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:14.795462+0000","last_change":"2024-09-10T20:39:56.460609+0000","last_active":"2024-09-10T20:40:14.795462+0000","last_peered":"2024-09-10T20:40:14.795462+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:40:14.795462+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3312,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-323}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":102,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.262885+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:39:58.262885+0000","last_peered":"2024-09-10T20:39:58.262885+0000","last_clean":"2024-09-10T20:39:58.262885+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:39:58.262885+0000","last_undegraded":"2024-09-10T20:39:58.262885+0000","last_fullsized":"2024-09-10T20:39:58.262885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28048,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.276958+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:58.276958+0000","last_peered":"2024-09-10T20:39:58.276958+0000","last_clean":"2024-09-10T20:39:58.276958+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:58.276958+0000","last_undegraded":"2024-09-10T20:39:58.276958+0000","last_fullsized":"2024-09-10T20:39:58.276958+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21105,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.338466+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:39:58.338466+0000","last_peered":"2024-09-10T20:39:58.338466+0000","last_clean":"2024-09-10T20:39:58.338466+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:39:58.338466+0000","last_undegraded":"2024-09-10T20:39:58.338466+0000","last_fullsized":"2024-09-10T20:39:58.338466+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263152+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:39:58.263152+0000","last_peered":"2024-09-10T20:39:58.263152+0000","last_clean":"2024-09-10T20:39:58.263152+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:39:58.263152+0000","last_undegraded":"2024-09-10T20:39:58.263152+0000","last_fullsized":"2024-09-10T20:39:58.263152+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27826,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263403+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:58.263403+0000","last_peered":"2024-09-10T20:39:58.263403+0000","last_clean":"2024-09-10T20:39:58.263403+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:58.263403+0000","last_undegraded":"2024-09-10T20:39:58.263403+0000","last_fullsized":"2024-09-10T20:39:58.263403+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277571+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:39:58.277571+0000","last_peered":"2024-09-10T20:39:58.277571+0000","last_clean":"2024-09-10T20:39:58.277571+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:39:58.277571+0000","last_undegraded":"2024-09-10T20:39:58.277571+0000","last_fullsized":"2024-09-10T20:39:58.277571+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21257,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.263453+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:58.263453+0000","last_peered":"2024-09-10T20:39:58.263453+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:58.263453+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":92,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263317+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:39:58.263317+0000","last_peered":"2024-09-10T20:39:58.263317+0000","last_clean":"2024-09-10T20:39:58.263317+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:39:58.263317+0000","last_undegraded":"2024-09-10T20:39:58.263317+0000","last_fullsized":"2024-09-10T20:39:58.263317+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263276+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:39:58.263276+0000","last_peered":"2024-09-10T20:39:58.263276+0000","last_clean":"2024-09-10T20:39:58.263276+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:39:58.263276+0000","last_undegraded":"2024-09-10T20:39:58.263276+0000","last_fullsized":"2024-09-10T20:39:58.263276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27656,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263068+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:58.263068+0000","last_peered":"2024-09-10T20:39:58.263068+0000","last_clean":"2024-09-10T20:39:58.263068+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:58.263068+0000","last_undegraded":"2024-09-10T20:39:58.263068+0000","last_fullsized":"2024-09-10T20:39:58.263068+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18140,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25009,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176676864,"data_stored":176637121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182539,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517164,"kb_used_data":57940,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854676,"statfs":{"total":96636764160,"available":96107188224,"internally_reserved":0,"allocated":59330560,"data_stored":59039347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986312,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480344,"kb_used_data":115460,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93891496,"statfs":{"total":96636764160,"available":96144891904,"internally_reserved":0,"allocated":118231040,"data_stored":117943904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659258,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491704,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880136,"statfs":{"total":96636764160,"available":96133259264,"internally_reserved":0,"allocated":499712,"data_stored":179595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502984366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117772288,"data_stored":117768675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:40:15.757 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:40:16.082 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:16.083 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:40:16.094 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":412,"stamp":"2024-09-10T20:40:15.065257+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18140,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25011,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1489212,"kb_used_data":173888,"kb_used_omap":12,"kb_used_meta":1315251,"kb_avail":281626308,"statfs":{"total":289910292480,"available":288385339392,"internally_reserved":0,"allocated":178061312,"data_stored":177162846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1346817026},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2819,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2819,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002506"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20893,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.263246+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:58.263246+0000","last_peered":"2024-09-10T20:39:58.263246+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:58.263246+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21075,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277759+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:58.277759+0000","last_peered":"2024-09-10T20:39:58.277759+0000","last_clean":"2024-09-10T20:39:58.277759+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:58.277759+0000","last_undegraded":"2024-09-10T20:39:58.277759+0000","last_fullsized":"2024-09-10T20:39:58.277759+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28087,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263157+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:58.263157+0000","last_peered":"2024-09-10T20:39:58.263157+0000","last_clean":"2024-09-10T20:39:58.263157+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:58.263157+0000","last_undegraded":"2024-09-10T20:39:58.263157+0000","last_fullsized":"2024-09-10T20:39:58.263157+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20707,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277394+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:58.277394+0000","last_peered":"2024-09-10T20:39:58.277394+0000","last_clean":"2024-09-10T20:39:58.277394+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:58.277394+0000","last_undegraded":"2024-09-10T20:39:58.277394+0000","last_fullsized":"2024-09-10T20:39:58.277394+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21219,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.265142+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:58.265142+0000","last_peered":"2024-09-10T20:39:58.265142+0000","last_clean":"2024-09-10T20:39:58.265142+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:58.265142+0000","last_undegraded":"2024-09-10T20:39:58.265142+0000","last_fullsized":"2024-09-10T20:39:58.265142+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20737,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.265623+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:58.265623+0000","last_peered":"2024-09-10T20:39:58.265623+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:58.265623+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20907,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.263985+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:58.263985+0000","last_peered":"2024-09-10T20:39:58.263985+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:58.263985+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20884,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.262813+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:58.262813+0000","last_peered":"2024-09-10T20:39:58.262813+0000","last_clean":"2024-09-10T20:39:58.262813+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:58.262813+0000","last_undegraded":"2024-09-10T20:39:58.262813+0000","last_fullsized":"2024-09-10T20:39:58.262813+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263001+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:39:58.263001+0000","last_peered":"2024-09-10T20:39:58.263001+0000","last_clean":"2024-09-10T20:39:58.263001+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:39:58.263001+0000","last_undegraded":"2024-09-10T20:39:58.263001+0000","last_fullsized":"2024-09-10T20:39:58.263001+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22126,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.262973+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:39:58.262973+0000","last_peered":"2024-09-10T20:39:58.262973+0000","last_clean":"2024-09-10T20:39:58.262973+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:39:58.262973+0000","last_undegraded":"2024-09-10T20:39:58.262973+0000","last_fullsized":"2024-09-10T20:39:58.262973+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277094+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:39:58.277094+0000","last_peered":"2024-09-10T20:39:58.277094+0000","last_clean":"2024-09-10T20:39:58.277094+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:39:58.277094+0000","last_undegraded":"2024-09-10T20:39:58.277094+0000","last_fullsized":"2024-09-10T20:39:58.277094+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":20960,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.265211+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:58.265211+0000","last_peered":"2024-09-10T20:39:58.265211+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:58.265211+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3564}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.264754+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:39:58.264754+0000","last_peered":"2024-09-10T20:39:58.264754+0000","last_clean":"2024-09-10T20:39:58.264754+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:39:58.264754+0000","last_undegraded":"2024-09-10T20:39:58.264754+0000","last_fullsized":"2024-09-10T20:39:58.264754+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":27342,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:14.795462+0000","last_change":"2024-09-10T20:39:56.460609+0000","last_active":"2024-09-10T20:40:14.795462+0000","last_peered":"2024-09-10T20:40:14.795462+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:40:14.795462+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3312,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-323}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":102,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.262885+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:39:58.262885+0000","last_peered":"2024-09-10T20:39:58.262885+0000","last_clean":"2024-09-10T20:39:58.262885+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:39:58.262885+0000","last_undegraded":"2024-09-10T20:39:58.262885+0000","last_fullsized":"2024-09-10T20:39:58.262885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28048,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.276958+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:58.276958+0000","last_peered":"2024-09-10T20:39:58.276958+0000","last_clean":"2024-09-10T20:39:58.276958+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:58.276958+0000","last_undegraded":"2024-09-10T20:39:58.276958+0000","last_fullsized":"2024-09-10T20:39:58.276958+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21105,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.338466+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:39:58.338466+0000","last_peered":"2024-09-10T20:39:58.338466+0000","last_clean":"2024-09-10T20:39:58.338466+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:39:58.338466+0000","last_undegraded":"2024-09-10T20:39:58.338466+0000","last_fullsized":"2024-09-10T20:39:58.338466+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263152+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:39:58.263152+0000","last_peered":"2024-09-10T20:39:58.263152+0000","last_clean":"2024-09-10T20:39:58.263152+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:39:58.263152+0000","last_undegraded":"2024-09-10T20:39:58.263152+0000","last_fullsized":"2024-09-10T20:39:58.263152+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27826,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263403+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:58.263403+0000","last_peered":"2024-09-10T20:39:58.263403+0000","last_clean":"2024-09-10T20:39:58.263403+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:58.263403+0000","last_undegraded":"2024-09-10T20:39:58.263403+0000","last_fullsized":"2024-09-10T20:39:58.263403+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277571+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:39:58.277571+0000","last_peered":"2024-09-10T20:39:58.277571+0000","last_clean":"2024-09-10T20:39:58.277571+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:39:58.277571+0000","last_undegraded":"2024-09-10T20:39:58.277571+0000","last_fullsized":"2024-09-10T20:39:58.277571+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21257,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.263453+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:58.263453+0000","last_peered":"2024-09-10T20:39:58.263453+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:58.263453+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":92,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263317+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:39:58.263317+0000","last_peered":"2024-09-10T20:39:58.263317+0000","last_clean":"2024-09-10T20:39:58.263317+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:39:58.263317+0000","last_undegraded":"2024-09-10T20:39:58.263317+0000","last_fullsized":"2024-09-10T20:39:58.263317+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263276+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:39:58.263276+0000","last_peered":"2024-09-10T20:39:58.263276+0000","last_clean":"2024-09-10T20:39:58.263276+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:39:58.263276+0000","last_undegraded":"2024-09-10T20:39:58.263276+0000","last_fullsized":"2024-09-10T20:39:58.263276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27656,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263068+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:58.263068+0000","last_peered":"2024-09-10T20:39:58.263068+0000","last_clean":"2024-09-10T20:39:58.263068+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:58.263068+0000","last_undegraded":"2024-09-10T20:39:58.263068+0000","last_fullsized":"2024-09-10T20:39:58.263068+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18140,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25009,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176676864,"data_stored":176637121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182539,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517164,"kb_used_data":57940,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854676,"statfs":{"total":96636764160,"available":96107188224,"internally_reserved":0,"allocated":59330560,"data_stored":59039347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986312,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480344,"kb_used_data":115460,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93891496,"statfs":{"total":96636764160,"available":96144891904,"internally_reserved":0,"allocated":118231040,"data_stored":117943904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659258,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491704,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880136,"statfs":{"total":96636764160,"available":96133259264,"internally_reserved":0,"allocated":499712,"data_stored":179595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502984366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117772288,"data_stored":117768675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:40:16.096 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:40:16.096 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'20658", 'reported_seq': 20893, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:58.263246+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:39:58.263246+0000', 'last_peered': '2024-09-10T20:39:58.263246+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:39:58.263246+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20658, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3541}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:16.097 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:40:16.097 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20471", 'reported_seq': 20737, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:58.265623+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:39:58.265623+0000', 'last_peered': '2024-09-10T20:39:58.265623+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:39:58.265623+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17335, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3561}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:16.097 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:40:16.097 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'20608", 'reported_seq': 20907, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:58.263985+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:39:58.263985+0000', 'last_peered': '2024-09-10T20:39:58.263985+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:39:58.263985+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17203, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3486}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:16.097 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:40:16.097 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20730", 'reported_seq': 20960, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:58.265211+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:39:58.265211+0000', 'last_peered': '2024-09-10T20:39:58.265211+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:39:58.265211+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20728, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3564}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:16.097 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:40:16.097 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "51'20565", 'reported_seq': 27342, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:14.795462+0000', 'last_change': '2024-09-10T20:39:56.460609+0000', 'last_active': '2024-09-10T20:40:14.795462+0000', 'last_peered': '2024-09-10T20:40:14.795462+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:40:14.795462+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18393, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3312, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -323}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:40:16.098 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:40:16.098 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "51'20986", 'reported_seq': 21257, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:58.263453+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:39:58.263453+0000', 'last_peered': '2024-09-10T20:39:58.263453+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:39:58.263453+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17813, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3655}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:16.098 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:40:16.487 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:16.487 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 18140/10 objects degraded (181400.000%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":728,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":56,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":5,"data_bytes":1468,"bytes_used":1524953088,"bytes_avail":288385339392,"bytes_total":289910292480,"degraded_objects":18140,"degraded_total":10,"degraded_ratio":1814,"recovering_objects_per_sec":234,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2819,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:40:16.502 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:40:16.502 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:40:16.829 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:16.829 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:40:16.843 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":413,"stamp":"2024-09-10T20:40:16.394685+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18140,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25011,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1489212,"kb_used_data":173888,"kb_used_omap":12,"kb_used_meta":1315251,"kb_avail":281626308,"statfs":{"total":289910292480,"available":288385339392,"internally_reserved":0,"allocated":178061312,"data_stored":177162846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1346817026},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1856,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1856,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.331568"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20894,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.264175+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:58.264175+0000","last_peered":"2024-09-10T20:39:58.264175+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:58.264175+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21075,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277759+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:58.277759+0000","last_peered":"2024-09-10T20:39:58.277759+0000","last_clean":"2024-09-10T20:39:58.277759+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:58.277759+0000","last_undegraded":"2024-09-10T20:39:58.277759+0000","last_fullsized":"2024-09-10T20:39:58.277759+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28087,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263157+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:39:58.263157+0000","last_peered":"2024-09-10T20:39:58.263157+0000","last_clean":"2024-09-10T20:39:58.263157+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:39:58.263157+0000","last_undegraded":"2024-09-10T20:39:58.263157+0000","last_fullsized":"2024-09-10T20:39:58.263157+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20707,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277394+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:58.277394+0000","last_peered":"2024-09-10T20:39:58.277394+0000","last_clean":"2024-09-10T20:39:58.277394+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:58.277394+0000","last_undegraded":"2024-09-10T20:39:58.277394+0000","last_fullsized":"2024-09-10T20:39:58.277394+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21219,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.265142+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:58.265142+0000","last_peered":"2024-09-10T20:39:58.265142+0000","last_clean":"2024-09-10T20:39:58.265142+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:58.265142+0000","last_undegraded":"2024-09-10T20:39:58.265142+0000","last_fullsized":"2024-09-10T20:39:58.265142+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20737,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.265623+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:58.265623+0000","last_peered":"2024-09-10T20:39:58.265623+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:58.265623+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20907,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.263985+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:58.263985+0000","last_peered":"2024-09-10T20:39:58.263985+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:58.263985+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20884,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.262813+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:58.262813+0000","last_peered":"2024-09-10T20:39:58.262813+0000","last_clean":"2024-09-10T20:39:58.262813+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:58.262813+0000","last_undegraded":"2024-09-10T20:39:58.262813+0000","last_fullsized":"2024-09-10T20:39:58.262813+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263001+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:39:58.263001+0000","last_peered":"2024-09-10T20:39:58.263001+0000","last_clean":"2024-09-10T20:39:58.263001+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:39:58.263001+0000","last_undegraded":"2024-09-10T20:39:58.263001+0000","last_fullsized":"2024-09-10T20:39:58.263001+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22126,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.262973+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:39:58.262973+0000","last_peered":"2024-09-10T20:39:58.262973+0000","last_clean":"2024-09-10T20:39:58.262973+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:39:58.262973+0000","last_undegraded":"2024-09-10T20:39:58.262973+0000","last_fullsized":"2024-09-10T20:39:58.262973+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277094+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:39:58.277094+0000","last_peered":"2024-09-10T20:39:58.277094+0000","last_clean":"2024-09-10T20:39:58.277094+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:39:58.277094+0000","last_undegraded":"2024-09-10T20:39:58.277094+0000","last_fullsized":"2024-09-10T20:39:58.277094+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":20960,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.265211+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:58.265211+0000","last_peered":"2024-09-10T20:39:58.265211+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:58.265211+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3564}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.264754+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:39:58.264754+0000","last_peered":"2024-09-10T20:39:58.264754+0000","last_clean":"2024-09-10T20:39:58.264754+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:39:58.264754+0000","last_undegraded":"2024-09-10T20:39:58.264754+0000","last_fullsized":"2024-09-10T20:39:58.264754+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":27342,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:14.795462+0000","last_change":"2024-09-10T20:39:56.460609+0000","last_active":"2024-09-10T20:40:14.795462+0000","last_peered":"2024-09-10T20:40:14.795462+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:40:14.795462+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3312,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-323}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":102,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.262885+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:39:58.262885+0000","last_peered":"2024-09-10T20:39:58.262885+0000","last_clean":"2024-09-10T20:39:58.262885+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:39:58.262885+0000","last_undegraded":"2024-09-10T20:39:58.262885+0000","last_fullsized":"2024-09-10T20:39:58.262885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28048,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.276958+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:58.276958+0000","last_peered":"2024-09-10T20:39:58.276958+0000","last_clean":"2024-09-10T20:39:58.276958+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:58.276958+0000","last_undegraded":"2024-09-10T20:39:58.276958+0000","last_fullsized":"2024-09-10T20:39:58.276958+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21105,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.338466+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:39:58.338466+0000","last_peered":"2024-09-10T20:39:58.338466+0000","last_clean":"2024-09-10T20:39:58.338466+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:39:58.338466+0000","last_undegraded":"2024-09-10T20:39:58.338466+0000","last_fullsized":"2024-09-10T20:39:58.338466+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263152+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:39:58.263152+0000","last_peered":"2024-09-10T20:39:58.263152+0000","last_clean":"2024-09-10T20:39:58.263152+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:39:58.263152+0000","last_undegraded":"2024-09-10T20:39:58.263152+0000","last_fullsized":"2024-09-10T20:39:58.263152+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27826,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263403+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:39:58.263403+0000","last_peered":"2024-09-10T20:39:58.263403+0000","last_clean":"2024-09-10T20:39:58.263403+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:39:58.263403+0000","last_undegraded":"2024-09-10T20:39:58.263403+0000","last_fullsized":"2024-09-10T20:39:58.263403+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277571+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:39:58.277571+0000","last_peered":"2024-09-10T20:39:58.277571+0000","last_clean":"2024-09-10T20:39:58.277571+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:39:58.277571+0000","last_undegraded":"2024-09-10T20:39:58.277571+0000","last_fullsized":"2024-09-10T20:39:58.277571+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21257,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.263453+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:58.263453+0000","last_peered":"2024-09-10T20:39:58.263453+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:58.263453+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":92,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263317+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:39:58.263317+0000","last_peered":"2024-09-10T20:39:58.263317+0000","last_clean":"2024-09-10T20:39:58.263317+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:39:58.263317+0000","last_undegraded":"2024-09-10T20:39:58.263317+0000","last_fullsized":"2024-09-10T20:39:58.263317+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263276+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:39:58.263276+0000","last_peered":"2024-09-10T20:39:58.263276+0000","last_clean":"2024-09-10T20:39:58.263276+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:39:58.263276+0000","last_undegraded":"2024-09-10T20:39:58.263276+0000","last_fullsized":"2024-09-10T20:39:58.263276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27656,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.263068+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:39:58.263068+0000","last_peered":"2024-09-10T20:39:58.263068+0000","last_clean":"2024-09-10T20:39:58.263068+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:39:58.263068+0000","last_undegraded":"2024-09-10T20:39:58.263068+0000","last_fullsized":"2024-09-10T20:39:58.263068+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18140,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25009,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176676864,"data_stored":176637121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182540,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517164,"kb_used_data":57940,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854676,"statfs":{"total":96636764160,"available":96107188224,"internally_reserved":0,"allocated":59330560,"data_stored":59039347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986312,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480344,"kb_used_data":115460,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93891496,"statfs":{"total":96636764160,"available":96144891904,"internally_reserved":0,"allocated":118231040,"data_stored":117943904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659258,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491704,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880136,"statfs":{"total":96636764160,"available":96133259264,"internally_reserved":0,"allocated":499712,"data_stored":179595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502984366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117772288,"data_stored":117768675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:40:19.845 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:40:20.147 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:20.148 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:40:20.160 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":416,"stamp":"2024-09-10T20:40:19.065935+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18140,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25011,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1480896,"kb_used_data":173888,"kb_used_omap":12,"kb_used_meta":1306931,"kb_avail":281634624,"statfs":{"total":289910292480,"available":288393854976,"internally_reserved":0,"allocated":178061312,"data_stored":177165155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1338297346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1046,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1046,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001614"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20894,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.264175+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:58.264175+0000","last_peered":"2024-09-10T20:39:58.264175+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:58.264175+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21075,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277759+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:58.277759+0000","last_peered":"2024-09-10T20:39:58.277759+0000","last_clean":"2024-09-10T20:39:58.277759+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:58.277759+0000","last_undegraded":"2024-09-10T20:39:58.277759+0000","last_fullsized":"2024-09-10T20:39:58.277759+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28089,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T20:40:16.406244+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:40:16.406244+0000","last_peered":"2024-09-10T20:40:16.406244+0000","last_clean":"2024-09-10T20:40:16.406244+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:40:16.406244+0000","last_undegraded":"2024-09-10T20:40:16.406244+0000","last_fullsized":"2024-09-10T20:40:16.406244+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20707,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277394+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:58.277394+0000","last_peered":"2024-09-10T20:39:58.277394+0000","last_clean":"2024-09-10T20:39:58.277394+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:58.277394+0000","last_undegraded":"2024-09-10T20:39:58.277394+0000","last_fullsized":"2024-09-10T20:39:58.277394+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21219,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.265142+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:58.265142+0000","last_peered":"2024-09-10T20:39:58.265142+0000","last_clean":"2024-09-10T20:39:58.265142+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:58.265142+0000","last_undegraded":"2024-09-10T20:39:58.265142+0000","last_fullsized":"2024-09-10T20:39:58.265142+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20737,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.265623+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:58.265623+0000","last_peered":"2024-09-10T20:39:58.265623+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:58.265623+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20907,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.263985+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:58.263985+0000","last_peered":"2024-09-10T20:39:58.263985+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:58.263985+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20884,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.262813+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:58.262813+0000","last_peered":"2024-09-10T20:39:58.262813+0000","last_clean":"2024-09-10T20:39:58.262813+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:58.262813+0000","last_undegraded":"2024-09-10T20:39:58.262813+0000","last_fullsized":"2024-09-10T20:39:58.262813+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":95,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T20:40:16.405950+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:40:16.405950+0000","last_peered":"2024-09-10T20:40:16.405950+0000","last_clean":"2024-09-10T20:40:16.405950+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:40:16.405950+0000","last_undegraded":"2024-09-10T20:40:16.405950+0000","last_fullsized":"2024-09-10T20:40:16.405950+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22128,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T20:40:16.406125+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:40:16.406125+0000","last_peered":"2024-09-10T20:40:16.406125+0000","last_clean":"2024-09-10T20:40:16.406125+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:40:16.406125+0000","last_undegraded":"2024-09-10T20:40:16.406125+0000","last_fullsized":"2024-09-10T20:40:16.406125+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277094+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:39:58.277094+0000","last_peered":"2024-09-10T20:39:58.277094+0000","last_clean":"2024-09-10T20:39:58.277094+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:39:58.277094+0000","last_undegraded":"2024-09-10T20:39:58.277094+0000","last_fullsized":"2024-09-10T20:39:58.277094+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":20960,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.265211+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:58.265211+0000","last_peered":"2024-09-10T20:39:58.265211+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:58.265211+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3564}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.264754+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:39:58.264754+0000","last_peered":"2024-09-10T20:39:58.264754+0000","last_clean":"2024-09-10T20:39:58.264754+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:39:58.264754+0000","last_undegraded":"2024-09-10T20:39:58.264754+0000","last_fullsized":"2024-09-10T20:39:58.264754+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":27342,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:14.795462+0000","last_change":"2024-09-10T20:39:56.460609+0000","last_active":"2024-09-10T20:40:14.795462+0000","last_peered":"2024-09-10T20:40:14.795462+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:40:14.795462+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3312,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-323}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":104,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T20:40:16.406149+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:40:16.406149+0000","last_peered":"2024-09-10T20:40:16.406149+0000","last_clean":"2024-09-10T20:40:16.406149+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:40:16.406149+0000","last_undegraded":"2024-09-10T20:40:16.406149+0000","last_fullsized":"2024-09-10T20:40:16.406149+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28048,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.276958+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:58.276958+0000","last_peered":"2024-09-10T20:39:58.276958+0000","last_clean":"2024-09-10T20:39:58.276958+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:58.276958+0000","last_undegraded":"2024-09-10T20:39:58.276958+0000","last_fullsized":"2024-09-10T20:39:58.276958+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21107,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T20:40:16.406467+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:40:16.406467+0000","last_peered":"2024-09-10T20:40:16.406467+0000","last_clean":"2024-09-10T20:40:16.406467+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:40:16.406467+0000","last_undegraded":"2024-09-10T20:40:16.406467+0000","last_fullsized":"2024-09-10T20:40:16.406467+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":94,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T20:40:16.406628+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:40:16.406628+0000","last_peered":"2024-09-10T20:40:16.406628+0000","last_clean":"2024-09-10T20:40:16.406628+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:40:16.406628+0000","last_undegraded":"2024-09-10T20:40:16.406628+0000","last_fullsized":"2024-09-10T20:40:16.406628+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27828,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T20:40:16.406384+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:40:16.406384+0000","last_peered":"2024-09-10T20:40:16.406384+0000","last_clean":"2024-09-10T20:40:16.406384+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:40:16.406384+0000","last_undegraded":"2024-09-10T20:40:16.406384+0000","last_fullsized":"2024-09-10T20:40:16.406384+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277571+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:39:58.277571+0000","last_peered":"2024-09-10T20:39:58.277571+0000","last_clean":"2024-09-10T20:39:58.277571+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:39:58.277571+0000","last_undegraded":"2024-09-10T20:39:58.277571+0000","last_fullsized":"2024-09-10T20:39:58.277571+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21257,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.263453+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:58.263453+0000","last_peered":"2024-09-10T20:39:58.263453+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:58.263453+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":94,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T20:40:16.406377+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:40:16.406377+0000","last_peered":"2024-09-10T20:40:16.406377+0000","last_clean":"2024-09-10T20:40:16.406377+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:40:16.406377+0000","last_undegraded":"2024-09-10T20:40:16.406377+0000","last_fullsized":"2024-09-10T20:40:16.406377+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":99,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T20:40:16.406379+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:40:16.406379+0000","last_peered":"2024-09-10T20:40:16.406379+0000","last_clean":"2024-09-10T20:40:16.406379+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:40:16.406379+0000","last_undegraded":"2024-09-10T20:40:16.406379+0000","last_fullsized":"2024-09-10T20:40:16.406379+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27658,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T20:40:16.406145+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:40:16.406145+0000","last_peered":"2024-09-10T20:40:16.406145+0000","last_clean":"2024-09-10T20:40:16.406145+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:40:16.406145+0000","last_undegraded":"2024-09-10T20:40:16.406145+0000","last_fullsized":"2024-09-10T20:40:16.406145+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18140,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25009,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176668672,"data_stored":176636155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182540,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517164,"kb_used_data":57940,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854676,"statfs":{"total":96636764160,"available":96107188224,"internally_reserved":0,"allocated":59330560,"data_stored":59039347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986312,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480344,"kb_used_data":115460,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93891496,"statfs":{"total":96636764160,"available":96144891904,"internally_reserved":0,"allocated":118231040,"data_stored":117943904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659259,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483388,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888452,"statfs":{"total":96636764160,"available":96141774848,"internally_reserved":0,"allocated":499712,"data_stored":181904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117772288,"data_stored":117768675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:40:20.162 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:40:20.489 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:20.490 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:40:20.503 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":416,"stamp":"2024-09-10T20:40:19.065935+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18140,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25011,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1480896,"kb_used_data":173888,"kb_used_omap":12,"kb_used_meta":1306931,"kb_avail":281634624,"statfs":{"total":289910292480,"available":288393854976,"internally_reserved":0,"allocated":178061312,"data_stored":177165155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1338297346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1046,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1046,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001614"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20894,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.264175+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:39:58.264175+0000","last_peered":"2024-09-10T20:39:58.264175+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:39:58.264175+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21075,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277759+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:39:58.277759+0000","last_peered":"2024-09-10T20:39:58.277759+0000","last_clean":"2024-09-10T20:39:58.277759+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:39:58.277759+0000","last_undegraded":"2024-09-10T20:39:58.277759+0000","last_fullsized":"2024-09-10T20:39:58.277759+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28089,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T20:40:16.406244+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:40:16.406244+0000","last_peered":"2024-09-10T20:40:16.406244+0000","last_clean":"2024-09-10T20:40:16.406244+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:40:16.406244+0000","last_undegraded":"2024-09-10T20:40:16.406244+0000","last_fullsized":"2024-09-10T20:40:16.406244+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20707,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277394+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:39:58.277394+0000","last_peered":"2024-09-10T20:39:58.277394+0000","last_clean":"2024-09-10T20:39:58.277394+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:39:58.277394+0000","last_undegraded":"2024-09-10T20:39:58.277394+0000","last_fullsized":"2024-09-10T20:39:58.277394+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21219,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.265142+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:39:58.265142+0000","last_peered":"2024-09-10T20:39:58.265142+0000","last_clean":"2024-09-10T20:39:58.265142+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:39:58.265142+0000","last_undegraded":"2024-09-10T20:39:58.265142+0000","last_fullsized":"2024-09-10T20:39:58.265142+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20737,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.265623+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:39:58.265623+0000","last_peered":"2024-09-10T20:39:58.265623+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:39:58.265623+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20907,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.263985+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:39:58.263985+0000","last_peered":"2024-09-10T20:39:58.263985+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:39:58.263985+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20884,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.262813+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:39:58.262813+0000","last_peered":"2024-09-10T20:39:58.262813+0000","last_clean":"2024-09-10T20:39:58.262813+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:39:58.262813+0000","last_undegraded":"2024-09-10T20:39:58.262813+0000","last_fullsized":"2024-09-10T20:39:58.262813+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":95,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T20:40:16.405950+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:40:16.405950+0000","last_peered":"2024-09-10T20:40:16.405950+0000","last_clean":"2024-09-10T20:40:16.405950+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:40:16.405950+0000","last_undegraded":"2024-09-10T20:40:16.405950+0000","last_fullsized":"2024-09-10T20:40:16.405950+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22128,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T20:40:16.406125+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:40:16.406125+0000","last_peered":"2024-09-10T20:40:16.406125+0000","last_clean":"2024-09-10T20:40:16.406125+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:40:16.406125+0000","last_undegraded":"2024-09-10T20:40:16.406125+0000","last_fullsized":"2024-09-10T20:40:16.406125+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277094+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:39:58.277094+0000","last_peered":"2024-09-10T20:39:58.277094+0000","last_clean":"2024-09-10T20:39:58.277094+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:39:58.277094+0000","last_undegraded":"2024-09-10T20:39:58.277094+0000","last_fullsized":"2024-09-10T20:39:58.277094+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":20960,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.265211+0000","last_change":"2024-09-10T20:30:05.765661+0000","last_active":"2024-09-10T20:39:58.265211+0000","last_peered":"2024-09-10T20:39:58.265211+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:39:58.265211+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3564}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.264754+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:39:58.264754+0000","last_peered":"2024-09-10T20:39:58.264754+0000","last_clean":"2024-09-10T20:39:58.264754+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:39:58.264754+0000","last_undegraded":"2024-09-10T20:39:58.264754+0000","last_fullsized":"2024-09-10T20:39:58.264754+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":27342,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:14.795462+0000","last_change":"2024-09-10T20:39:56.460609+0000","last_active":"2024-09-10T20:40:14.795462+0000","last_peered":"2024-09-10T20:40:14.795462+0000","last_clean":"2024-09-10T20:30:03.304792+0000","last_became_active":"2024-09-10T20:30:05.781091+0000","last_became_peered":"2024-09-10T20:30:05.781091+0000","last_unstale":"2024-09-10T20:40:14.795462+0000","last_undegraded":"2024-09-10T20:30:05.352989+0000","last_fullsized":"2024-09-10T20:30:05.312021+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3312,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-323}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":104,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T20:40:16.406149+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:40:16.406149+0000","last_peered":"2024-09-10T20:40:16.406149+0000","last_clean":"2024-09-10T20:40:16.406149+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:40:16.406149+0000","last_undegraded":"2024-09-10T20:40:16.406149+0000","last_fullsized":"2024-09-10T20:40:16.406149+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28048,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.276958+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:39:58.276958+0000","last_peered":"2024-09-10T20:39:58.276958+0000","last_clean":"2024-09-10T20:39:58.276958+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:39:58.276958+0000","last_undegraded":"2024-09-10T20:39:58.276958+0000","last_fullsized":"2024-09-10T20:39:58.276958+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21107,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T20:40:16.406467+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:40:16.406467+0000","last_peered":"2024-09-10T20:40:16.406467+0000","last_clean":"2024-09-10T20:40:16.406467+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:40:16.406467+0000","last_undegraded":"2024-09-10T20:40:16.406467+0000","last_fullsized":"2024-09-10T20:40:16.406467+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":94,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T20:40:16.406628+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:40:16.406628+0000","last_peered":"2024-09-10T20:40:16.406628+0000","last_clean":"2024-09-10T20:40:16.406628+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:40:16.406628+0000","last_undegraded":"2024-09-10T20:40:16.406628+0000","last_fullsized":"2024-09-10T20:40:16.406628+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27828,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T20:40:16.406384+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:40:16.406384+0000","last_peered":"2024-09-10T20:40:16.406384+0000","last_clean":"2024-09-10T20:40:16.406384+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:40:16.406384+0000","last_undegraded":"2024-09-10T20:40:16.406384+0000","last_fullsized":"2024-09-10T20:40:16.406384+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T20:39:58.277571+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:39:58.277571+0000","last_peered":"2024-09-10T20:39:58.277571+0000","last_clean":"2024-09-10T20:39:58.277571+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:39:58.277571+0000","last_undegraded":"2024-09-10T20:39:58.277571+0000","last_fullsized":"2024-09-10T20:39:58.277571+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21257,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:39:58.263453+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:39:58.263453+0000","last_peered":"2024-09-10T20:39:58.263453+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:39:58.263453+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":94,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T20:40:16.406377+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:40:16.406377+0000","last_peered":"2024-09-10T20:40:16.406377+0000","last_clean":"2024-09-10T20:40:16.406377+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:40:16.406377+0000","last_undegraded":"2024-09-10T20:40:16.406377+0000","last_fullsized":"2024-09-10T20:40:16.406377+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":99,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T20:40:16.406379+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:40:16.406379+0000","last_peered":"2024-09-10T20:40:16.406379+0000","last_clean":"2024-09-10T20:40:16.406379+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:40:16.406379+0000","last_undegraded":"2024-09-10T20:40:16.406379+0000","last_fullsized":"2024-09-10T20:40:16.406379+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27658,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T20:40:16.406145+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:40:16.406145+0000","last_peered":"2024-09-10T20:40:16.406145+0000","last_clean":"2024-09-10T20:40:16.406145+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:40:16.406145+0000","last_undegraded":"2024-09-10T20:40:16.406145+0000","last_fullsized":"2024-09-10T20:40:16.406145+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18140,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25009,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176668672,"data_stored":176636155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182540,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517164,"kb_used_data":57940,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854676,"statfs":{"total":96636764160,"available":96107188224,"internally_reserved":0,"allocated":59330560,"data_stored":59039347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986312,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480344,"kb_used_data":115460,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93891496,"statfs":{"total":96636764160,"available":96144891904,"internally_reserved":0,"allocated":118231040,"data_stored":117943904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659259,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483388,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888452,"statfs":{"total":96636764160,"available":96141774848,"internally_reserved":0,"allocated":499712,"data_stored":181904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117772288,"data_stored":117768675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:40:20.504 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:40:20.504 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'20658", 'reported_seq': 20894, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:58.264175+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:39:58.264175+0000', 'last_peered': '2024-09-10T20:39:58.264175+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:39:58.264175+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20658, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3541}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:20.504 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:40:20.504 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20471", 'reported_seq': 20737, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:58.265623+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:39:58.265623+0000', 'last_peered': '2024-09-10T20:39:58.265623+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:39:58.265623+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17335, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3561}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:20.504 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:40:20.504 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'20608", 'reported_seq': 20907, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:58.263985+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:39:58.263985+0000', 'last_peered': '2024-09-10T20:39:58.263985+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:39:58.263985+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17203, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3486}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:20.505 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:40:20.505 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20730", 'reported_seq': 20960, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:58.265211+0000', 'last_change': '2024-09-10T20:30:05.765661+0000', 'last_active': '2024-09-10T20:39:58.265211+0000', 'last_peered': '2024-09-10T20:39:58.265211+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:39:58.265211+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20728, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3564}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:20.505 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T20:40:20.505 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "51'20565", 'reported_seq': 27342, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:14.795462+0000', 'last_change': '2024-09-10T20:39:56.460609+0000', 'last_active': '2024-09-10T20:40:14.795462+0000', 'last_peered': '2024-09-10T20:40:14.795462+0000', 'last_clean': '2024-09-10T20:30:03.304792+0000', 'last_became_active': '2024-09-10T20:30:05.781091+0000', 'last_became_peered': '2024-09-10T20:30:05.781091+0000', 'last_unstale': '2024-09-10T20:40:14.795462+0000', 'last_undegraded': '2024-09-10T20:30:05.352989+0000', 'last_fullsized': '2024-09-10T20:30:05.312021+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2197", 'last_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:44.304225+0000', 'objects_scrubbed': 2167, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 689, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18393, 'num_write_kb': 32452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3312, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -323}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T20:40:20.505 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:40:20.505 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "51'20986", 'reported_seq': 21257, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:39:58.263453+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:39:58.263453+0000', 'last_peered': '2024-09-10T20:39:58.263453+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:39:58.263453+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17813, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3655}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:20.505 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:40:20.931 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:20.931 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 18140/10 objects degraded (181400.000%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":732,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":5,"data_bytes":1468,"bytes_used":1516437504,"bytes_avail":288393854976,"bytes_total":289910292480,"degraded_objects":18140,"degraded_total":10,"degraded_ratio":1814,"recovering_objects_per_sec":130,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1046,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:40:20.946 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:40:20.946 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:40:21.248 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:21.248 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:40:21.262 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":417,"stamp":"2024-09-10T20:40:21.066234+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25892,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1479864,"kb_used_data":172852,"kb_used_omap":12,"kb_used_meta":1306931,"kb_avail":281635656,"statfs":{"total":289910292480,"available":288394911744,"internally_reserved":0,"allocated":177000448,"data_stored":176084581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1338297346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1927,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1927,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001428"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20898,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.416306+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:40:17.416306+0000","last_peered":"2024-09-10T20:40:17.416306+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:40:17.416306+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21079,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418190+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:40:17.418190+0000","last_peered":"2024-09-10T20:40:17.418190+0000","last_clean":"2024-09-10T20:40:17.418190+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:40:17.418190+0000","last_undegraded":"2024-09-10T20:40:17.418190+0000","last_fullsized":"2024-09-10T20:40:17.418190+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28089,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T20:40:16.406244+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:40:16.406244+0000","last_peered":"2024-09-10T20:40:16.406244+0000","last_clean":"2024-09-10T20:40:16.406244+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:40:16.406244+0000","last_undegraded":"2024-09-10T20:40:16.406244+0000","last_fullsized":"2024-09-10T20:40:16.406244+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20711,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418346+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:40:17.418346+0000","last_peered":"2024-09-10T20:40:17.418346+0000","last_clean":"2024-09-10T20:40:17.418346+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:40:17.418346+0000","last_undegraded":"2024-09-10T20:40:17.418346+0000","last_fullsized":"2024-09-10T20:40:17.418346+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21223,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.414948+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:40:17.414948+0000","last_peered":"2024-09-10T20:40:17.414948+0000","last_clean":"2024-09-10T20:40:17.414948+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:40:17.414948+0000","last_undegraded":"2024-09-10T20:40:17.414948+0000","last_fullsized":"2024-09-10T20:40:17.414948+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20741,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.414623+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:40:17.414623+0000","last_peered":"2024-09-10T20:40:17.414623+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:40:17.414623+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20911,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.414726+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:40:17.414726+0000","last_peered":"2024-09-10T20:40:17.414726+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:40:17.414726+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20888,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.414444+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:40:17.414444+0000","last_peered":"2024-09-10T20:40:17.414444+0000","last_clean":"2024-09-10T20:40:17.414444+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:40:17.414444+0000","last_undegraded":"2024-09-10T20:40:17.414444+0000","last_fullsized":"2024-09-10T20:40:17.414444+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":95,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T20:40:16.405950+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:40:16.405950+0000","last_peered":"2024-09-10T20:40:16.405950+0000","last_clean":"2024-09-10T20:40:16.405950+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:40:16.405950+0000","last_undegraded":"2024-09-10T20:40:16.405950+0000","last_fullsized":"2024-09-10T20:40:16.405950+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22128,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T20:40:16.406125+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:40:16.406125+0000","last_peered":"2024-09-10T20:40:16.406125+0000","last_clean":"2024-09-10T20:40:16.406125+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:40:16.406125+0000","last_undegraded":"2024-09-10T20:40:16.406125+0000","last_fullsized":"2024-09-10T20:40:16.406125+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418023+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:40:17.418023+0000","last_peered":"2024-09-10T20:40:17.418023+0000","last_clean":"2024-09-10T20:40:17.418023+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:40:17.418023+0000","last_undegraded":"2024-09-10T20:40:17.418023+0000","last_fullsized":"2024-09-10T20:40:17.418023+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":22081,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:20.219691+0000","last_change":"2024-09-10T20:40:16.115711+0000","last_active":"2024-09-10T20:40:20.219691+0000","last_peered":"2024-09-10T20:40:20.219691+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:40:20.219691+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3010,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":609,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3008}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":128,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.414539+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:40:17.414539+0000","last_peered":"2024-09-10T20:40:17.414539+0000","last_clean":"2024-09-10T20:40:17.414539+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:40:17.414539+0000","last_undegraded":"2024-09-10T20:40:17.414539+0000","last_fullsized":"2024-09-10T20:40:17.414539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":27999,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.475534+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:40:17.475534+0000","last_peered":"2024-09-10T20:40:17.475534+0000","last_clean":"2024-09-10T20:40:17.475534+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:40:17.475534+0000","last_undegraded":"2024-09-10T20:40:17.475534+0000","last_fullsized":"2024-09-10T20:40:17.475534+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":104,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T20:40:16.406149+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:40:16.406149+0000","last_peered":"2024-09-10T20:40:16.406149+0000","last_clean":"2024-09-10T20:40:16.406149+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:40:16.406149+0000","last_undegraded":"2024-09-10T20:40:16.406149+0000","last_fullsized":"2024-09-10T20:40:16.406149+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28052,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417580+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:40:17.417580+0000","last_peered":"2024-09-10T20:40:17.417580+0000","last_clean":"2024-09-10T20:40:17.417580+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:40:17.417580+0000","last_undegraded":"2024-09-10T20:40:17.417580+0000","last_fullsized":"2024-09-10T20:40:17.417580+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21107,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T20:40:16.406467+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:40:16.406467+0000","last_peered":"2024-09-10T20:40:16.406467+0000","last_clean":"2024-09-10T20:40:16.406467+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:40:16.406467+0000","last_undegraded":"2024-09-10T20:40:16.406467+0000","last_fullsized":"2024-09-10T20:40:16.406467+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":94,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T20:40:16.406628+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:40:16.406628+0000","last_peered":"2024-09-10T20:40:16.406628+0000","last_clean":"2024-09-10T20:40:16.406628+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:40:16.406628+0000","last_undegraded":"2024-09-10T20:40:16.406628+0000","last_fullsized":"2024-09-10T20:40:16.406628+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27828,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T20:40:16.406384+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:40:16.406384+0000","last_peered":"2024-09-10T20:40:16.406384+0000","last_clean":"2024-09-10T20:40:16.406384+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:40:16.406384+0000","last_undegraded":"2024-09-10T20:40:16.406384+0000","last_fullsized":"2024-09-10T20:40:16.406384+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417903+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:40:17.417903+0000","last_peered":"2024-09-10T20:40:17.417903+0000","last_clean":"2024-09-10T20:40:17.417903+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:40:17.417903+0000","last_undegraded":"2024-09-10T20:40:17.417903+0000","last_fullsized":"2024-09-10T20:40:17.417903+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21261,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.415576+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:40:17.415576+0000","last_peered":"2024-09-10T20:40:17.415576+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:40:17.415576+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":94,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T20:40:16.406377+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:40:16.406377+0000","last_peered":"2024-09-10T20:40:16.406377+0000","last_clean":"2024-09-10T20:40:16.406377+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:40:16.406377+0000","last_undegraded":"2024-09-10T20:40:16.406377+0000","last_fullsized":"2024-09-10T20:40:16.406377+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":99,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T20:40:16.406379+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:40:16.406379+0000","last_peered":"2024-09-10T20:40:16.406379+0000","last_clean":"2024-09-10T20:40:16.406379+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:40:16.406379+0000","last_undegraded":"2024-09-10T20:40:16.406379+0000","last_fullsized":"2024-09-10T20:40:16.406379+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27658,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T20:40:16.406145+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:40:16.406145+0000","last_peered":"2024-09-10T20:40:16.406145+0000","last_clean":"2024-09-10T20:40:16.406145+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:40:16.406145+0000","last_undegraded":"2024-09-10T20:40:16.406145+0000","last_fullsized":"2024-09-10T20:40:16.406145+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25890,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":175206400,"data_stored":175173883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182541,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517180,"kb_used_data":57956,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854660,"statfs":{"total":96636764160,"available":96107171840,"internally_reserved":0,"allocated":59346944,"data_stored":59045876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986313,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479296,"kb_used_data":114408,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93892544,"statfs":{"total":96636764160,"available":96145965056,"internally_reserved":0,"allocated":117153792,"data_stored":116856801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659259,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483388,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888452,"statfs":{"total":96636764160,"available":96141774848,"internally_reserved":0,"allocated":499712,"data_stored":181904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116310016,"data_stored":116306403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:40:24.265 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:40:24.611 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:24.611 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:40:24.624 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":418,"stamp":"2024-09-10T20:40:23.066596+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25892,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1484028,"kb_used_data":172860,"kb_used_omap":12,"kb_used_meta":1311091,"kb_avail":281631492,"statfs":{"total":289910292480,"available":288390647808,"internally_reserved":0,"allocated":177008640,"data_stored":176087835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1342557186},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-881,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":881,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001338"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20898,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.416306+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:40:17.416306+0000","last_peered":"2024-09-10T20:40:17.416306+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:40:17.416306+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21079,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418190+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:40:17.418190+0000","last_peered":"2024-09-10T20:40:17.418190+0000","last_clean":"2024-09-10T20:40:17.418190+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:40:17.418190+0000","last_undegraded":"2024-09-10T20:40:17.418190+0000","last_fullsized":"2024-09-10T20:40:17.418190+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28091,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418033+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:40:17.418033+0000","last_peered":"2024-09-10T20:40:17.418033+0000","last_clean":"2024-09-10T20:40:17.418033+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:40:17.418033+0000","last_undegraded":"2024-09-10T20:40:17.418033+0000","last_fullsized":"2024-09-10T20:40:17.418033+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20711,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418346+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:40:17.418346+0000","last_peered":"2024-09-10T20:40:17.418346+0000","last_clean":"2024-09-10T20:40:17.418346+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:40:17.418346+0000","last_undegraded":"2024-09-10T20:40:17.418346+0000","last_fullsized":"2024-09-10T20:40:17.418346+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21223,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.414948+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:40:17.414948+0000","last_peered":"2024-09-10T20:40:17.414948+0000","last_clean":"2024-09-10T20:40:17.414948+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:40:17.414948+0000","last_undegraded":"2024-09-10T20:40:17.414948+0000","last_fullsized":"2024-09-10T20:40:17.414948+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20741,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.414623+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:40:17.414623+0000","last_peered":"2024-09-10T20:40:17.414623+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:40:17.414623+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20911,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.414726+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:40:17.414726+0000","last_peered":"2024-09-10T20:40:17.414726+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:40:17.414726+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20888,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.414444+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:40:17.414444+0000","last_peered":"2024-09-10T20:40:17.414444+0000","last_clean":"2024-09-10T20:40:17.414444+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:40:17.414444+0000","last_undegraded":"2024-09-10T20:40:17.414444+0000","last_fullsized":"2024-09-10T20:40:17.414444+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417460+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:40:17.417460+0000","last_peered":"2024-09-10T20:40:17.417460+0000","last_clean":"2024-09-10T20:40:17.417460+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:40:17.417460+0000","last_undegraded":"2024-09-10T20:40:17.417460+0000","last_fullsized":"2024-09-10T20:40:17.417460+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22130,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417939+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:40:17.417939+0000","last_peered":"2024-09-10T20:40:17.417939+0000","last_clean":"2024-09-10T20:40:17.417939+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:40:17.417939+0000","last_undegraded":"2024-09-10T20:40:17.417939+0000","last_fullsized":"2024-09-10T20:40:17.417939+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418023+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:40:17.418023+0000","last_peered":"2024-09-10T20:40:17.418023+0000","last_clean":"2024-09-10T20:40:17.418023+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:40:17.418023+0000","last_undegraded":"2024-09-10T20:40:17.418023+0000","last_fullsized":"2024-09-10T20:40:17.418023+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":22081,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:20.219691+0000","last_change":"2024-09-10T20:40:16.115711+0000","last_active":"2024-09-10T20:40:20.219691+0000","last_peered":"2024-09-10T20:40:20.219691+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:40:20.219691+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3010,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":609,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3008}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":128,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.414539+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:40:17.414539+0000","last_peered":"2024-09-10T20:40:17.414539+0000","last_clean":"2024-09-10T20:40:17.414539+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:40:17.414539+0000","last_undegraded":"2024-09-10T20:40:17.414539+0000","last_fullsized":"2024-09-10T20:40:17.414539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":27999,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.475534+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:40:17.475534+0000","last_peered":"2024-09-10T20:40:17.475534+0000","last_clean":"2024-09-10T20:40:17.475534+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:40:17.475534+0000","last_undegraded":"2024-09-10T20:40:17.475534+0000","last_fullsized":"2024-09-10T20:40:17.475534+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":106,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418322+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:40:17.418322+0000","last_peered":"2024-09-10T20:40:17.418322+0000","last_clean":"2024-09-10T20:40:17.418322+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:40:17.418322+0000","last_undegraded":"2024-09-10T20:40:17.418322+0000","last_fullsized":"2024-09-10T20:40:17.418322+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28052,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417580+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:40:17.417580+0000","last_peered":"2024-09-10T20:40:17.417580+0000","last_clean":"2024-09-10T20:40:17.417580+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:40:17.417580+0000","last_undegraded":"2024-09-10T20:40:17.417580+0000","last_fullsized":"2024-09-10T20:40:17.417580+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21109,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418310+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:40:17.418310+0000","last_peered":"2024-09-10T20:40:17.418310+0000","last_clean":"2024-09-10T20:40:17.418310+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:40:17.418310+0000","last_undegraded":"2024-09-10T20:40:17.418310+0000","last_fullsized":"2024-09-10T20:40:17.418310+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418039+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:40:17.418039+0000","last_peered":"2024-09-10T20:40:17.418039+0000","last_clean":"2024-09-10T20:40:17.418039+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:40:17.418039+0000","last_undegraded":"2024-09-10T20:40:17.418039+0000","last_fullsized":"2024-09-10T20:40:17.418039+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27830,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418188+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:40:17.418188+0000","last_peered":"2024-09-10T20:40:17.418188+0000","last_clean":"2024-09-10T20:40:17.418188+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:40:17.418188+0000","last_undegraded":"2024-09-10T20:40:17.418188+0000","last_fullsized":"2024-09-10T20:40:17.418188+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417903+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:40:17.417903+0000","last_peered":"2024-09-10T20:40:17.417903+0000","last_clean":"2024-09-10T20:40:17.417903+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:40:17.417903+0000","last_undegraded":"2024-09-10T20:40:17.417903+0000","last_fullsized":"2024-09-10T20:40:17.417903+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21261,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.415576+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:40:17.415576+0000","last_peered":"2024-09-10T20:40:17.415576+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:40:17.415576+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418131+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:40:17.418131+0000","last_peered":"2024-09-10T20:40:17.418131+0000","last_clean":"2024-09-10T20:40:17.418131+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:40:17.418131+0000","last_undegraded":"2024-09-10T20:40:17.418131+0000","last_fullsized":"2024-09-10T20:40:17.418131+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418224+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:40:17.418224+0000","last_peered":"2024-09-10T20:40:17.418224+0000","last_clean":"2024-09-10T20:40:17.418224+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:40:17.418224+0000","last_undegraded":"2024-09-10T20:40:17.418224+0000","last_fullsized":"2024-09-10T20:40:17.418224+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27660,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417945+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:40:17.417945+0000","last_peered":"2024-09-10T20:40:17.417945+0000","last_clean":"2024-09-10T20:40:17.417945+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:40:17.417945+0000","last_undegraded":"2024-09-10T20:40:17.417945+0000","last_fullsized":"2024-09-10T20:40:17.417945+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25890,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":175206400,"data_stored":175173883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182541,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517180,"kb_used_data":57956,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854660,"statfs":{"total":96636764160,"available":96107171840,"internally_reserved":0,"allocated":59346944,"data_stored":59045876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986313,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479296,"kb_used_data":114408,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93892544,"statfs":{"total":96636764160,"available":96145965056,"internally_reserved":0,"allocated":117153792,"data_stored":116856801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659260,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487552,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884288,"statfs":{"total":96636764160,"available":96137510912,"internally_reserved":0,"allocated":507904,"data_stored":185158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116310016,"data_stored":116306403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:40:24.625 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:40:24.953 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:24.954 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:40:24.968 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":418,"stamp":"2024-09-10T20:40:23.066596+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25892,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1484028,"kb_used_data":172860,"kb_used_omap":12,"kb_used_meta":1311091,"kb_avail":281631492,"statfs":{"total":289910292480,"available":288390647808,"internally_reserved":0,"allocated":177008640,"data_stored":176087835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1342557186},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-881,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":881,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001338"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20898,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.416306+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:40:17.416306+0000","last_peered":"2024-09-10T20:40:17.416306+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:40:17.416306+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21079,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418190+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:40:17.418190+0000","last_peered":"2024-09-10T20:40:17.418190+0000","last_clean":"2024-09-10T20:40:17.418190+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:40:17.418190+0000","last_undegraded":"2024-09-10T20:40:17.418190+0000","last_fullsized":"2024-09-10T20:40:17.418190+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28091,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418033+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:40:17.418033+0000","last_peered":"2024-09-10T20:40:17.418033+0000","last_clean":"2024-09-10T20:40:17.418033+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:40:17.418033+0000","last_undegraded":"2024-09-10T20:40:17.418033+0000","last_fullsized":"2024-09-10T20:40:17.418033+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20711,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418346+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:40:17.418346+0000","last_peered":"2024-09-10T20:40:17.418346+0000","last_clean":"2024-09-10T20:40:17.418346+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:40:17.418346+0000","last_undegraded":"2024-09-10T20:40:17.418346+0000","last_fullsized":"2024-09-10T20:40:17.418346+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21223,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.414948+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:40:17.414948+0000","last_peered":"2024-09-10T20:40:17.414948+0000","last_clean":"2024-09-10T20:40:17.414948+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:40:17.414948+0000","last_undegraded":"2024-09-10T20:40:17.414948+0000","last_fullsized":"2024-09-10T20:40:17.414948+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20741,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.414623+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:40:17.414623+0000","last_peered":"2024-09-10T20:40:17.414623+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:40:17.414623+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20911,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.414726+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:40:17.414726+0000","last_peered":"2024-09-10T20:40:17.414726+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:40:17.414726+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20888,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.414444+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:40:17.414444+0000","last_peered":"2024-09-10T20:40:17.414444+0000","last_clean":"2024-09-10T20:40:17.414444+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:40:17.414444+0000","last_undegraded":"2024-09-10T20:40:17.414444+0000","last_fullsized":"2024-09-10T20:40:17.414444+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417460+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:40:17.417460+0000","last_peered":"2024-09-10T20:40:17.417460+0000","last_clean":"2024-09-10T20:40:17.417460+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:40:17.417460+0000","last_undegraded":"2024-09-10T20:40:17.417460+0000","last_fullsized":"2024-09-10T20:40:17.417460+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22130,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417939+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:40:17.417939+0000","last_peered":"2024-09-10T20:40:17.417939+0000","last_clean":"2024-09-10T20:40:17.417939+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:40:17.417939+0000","last_undegraded":"2024-09-10T20:40:17.417939+0000","last_fullsized":"2024-09-10T20:40:17.417939+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418023+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:40:17.418023+0000","last_peered":"2024-09-10T20:40:17.418023+0000","last_clean":"2024-09-10T20:40:17.418023+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:40:17.418023+0000","last_undegraded":"2024-09-10T20:40:17.418023+0000","last_fullsized":"2024-09-10T20:40:17.418023+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":22081,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:20.219691+0000","last_change":"2024-09-10T20:40:16.115711+0000","last_active":"2024-09-10T20:40:20.219691+0000","last_peered":"2024-09-10T20:40:20.219691+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:40:20.219691+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3010,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":609,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3008}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":128,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.414539+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:40:17.414539+0000","last_peered":"2024-09-10T20:40:17.414539+0000","last_clean":"2024-09-10T20:40:17.414539+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:40:17.414539+0000","last_undegraded":"2024-09-10T20:40:17.414539+0000","last_fullsized":"2024-09-10T20:40:17.414539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":27999,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.475534+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:40:17.475534+0000","last_peered":"2024-09-10T20:40:17.475534+0000","last_clean":"2024-09-10T20:40:17.475534+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:40:17.475534+0000","last_undegraded":"2024-09-10T20:40:17.475534+0000","last_fullsized":"2024-09-10T20:40:17.475534+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":106,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418322+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:40:17.418322+0000","last_peered":"2024-09-10T20:40:17.418322+0000","last_clean":"2024-09-10T20:40:17.418322+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:40:17.418322+0000","last_undegraded":"2024-09-10T20:40:17.418322+0000","last_fullsized":"2024-09-10T20:40:17.418322+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28052,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417580+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:40:17.417580+0000","last_peered":"2024-09-10T20:40:17.417580+0000","last_clean":"2024-09-10T20:40:17.417580+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:40:17.417580+0000","last_undegraded":"2024-09-10T20:40:17.417580+0000","last_fullsized":"2024-09-10T20:40:17.417580+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21109,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418310+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:40:17.418310+0000","last_peered":"2024-09-10T20:40:17.418310+0000","last_clean":"2024-09-10T20:40:17.418310+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:40:17.418310+0000","last_undegraded":"2024-09-10T20:40:17.418310+0000","last_fullsized":"2024-09-10T20:40:17.418310+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418039+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:40:17.418039+0000","last_peered":"2024-09-10T20:40:17.418039+0000","last_clean":"2024-09-10T20:40:17.418039+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:40:17.418039+0000","last_undegraded":"2024-09-10T20:40:17.418039+0000","last_fullsized":"2024-09-10T20:40:17.418039+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27830,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418188+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:40:17.418188+0000","last_peered":"2024-09-10T20:40:17.418188+0000","last_clean":"2024-09-10T20:40:17.418188+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:40:17.418188+0000","last_undegraded":"2024-09-10T20:40:17.418188+0000","last_fullsized":"2024-09-10T20:40:17.418188+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417903+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:40:17.417903+0000","last_peered":"2024-09-10T20:40:17.417903+0000","last_clean":"2024-09-10T20:40:17.417903+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:40:17.417903+0000","last_undegraded":"2024-09-10T20:40:17.417903+0000","last_fullsized":"2024-09-10T20:40:17.417903+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21261,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.415576+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:40:17.415576+0000","last_peered":"2024-09-10T20:40:17.415576+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:40:17.415576+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418131+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:40:17.418131+0000","last_peered":"2024-09-10T20:40:17.418131+0000","last_clean":"2024-09-10T20:40:17.418131+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:40:17.418131+0000","last_undegraded":"2024-09-10T20:40:17.418131+0000","last_fullsized":"2024-09-10T20:40:17.418131+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418224+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:40:17.418224+0000","last_peered":"2024-09-10T20:40:17.418224+0000","last_clean":"2024-09-10T20:40:17.418224+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:40:17.418224+0000","last_undegraded":"2024-09-10T20:40:17.418224+0000","last_fullsized":"2024-09-10T20:40:17.418224+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27660,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417945+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:40:17.417945+0000","last_peered":"2024-09-10T20:40:17.417945+0000","last_clean":"2024-09-10T20:40:17.417945+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:40:17.417945+0000","last_undegraded":"2024-09-10T20:40:17.417945+0000","last_fullsized":"2024-09-10T20:40:17.417945+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25890,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":175206400,"data_stored":175173883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182541,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517180,"kb_used_data":57956,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854660,"statfs":{"total":96636764160,"available":96107171840,"internally_reserved":0,"allocated":59346944,"data_stored":59045876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986313,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479296,"kb_used_data":114408,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93892544,"statfs":{"total":96636764160,"available":96145965056,"internally_reserved":0,"allocated":117153792,"data_stored":116856801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659260,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487552,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884288,"statfs":{"total":96636764160,"available":96137510912,"internally_reserved":0,"allocated":507904,"data_stored":185158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116310016,"data_stored":116306403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:40:24.969 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:40:24.969 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'20658", 'reported_seq': 20898, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:17.416306+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:40:17.416306+0000', 'last_peered': '2024-09-10T20:40:17.416306+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:40:17.416306+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20658, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3541}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:24.969 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:40:24.969 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20471", 'reported_seq': 20741, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:17.414623+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:40:17.414623+0000', 'last_peered': '2024-09-10T20:40:17.414623+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:40:17.414623+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17335, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3561}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:24.969 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:40:24.969 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'20608", 'reported_seq': 20911, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:17.414726+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:40:17.414726+0000', 'last_peered': '2024-09-10T20:40:17.414726+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:40:17.414726+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17203, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3486}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:24.969 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:40:24.970 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20730", 'reported_seq': 22081, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:20.219691+0000', 'last_change': '2024-09-10T20:40:16.115711+0000', 'last_active': '2024-09-10T20:40:20.219691+0000', 'last_peered': '2024-09-10T20:40:20.219691+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:40:20.219691+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3010, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20728, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 609, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3008}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:24.970 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:40:24.970 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "51'20986", 'reported_seq': 21261, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:17.415576+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:40:17.415576+0000', 'last_peered': '2024-09-10T20:40:17.415576+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:40:17.415576+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17813, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3655}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:24.970 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:40:25.349 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:25.350 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 17259/10 objects degraded (172590.000%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":736,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":5,"data_bytes":1468,"bytes_used":1519644672,"bytes_avail":288390647808,"bytes_total":289910292480,"degraded_objects":17259,"degraded_total":10,"degraded_ratio":1725.9000000000001,"recovering_objects_per_sec":110,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":881,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:40:25.364 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:40:25.364 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:40:25.678 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:25.679 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:40:25.691 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":419,"stamp":"2024-09-10T20:40:25.067012+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25892,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1482220,"kb_used_data":171060,"kb_used_omap":12,"kb_used_meta":1311091,"kb_avail":281633300,"statfs":{"total":289910292480,"available":288392499200,"internally_reserved":0,"allocated":175165440,"data_stored":174244635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1342557186},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-881,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":881,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.672327"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20898,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.416306+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:40:17.416306+0000","last_peered":"2024-09-10T20:40:17.416306+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:40:17.416306+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21079,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418190+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:40:17.418190+0000","last_peered":"2024-09-10T20:40:17.418190+0000","last_clean":"2024-09-10T20:40:17.418190+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:40:17.418190+0000","last_undegraded":"2024-09-10T20:40:17.418190+0000","last_fullsized":"2024-09-10T20:40:17.418190+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28091,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418033+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:40:17.418033+0000","last_peered":"2024-09-10T20:40:17.418033+0000","last_clean":"2024-09-10T20:40:17.418033+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:40:17.418033+0000","last_undegraded":"2024-09-10T20:40:17.418033+0000","last_fullsized":"2024-09-10T20:40:17.418033+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20711,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418346+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:40:17.418346+0000","last_peered":"2024-09-10T20:40:17.418346+0000","last_clean":"2024-09-10T20:40:17.418346+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:40:17.418346+0000","last_undegraded":"2024-09-10T20:40:17.418346+0000","last_fullsized":"2024-09-10T20:40:17.418346+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21223,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.414948+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:40:17.414948+0000","last_peered":"2024-09-10T20:40:17.414948+0000","last_clean":"2024-09-10T20:40:17.414948+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:40:17.414948+0000","last_undegraded":"2024-09-10T20:40:17.414948+0000","last_fullsized":"2024-09-10T20:40:17.414948+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20741,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.414623+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:40:17.414623+0000","last_peered":"2024-09-10T20:40:17.414623+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:40:17.414623+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20911,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.414726+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:40:17.414726+0000","last_peered":"2024-09-10T20:40:17.414726+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:40:17.414726+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20888,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.414444+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:40:17.414444+0000","last_peered":"2024-09-10T20:40:17.414444+0000","last_clean":"2024-09-10T20:40:17.414444+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:40:17.414444+0000","last_undegraded":"2024-09-10T20:40:17.414444+0000","last_fullsized":"2024-09-10T20:40:17.414444+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417460+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:40:17.417460+0000","last_peered":"2024-09-10T20:40:17.417460+0000","last_clean":"2024-09-10T20:40:17.417460+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:40:17.417460+0000","last_undegraded":"2024-09-10T20:40:17.417460+0000","last_fullsized":"2024-09-10T20:40:17.417460+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22130,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417939+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:40:17.417939+0000","last_peered":"2024-09-10T20:40:17.417939+0000","last_clean":"2024-09-10T20:40:17.417939+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:40:17.417939+0000","last_undegraded":"2024-09-10T20:40:17.417939+0000","last_fullsized":"2024-09-10T20:40:17.417939+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418023+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:40:17.418023+0000","last_peered":"2024-09-10T20:40:17.418023+0000","last_clean":"2024-09-10T20:40:17.418023+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:40:17.418023+0000","last_undegraded":"2024-09-10T20:40:17.418023+0000","last_fullsized":"2024-09-10T20:40:17.418023+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":22081,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:20.219691+0000","last_change":"2024-09-10T20:40:16.115711+0000","last_active":"2024-09-10T20:40:20.219691+0000","last_peered":"2024-09-10T20:40:20.219691+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:40:20.219691+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3010,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":609,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3008}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":128,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.414539+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:40:17.414539+0000","last_peered":"2024-09-10T20:40:17.414539+0000","last_clean":"2024-09-10T20:40:17.414539+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:40:17.414539+0000","last_undegraded":"2024-09-10T20:40:17.414539+0000","last_fullsized":"2024-09-10T20:40:17.414539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":27999,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.475534+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:40:17.475534+0000","last_peered":"2024-09-10T20:40:17.475534+0000","last_clean":"2024-09-10T20:40:17.475534+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:40:17.475534+0000","last_undegraded":"2024-09-10T20:40:17.475534+0000","last_fullsized":"2024-09-10T20:40:17.475534+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":106,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418322+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:40:17.418322+0000","last_peered":"2024-09-10T20:40:17.418322+0000","last_clean":"2024-09-10T20:40:17.418322+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:40:17.418322+0000","last_undegraded":"2024-09-10T20:40:17.418322+0000","last_fullsized":"2024-09-10T20:40:17.418322+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28052,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417580+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:40:17.417580+0000","last_peered":"2024-09-10T20:40:17.417580+0000","last_clean":"2024-09-10T20:40:17.417580+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:40:17.417580+0000","last_undegraded":"2024-09-10T20:40:17.417580+0000","last_fullsized":"2024-09-10T20:40:17.417580+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21109,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418310+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:40:17.418310+0000","last_peered":"2024-09-10T20:40:17.418310+0000","last_clean":"2024-09-10T20:40:17.418310+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:40:17.418310+0000","last_undegraded":"2024-09-10T20:40:17.418310+0000","last_fullsized":"2024-09-10T20:40:17.418310+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418039+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:40:17.418039+0000","last_peered":"2024-09-10T20:40:17.418039+0000","last_clean":"2024-09-10T20:40:17.418039+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:40:17.418039+0000","last_undegraded":"2024-09-10T20:40:17.418039+0000","last_fullsized":"2024-09-10T20:40:17.418039+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27830,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418188+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:40:17.418188+0000","last_peered":"2024-09-10T20:40:17.418188+0000","last_clean":"2024-09-10T20:40:17.418188+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:40:17.418188+0000","last_undegraded":"2024-09-10T20:40:17.418188+0000","last_fullsized":"2024-09-10T20:40:17.418188+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417903+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:40:17.417903+0000","last_peered":"2024-09-10T20:40:17.417903+0000","last_clean":"2024-09-10T20:40:17.417903+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:40:17.417903+0000","last_undegraded":"2024-09-10T20:40:17.417903+0000","last_fullsized":"2024-09-10T20:40:17.417903+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21261,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.415576+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:40:17.415576+0000","last_peered":"2024-09-10T20:40:17.415576+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:40:17.415576+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418131+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:40:17.418131+0000","last_peered":"2024-09-10T20:40:17.418131+0000","last_clean":"2024-09-10T20:40:17.418131+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:40:17.418131+0000","last_undegraded":"2024-09-10T20:40:17.418131+0000","last_fullsized":"2024-09-10T20:40:17.418131+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418224+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:40:17.418224+0000","last_peered":"2024-09-10T20:40:17.418224+0000","last_clean":"2024-09-10T20:40:17.418224+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:40:17.418224+0000","last_undegraded":"2024-09-10T20:40:17.418224+0000","last_fullsized":"2024-09-10T20:40:17.418224+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27660,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417945+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:40:17.417945+0000","last_peered":"2024-09-10T20:40:17.417945+0000","last_clean":"2024-09-10T20:40:17.417945+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:40:17.417945+0000","last_undegraded":"2024-09-10T20:40:17.417945+0000","last_fullsized":"2024-09-10T20:40:17.417945+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25890,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":173363200,"data_stored":173330683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182541,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517180,"kb_used_data":57956,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854660,"statfs":{"total":96636764160,"available":96107171840,"internally_reserved":0,"allocated":59346944,"data_stored":59045876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986314,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":477488,"kb_used_data":112608,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93894352,"statfs":{"total":96636764160,"available":96147816448,"internally_reserved":0,"allocated":115310592,"data_stored":115013601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659260,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487552,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884288,"statfs":{"total":96636764160,"available":96137510912,"internally_reserved":0,"allocated":507904,"data_stored":185158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114466816,"data_stored":114463203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:40:28.693 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:40:29.016 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:29.017 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:40:29.029 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":420,"stamp":"2024-09-10T20:40:27.067395+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16634,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26517,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1482220,"kb_used_data":171060,"kb_used_omap":12,"kb_used_meta":1311091,"kb_avail":281633300,"statfs":{"total":289910292480,"available":288392499200,"internally_reserved":0,"allocated":175165440,"data_stored":174244635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1342557186},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1506,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1506,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001772"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20898,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.416306+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:40:17.416306+0000","last_peered":"2024-09-10T20:40:17.416306+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:40:17.416306+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21079,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418190+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:40:17.418190+0000","last_peered":"2024-09-10T20:40:17.418190+0000","last_clean":"2024-09-10T20:40:17.418190+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:40:17.418190+0000","last_undegraded":"2024-09-10T20:40:17.418190+0000","last_fullsized":"2024-09-10T20:40:17.418190+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28091,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418033+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:40:17.418033+0000","last_peered":"2024-09-10T20:40:17.418033+0000","last_clean":"2024-09-10T20:40:17.418033+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:40:17.418033+0000","last_undegraded":"2024-09-10T20:40:17.418033+0000","last_fullsized":"2024-09-10T20:40:17.418033+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20711,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418346+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:40:17.418346+0000","last_peered":"2024-09-10T20:40:17.418346+0000","last_clean":"2024-09-10T20:40:17.418346+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:40:17.418346+0000","last_undegraded":"2024-09-10T20:40:17.418346+0000","last_fullsized":"2024-09-10T20:40:17.418346+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21223,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.414948+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:40:17.414948+0000","last_peered":"2024-09-10T20:40:17.414948+0000","last_clean":"2024-09-10T20:40:17.414948+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:40:17.414948+0000","last_undegraded":"2024-09-10T20:40:17.414948+0000","last_fullsized":"2024-09-10T20:40:17.414948+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20741,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.414623+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:40:17.414623+0000","last_peered":"2024-09-10T20:40:17.414623+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:40:17.414623+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20911,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.414726+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:40:17.414726+0000","last_peered":"2024-09-10T20:40:17.414726+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:40:17.414726+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20888,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.414444+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:40:17.414444+0000","last_peered":"2024-09-10T20:40:17.414444+0000","last_clean":"2024-09-10T20:40:17.414444+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:40:17.414444+0000","last_undegraded":"2024-09-10T20:40:17.414444+0000","last_fullsized":"2024-09-10T20:40:17.414444+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417460+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:40:17.417460+0000","last_peered":"2024-09-10T20:40:17.417460+0000","last_clean":"2024-09-10T20:40:17.417460+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:40:17.417460+0000","last_undegraded":"2024-09-10T20:40:17.417460+0000","last_fullsized":"2024-09-10T20:40:17.417460+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22130,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417939+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:40:17.417939+0000","last_peered":"2024-09-10T20:40:17.417939+0000","last_clean":"2024-09-10T20:40:17.417939+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:40:17.417939+0000","last_undegraded":"2024-09-10T20:40:17.417939+0000","last_fullsized":"2024-09-10T20:40:17.417939+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418023+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:40:17.418023+0000","last_peered":"2024-09-10T20:40:17.418023+0000","last_clean":"2024-09-10T20:40:17.418023+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:40:17.418023+0000","last_undegraded":"2024-09-10T20:40:17.418023+0000","last_fullsized":"2024-09-10T20:40:17.418023+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":23331,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:25.223541+0000","last_change":"2024-09-10T20:40:16.115711+0000","last_active":"2024-09-10T20:40:25.223541+0000","last_peered":"2024-09-10T20:40:25.223541+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:40:25.223541+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2385,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1234,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2383}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":128,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.414539+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:40:17.414539+0000","last_peered":"2024-09-10T20:40:17.414539+0000","last_clean":"2024-09-10T20:40:17.414539+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:40:17.414539+0000","last_undegraded":"2024-09-10T20:40:17.414539+0000","last_fullsized":"2024-09-10T20:40:17.414539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":27999,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.475534+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:40:17.475534+0000","last_peered":"2024-09-10T20:40:17.475534+0000","last_clean":"2024-09-10T20:40:17.475534+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:40:17.475534+0000","last_undegraded":"2024-09-10T20:40:17.475534+0000","last_fullsized":"2024-09-10T20:40:17.475534+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":106,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418322+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:40:17.418322+0000","last_peered":"2024-09-10T20:40:17.418322+0000","last_clean":"2024-09-10T20:40:17.418322+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:40:17.418322+0000","last_undegraded":"2024-09-10T20:40:17.418322+0000","last_fullsized":"2024-09-10T20:40:17.418322+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28052,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417580+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:40:17.417580+0000","last_peered":"2024-09-10T20:40:17.417580+0000","last_clean":"2024-09-10T20:40:17.417580+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:40:17.417580+0000","last_undegraded":"2024-09-10T20:40:17.417580+0000","last_fullsized":"2024-09-10T20:40:17.417580+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21109,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418310+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:40:17.418310+0000","last_peered":"2024-09-10T20:40:17.418310+0000","last_clean":"2024-09-10T20:40:17.418310+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:40:17.418310+0000","last_undegraded":"2024-09-10T20:40:17.418310+0000","last_fullsized":"2024-09-10T20:40:17.418310+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418039+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:40:17.418039+0000","last_peered":"2024-09-10T20:40:17.418039+0000","last_clean":"2024-09-10T20:40:17.418039+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:40:17.418039+0000","last_undegraded":"2024-09-10T20:40:17.418039+0000","last_fullsized":"2024-09-10T20:40:17.418039+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27830,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418188+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:40:17.418188+0000","last_peered":"2024-09-10T20:40:17.418188+0000","last_clean":"2024-09-10T20:40:17.418188+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:40:17.418188+0000","last_undegraded":"2024-09-10T20:40:17.418188+0000","last_fullsized":"2024-09-10T20:40:17.418188+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417903+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:40:17.417903+0000","last_peered":"2024-09-10T20:40:17.417903+0000","last_clean":"2024-09-10T20:40:17.417903+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:40:17.417903+0000","last_undegraded":"2024-09-10T20:40:17.417903+0000","last_fullsized":"2024-09-10T20:40:17.417903+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21261,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.415576+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:40:17.415576+0000","last_peered":"2024-09-10T20:40:17.415576+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:40:17.415576+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418131+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:40:17.418131+0000","last_peered":"2024-09-10T20:40:17.418131+0000","last_clean":"2024-09-10T20:40:17.418131+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:40:17.418131+0000","last_undegraded":"2024-09-10T20:40:17.418131+0000","last_fullsized":"2024-09-10T20:40:17.418131+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418224+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:40:17.418224+0000","last_peered":"2024-09-10T20:40:17.418224+0000","last_clean":"2024-09-10T20:40:17.418224+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:40:17.418224+0000","last_undegraded":"2024-09-10T20:40:17.418224+0000","last_fullsized":"2024-09-10T20:40:17.418224+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27660,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417945+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:40:17.417945+0000","last_peered":"2024-09-10T20:40:17.417945+0000","last_clean":"2024-09-10T20:40:17.417945+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:40:17.417945+0000","last_undegraded":"2024-09-10T20:40:17.417945+0000","last_fullsized":"2024-09-10T20:40:17.417945+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16634,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26515,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":173363200,"data_stored":173330683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182542,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517180,"kb_used_data":57956,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854660,"statfs":{"total":96636764160,"available":96107171840,"internally_reserved":0,"allocated":59346944,"data_stored":59045876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986314,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":477488,"kb_used_data":112608,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93894352,"statfs":{"total":96636764160,"available":96147816448,"internally_reserved":0,"allocated":115310592,"data_stored":115013601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659260,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487552,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884288,"statfs":{"total":96636764160,"available":96137510912,"internally_reserved":0,"allocated":507904,"data_stored":185158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114466816,"data_stored":114463203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:40:29.032 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:40:29.337 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:29.338 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:40:29.350 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":421,"stamp":"2024-09-10T20:40:29.067735+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16634,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26517,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1490540,"kb_used_data":171060,"kb_used_omap":12,"kb_used_meta":1319411,"kb_avail":281624980,"statfs":{"total":289910292480,"available":288383979520,"internally_reserved":0,"allocated":175165440,"data_stored":174244635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1351076866},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1506,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1506,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.654641"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20898,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.416306+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:40:17.416306+0000","last_peered":"2024-09-10T20:40:17.416306+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:40:17.416306+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21079,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418190+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:40:17.418190+0000","last_peered":"2024-09-10T20:40:17.418190+0000","last_clean":"2024-09-10T20:40:17.418190+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:40:17.418190+0000","last_undegraded":"2024-09-10T20:40:17.418190+0000","last_fullsized":"2024-09-10T20:40:17.418190+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28091,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418033+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:40:17.418033+0000","last_peered":"2024-09-10T20:40:17.418033+0000","last_clean":"2024-09-10T20:40:17.418033+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:40:17.418033+0000","last_undegraded":"2024-09-10T20:40:17.418033+0000","last_fullsized":"2024-09-10T20:40:17.418033+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20711,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418346+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:40:17.418346+0000","last_peered":"2024-09-10T20:40:17.418346+0000","last_clean":"2024-09-10T20:40:17.418346+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:40:17.418346+0000","last_undegraded":"2024-09-10T20:40:17.418346+0000","last_fullsized":"2024-09-10T20:40:17.418346+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21223,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.414948+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:40:17.414948+0000","last_peered":"2024-09-10T20:40:17.414948+0000","last_clean":"2024-09-10T20:40:17.414948+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:40:17.414948+0000","last_undegraded":"2024-09-10T20:40:17.414948+0000","last_fullsized":"2024-09-10T20:40:17.414948+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20741,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.414623+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:40:17.414623+0000","last_peered":"2024-09-10T20:40:17.414623+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:40:17.414623+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20911,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.414726+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:40:17.414726+0000","last_peered":"2024-09-10T20:40:17.414726+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:40:17.414726+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20888,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.414444+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:40:17.414444+0000","last_peered":"2024-09-10T20:40:17.414444+0000","last_clean":"2024-09-10T20:40:17.414444+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:40:17.414444+0000","last_undegraded":"2024-09-10T20:40:17.414444+0000","last_fullsized":"2024-09-10T20:40:17.414444+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417460+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:40:17.417460+0000","last_peered":"2024-09-10T20:40:17.417460+0000","last_clean":"2024-09-10T20:40:17.417460+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:40:17.417460+0000","last_undegraded":"2024-09-10T20:40:17.417460+0000","last_fullsized":"2024-09-10T20:40:17.417460+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22130,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417939+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:40:17.417939+0000","last_peered":"2024-09-10T20:40:17.417939+0000","last_clean":"2024-09-10T20:40:17.417939+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:40:17.417939+0000","last_undegraded":"2024-09-10T20:40:17.417939+0000","last_fullsized":"2024-09-10T20:40:17.417939+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418023+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:40:17.418023+0000","last_peered":"2024-09-10T20:40:17.418023+0000","last_clean":"2024-09-10T20:40:17.418023+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:40:17.418023+0000","last_undegraded":"2024-09-10T20:40:17.418023+0000","last_fullsized":"2024-09-10T20:40:17.418023+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":23331,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:25.223541+0000","last_change":"2024-09-10T20:40:16.115711+0000","last_active":"2024-09-10T20:40:25.223541+0000","last_peered":"2024-09-10T20:40:25.223541+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:40:25.223541+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2385,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1234,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2383}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":128,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.414539+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:40:17.414539+0000","last_peered":"2024-09-10T20:40:17.414539+0000","last_clean":"2024-09-10T20:40:17.414539+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:40:17.414539+0000","last_undegraded":"2024-09-10T20:40:17.414539+0000","last_fullsized":"2024-09-10T20:40:17.414539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":27999,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.475534+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:40:17.475534+0000","last_peered":"2024-09-10T20:40:17.475534+0000","last_clean":"2024-09-10T20:40:17.475534+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:40:17.475534+0000","last_undegraded":"2024-09-10T20:40:17.475534+0000","last_fullsized":"2024-09-10T20:40:17.475534+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":106,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418322+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:40:17.418322+0000","last_peered":"2024-09-10T20:40:17.418322+0000","last_clean":"2024-09-10T20:40:17.418322+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:40:17.418322+0000","last_undegraded":"2024-09-10T20:40:17.418322+0000","last_fullsized":"2024-09-10T20:40:17.418322+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28052,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417580+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:40:17.417580+0000","last_peered":"2024-09-10T20:40:17.417580+0000","last_clean":"2024-09-10T20:40:17.417580+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:40:17.417580+0000","last_undegraded":"2024-09-10T20:40:17.417580+0000","last_fullsized":"2024-09-10T20:40:17.417580+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21109,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418310+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:40:17.418310+0000","last_peered":"2024-09-10T20:40:17.418310+0000","last_clean":"2024-09-10T20:40:17.418310+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:40:17.418310+0000","last_undegraded":"2024-09-10T20:40:17.418310+0000","last_fullsized":"2024-09-10T20:40:17.418310+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418039+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:40:17.418039+0000","last_peered":"2024-09-10T20:40:17.418039+0000","last_clean":"2024-09-10T20:40:17.418039+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:40:17.418039+0000","last_undegraded":"2024-09-10T20:40:17.418039+0000","last_fullsized":"2024-09-10T20:40:17.418039+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27830,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418188+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:40:17.418188+0000","last_peered":"2024-09-10T20:40:17.418188+0000","last_clean":"2024-09-10T20:40:17.418188+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:40:17.418188+0000","last_undegraded":"2024-09-10T20:40:17.418188+0000","last_fullsized":"2024-09-10T20:40:17.418188+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417903+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:40:17.417903+0000","last_peered":"2024-09-10T20:40:17.417903+0000","last_clean":"2024-09-10T20:40:17.417903+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:40:17.417903+0000","last_undegraded":"2024-09-10T20:40:17.417903+0000","last_fullsized":"2024-09-10T20:40:17.417903+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21261,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.415576+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:40:17.415576+0000","last_peered":"2024-09-10T20:40:17.415576+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:40:17.415576+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418131+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:40:17.418131+0000","last_peered":"2024-09-10T20:40:17.418131+0000","last_clean":"2024-09-10T20:40:17.418131+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:40:17.418131+0000","last_undegraded":"2024-09-10T20:40:17.418131+0000","last_fullsized":"2024-09-10T20:40:17.418131+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418224+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:40:17.418224+0000","last_peered":"2024-09-10T20:40:17.418224+0000","last_clean":"2024-09-10T20:40:17.418224+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:40:17.418224+0000","last_undegraded":"2024-09-10T20:40:17.418224+0000","last_fullsized":"2024-09-10T20:40:17.418224+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27660,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417945+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:40:17.417945+0000","last_peered":"2024-09-10T20:40:17.417945+0000","last_clean":"2024-09-10T20:40:17.417945+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:40:17.417945+0000","last_undegraded":"2024-09-10T20:40:17.417945+0000","last_fullsized":"2024-09-10T20:40:17.417945+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16634,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26515,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":173363200,"data_stored":173330683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182542,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517180,"kb_used_data":57956,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854660,"statfs":{"total":96636764160,"available":96107171840,"internally_reserved":0,"allocated":59346944,"data_stored":59045876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986314,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":477488,"kb_used_data":112608,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93894352,"statfs":{"total":96636764160,"available":96147816448,"internally_reserved":0,"allocated":115310592,"data_stored":115013601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659261,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":495872,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":495355,"kb_avail":93875968,"statfs":{"total":96636764160,"available":96128991232,"internally_reserved":0,"allocated":507904,"data_stored":185158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":507244206},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114466816,"data_stored":114463203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:40:29.352 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:40:29.352 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'20658", 'reported_seq': 20898, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:17.416306+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:40:17.416306+0000', 'last_peered': '2024-09-10T20:40:17.416306+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:40:17.416306+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20658, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3541}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:29.352 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:40:29.352 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20471", 'reported_seq': 20741, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:17.414623+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:40:17.414623+0000', 'last_peered': '2024-09-10T20:40:17.414623+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:40:17.414623+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17335, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3561}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:29.352 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:40:29.352 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'20608", 'reported_seq': 20911, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:17.414726+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:40:17.414726+0000', 'last_peered': '2024-09-10T20:40:17.414726+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:40:17.414726+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17203, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3486}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:29.352 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:40:29.352 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20730", 'reported_seq': 23331, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:25.223541+0000', 'last_change': '2024-09-10T20:40:16.115711+0000', 'last_active': '2024-09-10T20:40:25.223541+0000', 'last_peered': '2024-09-10T20:40:25.223541+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:40:25.223541+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2385, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20728, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1234, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2383}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:29.353 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:40:29.353 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "51'20986", 'reported_seq': 21261, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:17.415576+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:40:17.415576+0000', 'last_peered': '2024-09-10T20:40:17.415576+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:40:17.415576+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17813, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3655}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:29.353 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:40:29.734 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:29.735 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 16634/10 objects degraded (166340.000%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":741,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":5,"data_bytes":1468,"bytes_used":1526312960,"bytes_avail":288383979520,"bytes_total":289910292480,"degraded_objects":16634,"degraded_total":10,"degraded_ratio":1663.4000000000001,"recovering_objects_per_sec":129,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1506,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:40:29.748 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:40:29.748 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:40:30.052 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:30.053 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:40:30.065 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":421,"stamp":"2024-09-10T20:40:29.067735+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16634,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26517,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1490540,"kb_used_data":171060,"kb_used_omap":12,"kb_used_meta":1319411,"kb_avail":281624980,"statfs":{"total":289910292480,"available":288383979520,"internally_reserved":0,"allocated":175165440,"data_stored":174244635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1351076866},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1506,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1506,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.654641"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20898,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.416306+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:40:17.416306+0000","last_peered":"2024-09-10T20:40:17.416306+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:40:17.416306+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21079,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418190+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:40:17.418190+0000","last_peered":"2024-09-10T20:40:17.418190+0000","last_clean":"2024-09-10T20:40:17.418190+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:40:17.418190+0000","last_undegraded":"2024-09-10T20:40:17.418190+0000","last_fullsized":"2024-09-10T20:40:17.418190+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28091,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418033+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:40:17.418033+0000","last_peered":"2024-09-10T20:40:17.418033+0000","last_clean":"2024-09-10T20:40:17.418033+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:40:17.418033+0000","last_undegraded":"2024-09-10T20:40:17.418033+0000","last_fullsized":"2024-09-10T20:40:17.418033+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20711,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418346+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:40:17.418346+0000","last_peered":"2024-09-10T20:40:17.418346+0000","last_clean":"2024-09-10T20:40:17.418346+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:40:17.418346+0000","last_undegraded":"2024-09-10T20:40:17.418346+0000","last_fullsized":"2024-09-10T20:40:17.418346+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21223,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.414948+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:40:17.414948+0000","last_peered":"2024-09-10T20:40:17.414948+0000","last_clean":"2024-09-10T20:40:17.414948+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:40:17.414948+0000","last_undegraded":"2024-09-10T20:40:17.414948+0000","last_fullsized":"2024-09-10T20:40:17.414948+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20741,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.414623+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:40:17.414623+0000","last_peered":"2024-09-10T20:40:17.414623+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:40:17.414623+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20911,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.414726+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:40:17.414726+0000","last_peered":"2024-09-10T20:40:17.414726+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:40:17.414726+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20888,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.414444+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:40:17.414444+0000","last_peered":"2024-09-10T20:40:17.414444+0000","last_clean":"2024-09-10T20:40:17.414444+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:40:17.414444+0000","last_undegraded":"2024-09-10T20:40:17.414444+0000","last_fullsized":"2024-09-10T20:40:17.414444+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417460+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:40:17.417460+0000","last_peered":"2024-09-10T20:40:17.417460+0000","last_clean":"2024-09-10T20:40:17.417460+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:40:17.417460+0000","last_undegraded":"2024-09-10T20:40:17.417460+0000","last_fullsized":"2024-09-10T20:40:17.417460+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22130,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417939+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:40:17.417939+0000","last_peered":"2024-09-10T20:40:17.417939+0000","last_clean":"2024-09-10T20:40:17.417939+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:40:17.417939+0000","last_undegraded":"2024-09-10T20:40:17.417939+0000","last_fullsized":"2024-09-10T20:40:17.417939+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418023+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:40:17.418023+0000","last_peered":"2024-09-10T20:40:17.418023+0000","last_clean":"2024-09-10T20:40:17.418023+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:40:17.418023+0000","last_undegraded":"2024-09-10T20:40:17.418023+0000","last_fullsized":"2024-09-10T20:40:17.418023+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":23331,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:25.223541+0000","last_change":"2024-09-10T20:40:16.115711+0000","last_active":"2024-09-10T20:40:25.223541+0000","last_peered":"2024-09-10T20:40:25.223541+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:40:25.223541+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2385,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1234,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2383}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":128,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.414539+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:40:17.414539+0000","last_peered":"2024-09-10T20:40:17.414539+0000","last_clean":"2024-09-10T20:40:17.414539+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:40:17.414539+0000","last_undegraded":"2024-09-10T20:40:17.414539+0000","last_fullsized":"2024-09-10T20:40:17.414539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":27999,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.475534+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:40:17.475534+0000","last_peered":"2024-09-10T20:40:17.475534+0000","last_clean":"2024-09-10T20:40:17.475534+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:40:17.475534+0000","last_undegraded":"2024-09-10T20:40:17.475534+0000","last_fullsized":"2024-09-10T20:40:17.475534+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":106,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418322+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:40:17.418322+0000","last_peered":"2024-09-10T20:40:17.418322+0000","last_clean":"2024-09-10T20:40:17.418322+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:40:17.418322+0000","last_undegraded":"2024-09-10T20:40:17.418322+0000","last_fullsized":"2024-09-10T20:40:17.418322+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28052,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417580+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:40:17.417580+0000","last_peered":"2024-09-10T20:40:17.417580+0000","last_clean":"2024-09-10T20:40:17.417580+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:40:17.417580+0000","last_undegraded":"2024-09-10T20:40:17.417580+0000","last_fullsized":"2024-09-10T20:40:17.417580+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21109,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418310+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:40:17.418310+0000","last_peered":"2024-09-10T20:40:17.418310+0000","last_clean":"2024-09-10T20:40:17.418310+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:40:17.418310+0000","last_undegraded":"2024-09-10T20:40:17.418310+0000","last_fullsized":"2024-09-10T20:40:17.418310+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418039+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:40:17.418039+0000","last_peered":"2024-09-10T20:40:17.418039+0000","last_clean":"2024-09-10T20:40:17.418039+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:40:17.418039+0000","last_undegraded":"2024-09-10T20:40:17.418039+0000","last_fullsized":"2024-09-10T20:40:17.418039+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27830,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418188+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:40:17.418188+0000","last_peered":"2024-09-10T20:40:17.418188+0000","last_clean":"2024-09-10T20:40:17.418188+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:40:17.418188+0000","last_undegraded":"2024-09-10T20:40:17.418188+0000","last_fullsized":"2024-09-10T20:40:17.418188+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417903+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:40:17.417903+0000","last_peered":"2024-09-10T20:40:17.417903+0000","last_clean":"2024-09-10T20:40:17.417903+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:40:17.417903+0000","last_undegraded":"2024-09-10T20:40:17.417903+0000","last_fullsized":"2024-09-10T20:40:17.417903+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21261,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.415576+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:40:17.415576+0000","last_peered":"2024-09-10T20:40:17.415576+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:40:17.415576+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418131+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:40:17.418131+0000","last_peered":"2024-09-10T20:40:17.418131+0000","last_clean":"2024-09-10T20:40:17.418131+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:40:17.418131+0000","last_undegraded":"2024-09-10T20:40:17.418131+0000","last_fullsized":"2024-09-10T20:40:17.418131+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418224+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:40:17.418224+0000","last_peered":"2024-09-10T20:40:17.418224+0000","last_clean":"2024-09-10T20:40:17.418224+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:40:17.418224+0000","last_undegraded":"2024-09-10T20:40:17.418224+0000","last_fullsized":"2024-09-10T20:40:17.418224+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27660,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417945+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:40:17.417945+0000","last_peered":"2024-09-10T20:40:17.417945+0000","last_clean":"2024-09-10T20:40:17.417945+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:40:17.417945+0000","last_undegraded":"2024-09-10T20:40:17.417945+0000","last_fullsized":"2024-09-10T20:40:17.417945+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16634,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26515,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":173363200,"data_stored":173330683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182542,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517180,"kb_used_data":57956,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854660,"statfs":{"total":96636764160,"available":96107171840,"internally_reserved":0,"allocated":59346944,"data_stored":59045876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986314,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":477488,"kb_used_data":112608,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93894352,"statfs":{"total":96636764160,"available":96147816448,"internally_reserved":0,"allocated":115310592,"data_stored":115013601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659261,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":495872,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":495355,"kb_avail":93875968,"statfs":{"total":96636764160,"available":96128991232,"internally_reserved":0,"allocated":507904,"data_stored":185158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":507244206},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114466816,"data_stored":114463203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:40:33.068 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:40:33.373 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:33.373 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:40:33.386 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":423,"stamp":"2024-09-10T20:40:33.068383+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15887,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27264,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1475900,"kb_used_data":168900,"kb_used_omap":12,"kb_used_meta":1306931,"kb_avail":281639620,"statfs":{"total":289910292480,"available":288398970880,"internally_reserved":0,"allocated":172953600,"data_stored":172032795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1338297346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1372,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002149"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20898,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.416306+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:40:17.416306+0000","last_peered":"2024-09-10T20:40:17.416306+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:40:17.416306+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21080,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418235+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:40:17.418235+0000","last_peered":"2024-09-10T20:40:17.418235+0000","last_clean":"2024-09-10T20:40:17.418235+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:40:17.418235+0000","last_undegraded":"2024-09-10T20:40:17.418235+0000","last_fullsized":"2024-09-10T20:40:17.418235+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28091,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418033+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:40:17.418033+0000","last_peered":"2024-09-10T20:40:17.418033+0000","last_clean":"2024-09-10T20:40:17.418033+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:40:17.418033+0000","last_undegraded":"2024-09-10T20:40:17.418033+0000","last_fullsized":"2024-09-10T20:40:17.418033+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20712,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418390+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:40:17.418390+0000","last_peered":"2024-09-10T20:40:17.418390+0000","last_clean":"2024-09-10T20:40:17.418390+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:40:17.418390+0000","last_undegraded":"2024-09-10T20:40:17.418390+0000","last_fullsized":"2024-09-10T20:40:17.418390+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21223,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.414948+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:40:17.414948+0000","last_peered":"2024-09-10T20:40:17.414948+0000","last_clean":"2024-09-10T20:40:17.414948+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:40:17.414948+0000","last_undegraded":"2024-09-10T20:40:17.414948+0000","last_fullsized":"2024-09-10T20:40:17.414948+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20741,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.414623+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:40:17.414623+0000","last_peered":"2024-09-10T20:40:17.414623+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:40:17.414623+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20911,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.414726+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:40:17.414726+0000","last_peered":"2024-09-10T20:40:17.414726+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:40:17.414726+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20888,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.414444+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:40:17.414444+0000","last_peered":"2024-09-10T20:40:17.414444+0000","last_clean":"2024-09-10T20:40:17.414444+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:40:17.414444+0000","last_undegraded":"2024-09-10T20:40:17.414444+0000","last_fullsized":"2024-09-10T20:40:17.414444+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417460+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:40:17.417460+0000","last_peered":"2024-09-10T20:40:17.417460+0000","last_clean":"2024-09-10T20:40:17.417460+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:40:17.417460+0000","last_undegraded":"2024-09-10T20:40:17.417460+0000","last_fullsized":"2024-09-10T20:40:17.417460+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22130,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417939+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:40:17.417939+0000","last_peered":"2024-09-10T20:40:17.417939+0000","last_clean":"2024-09-10T20:40:17.417939+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:40:17.417939+0000","last_undegraded":"2024-09-10T20:40:17.417939+0000","last_fullsized":"2024-09-10T20:40:17.417939+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418023+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:40:17.418023+0000","last_peered":"2024-09-10T20:40:17.418023+0000","last_clean":"2024-09-10T20:40:17.418023+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:40:17.418023+0000","last_undegraded":"2024-09-10T20:40:17.418023+0000","last_fullsized":"2024-09-10T20:40:17.418023+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":24825,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:30.221656+0000","last_change":"2024-09-10T20:40:16.115711+0000","last_active":"2024-09-10T20:40:30.221656+0000","last_peered":"2024-09-10T20:40:30.221656+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:40:30.221656+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1638,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1981,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1636}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":128,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.414539+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:40:17.414539+0000","last_peered":"2024-09-10T20:40:17.414539+0000","last_clean":"2024-09-10T20:40:17.414539+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:40:17.414539+0000","last_undegraded":"2024-09-10T20:40:17.414539+0000","last_fullsized":"2024-09-10T20:40:17.414539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":27999,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.475534+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:40:17.475534+0000","last_peered":"2024-09-10T20:40:17.475534+0000","last_clean":"2024-09-10T20:40:17.475534+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:40:17.475534+0000","last_undegraded":"2024-09-10T20:40:17.475534+0000","last_fullsized":"2024-09-10T20:40:17.475534+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418359+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:40:17.418359+0000","last_peered":"2024-09-10T20:40:17.418359+0000","last_clean":"2024-09-10T20:40:17.418359+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:40:17.418359+0000","last_undegraded":"2024-09-10T20:40:17.418359+0000","last_fullsized":"2024-09-10T20:40:17.418359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28052,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417580+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:40:17.417580+0000","last_peered":"2024-09-10T20:40:17.417580+0000","last_clean":"2024-09-10T20:40:17.417580+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:40:17.417580+0000","last_undegraded":"2024-09-10T20:40:17.417580+0000","last_fullsized":"2024-09-10T20:40:17.417580+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21109,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418310+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:40:17.418310+0000","last_peered":"2024-09-10T20:40:17.418310+0000","last_clean":"2024-09-10T20:40:17.418310+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:40:17.418310+0000","last_undegraded":"2024-09-10T20:40:17.418310+0000","last_fullsized":"2024-09-10T20:40:17.418310+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418073+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:40:17.418073+0000","last_peered":"2024-09-10T20:40:17.418073+0000","last_clean":"2024-09-10T20:40:17.418073+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:40:17.418073+0000","last_undegraded":"2024-09-10T20:40:17.418073+0000","last_fullsized":"2024-09-10T20:40:17.418073+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27830,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418188+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:40:17.418188+0000","last_peered":"2024-09-10T20:40:17.418188+0000","last_clean":"2024-09-10T20:40:17.418188+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:40:17.418188+0000","last_undegraded":"2024-09-10T20:40:17.418188+0000","last_fullsized":"2024-09-10T20:40:17.418188+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417903+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:40:17.417903+0000","last_peered":"2024-09-10T20:40:17.417903+0000","last_clean":"2024-09-10T20:40:17.417903+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:40:17.417903+0000","last_undegraded":"2024-09-10T20:40:17.417903+0000","last_fullsized":"2024-09-10T20:40:17.417903+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21261,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.415576+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:40:17.415576+0000","last_peered":"2024-09-10T20:40:17.415576+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:40:17.415576+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418131+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:40:17.418131+0000","last_peered":"2024-09-10T20:40:17.418131+0000","last_clean":"2024-09-10T20:40:17.418131+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:40:17.418131+0000","last_undegraded":"2024-09-10T20:40:17.418131+0000","last_fullsized":"2024-09-10T20:40:17.418131+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418224+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:40:17.418224+0000","last_peered":"2024-09-10T20:40:17.418224+0000","last_clean":"2024-09-10T20:40:17.418224+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:40:17.418224+0000","last_undegraded":"2024-09-10T20:40:17.418224+0000","last_fullsized":"2024-09-10T20:40:17.418224+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27660,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417945+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:40:17.417945+0000","last_peered":"2024-09-10T20:40:17.417945+0000","last_clean":"2024-09-10T20:40:17.417945+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:40:17.417945+0000","last_undegraded":"2024-09-10T20:40:17.417945+0000","last_fullsized":"2024-09-10T20:40:17.417945+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15887,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27262,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":171520000,"data_stored":171487483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182543,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517180,"kb_used_data":57956,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854660,"statfs":{"total":96636764160,"available":96107171840,"internally_reserved":0,"allocated":59346944,"data_stored":59045876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986315,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":475328,"kb_used_data":110448,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93896512,"statfs":{"total":96636764160,"available":96150028288,"internally_reserved":0,"allocated":113098752,"data_stored":112801761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659262,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483392,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888448,"statfs":{"total":96636764160,"available":96141770752,"internally_reserved":0,"allocated":507904,"data_stored":185158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":112623616,"data_stored":112620003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:40:33.387 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:40:33.708 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:33.709 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:40:33.721 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":423,"stamp":"2024-09-10T20:40:33.068383+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15887,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27264,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1475900,"kb_used_data":168900,"kb_used_omap":12,"kb_used_meta":1306931,"kb_avail":281639620,"statfs":{"total":289910292480,"available":288398970880,"internally_reserved":0,"allocated":172953600,"data_stored":172032795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1338297346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1372,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002149"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20898,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.416306+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:40:17.416306+0000","last_peered":"2024-09-10T20:40:17.416306+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:40:17.416306+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21080,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418235+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:40:17.418235+0000","last_peered":"2024-09-10T20:40:17.418235+0000","last_clean":"2024-09-10T20:40:17.418235+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:40:17.418235+0000","last_undegraded":"2024-09-10T20:40:17.418235+0000","last_fullsized":"2024-09-10T20:40:17.418235+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28091,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418033+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:40:17.418033+0000","last_peered":"2024-09-10T20:40:17.418033+0000","last_clean":"2024-09-10T20:40:17.418033+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:40:17.418033+0000","last_undegraded":"2024-09-10T20:40:17.418033+0000","last_fullsized":"2024-09-10T20:40:17.418033+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20712,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418390+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:40:17.418390+0000","last_peered":"2024-09-10T20:40:17.418390+0000","last_clean":"2024-09-10T20:40:17.418390+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:40:17.418390+0000","last_undegraded":"2024-09-10T20:40:17.418390+0000","last_fullsized":"2024-09-10T20:40:17.418390+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21223,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.414948+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:40:17.414948+0000","last_peered":"2024-09-10T20:40:17.414948+0000","last_clean":"2024-09-10T20:40:17.414948+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:40:17.414948+0000","last_undegraded":"2024-09-10T20:40:17.414948+0000","last_fullsized":"2024-09-10T20:40:17.414948+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20741,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.414623+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:40:17.414623+0000","last_peered":"2024-09-10T20:40:17.414623+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:40:17.414623+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20911,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.414726+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:40:17.414726+0000","last_peered":"2024-09-10T20:40:17.414726+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:40:17.414726+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20888,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.414444+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:40:17.414444+0000","last_peered":"2024-09-10T20:40:17.414444+0000","last_clean":"2024-09-10T20:40:17.414444+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:40:17.414444+0000","last_undegraded":"2024-09-10T20:40:17.414444+0000","last_fullsized":"2024-09-10T20:40:17.414444+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417460+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:40:17.417460+0000","last_peered":"2024-09-10T20:40:17.417460+0000","last_clean":"2024-09-10T20:40:17.417460+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:40:17.417460+0000","last_undegraded":"2024-09-10T20:40:17.417460+0000","last_fullsized":"2024-09-10T20:40:17.417460+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22130,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417939+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:40:17.417939+0000","last_peered":"2024-09-10T20:40:17.417939+0000","last_clean":"2024-09-10T20:40:17.417939+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:40:17.417939+0000","last_undegraded":"2024-09-10T20:40:17.417939+0000","last_fullsized":"2024-09-10T20:40:17.417939+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418023+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:40:17.418023+0000","last_peered":"2024-09-10T20:40:17.418023+0000","last_clean":"2024-09-10T20:40:17.418023+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:40:17.418023+0000","last_undegraded":"2024-09-10T20:40:17.418023+0000","last_fullsized":"2024-09-10T20:40:17.418023+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":24825,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:30.221656+0000","last_change":"2024-09-10T20:40:16.115711+0000","last_active":"2024-09-10T20:40:30.221656+0000","last_peered":"2024-09-10T20:40:30.221656+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:40:30.221656+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1638,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1981,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1636}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":128,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.414539+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:40:17.414539+0000","last_peered":"2024-09-10T20:40:17.414539+0000","last_clean":"2024-09-10T20:40:17.414539+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:40:17.414539+0000","last_undegraded":"2024-09-10T20:40:17.414539+0000","last_fullsized":"2024-09-10T20:40:17.414539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":27999,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.475534+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:40:17.475534+0000","last_peered":"2024-09-10T20:40:17.475534+0000","last_clean":"2024-09-10T20:40:17.475534+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:40:17.475534+0000","last_undegraded":"2024-09-10T20:40:17.475534+0000","last_fullsized":"2024-09-10T20:40:17.475534+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418359+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:40:17.418359+0000","last_peered":"2024-09-10T20:40:17.418359+0000","last_clean":"2024-09-10T20:40:17.418359+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:40:17.418359+0000","last_undegraded":"2024-09-10T20:40:17.418359+0000","last_fullsized":"2024-09-10T20:40:17.418359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28052,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417580+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:40:17.417580+0000","last_peered":"2024-09-10T20:40:17.417580+0000","last_clean":"2024-09-10T20:40:17.417580+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:40:17.417580+0000","last_undegraded":"2024-09-10T20:40:17.417580+0000","last_fullsized":"2024-09-10T20:40:17.417580+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21109,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418310+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:40:17.418310+0000","last_peered":"2024-09-10T20:40:17.418310+0000","last_clean":"2024-09-10T20:40:17.418310+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:40:17.418310+0000","last_undegraded":"2024-09-10T20:40:17.418310+0000","last_fullsized":"2024-09-10T20:40:17.418310+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418073+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:40:17.418073+0000","last_peered":"2024-09-10T20:40:17.418073+0000","last_clean":"2024-09-10T20:40:17.418073+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:40:17.418073+0000","last_undegraded":"2024-09-10T20:40:17.418073+0000","last_fullsized":"2024-09-10T20:40:17.418073+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27830,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418188+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:40:17.418188+0000","last_peered":"2024-09-10T20:40:17.418188+0000","last_clean":"2024-09-10T20:40:17.418188+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:40:17.418188+0000","last_undegraded":"2024-09-10T20:40:17.418188+0000","last_fullsized":"2024-09-10T20:40:17.418188+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417903+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:40:17.417903+0000","last_peered":"2024-09-10T20:40:17.417903+0000","last_clean":"2024-09-10T20:40:17.417903+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:40:17.417903+0000","last_undegraded":"2024-09-10T20:40:17.417903+0000","last_fullsized":"2024-09-10T20:40:17.417903+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21261,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.415576+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:40:17.415576+0000","last_peered":"2024-09-10T20:40:17.415576+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:40:17.415576+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418131+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:40:17.418131+0000","last_peered":"2024-09-10T20:40:17.418131+0000","last_clean":"2024-09-10T20:40:17.418131+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:40:17.418131+0000","last_undegraded":"2024-09-10T20:40:17.418131+0000","last_fullsized":"2024-09-10T20:40:17.418131+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418224+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:40:17.418224+0000","last_peered":"2024-09-10T20:40:17.418224+0000","last_clean":"2024-09-10T20:40:17.418224+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:40:17.418224+0000","last_undegraded":"2024-09-10T20:40:17.418224+0000","last_fullsized":"2024-09-10T20:40:17.418224+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27660,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417945+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:40:17.417945+0000","last_peered":"2024-09-10T20:40:17.417945+0000","last_clean":"2024-09-10T20:40:17.417945+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:40:17.417945+0000","last_undegraded":"2024-09-10T20:40:17.417945+0000","last_fullsized":"2024-09-10T20:40:17.417945+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15887,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27262,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":171520000,"data_stored":171487483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182543,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517180,"kb_used_data":57956,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854660,"statfs":{"total":96636764160,"available":96107171840,"internally_reserved":0,"allocated":59346944,"data_stored":59045876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986315,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":475328,"kb_used_data":110448,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93896512,"statfs":{"total":96636764160,"available":96150028288,"internally_reserved":0,"allocated":113098752,"data_stored":112801761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659262,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483392,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888448,"statfs":{"total":96636764160,"available":96141770752,"internally_reserved":0,"allocated":507904,"data_stored":185158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":112623616,"data_stored":112620003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:40:33.722 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:40:33.722 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'20658", 'reported_seq': 20898, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:17.416306+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:40:17.416306+0000', 'last_peered': '2024-09-10T20:40:17.416306+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:40:17.416306+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20658, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3541}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:33.723 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:40:33.723 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20471", 'reported_seq': 20741, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:17.414623+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:40:17.414623+0000', 'last_peered': '2024-09-10T20:40:17.414623+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:40:17.414623+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17335, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3561}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:33.723 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:40:33.723 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'20608", 'reported_seq': 20911, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:17.414726+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:40:17.414726+0000', 'last_peered': '2024-09-10T20:40:17.414726+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:40:17.414726+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17203, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3486}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:33.723 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:40:33.723 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20730", 'reported_seq': 24825, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:30.221656+0000', 'last_change': '2024-09-10T20:40:16.115711+0000', 'last_active': '2024-09-10T20:40:30.221656+0000', 'last_peered': '2024-09-10T20:40:30.221656+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:40:30.221656+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1638, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20728, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1981, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1636}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:33.723 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:40:33.723 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "51'20986", 'reported_seq': 21261, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:17.415576+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:40:17.415576+0000', 'last_peered': '2024-09-10T20:40:17.415576+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:40:17.415576+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17813, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3655}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:33.723 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:40:34.106 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:34.106 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 15887/10 objects degraded (158870.000%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":745,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":5,"data_bytes":1468,"bytes_used":1511321600,"bytes_avail":288398970880,"bytes_total":289910292480,"degraded_objects":15887,"degraded_total":10,"degraded_ratio":1588.7,"recovering_objects_per_sec":114,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1372,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:40:34.119 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:40:34.120 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:40:34.448 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:34.448 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:40:34.461 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":423,"stamp":"2024-09-10T20:40:33.068383+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15887,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27264,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1475900,"kb_used_data":168900,"kb_used_omap":12,"kb_used_meta":1306931,"kb_avail":281639620,"statfs":{"total":289910292480,"available":288398970880,"internally_reserved":0,"allocated":172953600,"data_stored":172032795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1338297346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1372,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002149"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20898,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.416306+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:40:17.416306+0000","last_peered":"2024-09-10T20:40:17.416306+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:40:17.416306+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21080,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418235+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:40:17.418235+0000","last_peered":"2024-09-10T20:40:17.418235+0000","last_clean":"2024-09-10T20:40:17.418235+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:40:17.418235+0000","last_undegraded":"2024-09-10T20:40:17.418235+0000","last_fullsized":"2024-09-10T20:40:17.418235+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28091,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418033+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:40:17.418033+0000","last_peered":"2024-09-10T20:40:17.418033+0000","last_clean":"2024-09-10T20:40:17.418033+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:40:17.418033+0000","last_undegraded":"2024-09-10T20:40:17.418033+0000","last_fullsized":"2024-09-10T20:40:17.418033+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20712,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418390+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:40:17.418390+0000","last_peered":"2024-09-10T20:40:17.418390+0000","last_clean":"2024-09-10T20:40:17.418390+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:40:17.418390+0000","last_undegraded":"2024-09-10T20:40:17.418390+0000","last_fullsized":"2024-09-10T20:40:17.418390+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21223,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.414948+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:40:17.414948+0000","last_peered":"2024-09-10T20:40:17.414948+0000","last_clean":"2024-09-10T20:40:17.414948+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:40:17.414948+0000","last_undegraded":"2024-09-10T20:40:17.414948+0000","last_fullsized":"2024-09-10T20:40:17.414948+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20741,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.414623+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:40:17.414623+0000","last_peered":"2024-09-10T20:40:17.414623+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:40:17.414623+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20911,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.414726+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:40:17.414726+0000","last_peered":"2024-09-10T20:40:17.414726+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:40:17.414726+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20888,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.414444+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:40:17.414444+0000","last_peered":"2024-09-10T20:40:17.414444+0000","last_clean":"2024-09-10T20:40:17.414444+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:40:17.414444+0000","last_undegraded":"2024-09-10T20:40:17.414444+0000","last_fullsized":"2024-09-10T20:40:17.414444+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417460+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:40:17.417460+0000","last_peered":"2024-09-10T20:40:17.417460+0000","last_clean":"2024-09-10T20:40:17.417460+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:40:17.417460+0000","last_undegraded":"2024-09-10T20:40:17.417460+0000","last_fullsized":"2024-09-10T20:40:17.417460+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22130,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417939+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:40:17.417939+0000","last_peered":"2024-09-10T20:40:17.417939+0000","last_clean":"2024-09-10T20:40:17.417939+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:40:17.417939+0000","last_undegraded":"2024-09-10T20:40:17.417939+0000","last_fullsized":"2024-09-10T20:40:17.417939+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418023+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:40:17.418023+0000","last_peered":"2024-09-10T20:40:17.418023+0000","last_clean":"2024-09-10T20:40:17.418023+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:40:17.418023+0000","last_undegraded":"2024-09-10T20:40:17.418023+0000","last_fullsized":"2024-09-10T20:40:17.418023+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":24825,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:30.221656+0000","last_change":"2024-09-10T20:40:16.115711+0000","last_active":"2024-09-10T20:40:30.221656+0000","last_peered":"2024-09-10T20:40:30.221656+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:40:30.221656+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1638,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1981,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1636}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":128,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.414539+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:40:17.414539+0000","last_peered":"2024-09-10T20:40:17.414539+0000","last_clean":"2024-09-10T20:40:17.414539+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:40:17.414539+0000","last_undegraded":"2024-09-10T20:40:17.414539+0000","last_fullsized":"2024-09-10T20:40:17.414539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":27999,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.475534+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:40:17.475534+0000","last_peered":"2024-09-10T20:40:17.475534+0000","last_clean":"2024-09-10T20:40:17.475534+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:40:17.475534+0000","last_undegraded":"2024-09-10T20:40:17.475534+0000","last_fullsized":"2024-09-10T20:40:17.475534+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418359+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:40:17.418359+0000","last_peered":"2024-09-10T20:40:17.418359+0000","last_clean":"2024-09-10T20:40:17.418359+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:40:17.418359+0000","last_undegraded":"2024-09-10T20:40:17.418359+0000","last_fullsized":"2024-09-10T20:40:17.418359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28052,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417580+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:40:17.417580+0000","last_peered":"2024-09-10T20:40:17.417580+0000","last_clean":"2024-09-10T20:40:17.417580+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:40:17.417580+0000","last_undegraded":"2024-09-10T20:40:17.417580+0000","last_fullsized":"2024-09-10T20:40:17.417580+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21109,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418310+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:40:17.418310+0000","last_peered":"2024-09-10T20:40:17.418310+0000","last_clean":"2024-09-10T20:40:17.418310+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:40:17.418310+0000","last_undegraded":"2024-09-10T20:40:17.418310+0000","last_fullsized":"2024-09-10T20:40:17.418310+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418073+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:40:17.418073+0000","last_peered":"2024-09-10T20:40:17.418073+0000","last_clean":"2024-09-10T20:40:17.418073+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:40:17.418073+0000","last_undegraded":"2024-09-10T20:40:17.418073+0000","last_fullsized":"2024-09-10T20:40:17.418073+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27830,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418188+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:40:17.418188+0000","last_peered":"2024-09-10T20:40:17.418188+0000","last_clean":"2024-09-10T20:40:17.418188+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:40:17.418188+0000","last_undegraded":"2024-09-10T20:40:17.418188+0000","last_fullsized":"2024-09-10T20:40:17.418188+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417903+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:40:17.417903+0000","last_peered":"2024-09-10T20:40:17.417903+0000","last_clean":"2024-09-10T20:40:17.417903+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:40:17.417903+0000","last_undegraded":"2024-09-10T20:40:17.417903+0000","last_fullsized":"2024-09-10T20:40:17.417903+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21261,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.415576+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:40:17.415576+0000","last_peered":"2024-09-10T20:40:17.415576+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:40:17.415576+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418131+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:40:17.418131+0000","last_peered":"2024-09-10T20:40:17.418131+0000","last_clean":"2024-09-10T20:40:17.418131+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:40:17.418131+0000","last_undegraded":"2024-09-10T20:40:17.418131+0000","last_fullsized":"2024-09-10T20:40:17.418131+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418224+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:40:17.418224+0000","last_peered":"2024-09-10T20:40:17.418224+0000","last_clean":"2024-09-10T20:40:17.418224+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:40:17.418224+0000","last_undegraded":"2024-09-10T20:40:17.418224+0000","last_fullsized":"2024-09-10T20:40:17.418224+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27660,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417945+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:40:17.417945+0000","last_peered":"2024-09-10T20:40:17.417945+0000","last_clean":"2024-09-10T20:40:17.417945+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:40:17.417945+0000","last_undegraded":"2024-09-10T20:40:17.417945+0000","last_fullsized":"2024-09-10T20:40:17.417945+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15887,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27262,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":171520000,"data_stored":171487483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182543,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517180,"kb_used_data":57956,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854660,"statfs":{"total":96636764160,"available":96107171840,"internally_reserved":0,"allocated":59346944,"data_stored":59045876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986315,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":475328,"kb_used_data":110448,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93896512,"statfs":{"total":96636764160,"available":96150028288,"internally_reserved":0,"allocated":113098752,"data_stored":112801761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659262,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483392,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888448,"statfs":{"total":96636764160,"available":96141770752,"internally_reserved":0,"allocated":507904,"data_stored":185158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":112623616,"data_stored":112620003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:40:37.463 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:40:37.765 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:37.766 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:40:37.778 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":425,"stamp":"2024-09-10T20:40:37.069096+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14918,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28233,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1474100,"kb_used_data":167100,"kb_used_omap":12,"kb_used_meta":1306931,"kb_avail":281641420,"statfs":{"total":289910292480,"available":288400814080,"internally_reserved":0,"allocated":171110400,"data_stored":170189595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1338297346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2341,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002083"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20898,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.416306+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:40:17.416306+0000","last_peered":"2024-09-10T20:40:17.416306+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:40:17.416306+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21080,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418235+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:40:17.418235+0000","last_peered":"2024-09-10T20:40:17.418235+0000","last_clean":"2024-09-10T20:40:17.418235+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:40:17.418235+0000","last_undegraded":"2024-09-10T20:40:17.418235+0000","last_fullsized":"2024-09-10T20:40:17.418235+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28091,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418033+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:40:17.418033+0000","last_peered":"2024-09-10T20:40:17.418033+0000","last_clean":"2024-09-10T20:40:17.418033+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:40:17.418033+0000","last_undegraded":"2024-09-10T20:40:17.418033+0000","last_fullsized":"2024-09-10T20:40:17.418033+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20712,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418390+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:40:17.418390+0000","last_peered":"2024-09-10T20:40:17.418390+0000","last_clean":"2024-09-10T20:40:17.418390+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:40:17.418390+0000","last_undegraded":"2024-09-10T20:40:17.418390+0000","last_fullsized":"2024-09-10T20:40:17.418390+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21223,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.414948+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:40:17.414948+0000","last_peered":"2024-09-10T20:40:17.414948+0000","last_clean":"2024-09-10T20:40:17.414948+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:40:17.414948+0000","last_undegraded":"2024-09-10T20:40:17.414948+0000","last_fullsized":"2024-09-10T20:40:17.414948+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20741,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.414623+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:40:17.414623+0000","last_peered":"2024-09-10T20:40:17.414623+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:40:17.414623+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20911,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.414726+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:40:17.414726+0000","last_peered":"2024-09-10T20:40:17.414726+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:40:17.414726+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20888,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.414444+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:40:17.414444+0000","last_peered":"2024-09-10T20:40:17.414444+0000","last_clean":"2024-09-10T20:40:17.414444+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:40:17.414444+0000","last_undegraded":"2024-09-10T20:40:17.414444+0000","last_fullsized":"2024-09-10T20:40:17.414444+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417460+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:40:17.417460+0000","last_peered":"2024-09-10T20:40:17.417460+0000","last_clean":"2024-09-10T20:40:17.417460+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:40:17.417460+0000","last_undegraded":"2024-09-10T20:40:17.417460+0000","last_fullsized":"2024-09-10T20:40:17.417460+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22130,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417939+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:40:17.417939+0000","last_peered":"2024-09-10T20:40:17.417939+0000","last_clean":"2024-09-10T20:40:17.417939+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:40:17.417939+0000","last_undegraded":"2024-09-10T20:40:17.417939+0000","last_fullsized":"2024-09-10T20:40:17.417939+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418023+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:40:17.418023+0000","last_peered":"2024-09-10T20:40:17.418023+0000","last_clean":"2024-09-10T20:40:17.418023+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:40:17.418023+0000","last_undegraded":"2024-09-10T20:40:17.418023+0000","last_fullsized":"2024-09-10T20:40:17.418023+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":26763,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:35.225778+0000","last_change":"2024-09-10T20:40:16.115711+0000","last_active":"2024-09-10T20:40:35.225778+0000","last_peered":"2024-09-10T20:40:35.225778+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:40:35.225778+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":669,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2950,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-667}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":128,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.414539+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:40:17.414539+0000","last_peered":"2024-09-10T20:40:17.414539+0000","last_clean":"2024-09-10T20:40:17.414539+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:40:17.414539+0000","last_undegraded":"2024-09-10T20:40:17.414539+0000","last_fullsized":"2024-09-10T20:40:17.414539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":27999,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.475534+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:40:17.475534+0000","last_peered":"2024-09-10T20:40:17.475534+0000","last_clean":"2024-09-10T20:40:17.475534+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:40:17.475534+0000","last_undegraded":"2024-09-10T20:40:17.475534+0000","last_fullsized":"2024-09-10T20:40:17.475534+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418359+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:40:17.418359+0000","last_peered":"2024-09-10T20:40:17.418359+0000","last_clean":"2024-09-10T20:40:17.418359+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:40:17.418359+0000","last_undegraded":"2024-09-10T20:40:17.418359+0000","last_fullsized":"2024-09-10T20:40:17.418359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28052,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417580+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:40:17.417580+0000","last_peered":"2024-09-10T20:40:17.417580+0000","last_clean":"2024-09-10T20:40:17.417580+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:40:17.417580+0000","last_undegraded":"2024-09-10T20:40:17.417580+0000","last_fullsized":"2024-09-10T20:40:17.417580+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21109,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418310+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:40:17.418310+0000","last_peered":"2024-09-10T20:40:17.418310+0000","last_clean":"2024-09-10T20:40:17.418310+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:40:17.418310+0000","last_undegraded":"2024-09-10T20:40:17.418310+0000","last_fullsized":"2024-09-10T20:40:17.418310+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418073+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:40:17.418073+0000","last_peered":"2024-09-10T20:40:17.418073+0000","last_clean":"2024-09-10T20:40:17.418073+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:40:17.418073+0000","last_undegraded":"2024-09-10T20:40:17.418073+0000","last_fullsized":"2024-09-10T20:40:17.418073+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27830,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418188+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:40:17.418188+0000","last_peered":"2024-09-10T20:40:17.418188+0000","last_clean":"2024-09-10T20:40:17.418188+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:40:17.418188+0000","last_undegraded":"2024-09-10T20:40:17.418188+0000","last_fullsized":"2024-09-10T20:40:17.418188+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417903+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:40:17.417903+0000","last_peered":"2024-09-10T20:40:17.417903+0000","last_clean":"2024-09-10T20:40:17.417903+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:40:17.417903+0000","last_undegraded":"2024-09-10T20:40:17.417903+0000","last_fullsized":"2024-09-10T20:40:17.417903+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21261,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.415576+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:40:17.415576+0000","last_peered":"2024-09-10T20:40:17.415576+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:40:17.415576+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418131+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:40:17.418131+0000","last_peered":"2024-09-10T20:40:17.418131+0000","last_clean":"2024-09-10T20:40:17.418131+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:40:17.418131+0000","last_undegraded":"2024-09-10T20:40:17.418131+0000","last_fullsized":"2024-09-10T20:40:17.418131+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418224+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:40:17.418224+0000","last_peered":"2024-09-10T20:40:17.418224+0000","last_clean":"2024-09-10T20:40:17.418224+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:40:17.418224+0000","last_undegraded":"2024-09-10T20:40:17.418224+0000","last_fullsized":"2024-09-10T20:40:17.418224+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27660,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417945+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:40:17.417945+0000","last_peered":"2024-09-10T20:40:17.417945+0000","last_clean":"2024-09-10T20:40:17.417945+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:40:17.417945+0000","last_undegraded":"2024-09-10T20:40:17.417945+0000","last_fullsized":"2024-09-10T20:40:17.417945+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14918,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28231,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":169676800,"data_stored":169644283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182544,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517180,"kb_used_data":57956,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854660,"statfs":{"total":96636764160,"available":96107171840,"internally_reserved":0,"allocated":59346944,"data_stored":59045876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986316,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473528,"kb_used_data":108648,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93898312,"statfs":{"total":96636764160,"available":96151871488,"internally_reserved":0,"allocated":111255552,"data_stored":110958561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659262,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483392,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888448,"statfs":{"total":96636764160,"available":96141770752,"internally_reserved":0,"allocated":507904,"data_stored":185158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110780416,"data_stored":110776803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:40:37.780 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:40:38.074 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:38.075 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:40:38.087 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":425,"stamp":"2024-09-10T20:40:37.069096+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14918,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28233,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1474100,"kb_used_data":167100,"kb_used_omap":12,"kb_used_meta":1306931,"kb_avail":281641420,"statfs":{"total":289910292480,"available":288400814080,"internally_reserved":0,"allocated":171110400,"data_stored":170189595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1338297346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2341,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002083"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20898,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.416306+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:40:17.416306+0000","last_peered":"2024-09-10T20:40:17.416306+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:40:17.416306+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21080,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418235+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:40:17.418235+0000","last_peered":"2024-09-10T20:40:17.418235+0000","last_clean":"2024-09-10T20:40:17.418235+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:40:17.418235+0000","last_undegraded":"2024-09-10T20:40:17.418235+0000","last_fullsized":"2024-09-10T20:40:17.418235+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28091,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418033+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:40:17.418033+0000","last_peered":"2024-09-10T20:40:17.418033+0000","last_clean":"2024-09-10T20:40:17.418033+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:40:17.418033+0000","last_undegraded":"2024-09-10T20:40:17.418033+0000","last_fullsized":"2024-09-10T20:40:17.418033+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20712,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418390+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:40:17.418390+0000","last_peered":"2024-09-10T20:40:17.418390+0000","last_clean":"2024-09-10T20:40:17.418390+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:40:17.418390+0000","last_undegraded":"2024-09-10T20:40:17.418390+0000","last_fullsized":"2024-09-10T20:40:17.418390+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21223,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.414948+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:40:17.414948+0000","last_peered":"2024-09-10T20:40:17.414948+0000","last_clean":"2024-09-10T20:40:17.414948+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:40:17.414948+0000","last_undegraded":"2024-09-10T20:40:17.414948+0000","last_fullsized":"2024-09-10T20:40:17.414948+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20741,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.414623+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:40:17.414623+0000","last_peered":"2024-09-10T20:40:17.414623+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:40:17.414623+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20911,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.414726+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:40:17.414726+0000","last_peered":"2024-09-10T20:40:17.414726+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:40:17.414726+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20888,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.414444+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:40:17.414444+0000","last_peered":"2024-09-10T20:40:17.414444+0000","last_clean":"2024-09-10T20:40:17.414444+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:40:17.414444+0000","last_undegraded":"2024-09-10T20:40:17.414444+0000","last_fullsized":"2024-09-10T20:40:17.414444+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417460+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:40:17.417460+0000","last_peered":"2024-09-10T20:40:17.417460+0000","last_clean":"2024-09-10T20:40:17.417460+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:40:17.417460+0000","last_undegraded":"2024-09-10T20:40:17.417460+0000","last_fullsized":"2024-09-10T20:40:17.417460+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22130,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417939+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:40:17.417939+0000","last_peered":"2024-09-10T20:40:17.417939+0000","last_clean":"2024-09-10T20:40:17.417939+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:40:17.417939+0000","last_undegraded":"2024-09-10T20:40:17.417939+0000","last_fullsized":"2024-09-10T20:40:17.417939+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418023+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:40:17.418023+0000","last_peered":"2024-09-10T20:40:17.418023+0000","last_clean":"2024-09-10T20:40:17.418023+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:40:17.418023+0000","last_undegraded":"2024-09-10T20:40:17.418023+0000","last_fullsized":"2024-09-10T20:40:17.418023+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":26763,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:35.225778+0000","last_change":"2024-09-10T20:40:16.115711+0000","last_active":"2024-09-10T20:40:35.225778+0000","last_peered":"2024-09-10T20:40:35.225778+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:40:35.225778+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":669,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2950,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-667}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":128,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.414539+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:40:17.414539+0000","last_peered":"2024-09-10T20:40:17.414539+0000","last_clean":"2024-09-10T20:40:17.414539+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:40:17.414539+0000","last_undegraded":"2024-09-10T20:40:17.414539+0000","last_fullsized":"2024-09-10T20:40:17.414539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":27999,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.475534+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:40:17.475534+0000","last_peered":"2024-09-10T20:40:17.475534+0000","last_clean":"2024-09-10T20:40:17.475534+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:40:17.475534+0000","last_undegraded":"2024-09-10T20:40:17.475534+0000","last_fullsized":"2024-09-10T20:40:17.475534+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418359+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:40:17.418359+0000","last_peered":"2024-09-10T20:40:17.418359+0000","last_clean":"2024-09-10T20:40:17.418359+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:40:17.418359+0000","last_undegraded":"2024-09-10T20:40:17.418359+0000","last_fullsized":"2024-09-10T20:40:17.418359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28052,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417580+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:40:17.417580+0000","last_peered":"2024-09-10T20:40:17.417580+0000","last_clean":"2024-09-10T20:40:17.417580+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:40:17.417580+0000","last_undegraded":"2024-09-10T20:40:17.417580+0000","last_fullsized":"2024-09-10T20:40:17.417580+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21109,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418310+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:40:17.418310+0000","last_peered":"2024-09-10T20:40:17.418310+0000","last_clean":"2024-09-10T20:40:17.418310+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:40:17.418310+0000","last_undegraded":"2024-09-10T20:40:17.418310+0000","last_fullsized":"2024-09-10T20:40:17.418310+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418073+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:40:17.418073+0000","last_peered":"2024-09-10T20:40:17.418073+0000","last_clean":"2024-09-10T20:40:17.418073+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:40:17.418073+0000","last_undegraded":"2024-09-10T20:40:17.418073+0000","last_fullsized":"2024-09-10T20:40:17.418073+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27830,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418188+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:40:17.418188+0000","last_peered":"2024-09-10T20:40:17.418188+0000","last_clean":"2024-09-10T20:40:17.418188+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:40:17.418188+0000","last_undegraded":"2024-09-10T20:40:17.418188+0000","last_fullsized":"2024-09-10T20:40:17.418188+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417903+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:40:17.417903+0000","last_peered":"2024-09-10T20:40:17.417903+0000","last_clean":"2024-09-10T20:40:17.417903+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:40:17.417903+0000","last_undegraded":"2024-09-10T20:40:17.417903+0000","last_fullsized":"2024-09-10T20:40:17.417903+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21261,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.415576+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:40:17.415576+0000","last_peered":"2024-09-10T20:40:17.415576+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:40:17.415576+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418131+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:40:17.418131+0000","last_peered":"2024-09-10T20:40:17.418131+0000","last_clean":"2024-09-10T20:40:17.418131+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:40:17.418131+0000","last_undegraded":"2024-09-10T20:40:17.418131+0000","last_fullsized":"2024-09-10T20:40:17.418131+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418224+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:40:17.418224+0000","last_peered":"2024-09-10T20:40:17.418224+0000","last_clean":"2024-09-10T20:40:17.418224+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:40:17.418224+0000","last_undegraded":"2024-09-10T20:40:17.418224+0000","last_fullsized":"2024-09-10T20:40:17.418224+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27660,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417945+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:40:17.417945+0000","last_peered":"2024-09-10T20:40:17.417945+0000","last_clean":"2024-09-10T20:40:17.417945+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:40:17.417945+0000","last_undegraded":"2024-09-10T20:40:17.417945+0000","last_fullsized":"2024-09-10T20:40:17.417945+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14918,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28231,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":169676800,"data_stored":169644283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182544,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517180,"kb_used_data":57956,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854660,"statfs":{"total":96636764160,"available":96107171840,"internally_reserved":0,"allocated":59346944,"data_stored":59045876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986316,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473528,"kb_used_data":108648,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93898312,"statfs":{"total":96636764160,"available":96151871488,"internally_reserved":0,"allocated":111255552,"data_stored":110958561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659262,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483392,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888448,"statfs":{"total":96636764160,"available":96141770752,"internally_reserved":0,"allocated":507904,"data_stored":185158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110780416,"data_stored":110776803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:40:38.088 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:40:38.089 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'20658", 'reported_seq': 20898, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:17.416306+0000', 'last_change': '2024-09-10T20:30:05.767830+0000', 'last_active': '2024-09-10T20:40:17.416306+0000', 'last_peered': '2024-09-10T20:40:17.416306+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:40:17.416306+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20658, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3541}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:38.089 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:40:38.089 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20471", 'reported_seq': 20741, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:17.414623+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:40:17.414623+0000', 'last_peered': '2024-09-10T20:40:17.414623+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:40:17.414623+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17335, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3561}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:38.089 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:40:38.089 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'20608", 'reported_seq': 20911, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:17.414726+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:40:17.414726+0000', 'last_peered': '2024-09-10T20:40:17.414726+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:40:17.414726+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17203, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3486}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:38.089 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T20:40:38.089 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20730", 'reported_seq': 26763, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:35.225778+0000', 'last_change': '2024-09-10T20:40:16.115711+0000', 'last_active': '2024-09-10T20:40:35.225778+0000', 'last_peered': '2024-09-10T20:40:35.225778+0000', 'last_clean': '2024-09-10T20:30:03.306543+0000', 'last_became_active': '2024-09-10T20:30:05.760475+0000', 'last_became_peered': '2024-09-10T20:30:05.760475+0000', 'last_unstale': '2024-09-10T20:40:35.225778+0000', 'last_undegraded': '2024-09-10T20:30:05.392774+0000', 'last_fullsized': '2024-09-10T20:30:05.306720+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 2099, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 669, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20728, 'num_write_kb': 41448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2950, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -667}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:38.089 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:40:38.089 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "51'20986", 'reported_seq': 21261, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:17.415576+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:40:17.415576+0000', 'last_peered': '2024-09-10T20:40:17.415576+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:40:17.415576+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17813, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3655}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:38.090 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:40:38.511 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:38.512 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 14918/10 objects degraded (149180.000%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":750,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":58,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":5,"data_bytes":1468,"bytes_used":1509478400,"bytes_avail":288400814080,"bytes_total":289910292480,"degraded_objects":14918,"degraded_total":10,"degraded_ratio":1491.8,"recovering_objects_per_sec":195,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2341,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:40:38.526 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:40:38.526 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:40:38.896 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:38.897 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:40:38.910 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":426,"stamp":"2024-09-10T20:40:38.456643+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14918,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295416,"num_write_kb":519563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28233,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1482420,"kb_used_data":167100,"kb_used_omap":12,"kb_used_meta":1315251,"kb_avail":281633100,"statfs":{"total":289910292480,"available":288392294400,"internally_reserved":0,"allocated":171110400,"data_stored":170189595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1346817026},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1716,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1716,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.389248"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":20898,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.416306+0000","last_change":"2024-09-10T20:30:05.767830+0000","last_active":"2024-09-10T20:40:17.416306+0000","last_peered":"2024-09-10T20:40:17.416306+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:40:17.416306+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21080,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418235+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:40:17.418235+0000","last_peered":"2024-09-10T20:40:17.418235+0000","last_clean":"2024-09-10T20:40:17.418235+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:40:17.418235+0000","last_undegraded":"2024-09-10T20:40:17.418235+0000","last_fullsized":"2024-09-10T20:40:17.418235+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28091,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418033+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:40:17.418033+0000","last_peered":"2024-09-10T20:40:17.418033+0000","last_clean":"2024-09-10T20:40:17.418033+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:40:17.418033+0000","last_undegraded":"2024-09-10T20:40:17.418033+0000","last_fullsized":"2024-09-10T20:40:17.418033+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20712,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418390+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:40:17.418390+0000","last_peered":"2024-09-10T20:40:17.418390+0000","last_clean":"2024-09-10T20:40:17.418390+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:40:17.418390+0000","last_undegraded":"2024-09-10T20:40:17.418390+0000","last_fullsized":"2024-09-10T20:40:17.418390+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21223,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.414948+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:40:17.414948+0000","last_peered":"2024-09-10T20:40:17.414948+0000","last_clean":"2024-09-10T20:40:17.414948+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:40:17.414948+0000","last_undegraded":"2024-09-10T20:40:17.414948+0000","last_fullsized":"2024-09-10T20:40:17.414948+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20741,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.414623+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:40:17.414623+0000","last_peered":"2024-09-10T20:40:17.414623+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:40:17.414623+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20911,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.414726+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:40:17.414726+0000","last_peered":"2024-09-10T20:40:17.414726+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:40:17.414726+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20888,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.414444+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:40:17.414444+0000","last_peered":"2024-09-10T20:40:17.414444+0000","last_clean":"2024-09-10T20:40:17.414444+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:40:17.414444+0000","last_undegraded":"2024-09-10T20:40:17.414444+0000","last_fullsized":"2024-09-10T20:40:17.414444+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417460+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:40:17.417460+0000","last_peered":"2024-09-10T20:40:17.417460+0000","last_clean":"2024-09-10T20:40:17.417460+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:40:17.417460+0000","last_undegraded":"2024-09-10T20:40:17.417460+0000","last_fullsized":"2024-09-10T20:40:17.417460+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22130,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417939+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:40:17.417939+0000","last_peered":"2024-09-10T20:40:17.417939+0000","last_clean":"2024-09-10T20:40:17.417939+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:40:17.417939+0000","last_undegraded":"2024-09-10T20:40:17.417939+0000","last_fullsized":"2024-09-10T20:40:17.417939+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418023+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:40:17.418023+0000","last_peered":"2024-09-10T20:40:17.418023+0000","last_clean":"2024-09-10T20:40:17.418023+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:40:17.418023+0000","last_undegraded":"2024-09-10T20:40:17.418023+0000","last_fullsized":"2024-09-10T20:40:17.418023+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":26763,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:35.225778+0000","last_change":"2024-09-10T20:40:16.115711+0000","last_active":"2024-09-10T20:40:35.225778+0000","last_peered":"2024-09-10T20:40:35.225778+0000","last_clean":"2024-09-10T20:30:03.306543+0000","last_became_active":"2024-09-10T20:30:05.760475+0000","last_became_peered":"2024-09-10T20:30:05.760475+0000","last_unstale":"2024-09-10T20:40:35.225778+0000","last_undegraded":"2024-09-10T20:30:05.392774+0000","last_fullsized":"2024-09-10T20:30:05.306720+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":2099,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":669,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20728,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2950,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-667}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":128,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.414539+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:40:17.414539+0000","last_peered":"2024-09-10T20:40:17.414539+0000","last_clean":"2024-09-10T20:40:17.414539+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:40:17.414539+0000","last_undegraded":"2024-09-10T20:40:17.414539+0000","last_fullsized":"2024-09-10T20:40:17.414539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":27999,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.475534+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:40:17.475534+0000","last_peered":"2024-09-10T20:40:17.475534+0000","last_clean":"2024-09-10T20:40:17.475534+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:40:17.475534+0000","last_undegraded":"2024-09-10T20:40:17.475534+0000","last_fullsized":"2024-09-10T20:40:17.475534+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418359+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:40:17.418359+0000","last_peered":"2024-09-10T20:40:17.418359+0000","last_clean":"2024-09-10T20:40:17.418359+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:40:17.418359+0000","last_undegraded":"2024-09-10T20:40:17.418359+0000","last_fullsized":"2024-09-10T20:40:17.418359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28052,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417580+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:40:17.417580+0000","last_peered":"2024-09-10T20:40:17.417580+0000","last_clean":"2024-09-10T20:40:17.417580+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:40:17.417580+0000","last_undegraded":"2024-09-10T20:40:17.417580+0000","last_fullsized":"2024-09-10T20:40:17.417580+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21109,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418310+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:40:17.418310+0000","last_peered":"2024-09-10T20:40:17.418310+0000","last_clean":"2024-09-10T20:40:17.418310+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:40:17.418310+0000","last_undegraded":"2024-09-10T20:40:17.418310+0000","last_fullsized":"2024-09-10T20:40:17.418310+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418073+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:40:17.418073+0000","last_peered":"2024-09-10T20:40:17.418073+0000","last_clean":"2024-09-10T20:40:17.418073+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:40:17.418073+0000","last_undegraded":"2024-09-10T20:40:17.418073+0000","last_fullsized":"2024-09-10T20:40:17.418073+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27830,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418188+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:40:17.418188+0000","last_peered":"2024-09-10T20:40:17.418188+0000","last_clean":"2024-09-10T20:40:17.418188+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:40:17.418188+0000","last_undegraded":"2024-09-10T20:40:17.418188+0000","last_fullsized":"2024-09-10T20:40:17.418188+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417903+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:40:17.417903+0000","last_peered":"2024-09-10T20:40:17.417903+0000","last_clean":"2024-09-10T20:40:17.417903+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:40:17.417903+0000","last_undegraded":"2024-09-10T20:40:17.417903+0000","last_fullsized":"2024-09-10T20:40:17.417903+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21261,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:17.415576+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:40:17.415576+0000","last_peered":"2024-09-10T20:40:17.415576+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:40:17.415576+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418131+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:40:17.418131+0000","last_peered":"2024-09-10T20:40:17.418131+0000","last_clean":"2024-09-10T20:40:17.418131+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:40:17.418131+0000","last_undegraded":"2024-09-10T20:40:17.418131+0000","last_fullsized":"2024-09-10T20:40:17.418131+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418224+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:40:17.418224+0000","last_peered":"2024-09-10T20:40:17.418224+0000","last_clean":"2024-09-10T20:40:17.418224+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:40:17.418224+0000","last_undegraded":"2024-09-10T20:40:17.418224+0000","last_fullsized":"2024-09-10T20:40:17.418224+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27660,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417945+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:40:17.417945+0000","last_peered":"2024-09-10T20:40:17.417945+0000","last_clean":"2024-09-10T20:40:17.417945+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:40:17.417945+0000","last_undegraded":"2024-09-10T20:40:17.417945+0000","last_fullsized":"2024-09-10T20:40:17.417945+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14918,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":295414,"num_write_kb":519561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28231,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":169676800,"data_stored":169644283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182544,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517180,"kb_used_data":57956,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854660,"statfs":{"total":96636764160,"available":96107171840,"internally_reserved":0,"allocated":59346944,"data_stored":59045876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986316,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473528,"kb_used_data":108648,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93898312,"statfs":{"total":96636764160,"available":96151871488,"internally_reserved":0,"allocated":111255552,"data_stored":110958561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659263,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491712,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880128,"statfs":{"total":96636764160,"available":96133251072,"internally_reserved":0,"allocated":507904,"data_stored":185158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502984366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110780416,"data_stored":110776803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:40:41.913 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:40:42.267 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:42.268 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:40:42.280 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":429,"stamp":"2024-09-10T20:40:41.069709+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13899,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25633,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1480540,"kb_used_data":165216,"kb_used_omap":12,"kb_used_meta":1315251,"kb_avail":281634980,"statfs":{"total":289910292480,"available":288394219520,"internally_reserved":0,"allocated":169181184,"data_stored":168240569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1346817026},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-966,"num_objects":-2,"num_object_clones":0,"num_object_copies":-4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-20728,"num_write_kb":-41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001325"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":21603,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:40.227052+0000","last_change":"2024-09-10T20:40:37.857265+0000","last_active":"2024-09-10T20:40:40.227052+0000","last_peered":"2024-09-10T20:40:40.227052+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:40:40.227052+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3191,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":417,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3191}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21084,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.463153+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:40:39.463153+0000","last_peered":"2024-09-10T20:40:39.463153+0000","last_clean":"2024-09-10T20:40:39.463153+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:40:39.463153+0000","last_undegraded":"2024-09-10T20:40:39.463153+0000","last_fullsized":"2024-09-10T20:40:39.463153+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28091,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418033+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:40:17.418033+0000","last_peered":"2024-09-10T20:40:17.418033+0000","last_clean":"2024-09-10T20:40:17.418033+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:40:17.418033+0000","last_undegraded":"2024-09-10T20:40:17.418033+0000","last_fullsized":"2024-09-10T20:40:17.418033+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20716,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.463009+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:40:39.463009+0000","last_peered":"2024-09-10T20:40:39.463009+0000","last_clean":"2024-09-10T20:40:39.463009+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:40:39.463009+0000","last_undegraded":"2024-09-10T20:40:39.463009+0000","last_fullsized":"2024-09-10T20:40:39.463009+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21227,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462338+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:40:39.462338+0000","last_peered":"2024-09-10T20:40:39.462338+0000","last_clean":"2024-09-10T20:40:39.462338+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:40:39.462338+0000","last_undegraded":"2024-09-10T20:40:39.462338+0000","last_fullsized":"2024-09-10T20:40:39.462338+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20745,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.461976+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:40:39.461976+0000","last_peered":"2024-09-10T20:40:39.461976+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:40:39.461976+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20915,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.462003+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:40:39.462003+0000","last_peered":"2024-09-10T20:40:39.462003+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:40:39.462003+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20892,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461791+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:40:39.461791+0000","last_peered":"2024-09-10T20:40:39.461791+0000","last_clean":"2024-09-10T20:40:39.461791+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:40:39.461791+0000","last_undegraded":"2024-09-10T20:40:39.461791+0000","last_fullsized":"2024-09-10T20:40:39.461791+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417460+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:40:17.417460+0000","last_peered":"2024-09-10T20:40:17.417460+0000","last_clean":"2024-09-10T20:40:17.417460+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:40:17.417460+0000","last_undegraded":"2024-09-10T20:40:17.417460+0000","last_fullsized":"2024-09-10T20:40:17.417460+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22130,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417939+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:40:17.417939+0000","last_peered":"2024-09-10T20:40:17.417939+0000","last_clean":"2024-09-10T20:40:17.417939+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:40:17.417939+0000","last_undegraded":"2024-09-10T20:40:17.417939+0000","last_fullsized":"2024-09-10T20:40:17.417939+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462542+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:40:39.462542+0000","last_peered":"2024-09-10T20:40:39.462542+0000","last_clean":"2024-09-10T20:40:39.462542+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:40:39.462542+0000","last_undegraded":"2024-09-10T20:40:39.462542+0000","last_fullsized":"2024-09-10T20:40:39.462542+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28120,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.949385+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:40:39.949385+0000","last_peered":"2024-09-10T20:40:39.949385+0000","last_clean":"2024-09-10T20:40:39.949385+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:40:39.949385+0000","last_undegraded":"2024-09-10T20:40:39.949385+0000","last_fullsized":"2024-09-10T20:40:39.949385+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":132,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461665+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:40:39.461665+0000","last_peered":"2024-09-10T20:40:39.461665+0000","last_clean":"2024-09-10T20:40:39.461665+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:40:39.461665+0000","last_undegraded":"2024-09-10T20:40:39.461665+0000","last_fullsized":"2024-09-10T20:40:39.461665+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28003,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462607+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:40:39.462607+0000","last_peered":"2024-09-10T20:40:39.462607+0000","last_clean":"2024-09-10T20:40:39.462607+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:40:39.462607+0000","last_undegraded":"2024-09-10T20:40:39.462607+0000","last_fullsized":"2024-09-10T20:40:39.462607+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418359+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:40:17.418359+0000","last_peered":"2024-09-10T20:40:17.418359+0000","last_clean":"2024-09-10T20:40:17.418359+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:40:17.418359+0000","last_undegraded":"2024-09-10T20:40:17.418359+0000","last_fullsized":"2024-09-10T20:40:17.418359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28056,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461762+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:40:39.461762+0000","last_peered":"2024-09-10T20:40:39.461762+0000","last_clean":"2024-09-10T20:40:39.461762+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:40:39.461762+0000","last_undegraded":"2024-09-10T20:40:39.461762+0000","last_fullsized":"2024-09-10T20:40:39.461762+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21109,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418310+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:40:17.418310+0000","last_peered":"2024-09-10T20:40:17.418310+0000","last_clean":"2024-09-10T20:40:17.418310+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:40:17.418310+0000","last_undegraded":"2024-09-10T20:40:17.418310+0000","last_fullsized":"2024-09-10T20:40:17.418310+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418073+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:40:17.418073+0000","last_peered":"2024-09-10T20:40:17.418073+0000","last_clean":"2024-09-10T20:40:17.418073+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:40:17.418073+0000","last_undegraded":"2024-09-10T20:40:17.418073+0000","last_fullsized":"2024-09-10T20:40:17.418073+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27830,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418188+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:40:17.418188+0000","last_peered":"2024-09-10T20:40:17.418188+0000","last_clean":"2024-09-10T20:40:17.418188+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:40:17.418188+0000","last_undegraded":"2024-09-10T20:40:17.418188+0000","last_fullsized":"2024-09-10T20:40:17.418188+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":125,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462414+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:40:39.462414+0000","last_peered":"2024-09-10T20:40:39.462414+0000","last_clean":"2024-09-10T20:40:39.462414+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:40:39.462414+0000","last_undegraded":"2024-09-10T20:40:39.462414+0000","last_fullsized":"2024-09-10T20:40:39.462414+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21265,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.463026+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:40:39.463026+0000","last_peered":"2024-09-10T20:40:39.463026+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:40:39.463026+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418131+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:40:17.418131+0000","last_peered":"2024-09-10T20:40:17.418131+0000","last_clean":"2024-09-10T20:40:17.418131+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:40:17.418131+0000","last_undegraded":"2024-09-10T20:40:17.418131+0000","last_fullsized":"2024-09-10T20:40:17.418131+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418224+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:40:17.418224+0000","last_peered":"2024-09-10T20:40:17.418224+0000","last_clean":"2024-09-10T20:40:17.418224+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:40:17.418224+0000","last_undegraded":"2024-09-10T20:40:17.418224+0000","last_fullsized":"2024-09-10T20:40:17.418224+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27660,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417945+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:40:17.417945+0000","last_peered":"2024-09-10T20:40:17.417945+0000","last_clean":"2024-09-10T20:40:17.417945+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:40:17.417945+0000","last_undegraded":"2024-09-10T20:40:17.417945+0000","last_fullsized":"2024-09-10T20:40:17.417945+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13899,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25631,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":167714816,"data_stored":167682299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182545,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517192,"kb_used_data":57972,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854648,"statfs":{"total":96636764160,"available":96107159552,"internally_reserved":0,"allocated":59363328,"data_stored":59052355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986317,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":471636,"kb_used_data":106748,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93900204,"statfs":{"total":96636764160,"available":96153808896,"internally_reserved":0,"allocated":109309952,"data_stored":109003056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659263,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491712,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880128,"statfs":{"total":96636764160,"available":96133251072,"internally_reserved":0,"allocated":507904,"data_stored":185158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502984366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":108818432,"data_stored":108814819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:40:42.282 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:40:42.642 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:42.643 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:40:42.655 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":429,"stamp":"2024-09-10T20:40:41.069709+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13899,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25633,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1480540,"kb_used_data":165216,"kb_used_omap":12,"kb_used_meta":1315251,"kb_avail":281634980,"statfs":{"total":289910292480,"available":288394219520,"internally_reserved":0,"allocated":169181184,"data_stored":168240569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1346817026},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-966,"num_objects":-2,"num_object_clones":0,"num_object_copies":-4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-20728,"num_write_kb":-41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001325"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":21603,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:40.227052+0000","last_change":"2024-09-10T20:40:37.857265+0000","last_active":"2024-09-10T20:40:40.227052+0000","last_peered":"2024-09-10T20:40:40.227052+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:40:40.227052+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3191,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":417,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3191}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21084,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.463153+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:40:39.463153+0000","last_peered":"2024-09-10T20:40:39.463153+0000","last_clean":"2024-09-10T20:40:39.463153+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:40:39.463153+0000","last_undegraded":"2024-09-10T20:40:39.463153+0000","last_fullsized":"2024-09-10T20:40:39.463153+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28091,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418033+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:40:17.418033+0000","last_peered":"2024-09-10T20:40:17.418033+0000","last_clean":"2024-09-10T20:40:17.418033+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:40:17.418033+0000","last_undegraded":"2024-09-10T20:40:17.418033+0000","last_fullsized":"2024-09-10T20:40:17.418033+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20716,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.463009+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:40:39.463009+0000","last_peered":"2024-09-10T20:40:39.463009+0000","last_clean":"2024-09-10T20:40:39.463009+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:40:39.463009+0000","last_undegraded":"2024-09-10T20:40:39.463009+0000","last_fullsized":"2024-09-10T20:40:39.463009+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21227,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462338+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:40:39.462338+0000","last_peered":"2024-09-10T20:40:39.462338+0000","last_clean":"2024-09-10T20:40:39.462338+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:40:39.462338+0000","last_undegraded":"2024-09-10T20:40:39.462338+0000","last_fullsized":"2024-09-10T20:40:39.462338+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20745,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.461976+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:40:39.461976+0000","last_peered":"2024-09-10T20:40:39.461976+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:40:39.461976+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20915,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.462003+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:40:39.462003+0000","last_peered":"2024-09-10T20:40:39.462003+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:40:39.462003+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20892,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461791+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:40:39.461791+0000","last_peered":"2024-09-10T20:40:39.461791+0000","last_clean":"2024-09-10T20:40:39.461791+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:40:39.461791+0000","last_undegraded":"2024-09-10T20:40:39.461791+0000","last_fullsized":"2024-09-10T20:40:39.461791+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417460+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:40:17.417460+0000","last_peered":"2024-09-10T20:40:17.417460+0000","last_clean":"2024-09-10T20:40:17.417460+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:40:17.417460+0000","last_undegraded":"2024-09-10T20:40:17.417460+0000","last_fullsized":"2024-09-10T20:40:17.417460+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22130,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417939+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:40:17.417939+0000","last_peered":"2024-09-10T20:40:17.417939+0000","last_clean":"2024-09-10T20:40:17.417939+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:40:17.417939+0000","last_undegraded":"2024-09-10T20:40:17.417939+0000","last_fullsized":"2024-09-10T20:40:17.417939+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462542+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:40:39.462542+0000","last_peered":"2024-09-10T20:40:39.462542+0000","last_clean":"2024-09-10T20:40:39.462542+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:40:39.462542+0000","last_undegraded":"2024-09-10T20:40:39.462542+0000","last_fullsized":"2024-09-10T20:40:39.462542+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28120,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.949385+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:40:39.949385+0000","last_peered":"2024-09-10T20:40:39.949385+0000","last_clean":"2024-09-10T20:40:39.949385+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:40:39.949385+0000","last_undegraded":"2024-09-10T20:40:39.949385+0000","last_fullsized":"2024-09-10T20:40:39.949385+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":132,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461665+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:40:39.461665+0000","last_peered":"2024-09-10T20:40:39.461665+0000","last_clean":"2024-09-10T20:40:39.461665+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:40:39.461665+0000","last_undegraded":"2024-09-10T20:40:39.461665+0000","last_fullsized":"2024-09-10T20:40:39.461665+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28003,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462607+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:40:39.462607+0000","last_peered":"2024-09-10T20:40:39.462607+0000","last_clean":"2024-09-10T20:40:39.462607+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:40:39.462607+0000","last_undegraded":"2024-09-10T20:40:39.462607+0000","last_fullsized":"2024-09-10T20:40:39.462607+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418359+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:40:17.418359+0000","last_peered":"2024-09-10T20:40:17.418359+0000","last_clean":"2024-09-10T20:40:17.418359+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:40:17.418359+0000","last_undegraded":"2024-09-10T20:40:17.418359+0000","last_fullsized":"2024-09-10T20:40:17.418359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28056,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461762+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:40:39.461762+0000","last_peered":"2024-09-10T20:40:39.461762+0000","last_clean":"2024-09-10T20:40:39.461762+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:40:39.461762+0000","last_undegraded":"2024-09-10T20:40:39.461762+0000","last_fullsized":"2024-09-10T20:40:39.461762+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21109,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418310+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:40:17.418310+0000","last_peered":"2024-09-10T20:40:17.418310+0000","last_clean":"2024-09-10T20:40:17.418310+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:40:17.418310+0000","last_undegraded":"2024-09-10T20:40:17.418310+0000","last_fullsized":"2024-09-10T20:40:17.418310+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418073+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:40:17.418073+0000","last_peered":"2024-09-10T20:40:17.418073+0000","last_clean":"2024-09-10T20:40:17.418073+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:40:17.418073+0000","last_undegraded":"2024-09-10T20:40:17.418073+0000","last_fullsized":"2024-09-10T20:40:17.418073+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27830,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418188+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:40:17.418188+0000","last_peered":"2024-09-10T20:40:17.418188+0000","last_clean":"2024-09-10T20:40:17.418188+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:40:17.418188+0000","last_undegraded":"2024-09-10T20:40:17.418188+0000","last_fullsized":"2024-09-10T20:40:17.418188+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":125,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462414+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:40:39.462414+0000","last_peered":"2024-09-10T20:40:39.462414+0000","last_clean":"2024-09-10T20:40:39.462414+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:40:39.462414+0000","last_undegraded":"2024-09-10T20:40:39.462414+0000","last_fullsized":"2024-09-10T20:40:39.462414+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21265,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.463026+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:40:39.463026+0000","last_peered":"2024-09-10T20:40:39.463026+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:40:39.463026+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418131+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:40:17.418131+0000","last_peered":"2024-09-10T20:40:17.418131+0000","last_clean":"2024-09-10T20:40:17.418131+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:40:17.418131+0000","last_undegraded":"2024-09-10T20:40:17.418131+0000","last_fullsized":"2024-09-10T20:40:17.418131+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.418224+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:40:17.418224+0000","last_peered":"2024-09-10T20:40:17.418224+0000","last_clean":"2024-09-10T20:40:17.418224+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:40:17.418224+0000","last_undegraded":"2024-09-10T20:40:17.418224+0000","last_fullsized":"2024-09-10T20:40:17.418224+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27660,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T20:40:17.417945+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:40:17.417945+0000","last_peered":"2024-09-10T20:40:17.417945+0000","last_clean":"2024-09-10T20:40:17.417945+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:40:17.417945+0000","last_undegraded":"2024-09-10T20:40:17.417945+0000","last_fullsized":"2024-09-10T20:40:17.417945+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13899,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25631,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":167714816,"data_stored":167682299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182545,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517192,"kb_used_data":57972,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854648,"statfs":{"total":96636764160,"available":96107159552,"internally_reserved":0,"allocated":59363328,"data_stored":59052355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986317,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":471636,"kb_used_data":106748,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93900204,"statfs":{"total":96636764160,"available":96153808896,"internally_reserved":0,"allocated":109309952,"data_stored":109003056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659263,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491712,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880128,"statfs":{"total":96636764160,"available":96133251072,"internally_reserved":0,"allocated":507904,"data_stored":185158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502984366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":108818432,"data_stored":108814819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:40:42.656 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:40:42.656 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'20658", 'reported_seq': 21603, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:40.227052+0000', 'last_change': '2024-09-10T20:40:37.857265+0000', 'last_active': '2024-09-10T20:40:40.227052+0000', 'last_peered': '2024-09-10T20:40:40.227052+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:40:40.227052+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3191, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20658, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 417, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3191}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:42.656 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:40:42.656 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20471", 'reported_seq': 20745, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:39.461976+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:40:39.461976+0000', 'last_peered': '2024-09-10T20:40:39.461976+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:40:39.461976+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17335, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3561}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:42.656 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:40:42.656 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'20608", 'reported_seq': 20915, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:39.462003+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:40:39.462003+0000', 'last_peered': '2024-09-10T20:40:39.462003+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:40:39.462003+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17203, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3486}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:42.657 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:40:42.657 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "51'20986", 'reported_seq': 21265, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:39.463026+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:40:39.463026+0000', 'last_peered': '2024-09-10T20:40:39.463026+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:40:39.463026+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17813, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3655}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:42.657 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:40:43.080 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:43.080 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 13899/6 objects degraded (231650.000%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":754,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1516072960,"bytes_avail":288394219520,"bytes_total":289910292480,"degraded_objects":13899,"degraded_total":6,"degraded_ratio":2316.5},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:40:43.094 INFO:tasks.rep_lost_unfound_delete.ceph_manager:no progress seen, keeping timeout for now 2024-09-10T20:40:43.094 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:40:43.461 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:43.461 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:40:43.475 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":430,"stamp":"2024-09-10T20:40:43.070058+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13899,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25633,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472236,"kb_used_data":165232,"kb_used_omap":12,"kb_used_meta":1306931,"kb_avail":281643284,"statfs":{"total":289910292480,"available":288402722816,"internally_reserved":0,"allocated":169197568,"data_stored":168247048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1338297346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-966,"num_objects":-2,"num_object_clones":0,"num_object_copies":-4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-20728,"num_write_kb":-41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001309"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":21603,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:40.227052+0000","last_change":"2024-09-10T20:40:37.857265+0000","last_active":"2024-09-10T20:40:40.227052+0000","last_peered":"2024-09-10T20:40:40.227052+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:40:40.227052+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3191,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":417,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3191}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21084,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.463153+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:40:39.463153+0000","last_peered":"2024-09-10T20:40:39.463153+0000","last_clean":"2024-09-10T20:40:39.463153+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:40:39.463153+0000","last_undegraded":"2024-09-10T20:40:39.463153+0000","last_fullsized":"2024-09-10T20:40:39.463153+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28095,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464663+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:40:39.464663+0000","last_peered":"2024-09-10T20:40:39.464663+0000","last_clean":"2024-09-10T20:40:39.464663+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:40:39.464663+0000","last_undegraded":"2024-09-10T20:40:39.464663+0000","last_fullsized":"2024-09-10T20:40:39.464663+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20716,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.463009+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:40:39.463009+0000","last_peered":"2024-09-10T20:40:39.463009+0000","last_clean":"2024-09-10T20:40:39.463009+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:40:39.463009+0000","last_undegraded":"2024-09-10T20:40:39.463009+0000","last_fullsized":"2024-09-10T20:40:39.463009+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21227,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462338+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:40:39.462338+0000","last_peered":"2024-09-10T20:40:39.462338+0000","last_clean":"2024-09-10T20:40:39.462338+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:40:39.462338+0000","last_undegraded":"2024-09-10T20:40:39.462338+0000","last_fullsized":"2024-09-10T20:40:39.462338+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20745,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.461976+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:40:39.461976+0000","last_peered":"2024-09-10T20:40:39.461976+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:40:39.461976+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20915,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.462003+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:40:39.462003+0000","last_peered":"2024-09-10T20:40:39.462003+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:40:39.462003+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20892,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461791+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:40:39.461791+0000","last_peered":"2024-09-10T20:40:39.461791+0000","last_clean":"2024-09-10T20:40:39.461791+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:40:39.461791+0000","last_undegraded":"2024-09-10T20:40:39.461791+0000","last_fullsized":"2024-09-10T20:40:39.461791+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464374+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:40:39.464374+0000","last_peered":"2024-09-10T20:40:39.464374+0000","last_clean":"2024-09-10T20:40:39.464374+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:40:39.464374+0000","last_undegraded":"2024-09-10T20:40:39.464374+0000","last_fullsized":"2024-09-10T20:40:39.464374+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22134,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464659+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:40:39.464659+0000","last_peered":"2024-09-10T20:40:39.464659+0000","last_clean":"2024-09-10T20:40:39.464659+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:40:39.464659+0000","last_undegraded":"2024-09-10T20:40:39.464659+0000","last_fullsized":"2024-09-10T20:40:39.464659+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462542+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:40:39.462542+0000","last_peered":"2024-09-10T20:40:39.462542+0000","last_clean":"2024-09-10T20:40:39.462542+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:40:39.462542+0000","last_undegraded":"2024-09-10T20:40:39.462542+0000","last_fullsized":"2024-09-10T20:40:39.462542+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28120,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.949385+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:40:39.949385+0000","last_peered":"2024-09-10T20:40:39.949385+0000","last_clean":"2024-09-10T20:40:39.949385+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:40:39.949385+0000","last_undegraded":"2024-09-10T20:40:39.949385+0000","last_fullsized":"2024-09-10T20:40:39.949385+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":132,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461665+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:40:39.461665+0000","last_peered":"2024-09-10T20:40:39.461665+0000","last_clean":"2024-09-10T20:40:39.461665+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:40:39.461665+0000","last_undegraded":"2024-09-10T20:40:39.461665+0000","last_fullsized":"2024-09-10T20:40:39.461665+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28003,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462607+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:40:39.462607+0000","last_peered":"2024-09-10T20:40:39.462607+0000","last_clean":"2024-09-10T20:40:39.462607+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:40:39.462607+0000","last_undegraded":"2024-09-10T20:40:39.462607+0000","last_fullsized":"2024-09-10T20:40:39.462607+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464562+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:40:39.464562+0000","last_peered":"2024-09-10T20:40:39.464562+0000","last_clean":"2024-09-10T20:40:39.464562+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:40:39.464562+0000","last_undegraded":"2024-09-10T20:40:39.464562+0000","last_fullsized":"2024-09-10T20:40:39.464562+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28056,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461762+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:40:39.461762+0000","last_peered":"2024-09-10T20:40:39.461762+0000","last_clean":"2024-09-10T20:40:39.461762+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:40:39.461762+0000","last_undegraded":"2024-09-10T20:40:39.461762+0000","last_fullsized":"2024-09-10T20:40:39.461762+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21113,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464845+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:40:39.464845+0000","last_peered":"2024-09-10T20:40:39.464845+0000","last_clean":"2024-09-10T20:40:39.464845+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:40:39.464845+0000","last_undegraded":"2024-09-10T20:40:39.464845+0000","last_fullsized":"2024-09-10T20:40:39.464845+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464725+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:40:39.464725+0000","last_peered":"2024-09-10T20:40:39.464725+0000","last_clean":"2024-09-10T20:40:39.464725+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:40:39.464725+0000","last_undegraded":"2024-09-10T20:40:39.464725+0000","last_fullsized":"2024-09-10T20:40:39.464725+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27834,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464746+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:40:39.464746+0000","last_peered":"2024-09-10T20:40:39.464746+0000","last_clean":"2024-09-10T20:40:39.464746+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:40:39.464746+0000","last_undegraded":"2024-09-10T20:40:39.464746+0000","last_fullsized":"2024-09-10T20:40:39.464746+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":125,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462414+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:40:39.462414+0000","last_peered":"2024-09-10T20:40:39.462414+0000","last_clean":"2024-09-10T20:40:39.462414+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:40:39.462414+0000","last_undegraded":"2024-09-10T20:40:39.462414+0000","last_fullsized":"2024-09-10T20:40:39.462414+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21265,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.463026+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:40:39.463026+0000","last_peered":"2024-09-10T20:40:39.463026+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:40:39.463026+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464696+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:40:39.464696+0000","last_peered":"2024-09-10T20:40:39.464696+0000","last_clean":"2024-09-10T20:40:39.464696+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:40:39.464696+0000","last_undegraded":"2024-09-10T20:40:39.464696+0000","last_fullsized":"2024-09-10T20:40:39.464696+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464840+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:40:39.464840+0000","last_peered":"2024-09-10T20:40:39.464840+0000","last_clean":"2024-09-10T20:40:39.464840+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:40:39.464840+0000","last_undegraded":"2024-09-10T20:40:39.464840+0000","last_fullsized":"2024-09-10T20:40:39.464840+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27664,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464619+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:40:39.464619+0000","last_peered":"2024-09-10T20:40:39.464619+0000","last_clean":"2024-09-10T20:40:39.464619+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:40:39.464619+0000","last_undegraded":"2024-09-10T20:40:39.464619+0000","last_fullsized":"2024-09-10T20:40:39.464619+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13899,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25631,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":167714816,"data_stored":167682299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182545,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517192,"kb_used_data":57972,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854648,"statfs":{"total":96636764160,"available":96107159552,"internally_reserved":0,"allocated":59363328,"data_stored":59052355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986317,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":471636,"kb_used_data":106748,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93900204,"statfs":{"total":96636764160,"available":96153808896,"internally_reserved":0,"allocated":109309952,"data_stored":109003056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659264,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483408,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888432,"statfs":{"total":96636764160,"available":96141754368,"internally_reserved":0,"allocated":524288,"data_stored":191637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":108818432,"data_stored":108814819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:40:46.478 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:40:46.807 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:46.808 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:40:46.820 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":431,"stamp":"2024-09-10T20:40:45.070454+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13899,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25633,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1469828,"kb_used_data":162832,"kb_used_omap":12,"kb_used_meta":1306931,"kb_avail":281645692,"statfs":{"total":289910292480,"available":288405188608,"internally_reserved":0,"allocated":166739968,"data_stored":165789448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1338297346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-966,"num_objects":-2,"num_object_clones":0,"num_object_copies":-4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1019,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-20728,"num_write_kb":-41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2600,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001358"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":21603,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:40.227052+0000","last_change":"2024-09-10T20:40:37.857265+0000","last_active":"2024-09-10T20:40:40.227052+0000","last_peered":"2024-09-10T20:40:40.227052+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:40:40.227052+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3191,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":417,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3191}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21084,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.463153+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:40:39.463153+0000","last_peered":"2024-09-10T20:40:39.463153+0000","last_clean":"2024-09-10T20:40:39.463153+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:40:39.463153+0000","last_undegraded":"2024-09-10T20:40:39.463153+0000","last_fullsized":"2024-09-10T20:40:39.463153+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28095,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464663+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:40:39.464663+0000","last_peered":"2024-09-10T20:40:39.464663+0000","last_clean":"2024-09-10T20:40:39.464663+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:40:39.464663+0000","last_undegraded":"2024-09-10T20:40:39.464663+0000","last_fullsized":"2024-09-10T20:40:39.464663+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20716,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.463009+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:40:39.463009+0000","last_peered":"2024-09-10T20:40:39.463009+0000","last_clean":"2024-09-10T20:40:39.463009+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:40:39.463009+0000","last_undegraded":"2024-09-10T20:40:39.463009+0000","last_fullsized":"2024-09-10T20:40:39.463009+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21227,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462338+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:40:39.462338+0000","last_peered":"2024-09-10T20:40:39.462338+0000","last_clean":"2024-09-10T20:40:39.462338+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:40:39.462338+0000","last_undegraded":"2024-09-10T20:40:39.462338+0000","last_fullsized":"2024-09-10T20:40:39.462338+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20745,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.461976+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:40:39.461976+0000","last_peered":"2024-09-10T20:40:39.461976+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:40:39.461976+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20915,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.462003+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:40:39.462003+0000","last_peered":"2024-09-10T20:40:39.462003+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:40:39.462003+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20892,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461791+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:40:39.461791+0000","last_peered":"2024-09-10T20:40:39.461791+0000","last_clean":"2024-09-10T20:40:39.461791+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:40:39.461791+0000","last_undegraded":"2024-09-10T20:40:39.461791+0000","last_fullsized":"2024-09-10T20:40:39.461791+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464374+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:40:39.464374+0000","last_peered":"2024-09-10T20:40:39.464374+0000","last_clean":"2024-09-10T20:40:39.464374+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:40:39.464374+0000","last_undegraded":"2024-09-10T20:40:39.464374+0000","last_fullsized":"2024-09-10T20:40:39.464374+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22134,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464659+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:40:39.464659+0000","last_peered":"2024-09-10T20:40:39.464659+0000","last_clean":"2024-09-10T20:40:39.464659+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:40:39.464659+0000","last_undegraded":"2024-09-10T20:40:39.464659+0000","last_fullsized":"2024-09-10T20:40:39.464659+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462542+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:40:39.462542+0000","last_peered":"2024-09-10T20:40:39.462542+0000","last_clean":"2024-09-10T20:40:39.462542+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:40:39.462542+0000","last_undegraded":"2024-09-10T20:40:39.462542+0000","last_fullsized":"2024-09-10T20:40:39.462542+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28120,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.949385+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:40:39.949385+0000","last_peered":"2024-09-10T20:40:39.949385+0000","last_clean":"2024-09-10T20:40:39.949385+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:40:39.949385+0000","last_undegraded":"2024-09-10T20:40:39.949385+0000","last_fullsized":"2024-09-10T20:40:39.949385+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":132,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461665+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:40:39.461665+0000","last_peered":"2024-09-10T20:40:39.461665+0000","last_clean":"2024-09-10T20:40:39.461665+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:40:39.461665+0000","last_undegraded":"2024-09-10T20:40:39.461665+0000","last_fullsized":"2024-09-10T20:40:39.461665+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28003,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462607+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:40:39.462607+0000","last_peered":"2024-09-10T20:40:39.462607+0000","last_clean":"2024-09-10T20:40:39.462607+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:40:39.462607+0000","last_undegraded":"2024-09-10T20:40:39.462607+0000","last_fullsized":"2024-09-10T20:40:39.462607+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464562+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:40:39.464562+0000","last_peered":"2024-09-10T20:40:39.464562+0000","last_clean":"2024-09-10T20:40:39.464562+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:40:39.464562+0000","last_undegraded":"2024-09-10T20:40:39.464562+0000","last_fullsized":"2024-09-10T20:40:39.464562+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28056,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461762+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:40:39.461762+0000","last_peered":"2024-09-10T20:40:39.461762+0000","last_clean":"2024-09-10T20:40:39.461762+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:40:39.461762+0000","last_undegraded":"2024-09-10T20:40:39.461762+0000","last_fullsized":"2024-09-10T20:40:39.461762+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21113,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464845+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:40:39.464845+0000","last_peered":"2024-09-10T20:40:39.464845+0000","last_clean":"2024-09-10T20:40:39.464845+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:40:39.464845+0000","last_undegraded":"2024-09-10T20:40:39.464845+0000","last_fullsized":"2024-09-10T20:40:39.464845+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464725+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:40:39.464725+0000","last_peered":"2024-09-10T20:40:39.464725+0000","last_clean":"2024-09-10T20:40:39.464725+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:40:39.464725+0000","last_undegraded":"2024-09-10T20:40:39.464725+0000","last_fullsized":"2024-09-10T20:40:39.464725+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27834,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464746+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:40:39.464746+0000","last_peered":"2024-09-10T20:40:39.464746+0000","last_clean":"2024-09-10T20:40:39.464746+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:40:39.464746+0000","last_undegraded":"2024-09-10T20:40:39.464746+0000","last_fullsized":"2024-09-10T20:40:39.464746+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":125,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462414+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:40:39.462414+0000","last_peered":"2024-09-10T20:40:39.462414+0000","last_clean":"2024-09-10T20:40:39.462414+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:40:39.462414+0000","last_undegraded":"2024-09-10T20:40:39.462414+0000","last_fullsized":"2024-09-10T20:40:39.462414+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21265,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.463026+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:40:39.463026+0000","last_peered":"2024-09-10T20:40:39.463026+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:40:39.463026+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464696+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:40:39.464696+0000","last_peered":"2024-09-10T20:40:39.464696+0000","last_clean":"2024-09-10T20:40:39.464696+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:40:39.464696+0000","last_undegraded":"2024-09-10T20:40:39.464696+0000","last_fullsized":"2024-09-10T20:40:39.464696+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464840+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:40:39.464840+0000","last_peered":"2024-09-10T20:40:39.464840+0000","last_clean":"2024-09-10T20:40:39.464840+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:40:39.464840+0000","last_undegraded":"2024-09-10T20:40:39.464840+0000","last_fullsized":"2024-09-10T20:40:39.464840+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27664,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464619+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:40:39.464619+0000","last_peered":"2024-09-10T20:40:39.464619+0000","last_clean":"2024-09-10T20:40:39.464619+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:40:39.464619+0000","last_undegraded":"2024-09-10T20:40:39.464619+0000","last_fullsized":"2024-09-10T20:40:39.464619+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13899,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25631,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":165257216,"data_stored":165224699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182545,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517192,"kb_used_data":57972,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854648,"statfs":{"total":96636764160,"available":96107159552,"internally_reserved":0,"allocated":59363328,"data_stored":59052355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986318,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469228,"kb_used_data":104348,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93902612,"statfs":{"total":96636764160,"available":96156274688,"internally_reserved":0,"allocated":106852352,"data_stored":106545456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659264,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483408,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888432,"statfs":{"total":96636764160,"available":96141754368,"internally_reserved":0,"allocated":524288,"data_stored":191637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106360832,"data_stored":106357219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:40:46.822 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:40:47.131 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:47.131 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:40:47.145 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-09-10T20:40:47.070753+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12969,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26563,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1469828,"kb_used_data":162832,"kb_used_omap":12,"kb_used_meta":1306931,"kb_avail":281645692,"statfs":{"total":289910292480,"available":288405188608,"internally_reserved":0,"allocated":166739968,"data_stored":165789448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1338297346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-966,"num_objects":-2,"num_object_clones":0,"num_object_copies":-4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1949,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-20728,"num_write_kb":-41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1670,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.614109"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":23463,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:45.230672+0000","last_change":"2024-09-10T20:40:37.857265+0000","last_active":"2024-09-10T20:40:45.230672+0000","last_peered":"2024-09-10T20:40:45.230672+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:40:45.230672+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2261,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1347,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2261}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21084,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.463153+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:40:39.463153+0000","last_peered":"2024-09-10T20:40:39.463153+0000","last_clean":"2024-09-10T20:40:39.463153+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:40:39.463153+0000","last_undegraded":"2024-09-10T20:40:39.463153+0000","last_fullsized":"2024-09-10T20:40:39.463153+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28095,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464663+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:40:39.464663+0000","last_peered":"2024-09-10T20:40:39.464663+0000","last_clean":"2024-09-10T20:40:39.464663+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:40:39.464663+0000","last_undegraded":"2024-09-10T20:40:39.464663+0000","last_fullsized":"2024-09-10T20:40:39.464663+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20716,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.463009+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:40:39.463009+0000","last_peered":"2024-09-10T20:40:39.463009+0000","last_clean":"2024-09-10T20:40:39.463009+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:40:39.463009+0000","last_undegraded":"2024-09-10T20:40:39.463009+0000","last_fullsized":"2024-09-10T20:40:39.463009+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21227,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462338+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:40:39.462338+0000","last_peered":"2024-09-10T20:40:39.462338+0000","last_clean":"2024-09-10T20:40:39.462338+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:40:39.462338+0000","last_undegraded":"2024-09-10T20:40:39.462338+0000","last_fullsized":"2024-09-10T20:40:39.462338+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20745,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.461976+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:40:39.461976+0000","last_peered":"2024-09-10T20:40:39.461976+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:40:39.461976+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20915,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.462003+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:40:39.462003+0000","last_peered":"2024-09-10T20:40:39.462003+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:40:39.462003+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20892,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461791+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:40:39.461791+0000","last_peered":"2024-09-10T20:40:39.461791+0000","last_clean":"2024-09-10T20:40:39.461791+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:40:39.461791+0000","last_undegraded":"2024-09-10T20:40:39.461791+0000","last_fullsized":"2024-09-10T20:40:39.461791+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464374+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:40:39.464374+0000","last_peered":"2024-09-10T20:40:39.464374+0000","last_clean":"2024-09-10T20:40:39.464374+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:40:39.464374+0000","last_undegraded":"2024-09-10T20:40:39.464374+0000","last_fullsized":"2024-09-10T20:40:39.464374+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22134,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464659+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:40:39.464659+0000","last_peered":"2024-09-10T20:40:39.464659+0000","last_clean":"2024-09-10T20:40:39.464659+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:40:39.464659+0000","last_undegraded":"2024-09-10T20:40:39.464659+0000","last_fullsized":"2024-09-10T20:40:39.464659+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462542+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:40:39.462542+0000","last_peered":"2024-09-10T20:40:39.462542+0000","last_clean":"2024-09-10T20:40:39.462542+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:40:39.462542+0000","last_undegraded":"2024-09-10T20:40:39.462542+0000","last_fullsized":"2024-09-10T20:40:39.462542+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28120,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.949385+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:40:39.949385+0000","last_peered":"2024-09-10T20:40:39.949385+0000","last_clean":"2024-09-10T20:40:39.949385+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:40:39.949385+0000","last_undegraded":"2024-09-10T20:40:39.949385+0000","last_fullsized":"2024-09-10T20:40:39.949385+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":132,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461665+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:40:39.461665+0000","last_peered":"2024-09-10T20:40:39.461665+0000","last_clean":"2024-09-10T20:40:39.461665+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:40:39.461665+0000","last_undegraded":"2024-09-10T20:40:39.461665+0000","last_fullsized":"2024-09-10T20:40:39.461665+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28003,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462607+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:40:39.462607+0000","last_peered":"2024-09-10T20:40:39.462607+0000","last_clean":"2024-09-10T20:40:39.462607+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:40:39.462607+0000","last_undegraded":"2024-09-10T20:40:39.462607+0000","last_fullsized":"2024-09-10T20:40:39.462607+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464562+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:40:39.464562+0000","last_peered":"2024-09-10T20:40:39.464562+0000","last_clean":"2024-09-10T20:40:39.464562+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:40:39.464562+0000","last_undegraded":"2024-09-10T20:40:39.464562+0000","last_fullsized":"2024-09-10T20:40:39.464562+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28056,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461762+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:40:39.461762+0000","last_peered":"2024-09-10T20:40:39.461762+0000","last_clean":"2024-09-10T20:40:39.461762+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:40:39.461762+0000","last_undegraded":"2024-09-10T20:40:39.461762+0000","last_fullsized":"2024-09-10T20:40:39.461762+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21113,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464845+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:40:39.464845+0000","last_peered":"2024-09-10T20:40:39.464845+0000","last_clean":"2024-09-10T20:40:39.464845+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:40:39.464845+0000","last_undegraded":"2024-09-10T20:40:39.464845+0000","last_fullsized":"2024-09-10T20:40:39.464845+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464725+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:40:39.464725+0000","last_peered":"2024-09-10T20:40:39.464725+0000","last_clean":"2024-09-10T20:40:39.464725+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:40:39.464725+0000","last_undegraded":"2024-09-10T20:40:39.464725+0000","last_fullsized":"2024-09-10T20:40:39.464725+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27834,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464746+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:40:39.464746+0000","last_peered":"2024-09-10T20:40:39.464746+0000","last_clean":"2024-09-10T20:40:39.464746+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:40:39.464746+0000","last_undegraded":"2024-09-10T20:40:39.464746+0000","last_fullsized":"2024-09-10T20:40:39.464746+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":125,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462414+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:40:39.462414+0000","last_peered":"2024-09-10T20:40:39.462414+0000","last_clean":"2024-09-10T20:40:39.462414+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:40:39.462414+0000","last_undegraded":"2024-09-10T20:40:39.462414+0000","last_fullsized":"2024-09-10T20:40:39.462414+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21265,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.463026+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:40:39.463026+0000","last_peered":"2024-09-10T20:40:39.463026+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:40:39.463026+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464696+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:40:39.464696+0000","last_peered":"2024-09-10T20:40:39.464696+0000","last_clean":"2024-09-10T20:40:39.464696+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:40:39.464696+0000","last_undegraded":"2024-09-10T20:40:39.464696+0000","last_fullsized":"2024-09-10T20:40:39.464696+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464840+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:40:39.464840+0000","last_peered":"2024-09-10T20:40:39.464840+0000","last_clean":"2024-09-10T20:40:39.464840+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:40:39.464840+0000","last_undegraded":"2024-09-10T20:40:39.464840+0000","last_fullsized":"2024-09-10T20:40:39.464840+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27664,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464619+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:40:39.464619+0000","last_peered":"2024-09-10T20:40:39.464619+0000","last_clean":"2024-09-10T20:40:39.464619+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:40:39.464619+0000","last_undegraded":"2024-09-10T20:40:39.464619+0000","last_fullsized":"2024-09-10T20:40:39.464619+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12969,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26561,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":165257216,"data_stored":165224699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182546,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517192,"kb_used_data":57972,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854648,"statfs":{"total":96636764160,"available":96107159552,"internally_reserved":0,"allocated":59363328,"data_stored":59052355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986318,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469228,"kb_used_data":104348,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93902612,"statfs":{"total":96636764160,"available":96156274688,"internally_reserved":0,"allocated":106852352,"data_stored":106545456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659264,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483408,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888432,"statfs":{"total":96636764160,"available":96141754368,"internally_reserved":0,"allocated":524288,"data_stored":191637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106360832,"data_stored":106357219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:40:47.146 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:40:47.146 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'20658", 'reported_seq': 23463, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:45.230672+0000', 'last_change': '2024-09-10T20:40:37.857265+0000', 'last_active': '2024-09-10T20:40:45.230672+0000', 'last_peered': '2024-09-10T20:40:45.230672+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:40:45.230672+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2261, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20658, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1347, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2261}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:47.146 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:40:47.146 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20471", 'reported_seq': 20745, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:39.461976+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:40:39.461976+0000', 'last_peered': '2024-09-10T20:40:39.461976+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:40:39.461976+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17335, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3561}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:47.146 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:40:47.146 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'20608", 'reported_seq': 20915, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:39.462003+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:40:39.462003+0000', 'last_peered': '2024-09-10T20:40:39.462003+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:40:39.462003+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17203, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3486}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:47.147 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:40:47.147 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "51'20986", 'reported_seq': 21265, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:39.463026+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:40:39.463026+0000', 'last_peered': '2024-09-10T20:40:39.463026+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:40:39.463026+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17813, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3655}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:47.147 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:40:47.527 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:47.527 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 12969/6 objects degraded (216150.000%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":759,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1505103872,"bytes_avail":288405188608,"bytes_total":289910292480,"degraded_objects":12969,"degraded_total":6,"degraded_ratio":2161.5},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:40:47.542 INFO:tasks.rep_lost_unfound_delete.ceph_manager:no progress seen, keeping timeout for now 2024-09-10T20:40:47.543 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:40:47.895 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:47.895 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:40:47.908 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-09-10T20:40:47.070753+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12969,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26563,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1469828,"kb_used_data":162832,"kb_used_omap":12,"kb_used_meta":1306931,"kb_avail":281645692,"statfs":{"total":289910292480,"available":288405188608,"internally_reserved":0,"allocated":166739968,"data_stored":165789448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1338297346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-966,"num_objects":-2,"num_object_clones":0,"num_object_copies":-4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1949,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-20728,"num_write_kb":-41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1670,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.614109"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":23463,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:45.230672+0000","last_change":"2024-09-10T20:40:37.857265+0000","last_active":"2024-09-10T20:40:45.230672+0000","last_peered":"2024-09-10T20:40:45.230672+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:40:45.230672+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2261,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1347,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2261}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21084,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.463153+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:40:39.463153+0000","last_peered":"2024-09-10T20:40:39.463153+0000","last_clean":"2024-09-10T20:40:39.463153+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:40:39.463153+0000","last_undegraded":"2024-09-10T20:40:39.463153+0000","last_fullsized":"2024-09-10T20:40:39.463153+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28095,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464663+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:40:39.464663+0000","last_peered":"2024-09-10T20:40:39.464663+0000","last_clean":"2024-09-10T20:40:39.464663+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:40:39.464663+0000","last_undegraded":"2024-09-10T20:40:39.464663+0000","last_fullsized":"2024-09-10T20:40:39.464663+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20716,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.463009+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:40:39.463009+0000","last_peered":"2024-09-10T20:40:39.463009+0000","last_clean":"2024-09-10T20:40:39.463009+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:40:39.463009+0000","last_undegraded":"2024-09-10T20:40:39.463009+0000","last_fullsized":"2024-09-10T20:40:39.463009+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21227,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462338+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:40:39.462338+0000","last_peered":"2024-09-10T20:40:39.462338+0000","last_clean":"2024-09-10T20:40:39.462338+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:40:39.462338+0000","last_undegraded":"2024-09-10T20:40:39.462338+0000","last_fullsized":"2024-09-10T20:40:39.462338+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20745,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.461976+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:40:39.461976+0000","last_peered":"2024-09-10T20:40:39.461976+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:40:39.461976+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20915,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.462003+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:40:39.462003+0000","last_peered":"2024-09-10T20:40:39.462003+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:40:39.462003+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20892,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461791+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:40:39.461791+0000","last_peered":"2024-09-10T20:40:39.461791+0000","last_clean":"2024-09-10T20:40:39.461791+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:40:39.461791+0000","last_undegraded":"2024-09-10T20:40:39.461791+0000","last_fullsized":"2024-09-10T20:40:39.461791+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464374+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:40:39.464374+0000","last_peered":"2024-09-10T20:40:39.464374+0000","last_clean":"2024-09-10T20:40:39.464374+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:40:39.464374+0000","last_undegraded":"2024-09-10T20:40:39.464374+0000","last_fullsized":"2024-09-10T20:40:39.464374+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22134,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464659+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:40:39.464659+0000","last_peered":"2024-09-10T20:40:39.464659+0000","last_clean":"2024-09-10T20:40:39.464659+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:40:39.464659+0000","last_undegraded":"2024-09-10T20:40:39.464659+0000","last_fullsized":"2024-09-10T20:40:39.464659+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462542+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:40:39.462542+0000","last_peered":"2024-09-10T20:40:39.462542+0000","last_clean":"2024-09-10T20:40:39.462542+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:40:39.462542+0000","last_undegraded":"2024-09-10T20:40:39.462542+0000","last_fullsized":"2024-09-10T20:40:39.462542+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28120,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.949385+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:40:39.949385+0000","last_peered":"2024-09-10T20:40:39.949385+0000","last_clean":"2024-09-10T20:40:39.949385+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:40:39.949385+0000","last_undegraded":"2024-09-10T20:40:39.949385+0000","last_fullsized":"2024-09-10T20:40:39.949385+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":132,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461665+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:40:39.461665+0000","last_peered":"2024-09-10T20:40:39.461665+0000","last_clean":"2024-09-10T20:40:39.461665+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:40:39.461665+0000","last_undegraded":"2024-09-10T20:40:39.461665+0000","last_fullsized":"2024-09-10T20:40:39.461665+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28003,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462607+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:40:39.462607+0000","last_peered":"2024-09-10T20:40:39.462607+0000","last_clean":"2024-09-10T20:40:39.462607+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:40:39.462607+0000","last_undegraded":"2024-09-10T20:40:39.462607+0000","last_fullsized":"2024-09-10T20:40:39.462607+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464562+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:40:39.464562+0000","last_peered":"2024-09-10T20:40:39.464562+0000","last_clean":"2024-09-10T20:40:39.464562+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:40:39.464562+0000","last_undegraded":"2024-09-10T20:40:39.464562+0000","last_fullsized":"2024-09-10T20:40:39.464562+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28056,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461762+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:40:39.461762+0000","last_peered":"2024-09-10T20:40:39.461762+0000","last_clean":"2024-09-10T20:40:39.461762+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:40:39.461762+0000","last_undegraded":"2024-09-10T20:40:39.461762+0000","last_fullsized":"2024-09-10T20:40:39.461762+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21113,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464845+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:40:39.464845+0000","last_peered":"2024-09-10T20:40:39.464845+0000","last_clean":"2024-09-10T20:40:39.464845+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:40:39.464845+0000","last_undegraded":"2024-09-10T20:40:39.464845+0000","last_fullsized":"2024-09-10T20:40:39.464845+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464725+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:40:39.464725+0000","last_peered":"2024-09-10T20:40:39.464725+0000","last_clean":"2024-09-10T20:40:39.464725+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:40:39.464725+0000","last_undegraded":"2024-09-10T20:40:39.464725+0000","last_fullsized":"2024-09-10T20:40:39.464725+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27834,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464746+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:40:39.464746+0000","last_peered":"2024-09-10T20:40:39.464746+0000","last_clean":"2024-09-10T20:40:39.464746+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:40:39.464746+0000","last_undegraded":"2024-09-10T20:40:39.464746+0000","last_fullsized":"2024-09-10T20:40:39.464746+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":125,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462414+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:40:39.462414+0000","last_peered":"2024-09-10T20:40:39.462414+0000","last_clean":"2024-09-10T20:40:39.462414+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:40:39.462414+0000","last_undegraded":"2024-09-10T20:40:39.462414+0000","last_fullsized":"2024-09-10T20:40:39.462414+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21265,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.463026+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:40:39.463026+0000","last_peered":"2024-09-10T20:40:39.463026+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:40:39.463026+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464696+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:40:39.464696+0000","last_peered":"2024-09-10T20:40:39.464696+0000","last_clean":"2024-09-10T20:40:39.464696+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:40:39.464696+0000","last_undegraded":"2024-09-10T20:40:39.464696+0000","last_fullsized":"2024-09-10T20:40:39.464696+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464840+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:40:39.464840+0000","last_peered":"2024-09-10T20:40:39.464840+0000","last_clean":"2024-09-10T20:40:39.464840+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:40:39.464840+0000","last_undegraded":"2024-09-10T20:40:39.464840+0000","last_fullsized":"2024-09-10T20:40:39.464840+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27664,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464619+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:40:39.464619+0000","last_peered":"2024-09-10T20:40:39.464619+0000","last_clean":"2024-09-10T20:40:39.464619+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:40:39.464619+0000","last_undegraded":"2024-09-10T20:40:39.464619+0000","last_fullsized":"2024-09-10T20:40:39.464619+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12969,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26561,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":165257216,"data_stored":165224699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182546,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517192,"kb_used_data":57972,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854648,"statfs":{"total":96636764160,"available":96107159552,"internally_reserved":0,"allocated":59363328,"data_stored":59052355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986318,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469228,"kb_used_data":104348,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93902612,"statfs":{"total":96636764160,"available":96156274688,"internally_reserved":0,"allocated":106852352,"data_stored":106545456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659264,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483408,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888432,"statfs":{"total":96636764160,"available":96141754368,"internally_reserved":0,"allocated":524288,"data_stored":191637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106360832,"data_stored":106357219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:40:50.910 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:40:51.280 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:51.281 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:40:51.293 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":434,"stamp":"2024-09-10T20:40:51.071440+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12198,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27334,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1475748,"kb_used_data":160432,"kb_used_omap":12,"kb_used_meta":1315251,"kb_avail":281639772,"statfs":{"total":289910292480,"available":288399126528,"internally_reserved":0,"allocated":164282368,"data_stored":163331848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1346817026},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-966,"num_objects":-2,"num_object_clones":0,"num_object_copies":-4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2720,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-20728,"num_write_kb":-41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-899,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.610933"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":25005,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:50.232008+0000","last_change":"2024-09-10T20:40:37.857265+0000","last_active":"2024-09-10T20:40:50.232008+0000","last_peered":"2024-09-10T20:40:50.232008+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:40:50.232008+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1490,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2118,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1490}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21084,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.463153+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:40:39.463153+0000","last_peered":"2024-09-10T20:40:39.463153+0000","last_clean":"2024-09-10T20:40:39.463153+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:40:39.463153+0000","last_undegraded":"2024-09-10T20:40:39.463153+0000","last_fullsized":"2024-09-10T20:40:39.463153+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28095,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464663+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:40:39.464663+0000","last_peered":"2024-09-10T20:40:39.464663+0000","last_clean":"2024-09-10T20:40:39.464663+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:40:39.464663+0000","last_undegraded":"2024-09-10T20:40:39.464663+0000","last_fullsized":"2024-09-10T20:40:39.464663+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20716,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.463009+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:40:39.463009+0000","last_peered":"2024-09-10T20:40:39.463009+0000","last_clean":"2024-09-10T20:40:39.463009+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:40:39.463009+0000","last_undegraded":"2024-09-10T20:40:39.463009+0000","last_fullsized":"2024-09-10T20:40:39.463009+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21227,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462338+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:40:39.462338+0000","last_peered":"2024-09-10T20:40:39.462338+0000","last_clean":"2024-09-10T20:40:39.462338+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:40:39.462338+0000","last_undegraded":"2024-09-10T20:40:39.462338+0000","last_fullsized":"2024-09-10T20:40:39.462338+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20745,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.461976+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:40:39.461976+0000","last_peered":"2024-09-10T20:40:39.461976+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:40:39.461976+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20915,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.462003+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:40:39.462003+0000","last_peered":"2024-09-10T20:40:39.462003+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:40:39.462003+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20892,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461791+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:40:39.461791+0000","last_peered":"2024-09-10T20:40:39.461791+0000","last_clean":"2024-09-10T20:40:39.461791+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:40:39.461791+0000","last_undegraded":"2024-09-10T20:40:39.461791+0000","last_fullsized":"2024-09-10T20:40:39.461791+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464374+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:40:39.464374+0000","last_peered":"2024-09-10T20:40:39.464374+0000","last_clean":"2024-09-10T20:40:39.464374+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:40:39.464374+0000","last_undegraded":"2024-09-10T20:40:39.464374+0000","last_fullsized":"2024-09-10T20:40:39.464374+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22134,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464659+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:40:39.464659+0000","last_peered":"2024-09-10T20:40:39.464659+0000","last_clean":"2024-09-10T20:40:39.464659+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:40:39.464659+0000","last_undegraded":"2024-09-10T20:40:39.464659+0000","last_fullsized":"2024-09-10T20:40:39.464659+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462542+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:40:39.462542+0000","last_peered":"2024-09-10T20:40:39.462542+0000","last_clean":"2024-09-10T20:40:39.462542+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:40:39.462542+0000","last_undegraded":"2024-09-10T20:40:39.462542+0000","last_fullsized":"2024-09-10T20:40:39.462542+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28120,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.949385+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:40:39.949385+0000","last_peered":"2024-09-10T20:40:39.949385+0000","last_clean":"2024-09-10T20:40:39.949385+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:40:39.949385+0000","last_undegraded":"2024-09-10T20:40:39.949385+0000","last_fullsized":"2024-09-10T20:40:39.949385+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":132,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461665+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:40:39.461665+0000","last_peered":"2024-09-10T20:40:39.461665+0000","last_clean":"2024-09-10T20:40:39.461665+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:40:39.461665+0000","last_undegraded":"2024-09-10T20:40:39.461665+0000","last_fullsized":"2024-09-10T20:40:39.461665+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28003,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462607+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:40:39.462607+0000","last_peered":"2024-09-10T20:40:39.462607+0000","last_clean":"2024-09-10T20:40:39.462607+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:40:39.462607+0000","last_undegraded":"2024-09-10T20:40:39.462607+0000","last_fullsized":"2024-09-10T20:40:39.462607+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464562+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:40:39.464562+0000","last_peered":"2024-09-10T20:40:39.464562+0000","last_clean":"2024-09-10T20:40:39.464562+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:40:39.464562+0000","last_undegraded":"2024-09-10T20:40:39.464562+0000","last_fullsized":"2024-09-10T20:40:39.464562+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28056,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461762+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:40:39.461762+0000","last_peered":"2024-09-10T20:40:39.461762+0000","last_clean":"2024-09-10T20:40:39.461762+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:40:39.461762+0000","last_undegraded":"2024-09-10T20:40:39.461762+0000","last_fullsized":"2024-09-10T20:40:39.461762+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21113,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464845+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:40:39.464845+0000","last_peered":"2024-09-10T20:40:39.464845+0000","last_clean":"2024-09-10T20:40:39.464845+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:40:39.464845+0000","last_undegraded":"2024-09-10T20:40:39.464845+0000","last_fullsized":"2024-09-10T20:40:39.464845+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464725+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:40:39.464725+0000","last_peered":"2024-09-10T20:40:39.464725+0000","last_clean":"2024-09-10T20:40:39.464725+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:40:39.464725+0000","last_undegraded":"2024-09-10T20:40:39.464725+0000","last_fullsized":"2024-09-10T20:40:39.464725+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27834,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464746+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:40:39.464746+0000","last_peered":"2024-09-10T20:40:39.464746+0000","last_clean":"2024-09-10T20:40:39.464746+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:40:39.464746+0000","last_undegraded":"2024-09-10T20:40:39.464746+0000","last_fullsized":"2024-09-10T20:40:39.464746+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":125,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462414+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:40:39.462414+0000","last_peered":"2024-09-10T20:40:39.462414+0000","last_clean":"2024-09-10T20:40:39.462414+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:40:39.462414+0000","last_undegraded":"2024-09-10T20:40:39.462414+0000","last_fullsized":"2024-09-10T20:40:39.462414+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21265,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.463026+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:40:39.463026+0000","last_peered":"2024-09-10T20:40:39.463026+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:40:39.463026+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464696+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:40:39.464696+0000","last_peered":"2024-09-10T20:40:39.464696+0000","last_clean":"2024-09-10T20:40:39.464696+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:40:39.464696+0000","last_undegraded":"2024-09-10T20:40:39.464696+0000","last_fullsized":"2024-09-10T20:40:39.464696+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464840+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:40:39.464840+0000","last_peered":"2024-09-10T20:40:39.464840+0000","last_clean":"2024-09-10T20:40:39.464840+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:40:39.464840+0000","last_undegraded":"2024-09-10T20:40:39.464840+0000","last_fullsized":"2024-09-10T20:40:39.464840+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27665,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464669+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:40:39.464669+0000","last_peered":"2024-09-10T20:40:39.464669+0000","last_clean":"2024-09-10T20:40:39.464669+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:40:39.464669+0000","last_undegraded":"2024-09-10T20:40:39.464669+0000","last_fullsized":"2024-09-10T20:40:39.464669+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12198,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27332,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":162799616,"data_stored":162767099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182547,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517192,"kb_used_data":57972,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854648,"statfs":{"total":96636764160,"available":96107159552,"internally_reserved":0,"allocated":59363328,"data_stored":59052355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986319,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":466828,"kb_used_data":101948,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93905012,"statfs":{"total":96636764160,"available":96158732288,"internally_reserved":0,"allocated":104394752,"data_stored":104087856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659265,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491728,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880112,"statfs":{"total":96636764160,"available":96133234688,"internally_reserved":0,"allocated":524288,"data_stored":191637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502984366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":103903232,"data_stored":103899619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:40:51.294 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:40:51.603 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:51.603 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:40:51.617 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":434,"stamp":"2024-09-10T20:40:51.071440+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12198,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27334,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1475748,"kb_used_data":160432,"kb_used_omap":12,"kb_used_meta":1315251,"kb_avail":281639772,"statfs":{"total":289910292480,"available":288399126528,"internally_reserved":0,"allocated":164282368,"data_stored":163331848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1346817026},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-966,"num_objects":-2,"num_object_clones":0,"num_object_copies":-4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2720,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-20728,"num_write_kb":-41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-899,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.610933"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":25005,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:50.232008+0000","last_change":"2024-09-10T20:40:37.857265+0000","last_active":"2024-09-10T20:40:50.232008+0000","last_peered":"2024-09-10T20:40:50.232008+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:40:50.232008+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1490,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2118,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1490}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21084,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.463153+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:40:39.463153+0000","last_peered":"2024-09-10T20:40:39.463153+0000","last_clean":"2024-09-10T20:40:39.463153+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:40:39.463153+0000","last_undegraded":"2024-09-10T20:40:39.463153+0000","last_fullsized":"2024-09-10T20:40:39.463153+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28095,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464663+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:40:39.464663+0000","last_peered":"2024-09-10T20:40:39.464663+0000","last_clean":"2024-09-10T20:40:39.464663+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:40:39.464663+0000","last_undegraded":"2024-09-10T20:40:39.464663+0000","last_fullsized":"2024-09-10T20:40:39.464663+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20716,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.463009+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:40:39.463009+0000","last_peered":"2024-09-10T20:40:39.463009+0000","last_clean":"2024-09-10T20:40:39.463009+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:40:39.463009+0000","last_undegraded":"2024-09-10T20:40:39.463009+0000","last_fullsized":"2024-09-10T20:40:39.463009+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21227,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462338+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:40:39.462338+0000","last_peered":"2024-09-10T20:40:39.462338+0000","last_clean":"2024-09-10T20:40:39.462338+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:40:39.462338+0000","last_undegraded":"2024-09-10T20:40:39.462338+0000","last_fullsized":"2024-09-10T20:40:39.462338+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20745,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.461976+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:40:39.461976+0000","last_peered":"2024-09-10T20:40:39.461976+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:40:39.461976+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20915,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.462003+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:40:39.462003+0000","last_peered":"2024-09-10T20:40:39.462003+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:40:39.462003+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20892,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461791+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:40:39.461791+0000","last_peered":"2024-09-10T20:40:39.461791+0000","last_clean":"2024-09-10T20:40:39.461791+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:40:39.461791+0000","last_undegraded":"2024-09-10T20:40:39.461791+0000","last_fullsized":"2024-09-10T20:40:39.461791+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464374+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:40:39.464374+0000","last_peered":"2024-09-10T20:40:39.464374+0000","last_clean":"2024-09-10T20:40:39.464374+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:40:39.464374+0000","last_undegraded":"2024-09-10T20:40:39.464374+0000","last_fullsized":"2024-09-10T20:40:39.464374+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22134,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464659+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:40:39.464659+0000","last_peered":"2024-09-10T20:40:39.464659+0000","last_clean":"2024-09-10T20:40:39.464659+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:40:39.464659+0000","last_undegraded":"2024-09-10T20:40:39.464659+0000","last_fullsized":"2024-09-10T20:40:39.464659+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462542+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:40:39.462542+0000","last_peered":"2024-09-10T20:40:39.462542+0000","last_clean":"2024-09-10T20:40:39.462542+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:40:39.462542+0000","last_undegraded":"2024-09-10T20:40:39.462542+0000","last_fullsized":"2024-09-10T20:40:39.462542+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28120,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.949385+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:40:39.949385+0000","last_peered":"2024-09-10T20:40:39.949385+0000","last_clean":"2024-09-10T20:40:39.949385+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:40:39.949385+0000","last_undegraded":"2024-09-10T20:40:39.949385+0000","last_fullsized":"2024-09-10T20:40:39.949385+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":132,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461665+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:40:39.461665+0000","last_peered":"2024-09-10T20:40:39.461665+0000","last_clean":"2024-09-10T20:40:39.461665+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:40:39.461665+0000","last_undegraded":"2024-09-10T20:40:39.461665+0000","last_fullsized":"2024-09-10T20:40:39.461665+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28003,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462607+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:40:39.462607+0000","last_peered":"2024-09-10T20:40:39.462607+0000","last_clean":"2024-09-10T20:40:39.462607+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:40:39.462607+0000","last_undegraded":"2024-09-10T20:40:39.462607+0000","last_fullsized":"2024-09-10T20:40:39.462607+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464562+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:40:39.464562+0000","last_peered":"2024-09-10T20:40:39.464562+0000","last_clean":"2024-09-10T20:40:39.464562+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:40:39.464562+0000","last_undegraded":"2024-09-10T20:40:39.464562+0000","last_fullsized":"2024-09-10T20:40:39.464562+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28056,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461762+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:40:39.461762+0000","last_peered":"2024-09-10T20:40:39.461762+0000","last_clean":"2024-09-10T20:40:39.461762+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:40:39.461762+0000","last_undegraded":"2024-09-10T20:40:39.461762+0000","last_fullsized":"2024-09-10T20:40:39.461762+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21113,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464845+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:40:39.464845+0000","last_peered":"2024-09-10T20:40:39.464845+0000","last_clean":"2024-09-10T20:40:39.464845+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:40:39.464845+0000","last_undegraded":"2024-09-10T20:40:39.464845+0000","last_fullsized":"2024-09-10T20:40:39.464845+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464725+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:40:39.464725+0000","last_peered":"2024-09-10T20:40:39.464725+0000","last_clean":"2024-09-10T20:40:39.464725+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:40:39.464725+0000","last_undegraded":"2024-09-10T20:40:39.464725+0000","last_fullsized":"2024-09-10T20:40:39.464725+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27834,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464746+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:40:39.464746+0000","last_peered":"2024-09-10T20:40:39.464746+0000","last_clean":"2024-09-10T20:40:39.464746+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:40:39.464746+0000","last_undegraded":"2024-09-10T20:40:39.464746+0000","last_fullsized":"2024-09-10T20:40:39.464746+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":125,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462414+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:40:39.462414+0000","last_peered":"2024-09-10T20:40:39.462414+0000","last_clean":"2024-09-10T20:40:39.462414+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:40:39.462414+0000","last_undegraded":"2024-09-10T20:40:39.462414+0000","last_fullsized":"2024-09-10T20:40:39.462414+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21265,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.463026+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:40:39.463026+0000","last_peered":"2024-09-10T20:40:39.463026+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:40:39.463026+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464696+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:40:39.464696+0000","last_peered":"2024-09-10T20:40:39.464696+0000","last_clean":"2024-09-10T20:40:39.464696+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:40:39.464696+0000","last_undegraded":"2024-09-10T20:40:39.464696+0000","last_fullsized":"2024-09-10T20:40:39.464696+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464840+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:40:39.464840+0000","last_peered":"2024-09-10T20:40:39.464840+0000","last_clean":"2024-09-10T20:40:39.464840+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:40:39.464840+0000","last_undegraded":"2024-09-10T20:40:39.464840+0000","last_fullsized":"2024-09-10T20:40:39.464840+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27665,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464669+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:40:39.464669+0000","last_peered":"2024-09-10T20:40:39.464669+0000","last_clean":"2024-09-10T20:40:39.464669+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:40:39.464669+0000","last_undegraded":"2024-09-10T20:40:39.464669+0000","last_fullsized":"2024-09-10T20:40:39.464669+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12198,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27332,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":162799616,"data_stored":162767099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182547,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517192,"kb_used_data":57972,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854648,"statfs":{"total":96636764160,"available":96107159552,"internally_reserved":0,"allocated":59363328,"data_stored":59052355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986319,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":466828,"kb_used_data":101948,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93905012,"statfs":{"total":96636764160,"available":96158732288,"internally_reserved":0,"allocated":104394752,"data_stored":104087856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659265,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491728,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880112,"statfs":{"total":96636764160,"available":96133234688,"internally_reserved":0,"allocated":524288,"data_stored":191637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502984366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":103903232,"data_stored":103899619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:40:51.618 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:40:51.618 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'20658", 'reported_seq': 25005, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:50.232008+0000', 'last_change': '2024-09-10T20:40:37.857265+0000', 'last_active': '2024-09-10T20:40:50.232008+0000', 'last_peered': '2024-09-10T20:40:50.232008+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:40:50.232008+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1490, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20658, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2118, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1490}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:51.618 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:40:51.618 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20471", 'reported_seq': 20745, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:39.461976+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:40:39.461976+0000', 'last_peered': '2024-09-10T20:40:39.461976+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:40:39.461976+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17335, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3561}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:51.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:40:51.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'20608", 'reported_seq': 20915, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:39.462003+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:40:39.462003+0000', 'last_peered': '2024-09-10T20:40:39.462003+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:40:39.462003+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17203, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3486}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:51.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:40:51.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "51'20986", 'reported_seq': 21265, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:39.463026+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:40:39.463026+0000', 'last_peered': '2024-09-10T20:40:39.463026+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:40:39.463026+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17813, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3655}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:51.619 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:40:52.012 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:52.012 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 12198/6 objects degraded (203300.000%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":763,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1511165952,"bytes_avail":288399126528,"bytes_total":289910292480,"degraded_objects":12198,"degraded_total":6,"degraded_ratio":2033},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:40:52.026 INFO:tasks.rep_lost_unfound_delete.ceph_manager:no progress seen, keeping timeout for now 2024-09-10T20:40:52.026 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:40:52.333 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:52.334 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:40:52.347 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":434,"stamp":"2024-09-10T20:40:51.071440+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12198,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27334,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1475748,"kb_used_data":160432,"kb_used_omap":12,"kb_used_meta":1315251,"kb_avail":281639772,"statfs":{"total":289910292480,"available":288399126528,"internally_reserved":0,"allocated":164282368,"data_stored":163331848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1346817026},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-966,"num_objects":-2,"num_object_clones":0,"num_object_copies":-4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2720,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-20728,"num_write_kb":-41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-899,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.610933"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":25005,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:50.232008+0000","last_change":"2024-09-10T20:40:37.857265+0000","last_active":"2024-09-10T20:40:50.232008+0000","last_peered":"2024-09-10T20:40:50.232008+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:40:50.232008+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1490,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2118,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1490}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21084,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.463153+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:40:39.463153+0000","last_peered":"2024-09-10T20:40:39.463153+0000","last_clean":"2024-09-10T20:40:39.463153+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:40:39.463153+0000","last_undegraded":"2024-09-10T20:40:39.463153+0000","last_fullsized":"2024-09-10T20:40:39.463153+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28095,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464663+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:40:39.464663+0000","last_peered":"2024-09-10T20:40:39.464663+0000","last_clean":"2024-09-10T20:40:39.464663+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:40:39.464663+0000","last_undegraded":"2024-09-10T20:40:39.464663+0000","last_fullsized":"2024-09-10T20:40:39.464663+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20716,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.463009+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:40:39.463009+0000","last_peered":"2024-09-10T20:40:39.463009+0000","last_clean":"2024-09-10T20:40:39.463009+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:40:39.463009+0000","last_undegraded":"2024-09-10T20:40:39.463009+0000","last_fullsized":"2024-09-10T20:40:39.463009+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21227,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462338+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:40:39.462338+0000","last_peered":"2024-09-10T20:40:39.462338+0000","last_clean":"2024-09-10T20:40:39.462338+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:40:39.462338+0000","last_undegraded":"2024-09-10T20:40:39.462338+0000","last_fullsized":"2024-09-10T20:40:39.462338+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20745,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.461976+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:40:39.461976+0000","last_peered":"2024-09-10T20:40:39.461976+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:40:39.461976+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20915,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.462003+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:40:39.462003+0000","last_peered":"2024-09-10T20:40:39.462003+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:40:39.462003+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20892,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461791+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:40:39.461791+0000","last_peered":"2024-09-10T20:40:39.461791+0000","last_clean":"2024-09-10T20:40:39.461791+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:40:39.461791+0000","last_undegraded":"2024-09-10T20:40:39.461791+0000","last_fullsized":"2024-09-10T20:40:39.461791+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464374+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:40:39.464374+0000","last_peered":"2024-09-10T20:40:39.464374+0000","last_clean":"2024-09-10T20:40:39.464374+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:40:39.464374+0000","last_undegraded":"2024-09-10T20:40:39.464374+0000","last_fullsized":"2024-09-10T20:40:39.464374+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22134,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464659+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:40:39.464659+0000","last_peered":"2024-09-10T20:40:39.464659+0000","last_clean":"2024-09-10T20:40:39.464659+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:40:39.464659+0000","last_undegraded":"2024-09-10T20:40:39.464659+0000","last_fullsized":"2024-09-10T20:40:39.464659+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462542+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:40:39.462542+0000","last_peered":"2024-09-10T20:40:39.462542+0000","last_clean":"2024-09-10T20:40:39.462542+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:40:39.462542+0000","last_undegraded":"2024-09-10T20:40:39.462542+0000","last_fullsized":"2024-09-10T20:40:39.462542+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28120,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.949385+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:40:39.949385+0000","last_peered":"2024-09-10T20:40:39.949385+0000","last_clean":"2024-09-10T20:40:39.949385+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:40:39.949385+0000","last_undegraded":"2024-09-10T20:40:39.949385+0000","last_fullsized":"2024-09-10T20:40:39.949385+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":132,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461665+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:40:39.461665+0000","last_peered":"2024-09-10T20:40:39.461665+0000","last_clean":"2024-09-10T20:40:39.461665+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:40:39.461665+0000","last_undegraded":"2024-09-10T20:40:39.461665+0000","last_fullsized":"2024-09-10T20:40:39.461665+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28003,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462607+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:40:39.462607+0000","last_peered":"2024-09-10T20:40:39.462607+0000","last_clean":"2024-09-10T20:40:39.462607+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:40:39.462607+0000","last_undegraded":"2024-09-10T20:40:39.462607+0000","last_fullsized":"2024-09-10T20:40:39.462607+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464562+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:40:39.464562+0000","last_peered":"2024-09-10T20:40:39.464562+0000","last_clean":"2024-09-10T20:40:39.464562+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:40:39.464562+0000","last_undegraded":"2024-09-10T20:40:39.464562+0000","last_fullsized":"2024-09-10T20:40:39.464562+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28056,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461762+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:40:39.461762+0000","last_peered":"2024-09-10T20:40:39.461762+0000","last_clean":"2024-09-10T20:40:39.461762+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:40:39.461762+0000","last_undegraded":"2024-09-10T20:40:39.461762+0000","last_fullsized":"2024-09-10T20:40:39.461762+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21113,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464845+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:40:39.464845+0000","last_peered":"2024-09-10T20:40:39.464845+0000","last_clean":"2024-09-10T20:40:39.464845+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:40:39.464845+0000","last_undegraded":"2024-09-10T20:40:39.464845+0000","last_fullsized":"2024-09-10T20:40:39.464845+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464725+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:40:39.464725+0000","last_peered":"2024-09-10T20:40:39.464725+0000","last_clean":"2024-09-10T20:40:39.464725+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:40:39.464725+0000","last_undegraded":"2024-09-10T20:40:39.464725+0000","last_fullsized":"2024-09-10T20:40:39.464725+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27834,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464746+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:40:39.464746+0000","last_peered":"2024-09-10T20:40:39.464746+0000","last_clean":"2024-09-10T20:40:39.464746+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:40:39.464746+0000","last_undegraded":"2024-09-10T20:40:39.464746+0000","last_fullsized":"2024-09-10T20:40:39.464746+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":125,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462414+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:40:39.462414+0000","last_peered":"2024-09-10T20:40:39.462414+0000","last_clean":"2024-09-10T20:40:39.462414+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:40:39.462414+0000","last_undegraded":"2024-09-10T20:40:39.462414+0000","last_fullsized":"2024-09-10T20:40:39.462414+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21265,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.463026+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:40:39.463026+0000","last_peered":"2024-09-10T20:40:39.463026+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:40:39.463026+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464696+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:40:39.464696+0000","last_peered":"2024-09-10T20:40:39.464696+0000","last_clean":"2024-09-10T20:40:39.464696+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:40:39.464696+0000","last_undegraded":"2024-09-10T20:40:39.464696+0000","last_fullsized":"2024-09-10T20:40:39.464696+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464840+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:40:39.464840+0000","last_peered":"2024-09-10T20:40:39.464840+0000","last_clean":"2024-09-10T20:40:39.464840+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:40:39.464840+0000","last_undegraded":"2024-09-10T20:40:39.464840+0000","last_fullsized":"2024-09-10T20:40:39.464840+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27665,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464669+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:40:39.464669+0000","last_peered":"2024-09-10T20:40:39.464669+0000","last_clean":"2024-09-10T20:40:39.464669+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:40:39.464669+0000","last_undegraded":"2024-09-10T20:40:39.464669+0000","last_fullsized":"2024-09-10T20:40:39.464669+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12198,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27332,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":162799616,"data_stored":162767099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182547,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517192,"kb_used_data":57972,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854648,"statfs":{"total":96636764160,"available":96107159552,"internally_reserved":0,"allocated":59363328,"data_stored":59052355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986319,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":466828,"kb_used_data":101948,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93905012,"statfs":{"total":96636764160,"available":96158732288,"internally_reserved":0,"allocated":104394752,"data_stored":104087856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659265,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491728,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880112,"statfs":{"total":96636764160,"available":96133234688,"internally_reserved":0,"allocated":524288,"data_stored":191637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502984366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":103903232,"data_stored":103899619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:40:55.349 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:40:55.666 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:55.667 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:40:55.680 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":436,"stamp":"2024-09-10T20:40:55.072164+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12198,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27334,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1465508,"kb_used_data":158512,"kb_used_omap":12,"kb_used_meta":1306931,"kb_avail":281650012,"statfs":{"total":289910292480,"available":288409612288,"internally_reserved":0,"allocated":162316288,"data_stored":161365768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1338297346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1701,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1701,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002106"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":25005,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:50.232008+0000","last_change":"2024-09-10T20:40:37.857265+0000","last_active":"2024-09-10T20:40:50.232008+0000","last_peered":"2024-09-10T20:40:50.232008+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:40:50.232008+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1490,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2118,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1490}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21084,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.463153+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:40:39.463153+0000","last_peered":"2024-09-10T20:40:39.463153+0000","last_clean":"2024-09-10T20:40:39.463153+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:40:39.463153+0000","last_undegraded":"2024-09-10T20:40:39.463153+0000","last_fullsized":"2024-09-10T20:40:39.463153+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28095,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464663+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:40:39.464663+0000","last_peered":"2024-09-10T20:40:39.464663+0000","last_clean":"2024-09-10T20:40:39.464663+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:40:39.464663+0000","last_undegraded":"2024-09-10T20:40:39.464663+0000","last_fullsized":"2024-09-10T20:40:39.464663+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20716,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.463009+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:40:39.463009+0000","last_peered":"2024-09-10T20:40:39.463009+0000","last_clean":"2024-09-10T20:40:39.463009+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:40:39.463009+0000","last_undegraded":"2024-09-10T20:40:39.463009+0000","last_fullsized":"2024-09-10T20:40:39.463009+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21227,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462338+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:40:39.462338+0000","last_peered":"2024-09-10T20:40:39.462338+0000","last_clean":"2024-09-10T20:40:39.462338+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:40:39.462338+0000","last_undegraded":"2024-09-10T20:40:39.462338+0000","last_fullsized":"2024-09-10T20:40:39.462338+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20745,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.461976+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:40:39.461976+0000","last_peered":"2024-09-10T20:40:39.461976+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:40:39.461976+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20915,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.462003+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:40:39.462003+0000","last_peered":"2024-09-10T20:40:39.462003+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:40:39.462003+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20892,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461791+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:40:39.461791+0000","last_peered":"2024-09-10T20:40:39.461791+0000","last_clean":"2024-09-10T20:40:39.461791+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:40:39.461791+0000","last_undegraded":"2024-09-10T20:40:39.461791+0000","last_fullsized":"2024-09-10T20:40:39.461791+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464374+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:40:39.464374+0000","last_peered":"2024-09-10T20:40:39.464374+0000","last_clean":"2024-09-10T20:40:39.464374+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:40:39.464374+0000","last_undegraded":"2024-09-10T20:40:39.464374+0000","last_fullsized":"2024-09-10T20:40:39.464374+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22134,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464659+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:40:39.464659+0000","last_peered":"2024-09-10T20:40:39.464659+0000","last_clean":"2024-09-10T20:40:39.464659+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:40:39.464659+0000","last_undegraded":"2024-09-10T20:40:39.464659+0000","last_fullsized":"2024-09-10T20:40:39.464659+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462542+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:40:39.462542+0000","last_peered":"2024-09-10T20:40:39.462542+0000","last_clean":"2024-09-10T20:40:39.462542+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:40:39.462542+0000","last_undegraded":"2024-09-10T20:40:39.462542+0000","last_fullsized":"2024-09-10T20:40:39.462542+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28120,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.949385+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:40:39.949385+0000","last_peered":"2024-09-10T20:40:39.949385+0000","last_clean":"2024-09-10T20:40:39.949385+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:40:39.949385+0000","last_undegraded":"2024-09-10T20:40:39.949385+0000","last_fullsized":"2024-09-10T20:40:39.949385+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":132,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461665+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:40:39.461665+0000","last_peered":"2024-09-10T20:40:39.461665+0000","last_clean":"2024-09-10T20:40:39.461665+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:40:39.461665+0000","last_undegraded":"2024-09-10T20:40:39.461665+0000","last_fullsized":"2024-09-10T20:40:39.461665+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28003,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462607+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:40:39.462607+0000","last_peered":"2024-09-10T20:40:39.462607+0000","last_clean":"2024-09-10T20:40:39.462607+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:40:39.462607+0000","last_undegraded":"2024-09-10T20:40:39.462607+0000","last_fullsized":"2024-09-10T20:40:39.462607+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464562+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:40:39.464562+0000","last_peered":"2024-09-10T20:40:39.464562+0000","last_clean":"2024-09-10T20:40:39.464562+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:40:39.464562+0000","last_undegraded":"2024-09-10T20:40:39.464562+0000","last_fullsized":"2024-09-10T20:40:39.464562+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28056,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461762+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:40:39.461762+0000","last_peered":"2024-09-10T20:40:39.461762+0000","last_clean":"2024-09-10T20:40:39.461762+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:40:39.461762+0000","last_undegraded":"2024-09-10T20:40:39.461762+0000","last_fullsized":"2024-09-10T20:40:39.461762+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21113,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464845+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:40:39.464845+0000","last_peered":"2024-09-10T20:40:39.464845+0000","last_clean":"2024-09-10T20:40:39.464845+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:40:39.464845+0000","last_undegraded":"2024-09-10T20:40:39.464845+0000","last_fullsized":"2024-09-10T20:40:39.464845+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464725+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:40:39.464725+0000","last_peered":"2024-09-10T20:40:39.464725+0000","last_clean":"2024-09-10T20:40:39.464725+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:40:39.464725+0000","last_undegraded":"2024-09-10T20:40:39.464725+0000","last_fullsized":"2024-09-10T20:40:39.464725+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27834,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464746+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:40:39.464746+0000","last_peered":"2024-09-10T20:40:39.464746+0000","last_clean":"2024-09-10T20:40:39.464746+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:40:39.464746+0000","last_undegraded":"2024-09-10T20:40:39.464746+0000","last_fullsized":"2024-09-10T20:40:39.464746+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":125,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462414+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:40:39.462414+0000","last_peered":"2024-09-10T20:40:39.462414+0000","last_clean":"2024-09-10T20:40:39.462414+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:40:39.462414+0000","last_undegraded":"2024-09-10T20:40:39.462414+0000","last_fullsized":"2024-09-10T20:40:39.462414+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21265,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.463026+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:40:39.463026+0000","last_peered":"2024-09-10T20:40:39.463026+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:40:39.463026+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464696+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:40:39.464696+0000","last_peered":"2024-09-10T20:40:39.464696+0000","last_clean":"2024-09-10T20:40:39.464696+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:40:39.464696+0000","last_undegraded":"2024-09-10T20:40:39.464696+0000","last_fullsized":"2024-09-10T20:40:39.464696+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464840+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:40:39.464840+0000","last_peered":"2024-09-10T20:40:39.464840+0000","last_clean":"2024-09-10T20:40:39.464840+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:40:39.464840+0000","last_undegraded":"2024-09-10T20:40:39.464840+0000","last_fullsized":"2024-09-10T20:40:39.464840+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27665,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464669+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:40:39.464669+0000","last_peered":"2024-09-10T20:40:39.464669+0000","last_clean":"2024-09-10T20:40:39.464669+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:40:39.464669+0000","last_undegraded":"2024-09-10T20:40:39.464669+0000","last_fullsized":"2024-09-10T20:40:39.464669+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12198,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27332,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":160342016,"data_stored":160309499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182547,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517192,"kb_used_data":57972,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854648,"statfs":{"total":96636764160,"available":96107159552,"internally_reserved":0,"allocated":59363328,"data_stored":59052355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986320,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":464908,"kb_used_data":100028,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93906932,"statfs":{"total":96636764160,"available":96160698368,"internally_reserved":0,"allocated":102428672,"data_stored":102121776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659266,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483408,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888432,"statfs":{"total":96636764160,"available":96141754368,"internally_reserved":0,"allocated":524288,"data_stored":191637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":101445632,"data_stored":101442019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:40:55.681 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:40:55.986 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:55.987 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:40:55.999 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":436,"stamp":"2024-09-10T20:40:55.072164+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12198,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27334,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1465508,"kb_used_data":158512,"kb_used_omap":12,"kb_used_meta":1306931,"kb_avail":281650012,"statfs":{"total":289910292480,"available":288409612288,"internally_reserved":0,"allocated":162316288,"data_stored":161365768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1338297346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1701,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1701,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002106"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":25005,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:50.232008+0000","last_change":"2024-09-10T20:40:37.857265+0000","last_active":"2024-09-10T20:40:50.232008+0000","last_peered":"2024-09-10T20:40:50.232008+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:40:50.232008+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1490,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2118,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1490}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21084,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.463153+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:40:39.463153+0000","last_peered":"2024-09-10T20:40:39.463153+0000","last_clean":"2024-09-10T20:40:39.463153+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:40:39.463153+0000","last_undegraded":"2024-09-10T20:40:39.463153+0000","last_fullsized":"2024-09-10T20:40:39.463153+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28095,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464663+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:40:39.464663+0000","last_peered":"2024-09-10T20:40:39.464663+0000","last_clean":"2024-09-10T20:40:39.464663+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:40:39.464663+0000","last_undegraded":"2024-09-10T20:40:39.464663+0000","last_fullsized":"2024-09-10T20:40:39.464663+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20716,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.463009+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:40:39.463009+0000","last_peered":"2024-09-10T20:40:39.463009+0000","last_clean":"2024-09-10T20:40:39.463009+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:40:39.463009+0000","last_undegraded":"2024-09-10T20:40:39.463009+0000","last_fullsized":"2024-09-10T20:40:39.463009+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21227,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462338+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:40:39.462338+0000","last_peered":"2024-09-10T20:40:39.462338+0000","last_clean":"2024-09-10T20:40:39.462338+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:40:39.462338+0000","last_undegraded":"2024-09-10T20:40:39.462338+0000","last_fullsized":"2024-09-10T20:40:39.462338+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20745,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.461976+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:40:39.461976+0000","last_peered":"2024-09-10T20:40:39.461976+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:40:39.461976+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20915,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.462003+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:40:39.462003+0000","last_peered":"2024-09-10T20:40:39.462003+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:40:39.462003+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20892,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461791+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:40:39.461791+0000","last_peered":"2024-09-10T20:40:39.461791+0000","last_clean":"2024-09-10T20:40:39.461791+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:40:39.461791+0000","last_undegraded":"2024-09-10T20:40:39.461791+0000","last_fullsized":"2024-09-10T20:40:39.461791+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464374+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:40:39.464374+0000","last_peered":"2024-09-10T20:40:39.464374+0000","last_clean":"2024-09-10T20:40:39.464374+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:40:39.464374+0000","last_undegraded":"2024-09-10T20:40:39.464374+0000","last_fullsized":"2024-09-10T20:40:39.464374+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22134,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464659+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:40:39.464659+0000","last_peered":"2024-09-10T20:40:39.464659+0000","last_clean":"2024-09-10T20:40:39.464659+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:40:39.464659+0000","last_undegraded":"2024-09-10T20:40:39.464659+0000","last_fullsized":"2024-09-10T20:40:39.464659+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462542+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:40:39.462542+0000","last_peered":"2024-09-10T20:40:39.462542+0000","last_clean":"2024-09-10T20:40:39.462542+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:40:39.462542+0000","last_undegraded":"2024-09-10T20:40:39.462542+0000","last_fullsized":"2024-09-10T20:40:39.462542+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28120,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.949385+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:40:39.949385+0000","last_peered":"2024-09-10T20:40:39.949385+0000","last_clean":"2024-09-10T20:40:39.949385+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:40:39.949385+0000","last_undegraded":"2024-09-10T20:40:39.949385+0000","last_fullsized":"2024-09-10T20:40:39.949385+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":132,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461665+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:40:39.461665+0000","last_peered":"2024-09-10T20:40:39.461665+0000","last_clean":"2024-09-10T20:40:39.461665+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:40:39.461665+0000","last_undegraded":"2024-09-10T20:40:39.461665+0000","last_fullsized":"2024-09-10T20:40:39.461665+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28003,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462607+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:40:39.462607+0000","last_peered":"2024-09-10T20:40:39.462607+0000","last_clean":"2024-09-10T20:40:39.462607+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:40:39.462607+0000","last_undegraded":"2024-09-10T20:40:39.462607+0000","last_fullsized":"2024-09-10T20:40:39.462607+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464562+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:40:39.464562+0000","last_peered":"2024-09-10T20:40:39.464562+0000","last_clean":"2024-09-10T20:40:39.464562+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:40:39.464562+0000","last_undegraded":"2024-09-10T20:40:39.464562+0000","last_fullsized":"2024-09-10T20:40:39.464562+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28056,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461762+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:40:39.461762+0000","last_peered":"2024-09-10T20:40:39.461762+0000","last_clean":"2024-09-10T20:40:39.461762+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:40:39.461762+0000","last_undegraded":"2024-09-10T20:40:39.461762+0000","last_fullsized":"2024-09-10T20:40:39.461762+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21113,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464845+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:40:39.464845+0000","last_peered":"2024-09-10T20:40:39.464845+0000","last_clean":"2024-09-10T20:40:39.464845+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:40:39.464845+0000","last_undegraded":"2024-09-10T20:40:39.464845+0000","last_fullsized":"2024-09-10T20:40:39.464845+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464725+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:40:39.464725+0000","last_peered":"2024-09-10T20:40:39.464725+0000","last_clean":"2024-09-10T20:40:39.464725+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:40:39.464725+0000","last_undegraded":"2024-09-10T20:40:39.464725+0000","last_fullsized":"2024-09-10T20:40:39.464725+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27834,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464746+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:40:39.464746+0000","last_peered":"2024-09-10T20:40:39.464746+0000","last_clean":"2024-09-10T20:40:39.464746+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:40:39.464746+0000","last_undegraded":"2024-09-10T20:40:39.464746+0000","last_fullsized":"2024-09-10T20:40:39.464746+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":125,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462414+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:40:39.462414+0000","last_peered":"2024-09-10T20:40:39.462414+0000","last_clean":"2024-09-10T20:40:39.462414+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:40:39.462414+0000","last_undegraded":"2024-09-10T20:40:39.462414+0000","last_fullsized":"2024-09-10T20:40:39.462414+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21265,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.463026+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:40:39.463026+0000","last_peered":"2024-09-10T20:40:39.463026+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:40:39.463026+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464696+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:40:39.464696+0000","last_peered":"2024-09-10T20:40:39.464696+0000","last_clean":"2024-09-10T20:40:39.464696+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:40:39.464696+0000","last_undegraded":"2024-09-10T20:40:39.464696+0000","last_fullsized":"2024-09-10T20:40:39.464696+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464840+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:40:39.464840+0000","last_peered":"2024-09-10T20:40:39.464840+0000","last_clean":"2024-09-10T20:40:39.464840+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:40:39.464840+0000","last_undegraded":"2024-09-10T20:40:39.464840+0000","last_fullsized":"2024-09-10T20:40:39.464840+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27665,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464669+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:40:39.464669+0000","last_peered":"2024-09-10T20:40:39.464669+0000","last_clean":"2024-09-10T20:40:39.464669+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:40:39.464669+0000","last_undegraded":"2024-09-10T20:40:39.464669+0000","last_fullsized":"2024-09-10T20:40:39.464669+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12198,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27332,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":160342016,"data_stored":160309499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182547,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517192,"kb_used_data":57972,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854648,"statfs":{"total":96636764160,"available":96107159552,"internally_reserved":0,"allocated":59363328,"data_stored":59052355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986320,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":464908,"kb_used_data":100028,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93906932,"statfs":{"total":96636764160,"available":96160698368,"internally_reserved":0,"allocated":102428672,"data_stored":102121776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659266,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483408,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888432,"statfs":{"total":96636764160,"available":96141754368,"internally_reserved":0,"allocated":524288,"data_stored":191637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":101445632,"data_stored":101442019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:40:56.000 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:40:56.000 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'20658", 'reported_seq': 25005, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:50.232008+0000', 'last_change': '2024-09-10T20:40:37.857265+0000', 'last_active': '2024-09-10T20:40:50.232008+0000', 'last_peered': '2024-09-10T20:40:50.232008+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:40:50.232008+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1490, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20658, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2118, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1490}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:56.000 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:40:56.001 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20471", 'reported_seq': 20745, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:39.461976+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:40:39.461976+0000', 'last_peered': '2024-09-10T20:40:39.461976+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:40:39.461976+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17335, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3561}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:56.001 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:40:56.001 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'20608", 'reported_seq': 20915, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:39.462003+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:40:39.462003+0000', 'last_peered': '2024-09-10T20:40:39.462003+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:40:39.462003+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17203, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3486}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:56.001 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:40:56.001 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "51'20986", 'reported_seq': 21265, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:39.463026+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:40:39.463026+0000', 'last_peered': '2024-09-10T20:40:39.463026+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:40:39.463026+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17813, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3655}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:40:56.001 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:40:56.379 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:56.379 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 12198/6 objects degraded (203300.000%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":768,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1500680192,"bytes_avail":288409612288,"bytes_total":289910292480,"degraded_objects":12198,"degraded_total":6,"degraded_ratio":2033,"recovering_objects_per_sec":141,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1701,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:40:56.392 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:40:56.392 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:40:56.707 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:40:56.708 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:40:56.721 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":436,"stamp":"2024-09-10T20:40:55.072164+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12198,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27334,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1465508,"kb_used_data":158512,"kb_used_omap":12,"kb_used_meta":1306931,"kb_avail":281650012,"statfs":{"total":289910292480,"available":288409612288,"internally_reserved":0,"allocated":162316288,"data_stored":161365768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1338297346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1701,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1701,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002106"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":25005,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:50.232008+0000","last_change":"2024-09-10T20:40:37.857265+0000","last_active":"2024-09-10T20:40:50.232008+0000","last_peered":"2024-09-10T20:40:50.232008+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:40:50.232008+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1490,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2118,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1490}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21084,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.463153+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:40:39.463153+0000","last_peered":"2024-09-10T20:40:39.463153+0000","last_clean":"2024-09-10T20:40:39.463153+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:40:39.463153+0000","last_undegraded":"2024-09-10T20:40:39.463153+0000","last_fullsized":"2024-09-10T20:40:39.463153+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28095,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464663+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:40:39.464663+0000","last_peered":"2024-09-10T20:40:39.464663+0000","last_clean":"2024-09-10T20:40:39.464663+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:40:39.464663+0000","last_undegraded":"2024-09-10T20:40:39.464663+0000","last_fullsized":"2024-09-10T20:40:39.464663+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20716,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.463009+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:40:39.463009+0000","last_peered":"2024-09-10T20:40:39.463009+0000","last_clean":"2024-09-10T20:40:39.463009+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:40:39.463009+0000","last_undegraded":"2024-09-10T20:40:39.463009+0000","last_fullsized":"2024-09-10T20:40:39.463009+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21227,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462338+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:40:39.462338+0000","last_peered":"2024-09-10T20:40:39.462338+0000","last_clean":"2024-09-10T20:40:39.462338+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:40:39.462338+0000","last_undegraded":"2024-09-10T20:40:39.462338+0000","last_fullsized":"2024-09-10T20:40:39.462338+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20745,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.461976+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:40:39.461976+0000","last_peered":"2024-09-10T20:40:39.461976+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:40:39.461976+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20915,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.462003+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:40:39.462003+0000","last_peered":"2024-09-10T20:40:39.462003+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:40:39.462003+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20892,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461791+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:40:39.461791+0000","last_peered":"2024-09-10T20:40:39.461791+0000","last_clean":"2024-09-10T20:40:39.461791+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:40:39.461791+0000","last_undegraded":"2024-09-10T20:40:39.461791+0000","last_fullsized":"2024-09-10T20:40:39.461791+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464374+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:40:39.464374+0000","last_peered":"2024-09-10T20:40:39.464374+0000","last_clean":"2024-09-10T20:40:39.464374+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:40:39.464374+0000","last_undegraded":"2024-09-10T20:40:39.464374+0000","last_fullsized":"2024-09-10T20:40:39.464374+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22134,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464659+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:40:39.464659+0000","last_peered":"2024-09-10T20:40:39.464659+0000","last_clean":"2024-09-10T20:40:39.464659+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:40:39.464659+0000","last_undegraded":"2024-09-10T20:40:39.464659+0000","last_fullsized":"2024-09-10T20:40:39.464659+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462542+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:40:39.462542+0000","last_peered":"2024-09-10T20:40:39.462542+0000","last_clean":"2024-09-10T20:40:39.462542+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:40:39.462542+0000","last_undegraded":"2024-09-10T20:40:39.462542+0000","last_fullsized":"2024-09-10T20:40:39.462542+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28120,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.949385+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:40:39.949385+0000","last_peered":"2024-09-10T20:40:39.949385+0000","last_clean":"2024-09-10T20:40:39.949385+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:40:39.949385+0000","last_undegraded":"2024-09-10T20:40:39.949385+0000","last_fullsized":"2024-09-10T20:40:39.949385+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":132,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461665+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:40:39.461665+0000","last_peered":"2024-09-10T20:40:39.461665+0000","last_clean":"2024-09-10T20:40:39.461665+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:40:39.461665+0000","last_undegraded":"2024-09-10T20:40:39.461665+0000","last_fullsized":"2024-09-10T20:40:39.461665+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28003,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462607+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:40:39.462607+0000","last_peered":"2024-09-10T20:40:39.462607+0000","last_clean":"2024-09-10T20:40:39.462607+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:40:39.462607+0000","last_undegraded":"2024-09-10T20:40:39.462607+0000","last_fullsized":"2024-09-10T20:40:39.462607+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464562+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:40:39.464562+0000","last_peered":"2024-09-10T20:40:39.464562+0000","last_clean":"2024-09-10T20:40:39.464562+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:40:39.464562+0000","last_undegraded":"2024-09-10T20:40:39.464562+0000","last_fullsized":"2024-09-10T20:40:39.464562+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28056,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461762+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:40:39.461762+0000","last_peered":"2024-09-10T20:40:39.461762+0000","last_clean":"2024-09-10T20:40:39.461762+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:40:39.461762+0000","last_undegraded":"2024-09-10T20:40:39.461762+0000","last_fullsized":"2024-09-10T20:40:39.461762+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21113,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464845+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:40:39.464845+0000","last_peered":"2024-09-10T20:40:39.464845+0000","last_clean":"2024-09-10T20:40:39.464845+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:40:39.464845+0000","last_undegraded":"2024-09-10T20:40:39.464845+0000","last_fullsized":"2024-09-10T20:40:39.464845+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464725+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:40:39.464725+0000","last_peered":"2024-09-10T20:40:39.464725+0000","last_clean":"2024-09-10T20:40:39.464725+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:40:39.464725+0000","last_undegraded":"2024-09-10T20:40:39.464725+0000","last_fullsized":"2024-09-10T20:40:39.464725+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27834,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464746+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:40:39.464746+0000","last_peered":"2024-09-10T20:40:39.464746+0000","last_clean":"2024-09-10T20:40:39.464746+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:40:39.464746+0000","last_undegraded":"2024-09-10T20:40:39.464746+0000","last_fullsized":"2024-09-10T20:40:39.464746+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":125,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462414+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:40:39.462414+0000","last_peered":"2024-09-10T20:40:39.462414+0000","last_clean":"2024-09-10T20:40:39.462414+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:40:39.462414+0000","last_undegraded":"2024-09-10T20:40:39.462414+0000","last_fullsized":"2024-09-10T20:40:39.462414+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21265,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.463026+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:40:39.463026+0000","last_peered":"2024-09-10T20:40:39.463026+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:40:39.463026+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464696+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:40:39.464696+0000","last_peered":"2024-09-10T20:40:39.464696+0000","last_clean":"2024-09-10T20:40:39.464696+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:40:39.464696+0000","last_undegraded":"2024-09-10T20:40:39.464696+0000","last_fullsized":"2024-09-10T20:40:39.464696+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464840+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:40:39.464840+0000","last_peered":"2024-09-10T20:40:39.464840+0000","last_clean":"2024-09-10T20:40:39.464840+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:40:39.464840+0000","last_undegraded":"2024-09-10T20:40:39.464840+0000","last_fullsized":"2024-09-10T20:40:39.464840+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27665,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464669+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:40:39.464669+0000","last_peered":"2024-09-10T20:40:39.464669+0000","last_clean":"2024-09-10T20:40:39.464669+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:40:39.464669+0000","last_undegraded":"2024-09-10T20:40:39.464669+0000","last_fullsized":"2024-09-10T20:40:39.464669+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12198,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27332,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":160342016,"data_stored":160309499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182547,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517192,"kb_used_data":57972,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854648,"statfs":{"total":96636764160,"available":96107159552,"internally_reserved":0,"allocated":59363328,"data_stored":59052355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986320,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":464908,"kb_used_data":100028,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93906932,"statfs":{"total":96636764160,"available":96160698368,"internally_reserved":0,"allocated":102428672,"data_stored":102121776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659266,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483408,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888432,"statfs":{"total":96636764160,"available":96141754368,"internally_reserved":0,"allocated":524288,"data_stored":191637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":101445632,"data_stored":101442019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:40:59.723 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:00.155 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:00.156 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:00.170 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":440,"stamp":"2024-09-10T20:40:59.072902+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11213,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28319,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1473828,"kb_used_data":158512,"kb_used_omap":12,"kb_used_meta":1315251,"kb_avail":281641692,"statfs":{"total":289910292480,"available":288401092608,"internally_reserved":0,"allocated":162316288,"data_stored":161365768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1346817026},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-985,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":985,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001461"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":26975,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:55.229854+0000","last_change":"2024-09-10T20:40:37.857265+0000","last_active":"2024-09-10T20:40:55.229854+0000","last_peered":"2024-09-10T20:40:55.229854+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:40:55.229854+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":505,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3103,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-505}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21084,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.463153+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:40:39.463153+0000","last_peered":"2024-09-10T20:40:39.463153+0000","last_clean":"2024-09-10T20:40:39.463153+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:40:39.463153+0000","last_undegraded":"2024-09-10T20:40:39.463153+0000","last_fullsized":"2024-09-10T20:40:39.463153+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28095,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464663+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:40:39.464663+0000","last_peered":"2024-09-10T20:40:39.464663+0000","last_clean":"2024-09-10T20:40:39.464663+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:40:39.464663+0000","last_undegraded":"2024-09-10T20:40:39.464663+0000","last_fullsized":"2024-09-10T20:40:39.464663+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20716,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.463009+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:40:39.463009+0000","last_peered":"2024-09-10T20:40:39.463009+0000","last_clean":"2024-09-10T20:40:39.463009+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:40:39.463009+0000","last_undegraded":"2024-09-10T20:40:39.463009+0000","last_fullsized":"2024-09-10T20:40:39.463009+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21227,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462338+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:40:39.462338+0000","last_peered":"2024-09-10T20:40:39.462338+0000","last_clean":"2024-09-10T20:40:39.462338+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:40:39.462338+0000","last_undegraded":"2024-09-10T20:40:39.462338+0000","last_fullsized":"2024-09-10T20:40:39.462338+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20745,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.461976+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:40:39.461976+0000","last_peered":"2024-09-10T20:40:39.461976+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:40:39.461976+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20916,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.464521+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:40:39.464521+0000","last_peered":"2024-09-10T20:40:39.464521+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:40:39.464521+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20892,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461791+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:40:39.461791+0000","last_peered":"2024-09-10T20:40:39.461791+0000","last_clean":"2024-09-10T20:40:39.461791+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:40:39.461791+0000","last_undegraded":"2024-09-10T20:40:39.461791+0000","last_fullsized":"2024-09-10T20:40:39.461791+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464374+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:40:39.464374+0000","last_peered":"2024-09-10T20:40:39.464374+0000","last_clean":"2024-09-10T20:40:39.464374+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:40:39.464374+0000","last_undegraded":"2024-09-10T20:40:39.464374+0000","last_fullsized":"2024-09-10T20:40:39.464374+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22134,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464659+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:40:39.464659+0000","last_peered":"2024-09-10T20:40:39.464659+0000","last_clean":"2024-09-10T20:40:39.464659+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:40:39.464659+0000","last_undegraded":"2024-09-10T20:40:39.464659+0000","last_fullsized":"2024-09-10T20:40:39.464659+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462542+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:40:39.462542+0000","last_peered":"2024-09-10T20:40:39.462542+0000","last_clean":"2024-09-10T20:40:39.462542+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:40:39.462542+0000","last_undegraded":"2024-09-10T20:40:39.462542+0000","last_fullsized":"2024-09-10T20:40:39.462542+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28120,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.949385+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:40:39.949385+0000","last_peered":"2024-09-10T20:40:39.949385+0000","last_clean":"2024-09-10T20:40:39.949385+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:40:39.949385+0000","last_undegraded":"2024-09-10T20:40:39.949385+0000","last_fullsized":"2024-09-10T20:40:39.949385+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":132,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461665+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:40:39.461665+0000","last_peered":"2024-09-10T20:40:39.461665+0000","last_clean":"2024-09-10T20:40:39.461665+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:40:39.461665+0000","last_undegraded":"2024-09-10T20:40:39.461665+0000","last_fullsized":"2024-09-10T20:40:39.461665+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28003,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462607+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:40:39.462607+0000","last_peered":"2024-09-10T20:40:39.462607+0000","last_clean":"2024-09-10T20:40:39.462607+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:40:39.462607+0000","last_undegraded":"2024-09-10T20:40:39.462607+0000","last_fullsized":"2024-09-10T20:40:39.462607+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464562+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:40:39.464562+0000","last_peered":"2024-09-10T20:40:39.464562+0000","last_clean":"2024-09-10T20:40:39.464562+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:40:39.464562+0000","last_undegraded":"2024-09-10T20:40:39.464562+0000","last_fullsized":"2024-09-10T20:40:39.464562+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28056,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461762+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:40:39.461762+0000","last_peered":"2024-09-10T20:40:39.461762+0000","last_clean":"2024-09-10T20:40:39.461762+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:40:39.461762+0000","last_undegraded":"2024-09-10T20:40:39.461762+0000","last_fullsized":"2024-09-10T20:40:39.461762+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21113,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464845+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:40:39.464845+0000","last_peered":"2024-09-10T20:40:39.464845+0000","last_clean":"2024-09-10T20:40:39.464845+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:40:39.464845+0000","last_undegraded":"2024-09-10T20:40:39.464845+0000","last_fullsized":"2024-09-10T20:40:39.464845+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464725+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:40:39.464725+0000","last_peered":"2024-09-10T20:40:39.464725+0000","last_clean":"2024-09-10T20:40:39.464725+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:40:39.464725+0000","last_undegraded":"2024-09-10T20:40:39.464725+0000","last_fullsized":"2024-09-10T20:40:39.464725+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27834,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464746+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:40:39.464746+0000","last_peered":"2024-09-10T20:40:39.464746+0000","last_clean":"2024-09-10T20:40:39.464746+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:40:39.464746+0000","last_undegraded":"2024-09-10T20:40:39.464746+0000","last_fullsized":"2024-09-10T20:40:39.464746+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":125,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462414+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:40:39.462414+0000","last_peered":"2024-09-10T20:40:39.462414+0000","last_clean":"2024-09-10T20:40:39.462414+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:40:39.462414+0000","last_undegraded":"2024-09-10T20:40:39.462414+0000","last_fullsized":"2024-09-10T20:40:39.462414+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21265,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.463026+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:40:39.463026+0000","last_peered":"2024-09-10T20:40:39.463026+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:40:39.463026+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464696+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:40:39.464696+0000","last_peered":"2024-09-10T20:40:39.464696+0000","last_clean":"2024-09-10T20:40:39.464696+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:40:39.464696+0000","last_undegraded":"2024-09-10T20:40:39.464696+0000","last_fullsized":"2024-09-10T20:40:39.464696+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464922+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:40:39.464922+0000","last_peered":"2024-09-10T20:40:39.464922+0000","last_clean":"2024-09-10T20:40:39.464922+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:40:39.464922+0000","last_undegraded":"2024-09-10T20:40:39.464922+0000","last_fullsized":"2024-09-10T20:40:39.464922+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27665,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464669+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:40:39.464669+0000","last_peered":"2024-09-10T20:40:39.464669+0000","last_clean":"2024-09-10T20:40:39.464669+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:40:39.464669+0000","last_undegraded":"2024-09-10T20:40:39.464669+0000","last_fullsized":"2024-09-10T20:40:39.464669+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11213,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28317,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":160342016,"data_stored":160309499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182548,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517192,"kb_used_data":57972,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854648,"statfs":{"total":96636764160,"available":96107159552,"internally_reserved":0,"allocated":59363328,"data_stored":59052355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986320,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":464908,"kb_used_data":100028,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93906932,"statfs":{"total":96636764160,"available":96160698368,"internally_reserved":0,"allocated":102428672,"data_stored":102121776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659267,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491728,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880112,"statfs":{"total":96636764160,"available":96133234688,"internally_reserved":0,"allocated":524288,"data_stored":191637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502984366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":101445632,"data_stored":101442019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:41:00.172 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:00.513 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:00.513 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:00.525 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":440,"stamp":"2024-09-10T20:40:59.072902+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11213,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28319,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1473828,"kb_used_data":158512,"kb_used_omap":12,"kb_used_meta":1315251,"kb_avail":281641692,"statfs":{"total":289910292480,"available":288401092608,"internally_reserved":0,"allocated":162316288,"data_stored":161365768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1346817026},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-985,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":985,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001461"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":26975,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:55.229854+0000","last_change":"2024-09-10T20:40:37.857265+0000","last_active":"2024-09-10T20:40:55.229854+0000","last_peered":"2024-09-10T20:40:55.229854+0000","last_clean":"2024-09-10T20:30:03.300566+0000","last_became_active":"2024-09-10T20:30:05.760498+0000","last_became_peered":"2024-09-10T20:30:05.760498+0000","last_unstale":"2024-09-10T20:40:55.229854+0000","last_undegraded":"2024-09-10T20:30:05.381652+0000","last_fullsized":"2024-09-10T20:30:05.306166+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":505,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3103,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-505}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21084,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.463153+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:40:39.463153+0000","last_peered":"2024-09-10T20:40:39.463153+0000","last_clean":"2024-09-10T20:40:39.463153+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:40:39.463153+0000","last_undegraded":"2024-09-10T20:40:39.463153+0000","last_fullsized":"2024-09-10T20:40:39.463153+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28095,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464663+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:40:39.464663+0000","last_peered":"2024-09-10T20:40:39.464663+0000","last_clean":"2024-09-10T20:40:39.464663+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:40:39.464663+0000","last_undegraded":"2024-09-10T20:40:39.464663+0000","last_fullsized":"2024-09-10T20:40:39.464663+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20716,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.463009+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:40:39.463009+0000","last_peered":"2024-09-10T20:40:39.463009+0000","last_clean":"2024-09-10T20:40:39.463009+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:40:39.463009+0000","last_undegraded":"2024-09-10T20:40:39.463009+0000","last_fullsized":"2024-09-10T20:40:39.463009+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21227,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462338+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:40:39.462338+0000","last_peered":"2024-09-10T20:40:39.462338+0000","last_clean":"2024-09-10T20:40:39.462338+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:40:39.462338+0000","last_undegraded":"2024-09-10T20:40:39.462338+0000","last_fullsized":"2024-09-10T20:40:39.462338+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20745,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.461976+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:40:39.461976+0000","last_peered":"2024-09-10T20:40:39.461976+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:40:39.461976+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20916,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.464521+0000","last_change":"2024-09-10T20:30:05.789134+0000","last_active":"2024-09-10T20:40:39.464521+0000","last_peered":"2024-09-10T20:40:39.464521+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:40:39.464521+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20892,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461791+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:40:39.461791+0000","last_peered":"2024-09-10T20:40:39.461791+0000","last_clean":"2024-09-10T20:40:39.461791+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:40:39.461791+0000","last_undegraded":"2024-09-10T20:40:39.461791+0000","last_fullsized":"2024-09-10T20:40:39.461791+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464374+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:40:39.464374+0000","last_peered":"2024-09-10T20:40:39.464374+0000","last_clean":"2024-09-10T20:40:39.464374+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:40:39.464374+0000","last_undegraded":"2024-09-10T20:40:39.464374+0000","last_fullsized":"2024-09-10T20:40:39.464374+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22134,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464659+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:40:39.464659+0000","last_peered":"2024-09-10T20:40:39.464659+0000","last_clean":"2024-09-10T20:40:39.464659+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:40:39.464659+0000","last_undegraded":"2024-09-10T20:40:39.464659+0000","last_fullsized":"2024-09-10T20:40:39.464659+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462542+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:40:39.462542+0000","last_peered":"2024-09-10T20:40:39.462542+0000","last_clean":"2024-09-10T20:40:39.462542+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:40:39.462542+0000","last_undegraded":"2024-09-10T20:40:39.462542+0000","last_fullsized":"2024-09-10T20:40:39.462542+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28120,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.949385+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:40:39.949385+0000","last_peered":"2024-09-10T20:40:39.949385+0000","last_clean":"2024-09-10T20:40:39.949385+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:40:39.949385+0000","last_undegraded":"2024-09-10T20:40:39.949385+0000","last_fullsized":"2024-09-10T20:40:39.949385+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":132,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461665+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:40:39.461665+0000","last_peered":"2024-09-10T20:40:39.461665+0000","last_clean":"2024-09-10T20:40:39.461665+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:40:39.461665+0000","last_undegraded":"2024-09-10T20:40:39.461665+0000","last_fullsized":"2024-09-10T20:40:39.461665+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28003,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462607+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:40:39.462607+0000","last_peered":"2024-09-10T20:40:39.462607+0000","last_clean":"2024-09-10T20:40:39.462607+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:40:39.462607+0000","last_undegraded":"2024-09-10T20:40:39.462607+0000","last_fullsized":"2024-09-10T20:40:39.462607+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464562+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:40:39.464562+0000","last_peered":"2024-09-10T20:40:39.464562+0000","last_clean":"2024-09-10T20:40:39.464562+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:40:39.464562+0000","last_undegraded":"2024-09-10T20:40:39.464562+0000","last_fullsized":"2024-09-10T20:40:39.464562+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28056,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.461762+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:40:39.461762+0000","last_peered":"2024-09-10T20:40:39.461762+0000","last_clean":"2024-09-10T20:40:39.461762+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:40:39.461762+0000","last_undegraded":"2024-09-10T20:40:39.461762+0000","last_fullsized":"2024-09-10T20:40:39.461762+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21113,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464845+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:40:39.464845+0000","last_peered":"2024-09-10T20:40:39.464845+0000","last_clean":"2024-09-10T20:40:39.464845+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:40:39.464845+0000","last_undegraded":"2024-09-10T20:40:39.464845+0000","last_fullsized":"2024-09-10T20:40:39.464845+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464725+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:40:39.464725+0000","last_peered":"2024-09-10T20:40:39.464725+0000","last_clean":"2024-09-10T20:40:39.464725+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:40:39.464725+0000","last_undegraded":"2024-09-10T20:40:39.464725+0000","last_fullsized":"2024-09-10T20:40:39.464725+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27834,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464746+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:40:39.464746+0000","last_peered":"2024-09-10T20:40:39.464746+0000","last_clean":"2024-09-10T20:40:39.464746+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:40:39.464746+0000","last_undegraded":"2024-09-10T20:40:39.464746+0000","last_fullsized":"2024-09-10T20:40:39.464746+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":125,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.462414+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:40:39.462414+0000","last_peered":"2024-09-10T20:40:39.462414+0000","last_clean":"2024-09-10T20:40:39.462414+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:40:39.462414+0000","last_undegraded":"2024-09-10T20:40:39.462414+0000","last_fullsized":"2024-09-10T20:40:39.462414+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21265,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:39.463026+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:40:39.463026+0000","last_peered":"2024-09-10T20:40:39.463026+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:40:39.463026+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464696+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:40:39.464696+0000","last_peered":"2024-09-10T20:40:39.464696+0000","last_clean":"2024-09-10T20:40:39.464696+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:40:39.464696+0000","last_undegraded":"2024-09-10T20:40:39.464696+0000","last_fullsized":"2024-09-10T20:40:39.464696+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464922+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:40:39.464922+0000","last_peered":"2024-09-10T20:40:39.464922+0000","last_clean":"2024-09-10T20:40:39.464922+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:40:39.464922+0000","last_undegraded":"2024-09-10T20:40:39.464922+0000","last_fullsized":"2024-09-10T20:40:39.464922+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27665,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464669+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:40:39.464669+0000","last_peered":"2024-09-10T20:40:39.464669+0000","last_clean":"2024-09-10T20:40:39.464669+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:40:39.464669+0000","last_undegraded":"2024-09-10T20:40:39.464669+0000","last_fullsized":"2024-09-10T20:40:39.464669+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11213,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28317,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":160342016,"data_stored":160309499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182548,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517192,"kb_used_data":57972,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854648,"statfs":{"total":96636764160,"available":96107159552,"internally_reserved":0,"allocated":59363328,"data_stored":59052355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986320,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":464908,"kb_used_data":100028,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93906932,"statfs":{"total":96636764160,"available":96160698368,"internally_reserved":0,"allocated":102428672,"data_stored":102121776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659267,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491728,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880112,"statfs":{"total":96636764160,"available":96133234688,"internally_reserved":0,"allocated":524288,"data_stored":191637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502984366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":101445632,"data_stored":101442019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:41:00.526 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T20:41:00.526 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'20658", 'reported_seq': 26975, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:55.229854+0000', 'last_change': '2024-09-10T20:40:37.857265+0000', 'last_active': '2024-09-10T20:40:55.229854+0000', 'last_peered': '2024-09-10T20:40:55.229854+0000', 'last_clean': '2024-09-10T20:30:03.300566+0000', 'last_became_active': '2024-09-10T20:30:05.760498+0000', 'last_became_peered': '2024-09-10T20:30:05.760498+0000', 'last_unstale': '2024-09-10T20:40:55.229854+0000', 'last_undegraded': '2024-09-10T20:30:05.381652+0000', 'last_fullsized': '2024-09-10T20:30:05.306166+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'objects_scrubbed': 0, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 505, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20658, 'num_write_kb': 41316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3103, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -505}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:41:00.527 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:41:00.527 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20471", 'reported_seq': 20745, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:39.461976+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:40:39.461976+0000', 'last_peered': '2024-09-10T20:40:39.461976+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:40:39.461976+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17335, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3561}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:41:00.527 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:41:00.527 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'20608", 'reported_seq': 20916, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:39.464521+0000', 'last_change': '2024-09-10T20:30:05.789134+0000', 'last_active': '2024-09-10T20:40:39.464521+0000', 'last_peered': '2024-09-10T20:40:39.464521+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:40:39.464521+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17203, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3486}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:41:00.527 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:41:00.527 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "51'20986", 'reported_seq': 21265, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:39.463026+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:40:39.463026+0000', 'last_peered': '2024-09-10T20:40:39.463026+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:40:39.463026+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17813, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3655}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:41:00.527 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:41:00.954 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:00.954 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 11213/6 objects degraded (186883.333%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":772,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1509199872,"bytes_avail":288401092608,"bytes_total":289910292480,"degraded_objects":11213,"degraded_total":6,"degraded_ratio":1868.833333333333,"recovering_objects_per_sec":123,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":985,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:41:00.968 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:41:00.968 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:01.356 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:01.357 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:01.370 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":441,"stamp":"2024-09-10T20:41:01.073244+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10199,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29333,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1471352,"kb_used_data":156028,"kb_used_omap":12,"kb_used_meta":1315251,"kb_avail":281644168,"statfs":{"total":289910292480,"available":288403628032,"internally_reserved":0,"allocated":159772672,"data_stored":158802242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1346817026},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1999,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1999,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001470"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":27996,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.617469+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:40:58.617469+0000","last_peered":"2024-09-10T20:40:58.617469+0000","last_clean":"2024-09-10T20:40:58.617469+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:40:58.617469+0000","last_undegraded":"2024-09-10T20:40:58.617469+0000","last_fullsized":"2024-09-10T20:40:58.617469+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21088,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707761+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:40:58.707761+0000","last_peered":"2024-09-10T20:40:58.707761+0000","last_clean":"2024-09-10T20:40:58.707761+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:40:58.707761+0000","last_undegraded":"2024-09-10T20:40:58.707761+0000","last_fullsized":"2024-09-10T20:40:58.707761+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28095,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464663+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:40:39.464663+0000","last_peered":"2024-09-10T20:40:39.464663+0000","last_clean":"2024-09-10T20:40:39.464663+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:40:39.464663+0000","last_undegraded":"2024-09-10T20:40:39.464663+0000","last_fullsized":"2024-09-10T20:40:39.464663+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20720,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707714+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:40:58.707714+0000","last_peered":"2024-09-10T20:40:58.707714+0000","last_clean":"2024-09-10T20:40:58.707714+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:40:58.707714+0000","last_undegraded":"2024-09-10T20:40:58.707714+0000","last_fullsized":"2024-09-10T20:40:58.707714+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21231,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558744+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:40:58.558744+0000","last_peered":"2024-09-10T20:40:58.558744+0000","last_clean":"2024-09-10T20:40:58.558744+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:40:58.558744+0000","last_undegraded":"2024-09-10T20:40:58.558744+0000","last_fullsized":"2024-09-10T20:40:58.558744+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20749,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:58.558601+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:40:58.558601+0000","last_peered":"2024-09-10T20:40:58.558601+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:40:58.558601+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":21939,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:00.234949+0000","last_change":"2024-09-10T20:40:57.471908+0000","last_active":"2024-09-10T20:41:00.234949+0000","last_peered":"2024-09-10T20:41:00.234949+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:41:00.234949+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2979}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20896,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558180+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:40:58.558180+0000","last_peered":"2024-09-10T20:40:58.558180+0000","last_clean":"2024-09-10T20:40:58.558180+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:40:58.558180+0000","last_undegraded":"2024-09-10T20:40:58.558180+0000","last_fullsized":"2024-09-10T20:40:58.558180+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464374+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:40:39.464374+0000","last_peered":"2024-09-10T20:40:39.464374+0000","last_clean":"2024-09-10T20:40:39.464374+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:40:39.464374+0000","last_undegraded":"2024-09-10T20:40:39.464374+0000","last_fullsized":"2024-09-10T20:40:39.464374+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22134,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464659+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:40:39.464659+0000","last_peered":"2024-09-10T20:40:39.464659+0000","last_clean":"2024-09-10T20:40:39.464659+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:40:39.464659+0000","last_undegraded":"2024-09-10T20:40:39.464659+0000","last_fullsized":"2024-09-10T20:40:39.464659+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":129,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707515+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:40:58.707515+0000","last_peered":"2024-09-10T20:40:58.707515+0000","last_clean":"2024-09-10T20:40:58.707515+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:40:58.707515+0000","last_undegraded":"2024-09-10T20:40:58.707515+0000","last_fullsized":"2024-09-10T20:40:58.707515+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28124,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558664+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:40:58.558664+0000","last_peered":"2024-09-10T20:40:58.558664+0000","last_clean":"2024-09-10T20:40:58.558664+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:40:58.558664+0000","last_undegraded":"2024-09-10T20:40:58.558664+0000","last_fullsized":"2024-09-10T20:40:58.558664+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558561+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:40:58.558561+0000","last_peered":"2024-09-10T20:40:58.558561+0000","last_clean":"2024-09-10T20:40:58.558561+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:40:58.558561+0000","last_undegraded":"2024-09-10T20:40:58.558561+0000","last_fullsized":"2024-09-10T20:40:58.558561+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28007,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707523+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:40:58.707523+0000","last_peered":"2024-09-10T20:40:58.707523+0000","last_clean":"2024-09-10T20:40:58.707523+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:40:58.707523+0000","last_undegraded":"2024-09-10T20:40:58.707523+0000","last_fullsized":"2024-09-10T20:40:58.707523+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464562+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:40:39.464562+0000","last_peered":"2024-09-10T20:40:39.464562+0000","last_clean":"2024-09-10T20:40:39.464562+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:40:39.464562+0000","last_undegraded":"2024-09-10T20:40:39.464562+0000","last_fullsized":"2024-09-10T20:40:39.464562+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28060,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707442+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:40:58.707442+0000","last_peered":"2024-09-10T20:40:58.707442+0000","last_clean":"2024-09-10T20:40:58.707442+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:40:58.707442+0000","last_undegraded":"2024-09-10T20:40:58.707442+0000","last_fullsized":"2024-09-10T20:40:58.707442+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21113,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464845+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:40:39.464845+0000","last_peered":"2024-09-10T20:40:39.464845+0000","last_clean":"2024-09-10T20:40:39.464845+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:40:39.464845+0000","last_undegraded":"2024-09-10T20:40:39.464845+0000","last_fullsized":"2024-09-10T20:40:39.464845+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464725+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:40:39.464725+0000","last_peered":"2024-09-10T20:40:39.464725+0000","last_clean":"2024-09-10T20:40:39.464725+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:40:39.464725+0000","last_undegraded":"2024-09-10T20:40:39.464725+0000","last_fullsized":"2024-09-10T20:40:39.464725+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27834,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464746+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:40:39.464746+0000","last_peered":"2024-09-10T20:40:39.464746+0000","last_clean":"2024-09-10T20:40:39.464746+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:40:39.464746+0000","last_undegraded":"2024-09-10T20:40:39.464746+0000","last_fullsized":"2024-09-10T20:40:39.464746+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":129,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707683+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:40:58.707683+0000","last_peered":"2024-09-10T20:40:58.707683+0000","last_clean":"2024-09-10T20:40:58.707683+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:40:58.707683+0000","last_undegraded":"2024-09-10T20:40:58.707683+0000","last_fullsized":"2024-09-10T20:40:58.707683+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21269,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:58.558883+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:40:58.558883+0000","last_peered":"2024-09-10T20:40:58.558883+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:40:58.558883+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464696+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:40:39.464696+0000","last_peered":"2024-09-10T20:40:39.464696+0000","last_clean":"2024-09-10T20:40:39.464696+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:40:39.464696+0000","last_undegraded":"2024-09-10T20:40:39.464696+0000","last_fullsized":"2024-09-10T20:40:39.464696+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464922+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:40:39.464922+0000","last_peered":"2024-09-10T20:40:39.464922+0000","last_clean":"2024-09-10T20:40:39.464922+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:40:39.464922+0000","last_undegraded":"2024-09-10T20:40:39.464922+0000","last_fullsized":"2024-09-10T20:40:39.464922+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27665,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T20:40:39.464669+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:40:39.464669+0000","last_peered":"2024-09-10T20:40:39.464669+0000","last_clean":"2024-09-10T20:40:39.464669+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:40:39.464669+0000","last_undegraded":"2024-09-10T20:40:39.464669+0000","last_fullsized":"2024-09-10T20:40:39.464669+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10199,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29331,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":157642752,"data_stored":157610235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182549,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517212,"kb_used_data":57988,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854628,"statfs":{"total":96636764160,"available":96107139072,"internally_reserved":0,"allocated":59379712,"data_stored":59058784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986321,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":462412,"kb_used_data":97528,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93909428,"statfs":{"total":96636764160,"available":96163254272,"internally_reserved":0,"allocated":99868672,"data_stored":99551821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659267,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491728,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880112,"statfs":{"total":96636764160,"available":96133234688,"internally_reserved":0,"allocated":524288,"data_stored":191637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502984366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98746368,"data_stored":98742755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:41:04.372 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:04.676 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:04.677 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:04.689 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":442,"stamp":"2024-09-10T20:41:03.073568+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10199,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29333,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1463040,"kb_used_data":156036,"kb_used_omap":12,"kb_used_meta":1306931,"kb_avail":281652480,"statfs":{"total":289910292480,"available":288412139520,"internally_reserved":0,"allocated":159780864,"data_stored":158807705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1338297346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1999,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1999,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001403"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":27996,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.617469+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:40:58.617469+0000","last_peered":"2024-09-10T20:40:58.617469+0000","last_clean":"2024-09-10T20:40:58.617469+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:40:58.617469+0000","last_undegraded":"2024-09-10T20:40:58.617469+0000","last_fullsized":"2024-09-10T20:40:58.617469+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21088,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707761+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:40:58.707761+0000","last_peered":"2024-09-10T20:40:58.707761+0000","last_clean":"2024-09-10T20:40:58.707761+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:40:58.707761+0000","last_undegraded":"2024-09-10T20:40:58.707761+0000","last_fullsized":"2024-09-10T20:40:58.707761+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28099,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562751+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:40:58.562751+0000","last_peered":"2024-09-10T20:40:58.562751+0000","last_clean":"2024-09-10T20:40:58.562751+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:40:58.562751+0000","last_undegraded":"2024-09-10T20:40:58.562751+0000","last_fullsized":"2024-09-10T20:40:58.562751+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20720,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707714+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:40:58.707714+0000","last_peered":"2024-09-10T20:40:58.707714+0000","last_clean":"2024-09-10T20:40:58.707714+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:40:58.707714+0000","last_undegraded":"2024-09-10T20:40:58.707714+0000","last_fullsized":"2024-09-10T20:40:58.707714+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21231,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558744+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:40:58.558744+0000","last_peered":"2024-09-10T20:40:58.558744+0000","last_clean":"2024-09-10T20:40:58.558744+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:40:58.558744+0000","last_undegraded":"2024-09-10T20:40:58.558744+0000","last_fullsized":"2024-09-10T20:40:58.558744+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20749,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:58.558601+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:40:58.558601+0000","last_peered":"2024-09-10T20:40:58.558601+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:40:58.558601+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":21939,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:00.234949+0000","last_change":"2024-09-10T20:40:57.471908+0000","last_active":"2024-09-10T20:41:00.234949+0000","last_peered":"2024-09-10T20:41:00.234949+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:41:00.234949+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2979}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20896,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558180+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:40:58.558180+0000","last_peered":"2024-09-10T20:40:58.558180+0000","last_clean":"2024-09-10T20:40:58.558180+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:40:58.558180+0000","last_undegraded":"2024-09-10T20:40:58.558180+0000","last_fullsized":"2024-09-10T20:40:58.558180+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562632+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:40:58.562632+0000","last_peered":"2024-09-10T20:40:58.562632+0000","last_clean":"2024-09-10T20:40:58.562632+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:40:58.562632+0000","last_undegraded":"2024-09-10T20:40:58.562632+0000","last_fullsized":"2024-09-10T20:40:58.562632+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22138,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562862+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:40:58.562862+0000","last_peered":"2024-09-10T20:40:58.562862+0000","last_clean":"2024-09-10T20:40:58.562862+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:40:58.562862+0000","last_undegraded":"2024-09-10T20:40:58.562862+0000","last_fullsized":"2024-09-10T20:40:58.562862+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":129,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707515+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:40:58.707515+0000","last_peered":"2024-09-10T20:40:58.707515+0000","last_clean":"2024-09-10T20:40:58.707515+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:40:58.707515+0000","last_undegraded":"2024-09-10T20:40:58.707515+0000","last_fullsized":"2024-09-10T20:40:58.707515+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28124,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558664+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:40:58.558664+0000","last_peered":"2024-09-10T20:40:58.558664+0000","last_clean":"2024-09-10T20:40:58.558664+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:40:58.558664+0000","last_undegraded":"2024-09-10T20:40:58.558664+0000","last_fullsized":"2024-09-10T20:40:58.558664+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558561+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:40:58.558561+0000","last_peered":"2024-09-10T20:40:58.558561+0000","last_clean":"2024-09-10T20:40:58.558561+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:40:58.558561+0000","last_undegraded":"2024-09-10T20:40:58.558561+0000","last_fullsized":"2024-09-10T20:40:58.558561+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28007,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707523+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:40:58.707523+0000","last_peered":"2024-09-10T20:40:58.707523+0000","last_clean":"2024-09-10T20:40:58.707523+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:40:58.707523+0000","last_undegraded":"2024-09-10T20:40:58.707523+0000","last_fullsized":"2024-09-10T20:40:58.707523+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562762+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:40:58.562762+0000","last_peered":"2024-09-10T20:40:58.562762+0000","last_clean":"2024-09-10T20:40:58.562762+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:40:58.562762+0000","last_undegraded":"2024-09-10T20:40:58.562762+0000","last_fullsized":"2024-09-10T20:40:58.562762+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28060,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707442+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:40:58.707442+0000","last_peered":"2024-09-10T20:40:58.707442+0000","last_clean":"2024-09-10T20:40:58.707442+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:40:58.707442+0000","last_undegraded":"2024-09-10T20:40:58.707442+0000","last_fullsized":"2024-09-10T20:40:58.707442+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21117,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.563002+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:40:58.563002+0000","last_peered":"2024-09-10T20:40:58.563002+0000","last_clean":"2024-09-10T20:40:58.563002+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:40:58.563002+0000","last_undegraded":"2024-09-10T20:40:58.563002+0000","last_fullsized":"2024-09-10T20:40:58.563002+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562986+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:40:58.562986+0000","last_peered":"2024-09-10T20:40:58.562986+0000","last_clean":"2024-09-10T20:40:58.562986+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:40:58.562986+0000","last_undegraded":"2024-09-10T20:40:58.562986+0000","last_fullsized":"2024-09-10T20:40:58.562986+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27839,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562924+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:40:58.562924+0000","last_peered":"2024-09-10T20:40:58.562924+0000","last_clean":"2024-09-10T20:40:58.562924+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:40:58.562924+0000","last_undegraded":"2024-09-10T20:40:58.562924+0000","last_fullsized":"2024-09-10T20:40:58.562924+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":129,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707683+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:40:58.707683+0000","last_peered":"2024-09-10T20:40:58.707683+0000","last_clean":"2024-09-10T20:40:58.707683+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:40:58.707683+0000","last_undegraded":"2024-09-10T20:40:58.707683+0000","last_fullsized":"2024-09-10T20:40:58.707683+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21269,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:58.558883+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:40:58.558883+0000","last_peered":"2024-09-10T20:40:58.558883+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:40:58.558883+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562868+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:40:58.562868+0000","last_peered":"2024-09-10T20:40:58.562868+0000","last_clean":"2024-09-10T20:40:58.562868+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:40:58.562868+0000","last_undegraded":"2024-09-10T20:40:58.562868+0000","last_fullsized":"2024-09-10T20:40:58.562868+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562953+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:40:58.562953+0000","last_peered":"2024-09-10T20:40:58.562953+0000","last_clean":"2024-09-10T20:40:58.562953+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:40:58.562953+0000","last_undegraded":"2024-09-10T20:40:58.562953+0000","last_fullsized":"2024-09-10T20:40:58.562953+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27669,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562986+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:40:58.562986+0000","last_peered":"2024-09-10T20:40:58.562986+0000","last_clean":"2024-09-10T20:40:58.562986+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:40:58.562986+0000","last_undegraded":"2024-09-10T20:40:58.562986+0000","last_fullsized":"2024-09-10T20:40:58.562986+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10199,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29331,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":157634560,"data_stored":157609269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182549,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517212,"kb_used_data":57988,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854628,"statfs":{"total":96636764160,"available":96107139072,"internally_reserved":0,"allocated":59379712,"data_stored":59058784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986321,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":462412,"kb_used_data":97528,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93909428,"statfs":{"total":96636764160,"available":96163254272,"internally_reserved":0,"allocated":99868672,"data_stored":99551821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659268,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483416,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888424,"statfs":{"total":96636764160,"available":96141746176,"internally_reserved":0,"allocated":532480,"data_stored":197100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98746368,"data_stored":98742755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:41:04.691 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:04.994 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:04.995 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:05.007 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":442,"stamp":"2024-09-10T20:41:03.073568+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10199,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29333,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1463040,"kb_used_data":156036,"kb_used_omap":12,"kb_used_meta":1306931,"kb_avail":281652480,"statfs":{"total":289910292480,"available":288412139520,"internally_reserved":0,"allocated":159780864,"data_stored":158807705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1338297346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1999,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1999,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001403"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":27996,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.617469+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:40:58.617469+0000","last_peered":"2024-09-10T20:40:58.617469+0000","last_clean":"2024-09-10T20:40:58.617469+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:40:58.617469+0000","last_undegraded":"2024-09-10T20:40:58.617469+0000","last_fullsized":"2024-09-10T20:40:58.617469+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21088,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707761+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:40:58.707761+0000","last_peered":"2024-09-10T20:40:58.707761+0000","last_clean":"2024-09-10T20:40:58.707761+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:40:58.707761+0000","last_undegraded":"2024-09-10T20:40:58.707761+0000","last_fullsized":"2024-09-10T20:40:58.707761+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28099,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562751+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:40:58.562751+0000","last_peered":"2024-09-10T20:40:58.562751+0000","last_clean":"2024-09-10T20:40:58.562751+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:40:58.562751+0000","last_undegraded":"2024-09-10T20:40:58.562751+0000","last_fullsized":"2024-09-10T20:40:58.562751+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20720,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707714+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:40:58.707714+0000","last_peered":"2024-09-10T20:40:58.707714+0000","last_clean":"2024-09-10T20:40:58.707714+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:40:58.707714+0000","last_undegraded":"2024-09-10T20:40:58.707714+0000","last_fullsized":"2024-09-10T20:40:58.707714+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21231,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558744+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:40:58.558744+0000","last_peered":"2024-09-10T20:40:58.558744+0000","last_clean":"2024-09-10T20:40:58.558744+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:40:58.558744+0000","last_undegraded":"2024-09-10T20:40:58.558744+0000","last_fullsized":"2024-09-10T20:40:58.558744+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20749,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:58.558601+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:40:58.558601+0000","last_peered":"2024-09-10T20:40:58.558601+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:40:58.558601+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":21939,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:00.234949+0000","last_change":"2024-09-10T20:40:57.471908+0000","last_active":"2024-09-10T20:41:00.234949+0000","last_peered":"2024-09-10T20:41:00.234949+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:41:00.234949+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2979}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20896,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558180+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:40:58.558180+0000","last_peered":"2024-09-10T20:40:58.558180+0000","last_clean":"2024-09-10T20:40:58.558180+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:40:58.558180+0000","last_undegraded":"2024-09-10T20:40:58.558180+0000","last_fullsized":"2024-09-10T20:40:58.558180+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562632+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:40:58.562632+0000","last_peered":"2024-09-10T20:40:58.562632+0000","last_clean":"2024-09-10T20:40:58.562632+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:40:58.562632+0000","last_undegraded":"2024-09-10T20:40:58.562632+0000","last_fullsized":"2024-09-10T20:40:58.562632+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22138,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562862+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:40:58.562862+0000","last_peered":"2024-09-10T20:40:58.562862+0000","last_clean":"2024-09-10T20:40:58.562862+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:40:58.562862+0000","last_undegraded":"2024-09-10T20:40:58.562862+0000","last_fullsized":"2024-09-10T20:40:58.562862+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":129,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707515+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:40:58.707515+0000","last_peered":"2024-09-10T20:40:58.707515+0000","last_clean":"2024-09-10T20:40:58.707515+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:40:58.707515+0000","last_undegraded":"2024-09-10T20:40:58.707515+0000","last_fullsized":"2024-09-10T20:40:58.707515+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28124,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558664+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:40:58.558664+0000","last_peered":"2024-09-10T20:40:58.558664+0000","last_clean":"2024-09-10T20:40:58.558664+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:40:58.558664+0000","last_undegraded":"2024-09-10T20:40:58.558664+0000","last_fullsized":"2024-09-10T20:40:58.558664+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558561+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:40:58.558561+0000","last_peered":"2024-09-10T20:40:58.558561+0000","last_clean":"2024-09-10T20:40:58.558561+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:40:58.558561+0000","last_undegraded":"2024-09-10T20:40:58.558561+0000","last_fullsized":"2024-09-10T20:40:58.558561+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28007,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707523+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:40:58.707523+0000","last_peered":"2024-09-10T20:40:58.707523+0000","last_clean":"2024-09-10T20:40:58.707523+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:40:58.707523+0000","last_undegraded":"2024-09-10T20:40:58.707523+0000","last_fullsized":"2024-09-10T20:40:58.707523+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562762+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:40:58.562762+0000","last_peered":"2024-09-10T20:40:58.562762+0000","last_clean":"2024-09-10T20:40:58.562762+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:40:58.562762+0000","last_undegraded":"2024-09-10T20:40:58.562762+0000","last_fullsized":"2024-09-10T20:40:58.562762+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28060,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707442+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:40:58.707442+0000","last_peered":"2024-09-10T20:40:58.707442+0000","last_clean":"2024-09-10T20:40:58.707442+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:40:58.707442+0000","last_undegraded":"2024-09-10T20:40:58.707442+0000","last_fullsized":"2024-09-10T20:40:58.707442+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21117,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.563002+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:40:58.563002+0000","last_peered":"2024-09-10T20:40:58.563002+0000","last_clean":"2024-09-10T20:40:58.563002+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:40:58.563002+0000","last_undegraded":"2024-09-10T20:40:58.563002+0000","last_fullsized":"2024-09-10T20:40:58.563002+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562986+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:40:58.562986+0000","last_peered":"2024-09-10T20:40:58.562986+0000","last_clean":"2024-09-10T20:40:58.562986+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:40:58.562986+0000","last_undegraded":"2024-09-10T20:40:58.562986+0000","last_fullsized":"2024-09-10T20:40:58.562986+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27839,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562924+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:40:58.562924+0000","last_peered":"2024-09-10T20:40:58.562924+0000","last_clean":"2024-09-10T20:40:58.562924+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:40:58.562924+0000","last_undegraded":"2024-09-10T20:40:58.562924+0000","last_fullsized":"2024-09-10T20:40:58.562924+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":129,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707683+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:40:58.707683+0000","last_peered":"2024-09-10T20:40:58.707683+0000","last_clean":"2024-09-10T20:40:58.707683+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:40:58.707683+0000","last_undegraded":"2024-09-10T20:40:58.707683+0000","last_fullsized":"2024-09-10T20:40:58.707683+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21269,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:58.558883+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:40:58.558883+0000","last_peered":"2024-09-10T20:40:58.558883+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:40:58.558883+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562868+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:40:58.562868+0000","last_peered":"2024-09-10T20:40:58.562868+0000","last_clean":"2024-09-10T20:40:58.562868+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:40:58.562868+0000","last_undegraded":"2024-09-10T20:40:58.562868+0000","last_fullsized":"2024-09-10T20:40:58.562868+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562953+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:40:58.562953+0000","last_peered":"2024-09-10T20:40:58.562953+0000","last_clean":"2024-09-10T20:40:58.562953+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:40:58.562953+0000","last_undegraded":"2024-09-10T20:40:58.562953+0000","last_fullsized":"2024-09-10T20:40:58.562953+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27669,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562986+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:40:58.562986+0000","last_peered":"2024-09-10T20:40:58.562986+0000","last_clean":"2024-09-10T20:40:58.562986+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:40:58.562986+0000","last_undegraded":"2024-09-10T20:40:58.562986+0000","last_fullsized":"2024-09-10T20:40:58.562986+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10199,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29331,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":157634560,"data_stored":157609269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182549,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517212,"kb_used_data":57988,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854628,"statfs":{"total":96636764160,"available":96107139072,"internally_reserved":0,"allocated":59379712,"data_stored":59058784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986321,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":462412,"kb_used_data":97528,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93909428,"statfs":{"total":96636764160,"available":96163254272,"internally_reserved":0,"allocated":99868672,"data_stored":99551821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659268,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483416,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888424,"statfs":{"total":96636764160,"available":96141746176,"internally_reserved":0,"allocated":532480,"data_stored":197100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98746368,"data_stored":98742755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:41:05.008 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:41:05.008 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20471", 'reported_seq': 20749, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:58.558601+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:40:58.558601+0000', 'last_peered': '2024-09-10T20:40:58.558601+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:40:58.558601+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17335, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3561}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:41:05.009 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:41:05.009 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'20608", 'reported_seq': 21939, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:41:00.234949+0000', 'last_change': '2024-09-10T20:40:57.471908+0000', 'last_active': '2024-09-10T20:41:00.234949+0000', 'last_peered': '2024-09-10T20:41:00.234949+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:41:00.234949+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2979, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17203, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 578, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2979}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:41:05.009 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:41:05.009 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "51'20986", 'reported_seq': 21269, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:58.558883+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:40:58.558883+0000', 'last_peered': '2024-09-10T20:40:58.558883+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:40:58.558883+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17813, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3655}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:41:05.009 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:41:05.387 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:05.387 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 10199/6 objects degraded (169983.333%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":777,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1498152960,"bytes_avail":288412139520,"bytes_total":289910292480,"degraded_objects":10199,"degraded_total":6,"degraded_ratio":1699.833333333333,"recovering_objects_per_sec":249,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1999,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:41:05.400 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:41:05.401 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:05.699 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:05.699 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:05.712 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":443,"stamp":"2024-09-10T20:41:05.073988+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10199,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29333,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1460036,"kb_used_data":153036,"kb_used_omap":12,"kb_used_meta":1306931,"kb_avail":281655484,"statfs":{"total":289910292480,"available":288415215616,"internally_reserved":0,"allocated":156708864,"data_stored":155735705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1338297346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1014,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1014,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001409"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":27996,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.617469+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:40:58.617469+0000","last_peered":"2024-09-10T20:40:58.617469+0000","last_clean":"2024-09-10T20:40:58.617469+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:40:58.617469+0000","last_undegraded":"2024-09-10T20:40:58.617469+0000","last_fullsized":"2024-09-10T20:40:58.617469+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21088,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707761+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:40:58.707761+0000","last_peered":"2024-09-10T20:40:58.707761+0000","last_clean":"2024-09-10T20:40:58.707761+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:40:58.707761+0000","last_undegraded":"2024-09-10T20:40:58.707761+0000","last_fullsized":"2024-09-10T20:40:58.707761+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28099,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562751+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:40:58.562751+0000","last_peered":"2024-09-10T20:40:58.562751+0000","last_clean":"2024-09-10T20:40:58.562751+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:40:58.562751+0000","last_undegraded":"2024-09-10T20:40:58.562751+0000","last_fullsized":"2024-09-10T20:40:58.562751+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20720,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707714+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:40:58.707714+0000","last_peered":"2024-09-10T20:40:58.707714+0000","last_clean":"2024-09-10T20:40:58.707714+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:40:58.707714+0000","last_undegraded":"2024-09-10T20:40:58.707714+0000","last_fullsized":"2024-09-10T20:40:58.707714+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21231,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558744+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:40:58.558744+0000","last_peered":"2024-09-10T20:40:58.558744+0000","last_clean":"2024-09-10T20:40:58.558744+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:40:58.558744+0000","last_undegraded":"2024-09-10T20:40:58.558744+0000","last_fullsized":"2024-09-10T20:40:58.558744+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20749,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:58.558601+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:40:58.558601+0000","last_peered":"2024-09-10T20:40:58.558601+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:40:58.558601+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":21939,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:00.234949+0000","last_change":"2024-09-10T20:40:57.471908+0000","last_active":"2024-09-10T20:41:00.234949+0000","last_peered":"2024-09-10T20:41:00.234949+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:41:00.234949+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2979}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20896,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558180+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:40:58.558180+0000","last_peered":"2024-09-10T20:40:58.558180+0000","last_clean":"2024-09-10T20:40:58.558180+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:40:58.558180+0000","last_undegraded":"2024-09-10T20:40:58.558180+0000","last_fullsized":"2024-09-10T20:40:58.558180+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562632+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:40:58.562632+0000","last_peered":"2024-09-10T20:40:58.562632+0000","last_clean":"2024-09-10T20:40:58.562632+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:40:58.562632+0000","last_undegraded":"2024-09-10T20:40:58.562632+0000","last_fullsized":"2024-09-10T20:40:58.562632+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22138,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562862+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:40:58.562862+0000","last_peered":"2024-09-10T20:40:58.562862+0000","last_clean":"2024-09-10T20:40:58.562862+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:40:58.562862+0000","last_undegraded":"2024-09-10T20:40:58.562862+0000","last_fullsized":"2024-09-10T20:40:58.562862+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":129,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707515+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:40:58.707515+0000","last_peered":"2024-09-10T20:40:58.707515+0000","last_clean":"2024-09-10T20:40:58.707515+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:40:58.707515+0000","last_undegraded":"2024-09-10T20:40:58.707515+0000","last_fullsized":"2024-09-10T20:40:58.707515+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28124,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558664+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:40:58.558664+0000","last_peered":"2024-09-10T20:40:58.558664+0000","last_clean":"2024-09-10T20:40:58.558664+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:40:58.558664+0000","last_undegraded":"2024-09-10T20:40:58.558664+0000","last_fullsized":"2024-09-10T20:40:58.558664+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558561+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:40:58.558561+0000","last_peered":"2024-09-10T20:40:58.558561+0000","last_clean":"2024-09-10T20:40:58.558561+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:40:58.558561+0000","last_undegraded":"2024-09-10T20:40:58.558561+0000","last_fullsized":"2024-09-10T20:40:58.558561+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28007,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707523+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:40:58.707523+0000","last_peered":"2024-09-10T20:40:58.707523+0000","last_clean":"2024-09-10T20:40:58.707523+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:40:58.707523+0000","last_undegraded":"2024-09-10T20:40:58.707523+0000","last_fullsized":"2024-09-10T20:40:58.707523+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562762+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:40:58.562762+0000","last_peered":"2024-09-10T20:40:58.562762+0000","last_clean":"2024-09-10T20:40:58.562762+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:40:58.562762+0000","last_undegraded":"2024-09-10T20:40:58.562762+0000","last_fullsized":"2024-09-10T20:40:58.562762+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28060,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707442+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:40:58.707442+0000","last_peered":"2024-09-10T20:40:58.707442+0000","last_clean":"2024-09-10T20:40:58.707442+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:40:58.707442+0000","last_undegraded":"2024-09-10T20:40:58.707442+0000","last_fullsized":"2024-09-10T20:40:58.707442+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21117,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.563002+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:40:58.563002+0000","last_peered":"2024-09-10T20:40:58.563002+0000","last_clean":"2024-09-10T20:40:58.563002+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:40:58.563002+0000","last_undegraded":"2024-09-10T20:40:58.563002+0000","last_fullsized":"2024-09-10T20:40:58.563002+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562986+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:40:58.562986+0000","last_peered":"2024-09-10T20:40:58.562986+0000","last_clean":"2024-09-10T20:40:58.562986+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:40:58.562986+0000","last_undegraded":"2024-09-10T20:40:58.562986+0000","last_fullsized":"2024-09-10T20:40:58.562986+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27839,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562924+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:40:58.562924+0000","last_peered":"2024-09-10T20:40:58.562924+0000","last_clean":"2024-09-10T20:40:58.562924+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:40:58.562924+0000","last_undegraded":"2024-09-10T20:40:58.562924+0000","last_fullsized":"2024-09-10T20:40:58.562924+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":129,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707683+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:40:58.707683+0000","last_peered":"2024-09-10T20:40:58.707683+0000","last_clean":"2024-09-10T20:40:58.707683+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:40:58.707683+0000","last_undegraded":"2024-09-10T20:40:58.707683+0000","last_fullsized":"2024-09-10T20:40:58.707683+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21269,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:58.558883+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:40:58.558883+0000","last_peered":"2024-09-10T20:40:58.558883+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:40:58.558883+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562868+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:40:58.562868+0000","last_peered":"2024-09-10T20:40:58.562868+0000","last_clean":"2024-09-10T20:40:58.562868+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:40:58.562868+0000","last_undegraded":"2024-09-10T20:40:58.562868+0000","last_fullsized":"2024-09-10T20:40:58.562868+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562953+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:40:58.562953+0000","last_peered":"2024-09-10T20:40:58.562953+0000","last_clean":"2024-09-10T20:40:58.562953+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:40:58.562953+0000","last_undegraded":"2024-09-10T20:40:58.562953+0000","last_fullsized":"2024-09-10T20:40:58.562953+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27669,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562986+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:40:58.562986+0000","last_peered":"2024-09-10T20:40:58.562986+0000","last_clean":"2024-09-10T20:40:58.562986+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:40:58.562986+0000","last_undegraded":"2024-09-10T20:40:58.562986+0000","last_fullsized":"2024-09-10T20:40:58.562986+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10199,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29331,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":154562560,"data_stored":154537269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182549,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517212,"kb_used_data":57988,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854628,"statfs":{"total":96636764160,"available":96107139072,"internally_reserved":0,"allocated":59379712,"data_stored":59058784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986322,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":459408,"kb_used_data":94528,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93912432,"statfs":{"total":96636764160,"available":96166330368,"internally_reserved":0,"allocated":96796672,"data_stored":96479821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659268,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483416,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888424,"statfs":{"total":96636764160,"available":96141746176,"internally_reserved":0,"allocated":532480,"data_stored":197100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":95674368,"data_stored":95670755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:41:08.713 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:09.036 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:09.037 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:09.049 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":444,"stamp":"2024-09-10T20:41:07.074359+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9329,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30203,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1460036,"kb_used_data":153036,"kb_used_omap":12,"kb_used_meta":1306931,"kb_avail":281655484,"statfs":{"total":289910292480,"available":288415215616,"internally_reserved":0,"allocated":156708864,"data_stored":155735705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1338297346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1884,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1884,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.526910"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":27996,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.617469+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:40:58.617469+0000","last_peered":"2024-09-10T20:40:58.617469+0000","last_clean":"2024-09-10T20:40:58.617469+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:40:58.617469+0000","last_undegraded":"2024-09-10T20:40:58.617469+0000","last_fullsized":"2024-09-10T20:40:58.617469+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21088,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707761+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:40:58.707761+0000","last_peered":"2024-09-10T20:40:58.707761+0000","last_clean":"2024-09-10T20:40:58.707761+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:40:58.707761+0000","last_undegraded":"2024-09-10T20:40:58.707761+0000","last_fullsized":"2024-09-10T20:40:58.707761+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28099,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562751+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:40:58.562751+0000","last_peered":"2024-09-10T20:40:58.562751+0000","last_clean":"2024-09-10T20:40:58.562751+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:40:58.562751+0000","last_undegraded":"2024-09-10T20:40:58.562751+0000","last_fullsized":"2024-09-10T20:40:58.562751+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20720,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707714+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:40:58.707714+0000","last_peered":"2024-09-10T20:40:58.707714+0000","last_clean":"2024-09-10T20:40:58.707714+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:40:58.707714+0000","last_undegraded":"2024-09-10T20:40:58.707714+0000","last_fullsized":"2024-09-10T20:40:58.707714+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21231,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558744+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:40:58.558744+0000","last_peered":"2024-09-10T20:40:58.558744+0000","last_clean":"2024-09-10T20:40:58.558744+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:40:58.558744+0000","last_undegraded":"2024-09-10T20:40:58.558744+0000","last_fullsized":"2024-09-10T20:40:58.558744+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20749,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:58.558601+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:40:58.558601+0000","last_peered":"2024-09-10T20:40:58.558601+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:40:58.558601+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":23679,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:05.235187+0000","last_change":"2024-09-10T20:40:57.471908+0000","last_active":"2024-09-10T20:41:05.235187+0000","last_peered":"2024-09-10T20:41:05.235187+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:41:05.235187+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2109,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1448,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20896,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558180+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:40:58.558180+0000","last_peered":"2024-09-10T20:40:58.558180+0000","last_clean":"2024-09-10T20:40:58.558180+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:40:58.558180+0000","last_undegraded":"2024-09-10T20:40:58.558180+0000","last_fullsized":"2024-09-10T20:40:58.558180+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562632+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:40:58.562632+0000","last_peered":"2024-09-10T20:40:58.562632+0000","last_clean":"2024-09-10T20:40:58.562632+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:40:58.562632+0000","last_undegraded":"2024-09-10T20:40:58.562632+0000","last_fullsized":"2024-09-10T20:40:58.562632+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22138,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562862+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:40:58.562862+0000","last_peered":"2024-09-10T20:40:58.562862+0000","last_clean":"2024-09-10T20:40:58.562862+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:40:58.562862+0000","last_undegraded":"2024-09-10T20:40:58.562862+0000","last_fullsized":"2024-09-10T20:40:58.562862+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":129,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707515+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:40:58.707515+0000","last_peered":"2024-09-10T20:40:58.707515+0000","last_clean":"2024-09-10T20:40:58.707515+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:40:58.707515+0000","last_undegraded":"2024-09-10T20:40:58.707515+0000","last_fullsized":"2024-09-10T20:40:58.707515+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28124,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558664+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:40:58.558664+0000","last_peered":"2024-09-10T20:40:58.558664+0000","last_clean":"2024-09-10T20:40:58.558664+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:40:58.558664+0000","last_undegraded":"2024-09-10T20:40:58.558664+0000","last_fullsized":"2024-09-10T20:40:58.558664+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558561+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:40:58.558561+0000","last_peered":"2024-09-10T20:40:58.558561+0000","last_clean":"2024-09-10T20:40:58.558561+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:40:58.558561+0000","last_undegraded":"2024-09-10T20:40:58.558561+0000","last_fullsized":"2024-09-10T20:40:58.558561+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28007,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707523+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:40:58.707523+0000","last_peered":"2024-09-10T20:40:58.707523+0000","last_clean":"2024-09-10T20:40:58.707523+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:40:58.707523+0000","last_undegraded":"2024-09-10T20:40:58.707523+0000","last_fullsized":"2024-09-10T20:40:58.707523+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562762+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:40:58.562762+0000","last_peered":"2024-09-10T20:40:58.562762+0000","last_clean":"2024-09-10T20:40:58.562762+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:40:58.562762+0000","last_undegraded":"2024-09-10T20:40:58.562762+0000","last_fullsized":"2024-09-10T20:40:58.562762+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28060,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707442+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:40:58.707442+0000","last_peered":"2024-09-10T20:40:58.707442+0000","last_clean":"2024-09-10T20:40:58.707442+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:40:58.707442+0000","last_undegraded":"2024-09-10T20:40:58.707442+0000","last_fullsized":"2024-09-10T20:40:58.707442+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21117,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.563002+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:40:58.563002+0000","last_peered":"2024-09-10T20:40:58.563002+0000","last_clean":"2024-09-10T20:40:58.563002+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:40:58.563002+0000","last_undegraded":"2024-09-10T20:40:58.563002+0000","last_fullsized":"2024-09-10T20:40:58.563002+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562986+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:40:58.562986+0000","last_peered":"2024-09-10T20:40:58.562986+0000","last_clean":"2024-09-10T20:40:58.562986+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:40:58.562986+0000","last_undegraded":"2024-09-10T20:40:58.562986+0000","last_fullsized":"2024-09-10T20:40:58.562986+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27839,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562924+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:40:58.562924+0000","last_peered":"2024-09-10T20:40:58.562924+0000","last_clean":"2024-09-10T20:40:58.562924+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:40:58.562924+0000","last_undegraded":"2024-09-10T20:40:58.562924+0000","last_fullsized":"2024-09-10T20:40:58.562924+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":129,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707683+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:40:58.707683+0000","last_peered":"2024-09-10T20:40:58.707683+0000","last_clean":"2024-09-10T20:40:58.707683+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:40:58.707683+0000","last_undegraded":"2024-09-10T20:40:58.707683+0000","last_fullsized":"2024-09-10T20:40:58.707683+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21269,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:58.558883+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:40:58.558883+0000","last_peered":"2024-09-10T20:40:58.558883+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:40:58.558883+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562868+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:40:58.562868+0000","last_peered":"2024-09-10T20:40:58.562868+0000","last_clean":"2024-09-10T20:40:58.562868+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:40:58.562868+0000","last_undegraded":"2024-09-10T20:40:58.562868+0000","last_fullsized":"2024-09-10T20:40:58.562868+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562953+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:40:58.562953+0000","last_peered":"2024-09-10T20:40:58.562953+0000","last_clean":"2024-09-10T20:40:58.562953+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:40:58.562953+0000","last_undegraded":"2024-09-10T20:40:58.562953+0000","last_fullsized":"2024-09-10T20:40:58.562953+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27669,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562986+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:40:58.562986+0000","last_peered":"2024-09-10T20:40:58.562986+0000","last_clean":"2024-09-10T20:40:58.562986+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:40:58.562986+0000","last_undegraded":"2024-09-10T20:40:58.562986+0000","last_fullsized":"2024-09-10T20:40:58.562986+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9329,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30201,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":154562560,"data_stored":154537269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182550,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517212,"kb_used_data":57988,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854628,"statfs":{"total":96636764160,"available":96107139072,"internally_reserved":0,"allocated":59379712,"data_stored":59058784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986322,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":459408,"kb_used_data":94528,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93912432,"statfs":{"total":96636764160,"available":96166330368,"internally_reserved":0,"allocated":96796672,"data_stored":96479821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659268,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483416,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888424,"statfs":{"total":96636764160,"available":96141746176,"internally_reserved":0,"allocated":532480,"data_stored":197100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":95674368,"data_stored":95670755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:41:09.051 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:09.353 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:09.353 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:09.365 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":445,"stamp":"2024-09-10T20:41:09.074660+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9329,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30203,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1482244,"kb_used_data":153036,"kb_used_omap":12,"kb_used_meta":1329139,"kb_avail":281633276,"statfs":{"total":289910292480,"available":288392474624,"internally_reserved":0,"allocated":156708864,"data_stored":155735705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1361038338},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1884,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1884,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.517503"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":27996,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.617469+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:40:58.617469+0000","last_peered":"2024-09-10T20:40:58.617469+0000","last_clean":"2024-09-10T20:40:58.617469+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:40:58.617469+0000","last_undegraded":"2024-09-10T20:40:58.617469+0000","last_fullsized":"2024-09-10T20:40:58.617469+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21088,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707761+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:40:58.707761+0000","last_peered":"2024-09-10T20:40:58.707761+0000","last_clean":"2024-09-10T20:40:58.707761+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:40:58.707761+0000","last_undegraded":"2024-09-10T20:40:58.707761+0000","last_fullsized":"2024-09-10T20:40:58.707761+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28099,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562751+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:40:58.562751+0000","last_peered":"2024-09-10T20:40:58.562751+0000","last_clean":"2024-09-10T20:40:58.562751+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:40:58.562751+0000","last_undegraded":"2024-09-10T20:40:58.562751+0000","last_fullsized":"2024-09-10T20:40:58.562751+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20720,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707714+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:40:58.707714+0000","last_peered":"2024-09-10T20:40:58.707714+0000","last_clean":"2024-09-10T20:40:58.707714+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:40:58.707714+0000","last_undegraded":"2024-09-10T20:40:58.707714+0000","last_fullsized":"2024-09-10T20:40:58.707714+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21231,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558744+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:40:58.558744+0000","last_peered":"2024-09-10T20:40:58.558744+0000","last_clean":"2024-09-10T20:40:58.558744+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:40:58.558744+0000","last_undegraded":"2024-09-10T20:40:58.558744+0000","last_fullsized":"2024-09-10T20:40:58.558744+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20749,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:58.558601+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:40:58.558601+0000","last_peered":"2024-09-10T20:40:58.558601+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:40:58.558601+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":23679,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:05.235187+0000","last_change":"2024-09-10T20:40:57.471908+0000","last_active":"2024-09-10T20:41:05.235187+0000","last_peered":"2024-09-10T20:41:05.235187+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:41:05.235187+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2109,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1448,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20896,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558180+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:40:58.558180+0000","last_peered":"2024-09-10T20:40:58.558180+0000","last_clean":"2024-09-10T20:40:58.558180+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:40:58.558180+0000","last_undegraded":"2024-09-10T20:40:58.558180+0000","last_fullsized":"2024-09-10T20:40:58.558180+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562632+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:40:58.562632+0000","last_peered":"2024-09-10T20:40:58.562632+0000","last_clean":"2024-09-10T20:40:58.562632+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:40:58.562632+0000","last_undegraded":"2024-09-10T20:40:58.562632+0000","last_fullsized":"2024-09-10T20:40:58.562632+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22138,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562862+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:40:58.562862+0000","last_peered":"2024-09-10T20:40:58.562862+0000","last_clean":"2024-09-10T20:40:58.562862+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:40:58.562862+0000","last_undegraded":"2024-09-10T20:40:58.562862+0000","last_fullsized":"2024-09-10T20:40:58.562862+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":129,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707515+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:40:58.707515+0000","last_peered":"2024-09-10T20:40:58.707515+0000","last_clean":"2024-09-10T20:40:58.707515+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:40:58.707515+0000","last_undegraded":"2024-09-10T20:40:58.707515+0000","last_fullsized":"2024-09-10T20:40:58.707515+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28124,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558664+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:40:58.558664+0000","last_peered":"2024-09-10T20:40:58.558664+0000","last_clean":"2024-09-10T20:40:58.558664+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:40:58.558664+0000","last_undegraded":"2024-09-10T20:40:58.558664+0000","last_fullsized":"2024-09-10T20:40:58.558664+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558561+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:40:58.558561+0000","last_peered":"2024-09-10T20:40:58.558561+0000","last_clean":"2024-09-10T20:40:58.558561+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:40:58.558561+0000","last_undegraded":"2024-09-10T20:40:58.558561+0000","last_fullsized":"2024-09-10T20:40:58.558561+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28007,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707523+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:40:58.707523+0000","last_peered":"2024-09-10T20:40:58.707523+0000","last_clean":"2024-09-10T20:40:58.707523+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:40:58.707523+0000","last_undegraded":"2024-09-10T20:40:58.707523+0000","last_fullsized":"2024-09-10T20:40:58.707523+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562762+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:40:58.562762+0000","last_peered":"2024-09-10T20:40:58.562762+0000","last_clean":"2024-09-10T20:40:58.562762+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:40:58.562762+0000","last_undegraded":"2024-09-10T20:40:58.562762+0000","last_fullsized":"2024-09-10T20:40:58.562762+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28060,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707442+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:40:58.707442+0000","last_peered":"2024-09-10T20:40:58.707442+0000","last_clean":"2024-09-10T20:40:58.707442+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:40:58.707442+0000","last_undegraded":"2024-09-10T20:40:58.707442+0000","last_fullsized":"2024-09-10T20:40:58.707442+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21117,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.563002+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:40:58.563002+0000","last_peered":"2024-09-10T20:40:58.563002+0000","last_clean":"2024-09-10T20:40:58.563002+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:40:58.563002+0000","last_undegraded":"2024-09-10T20:40:58.563002+0000","last_fullsized":"2024-09-10T20:40:58.563002+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562986+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:40:58.562986+0000","last_peered":"2024-09-10T20:40:58.562986+0000","last_clean":"2024-09-10T20:40:58.562986+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:40:58.562986+0000","last_undegraded":"2024-09-10T20:40:58.562986+0000","last_fullsized":"2024-09-10T20:40:58.562986+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27839,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562924+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:40:58.562924+0000","last_peered":"2024-09-10T20:40:58.562924+0000","last_clean":"2024-09-10T20:40:58.562924+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:40:58.562924+0000","last_undegraded":"2024-09-10T20:40:58.562924+0000","last_fullsized":"2024-09-10T20:40:58.562924+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":129,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707683+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:40:58.707683+0000","last_peered":"2024-09-10T20:40:58.707683+0000","last_clean":"2024-09-10T20:40:58.707683+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:40:58.707683+0000","last_undegraded":"2024-09-10T20:40:58.707683+0000","last_fullsized":"2024-09-10T20:40:58.707683+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21269,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:58.558883+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:40:58.558883+0000","last_peered":"2024-09-10T20:40:58.558883+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:40:58.558883+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562868+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:40:58.562868+0000","last_peered":"2024-09-10T20:40:58.562868+0000","last_clean":"2024-09-10T20:40:58.562868+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:40:58.562868+0000","last_undegraded":"2024-09-10T20:40:58.562868+0000","last_fullsized":"2024-09-10T20:40:58.562868+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562953+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:40:58.562953+0000","last_peered":"2024-09-10T20:40:58.562953+0000","last_clean":"2024-09-10T20:40:58.562953+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:40:58.562953+0000","last_undegraded":"2024-09-10T20:40:58.562953+0000","last_fullsized":"2024-09-10T20:40:58.562953+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27669,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562986+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:40:58.562986+0000","last_peered":"2024-09-10T20:40:58.562986+0000","last_clean":"2024-09-10T20:40:58.562986+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:40:58.562986+0000","last_undegraded":"2024-09-10T20:40:58.562986+0000","last_fullsized":"2024-09-10T20:40:58.562986+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9329,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30201,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":154562560,"data_stored":154537269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182550,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517212,"kb_used_data":57988,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854628,"statfs":{"total":96636764160,"available":96107139072,"internally_reserved":0,"allocated":59379712,"data_stored":59058784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986322,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":459408,"kb_used_data":94528,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93912432,"statfs":{"total":96636764160,"available":96166330368,"internally_reserved":0,"allocated":96796672,"data_stored":96479821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659269,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505624,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866216,"statfs":{"total":96636764160,"available":96119005184,"internally_reserved":0,"allocated":532480,"data_stored":197100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":517205678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":95674368,"data_stored":95670755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:41:09.366 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:41:09.367 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20471", 'reported_seq': 20749, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:58.558601+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:40:58.558601+0000', 'last_peered': '2024-09-10T20:40:58.558601+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:40:58.558601+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17335, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3561}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:41:09.367 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:41:09.367 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'20608", 'reported_seq': 23679, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:41:05.235187+0000', 'last_change': '2024-09-10T20:40:57.471908+0000', 'last_active': '2024-09-10T20:41:05.235187+0000', 'last_peered': '2024-09-10T20:41:05.235187+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:41:05.235187+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2109, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17203, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1448, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2109}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:41:09.367 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:41:09.367 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "51'20986", 'reported_seq': 21269, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:58.558883+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:40:58.558883+0000', 'last_peered': '2024-09-10T20:40:58.558883+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:40:58.558883+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17813, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3655}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:41:09.367 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:41:09.759 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:09.759 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 9329/6 objects degraded (155483.333%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":781,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1517817856,"bytes_avail":288392474624,"bytes_total":289910292480,"degraded_objects":9329,"degraded_total":6,"degraded_ratio":1554.833333333333,"recovering_objects_per_sec":179,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1884,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:41:09.775 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:41:09.775 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:10.083 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:10.083 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:10.095 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":445,"stamp":"2024-09-10T20:41:09.074660+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9329,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30203,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1482244,"kb_used_data":153036,"kb_used_omap":12,"kb_used_meta":1329139,"kb_avail":281633276,"statfs":{"total":289910292480,"available":288392474624,"internally_reserved":0,"allocated":156708864,"data_stored":155735705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1361038338},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1884,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1884,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.517503"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":27996,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.617469+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:40:58.617469+0000","last_peered":"2024-09-10T20:40:58.617469+0000","last_clean":"2024-09-10T20:40:58.617469+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:40:58.617469+0000","last_undegraded":"2024-09-10T20:40:58.617469+0000","last_fullsized":"2024-09-10T20:40:58.617469+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21088,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707761+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:40:58.707761+0000","last_peered":"2024-09-10T20:40:58.707761+0000","last_clean":"2024-09-10T20:40:58.707761+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:40:58.707761+0000","last_undegraded":"2024-09-10T20:40:58.707761+0000","last_fullsized":"2024-09-10T20:40:58.707761+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28099,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562751+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:40:58.562751+0000","last_peered":"2024-09-10T20:40:58.562751+0000","last_clean":"2024-09-10T20:40:58.562751+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:40:58.562751+0000","last_undegraded":"2024-09-10T20:40:58.562751+0000","last_fullsized":"2024-09-10T20:40:58.562751+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20720,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707714+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:40:58.707714+0000","last_peered":"2024-09-10T20:40:58.707714+0000","last_clean":"2024-09-10T20:40:58.707714+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:40:58.707714+0000","last_undegraded":"2024-09-10T20:40:58.707714+0000","last_fullsized":"2024-09-10T20:40:58.707714+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21231,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558744+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:40:58.558744+0000","last_peered":"2024-09-10T20:40:58.558744+0000","last_clean":"2024-09-10T20:40:58.558744+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:40:58.558744+0000","last_undegraded":"2024-09-10T20:40:58.558744+0000","last_fullsized":"2024-09-10T20:40:58.558744+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20749,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:58.558601+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:40:58.558601+0000","last_peered":"2024-09-10T20:40:58.558601+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:40:58.558601+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":23679,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:05.235187+0000","last_change":"2024-09-10T20:40:57.471908+0000","last_active":"2024-09-10T20:41:05.235187+0000","last_peered":"2024-09-10T20:41:05.235187+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:41:05.235187+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2109,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1448,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20896,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558180+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:40:58.558180+0000","last_peered":"2024-09-10T20:40:58.558180+0000","last_clean":"2024-09-10T20:40:58.558180+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:40:58.558180+0000","last_undegraded":"2024-09-10T20:40:58.558180+0000","last_fullsized":"2024-09-10T20:40:58.558180+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562632+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:40:58.562632+0000","last_peered":"2024-09-10T20:40:58.562632+0000","last_clean":"2024-09-10T20:40:58.562632+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:40:58.562632+0000","last_undegraded":"2024-09-10T20:40:58.562632+0000","last_fullsized":"2024-09-10T20:40:58.562632+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22138,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562862+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:40:58.562862+0000","last_peered":"2024-09-10T20:40:58.562862+0000","last_clean":"2024-09-10T20:40:58.562862+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:40:58.562862+0000","last_undegraded":"2024-09-10T20:40:58.562862+0000","last_fullsized":"2024-09-10T20:40:58.562862+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":129,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707515+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:40:58.707515+0000","last_peered":"2024-09-10T20:40:58.707515+0000","last_clean":"2024-09-10T20:40:58.707515+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:40:58.707515+0000","last_undegraded":"2024-09-10T20:40:58.707515+0000","last_fullsized":"2024-09-10T20:40:58.707515+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28124,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558664+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:40:58.558664+0000","last_peered":"2024-09-10T20:40:58.558664+0000","last_clean":"2024-09-10T20:40:58.558664+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:40:58.558664+0000","last_undegraded":"2024-09-10T20:40:58.558664+0000","last_fullsized":"2024-09-10T20:40:58.558664+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558561+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:40:58.558561+0000","last_peered":"2024-09-10T20:40:58.558561+0000","last_clean":"2024-09-10T20:40:58.558561+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:40:58.558561+0000","last_undegraded":"2024-09-10T20:40:58.558561+0000","last_fullsized":"2024-09-10T20:40:58.558561+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28007,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707523+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:40:58.707523+0000","last_peered":"2024-09-10T20:40:58.707523+0000","last_clean":"2024-09-10T20:40:58.707523+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:40:58.707523+0000","last_undegraded":"2024-09-10T20:40:58.707523+0000","last_fullsized":"2024-09-10T20:40:58.707523+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562762+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:40:58.562762+0000","last_peered":"2024-09-10T20:40:58.562762+0000","last_clean":"2024-09-10T20:40:58.562762+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:40:58.562762+0000","last_undegraded":"2024-09-10T20:40:58.562762+0000","last_fullsized":"2024-09-10T20:40:58.562762+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28060,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707442+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:40:58.707442+0000","last_peered":"2024-09-10T20:40:58.707442+0000","last_clean":"2024-09-10T20:40:58.707442+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:40:58.707442+0000","last_undegraded":"2024-09-10T20:40:58.707442+0000","last_fullsized":"2024-09-10T20:40:58.707442+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21117,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.563002+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:40:58.563002+0000","last_peered":"2024-09-10T20:40:58.563002+0000","last_clean":"2024-09-10T20:40:58.563002+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:40:58.563002+0000","last_undegraded":"2024-09-10T20:40:58.563002+0000","last_fullsized":"2024-09-10T20:40:58.563002+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562986+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:40:58.562986+0000","last_peered":"2024-09-10T20:40:58.562986+0000","last_clean":"2024-09-10T20:40:58.562986+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:40:58.562986+0000","last_undegraded":"2024-09-10T20:40:58.562986+0000","last_fullsized":"2024-09-10T20:40:58.562986+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27839,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562924+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:40:58.562924+0000","last_peered":"2024-09-10T20:40:58.562924+0000","last_clean":"2024-09-10T20:40:58.562924+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:40:58.562924+0000","last_undegraded":"2024-09-10T20:40:58.562924+0000","last_fullsized":"2024-09-10T20:40:58.562924+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":129,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707683+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:40:58.707683+0000","last_peered":"2024-09-10T20:40:58.707683+0000","last_clean":"2024-09-10T20:40:58.707683+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:40:58.707683+0000","last_undegraded":"2024-09-10T20:40:58.707683+0000","last_fullsized":"2024-09-10T20:40:58.707683+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21269,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:58.558883+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:40:58.558883+0000","last_peered":"2024-09-10T20:40:58.558883+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:40:58.558883+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562868+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:40:58.562868+0000","last_peered":"2024-09-10T20:40:58.562868+0000","last_clean":"2024-09-10T20:40:58.562868+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:40:58.562868+0000","last_undegraded":"2024-09-10T20:40:58.562868+0000","last_fullsized":"2024-09-10T20:40:58.562868+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562953+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:40:58.562953+0000","last_peered":"2024-09-10T20:40:58.562953+0000","last_clean":"2024-09-10T20:40:58.562953+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:40:58.562953+0000","last_undegraded":"2024-09-10T20:40:58.562953+0000","last_fullsized":"2024-09-10T20:40:58.562953+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27669,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562986+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:40:58.562986+0000","last_peered":"2024-09-10T20:40:58.562986+0000","last_clean":"2024-09-10T20:40:58.562986+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:40:58.562986+0000","last_undegraded":"2024-09-10T20:40:58.562986+0000","last_fullsized":"2024-09-10T20:40:58.562986+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9329,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30201,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":154562560,"data_stored":154537269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182550,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517212,"kb_used_data":57988,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854628,"statfs":{"total":96636764160,"available":96107139072,"internally_reserved":0,"allocated":59379712,"data_stored":59058784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986322,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":459408,"kb_used_data":94528,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93912432,"statfs":{"total":96636764160,"available":96166330368,"internally_reserved":0,"allocated":96796672,"data_stored":96479821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659269,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505624,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866216,"statfs":{"total":96636764160,"available":96119005184,"internally_reserved":0,"allocated":532480,"data_stored":197100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":517205678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":95674368,"data_stored":95670755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:41:13.097 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:13.404 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:13.405 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:13.417 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":447,"stamp":"2024-09-10T20:41:13.075302+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8510,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31022,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1470924,"kb_used_data":150036,"kb_used_omap":12,"kb_used_meta":1320819,"kb_avail":281644596,"statfs":{"total":289910292480,"available":288404066304,"internally_reserved":0,"allocated":153636864,"data_stored":152663705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1352518658},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1689,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1689,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002058"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":27996,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.617469+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:40:58.617469+0000","last_peered":"2024-09-10T20:40:58.617469+0000","last_clean":"2024-09-10T20:40:58.617469+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:40:58.617469+0000","last_undegraded":"2024-09-10T20:40:58.617469+0000","last_fullsized":"2024-09-10T20:40:58.617469+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21088,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707761+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:40:58.707761+0000","last_peered":"2024-09-10T20:40:58.707761+0000","last_clean":"2024-09-10T20:40:58.707761+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:40:58.707761+0000","last_undegraded":"2024-09-10T20:40:58.707761+0000","last_fullsized":"2024-09-10T20:40:58.707761+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28099,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562751+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:40:58.562751+0000","last_peered":"2024-09-10T20:40:58.562751+0000","last_clean":"2024-09-10T20:40:58.562751+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:40:58.562751+0000","last_undegraded":"2024-09-10T20:40:58.562751+0000","last_fullsized":"2024-09-10T20:40:58.562751+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20720,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707714+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:40:58.707714+0000","last_peered":"2024-09-10T20:40:58.707714+0000","last_clean":"2024-09-10T20:40:58.707714+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:40:58.707714+0000","last_undegraded":"2024-09-10T20:40:58.707714+0000","last_fullsized":"2024-09-10T20:40:58.707714+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21231,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558744+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:40:58.558744+0000","last_peered":"2024-09-10T20:40:58.558744+0000","last_clean":"2024-09-10T20:40:58.558744+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:40:58.558744+0000","last_undegraded":"2024-09-10T20:40:58.558744+0000","last_fullsized":"2024-09-10T20:40:58.558744+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20749,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:58.558601+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:40:58.558601+0000","last_peered":"2024-09-10T20:40:58.558601+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:40:58.558601+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":25317,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:10.235262+0000","last_change":"2024-09-10T20:40:57.471908+0000","last_active":"2024-09-10T20:41:10.235262+0000","last_peered":"2024-09-10T20:41:10.235262+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:41:10.235262+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1290,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2267,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1290}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20896,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558180+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:40:58.558180+0000","last_peered":"2024-09-10T20:40:58.558180+0000","last_clean":"2024-09-10T20:40:58.558180+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:40:58.558180+0000","last_undegraded":"2024-09-10T20:40:58.558180+0000","last_fullsized":"2024-09-10T20:40:58.558180+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562632+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:40:58.562632+0000","last_peered":"2024-09-10T20:40:58.562632+0000","last_clean":"2024-09-10T20:40:58.562632+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:40:58.562632+0000","last_undegraded":"2024-09-10T20:40:58.562632+0000","last_fullsized":"2024-09-10T20:40:58.562632+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22138,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562862+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:40:58.562862+0000","last_peered":"2024-09-10T20:40:58.562862+0000","last_clean":"2024-09-10T20:40:58.562862+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:40:58.562862+0000","last_undegraded":"2024-09-10T20:40:58.562862+0000","last_fullsized":"2024-09-10T20:40:58.562862+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":129,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707515+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:40:58.707515+0000","last_peered":"2024-09-10T20:40:58.707515+0000","last_clean":"2024-09-10T20:40:58.707515+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:40:58.707515+0000","last_undegraded":"2024-09-10T20:40:58.707515+0000","last_fullsized":"2024-09-10T20:40:58.707515+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28124,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558664+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:40:58.558664+0000","last_peered":"2024-09-10T20:40:58.558664+0000","last_clean":"2024-09-10T20:40:58.558664+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:40:58.558664+0000","last_undegraded":"2024-09-10T20:40:58.558664+0000","last_fullsized":"2024-09-10T20:40:58.558664+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558561+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:40:58.558561+0000","last_peered":"2024-09-10T20:40:58.558561+0000","last_clean":"2024-09-10T20:40:58.558561+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:40:58.558561+0000","last_undegraded":"2024-09-10T20:40:58.558561+0000","last_fullsized":"2024-09-10T20:40:58.558561+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28007,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707523+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:40:58.707523+0000","last_peered":"2024-09-10T20:40:58.707523+0000","last_clean":"2024-09-10T20:40:58.707523+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:40:58.707523+0000","last_undegraded":"2024-09-10T20:40:58.707523+0000","last_fullsized":"2024-09-10T20:40:58.707523+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562762+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:40:58.562762+0000","last_peered":"2024-09-10T20:40:58.562762+0000","last_clean":"2024-09-10T20:40:58.562762+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:40:58.562762+0000","last_undegraded":"2024-09-10T20:40:58.562762+0000","last_fullsized":"2024-09-10T20:40:58.562762+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28060,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707442+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:40:58.707442+0000","last_peered":"2024-09-10T20:40:58.707442+0000","last_clean":"2024-09-10T20:40:58.707442+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:40:58.707442+0000","last_undegraded":"2024-09-10T20:40:58.707442+0000","last_fullsized":"2024-09-10T20:40:58.707442+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21117,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.563002+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:40:58.563002+0000","last_peered":"2024-09-10T20:40:58.563002+0000","last_clean":"2024-09-10T20:40:58.563002+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:40:58.563002+0000","last_undegraded":"2024-09-10T20:40:58.563002+0000","last_fullsized":"2024-09-10T20:40:58.563002+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562986+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:40:58.562986+0000","last_peered":"2024-09-10T20:40:58.562986+0000","last_clean":"2024-09-10T20:40:58.562986+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:40:58.562986+0000","last_undegraded":"2024-09-10T20:40:58.562986+0000","last_fullsized":"2024-09-10T20:40:58.562986+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27839,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562924+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:40:58.562924+0000","last_peered":"2024-09-10T20:40:58.562924+0000","last_clean":"2024-09-10T20:40:58.562924+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:40:58.562924+0000","last_undegraded":"2024-09-10T20:40:58.562924+0000","last_fullsized":"2024-09-10T20:40:58.562924+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":129,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707683+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:40:58.707683+0000","last_peered":"2024-09-10T20:40:58.707683+0000","last_clean":"2024-09-10T20:40:58.707683+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:40:58.707683+0000","last_undegraded":"2024-09-10T20:40:58.707683+0000","last_fullsized":"2024-09-10T20:40:58.707683+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21269,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:58.558883+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:40:58.558883+0000","last_peered":"2024-09-10T20:40:58.558883+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:40:58.558883+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562868+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:40:58.562868+0000","last_peered":"2024-09-10T20:40:58.562868+0000","last_clean":"2024-09-10T20:40:58.562868+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:40:58.562868+0000","last_undegraded":"2024-09-10T20:40:58.562868+0000","last_fullsized":"2024-09-10T20:40:58.562868+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562953+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:40:58.562953+0000","last_peered":"2024-09-10T20:40:58.562953+0000","last_clean":"2024-09-10T20:40:58.562953+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:40:58.562953+0000","last_undegraded":"2024-09-10T20:40:58.562953+0000","last_fullsized":"2024-09-10T20:40:58.562953+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27669,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562986+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:40:58.562986+0000","last_peered":"2024-09-10T20:40:58.562986+0000","last_clean":"2024-09-10T20:40:58.562986+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:40:58.562986+0000","last_undegraded":"2024-09-10T20:40:58.562986+0000","last_fullsized":"2024-09-10T20:40:58.562986+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8510,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31020,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":151490560,"data_stored":151465269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182551,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517212,"kb_used_data":57988,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854628,"statfs":{"total":96636764160,"available":96107139072,"internally_reserved":0,"allocated":59379712,"data_stored":59058784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986323,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":456408,"kb_used_data":91528,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93915432,"statfs":{"total":96636764160,"available":96169402368,"internally_reserved":0,"allocated":93724672,"data_stored":93407821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659270,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497304,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874536,"statfs":{"total":96636764160,"available":96127524864,"internally_reserved":0,"allocated":532480,"data_stored":197100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":92602368,"data_stored":92598755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:41:13.418 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:13.723 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:13.724 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:13.736 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":447,"stamp":"2024-09-10T20:41:13.075302+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8510,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31022,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1470924,"kb_used_data":150036,"kb_used_omap":12,"kb_used_meta":1320819,"kb_avail":281644596,"statfs":{"total":289910292480,"available":288404066304,"internally_reserved":0,"allocated":153636864,"data_stored":152663705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1352518658},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1689,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1689,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002058"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":27996,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.617469+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:40:58.617469+0000","last_peered":"2024-09-10T20:40:58.617469+0000","last_clean":"2024-09-10T20:40:58.617469+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:40:58.617469+0000","last_undegraded":"2024-09-10T20:40:58.617469+0000","last_fullsized":"2024-09-10T20:40:58.617469+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21088,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707761+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:40:58.707761+0000","last_peered":"2024-09-10T20:40:58.707761+0000","last_clean":"2024-09-10T20:40:58.707761+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:40:58.707761+0000","last_undegraded":"2024-09-10T20:40:58.707761+0000","last_fullsized":"2024-09-10T20:40:58.707761+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28099,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562751+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:40:58.562751+0000","last_peered":"2024-09-10T20:40:58.562751+0000","last_clean":"2024-09-10T20:40:58.562751+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:40:58.562751+0000","last_undegraded":"2024-09-10T20:40:58.562751+0000","last_fullsized":"2024-09-10T20:40:58.562751+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20720,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707714+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:40:58.707714+0000","last_peered":"2024-09-10T20:40:58.707714+0000","last_clean":"2024-09-10T20:40:58.707714+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:40:58.707714+0000","last_undegraded":"2024-09-10T20:40:58.707714+0000","last_fullsized":"2024-09-10T20:40:58.707714+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21231,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558744+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:40:58.558744+0000","last_peered":"2024-09-10T20:40:58.558744+0000","last_clean":"2024-09-10T20:40:58.558744+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:40:58.558744+0000","last_undegraded":"2024-09-10T20:40:58.558744+0000","last_fullsized":"2024-09-10T20:40:58.558744+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20749,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:58.558601+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:40:58.558601+0000","last_peered":"2024-09-10T20:40:58.558601+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:40:58.558601+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":25317,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:10.235262+0000","last_change":"2024-09-10T20:40:57.471908+0000","last_active":"2024-09-10T20:41:10.235262+0000","last_peered":"2024-09-10T20:41:10.235262+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:41:10.235262+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1290,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2267,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1290}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20896,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558180+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:40:58.558180+0000","last_peered":"2024-09-10T20:40:58.558180+0000","last_clean":"2024-09-10T20:40:58.558180+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:40:58.558180+0000","last_undegraded":"2024-09-10T20:40:58.558180+0000","last_fullsized":"2024-09-10T20:40:58.558180+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562632+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:40:58.562632+0000","last_peered":"2024-09-10T20:40:58.562632+0000","last_clean":"2024-09-10T20:40:58.562632+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:40:58.562632+0000","last_undegraded":"2024-09-10T20:40:58.562632+0000","last_fullsized":"2024-09-10T20:40:58.562632+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22138,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562862+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:40:58.562862+0000","last_peered":"2024-09-10T20:40:58.562862+0000","last_clean":"2024-09-10T20:40:58.562862+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:40:58.562862+0000","last_undegraded":"2024-09-10T20:40:58.562862+0000","last_fullsized":"2024-09-10T20:40:58.562862+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":129,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707515+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:40:58.707515+0000","last_peered":"2024-09-10T20:40:58.707515+0000","last_clean":"2024-09-10T20:40:58.707515+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:40:58.707515+0000","last_undegraded":"2024-09-10T20:40:58.707515+0000","last_fullsized":"2024-09-10T20:40:58.707515+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28124,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558664+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:40:58.558664+0000","last_peered":"2024-09-10T20:40:58.558664+0000","last_clean":"2024-09-10T20:40:58.558664+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:40:58.558664+0000","last_undegraded":"2024-09-10T20:40:58.558664+0000","last_fullsized":"2024-09-10T20:40:58.558664+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558561+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:40:58.558561+0000","last_peered":"2024-09-10T20:40:58.558561+0000","last_clean":"2024-09-10T20:40:58.558561+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:40:58.558561+0000","last_undegraded":"2024-09-10T20:40:58.558561+0000","last_fullsized":"2024-09-10T20:40:58.558561+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28007,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707523+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:40:58.707523+0000","last_peered":"2024-09-10T20:40:58.707523+0000","last_clean":"2024-09-10T20:40:58.707523+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:40:58.707523+0000","last_undegraded":"2024-09-10T20:40:58.707523+0000","last_fullsized":"2024-09-10T20:40:58.707523+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562762+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:40:58.562762+0000","last_peered":"2024-09-10T20:40:58.562762+0000","last_clean":"2024-09-10T20:40:58.562762+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:40:58.562762+0000","last_undegraded":"2024-09-10T20:40:58.562762+0000","last_fullsized":"2024-09-10T20:40:58.562762+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28060,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707442+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:40:58.707442+0000","last_peered":"2024-09-10T20:40:58.707442+0000","last_clean":"2024-09-10T20:40:58.707442+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:40:58.707442+0000","last_undegraded":"2024-09-10T20:40:58.707442+0000","last_fullsized":"2024-09-10T20:40:58.707442+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21117,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.563002+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:40:58.563002+0000","last_peered":"2024-09-10T20:40:58.563002+0000","last_clean":"2024-09-10T20:40:58.563002+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:40:58.563002+0000","last_undegraded":"2024-09-10T20:40:58.563002+0000","last_fullsized":"2024-09-10T20:40:58.563002+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562986+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:40:58.562986+0000","last_peered":"2024-09-10T20:40:58.562986+0000","last_clean":"2024-09-10T20:40:58.562986+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:40:58.562986+0000","last_undegraded":"2024-09-10T20:40:58.562986+0000","last_fullsized":"2024-09-10T20:40:58.562986+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27839,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562924+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:40:58.562924+0000","last_peered":"2024-09-10T20:40:58.562924+0000","last_clean":"2024-09-10T20:40:58.562924+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:40:58.562924+0000","last_undegraded":"2024-09-10T20:40:58.562924+0000","last_fullsized":"2024-09-10T20:40:58.562924+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":129,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707683+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:40:58.707683+0000","last_peered":"2024-09-10T20:40:58.707683+0000","last_clean":"2024-09-10T20:40:58.707683+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:40:58.707683+0000","last_undegraded":"2024-09-10T20:40:58.707683+0000","last_fullsized":"2024-09-10T20:40:58.707683+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21269,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:58.558883+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:40:58.558883+0000","last_peered":"2024-09-10T20:40:58.558883+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:40:58.558883+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562868+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:40:58.562868+0000","last_peered":"2024-09-10T20:40:58.562868+0000","last_clean":"2024-09-10T20:40:58.562868+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:40:58.562868+0000","last_undegraded":"2024-09-10T20:40:58.562868+0000","last_fullsized":"2024-09-10T20:40:58.562868+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562953+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:40:58.562953+0000","last_peered":"2024-09-10T20:40:58.562953+0000","last_clean":"2024-09-10T20:40:58.562953+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:40:58.562953+0000","last_undegraded":"2024-09-10T20:40:58.562953+0000","last_fullsized":"2024-09-10T20:40:58.562953+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27669,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562986+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:40:58.562986+0000","last_peered":"2024-09-10T20:40:58.562986+0000","last_clean":"2024-09-10T20:40:58.562986+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:40:58.562986+0000","last_undegraded":"2024-09-10T20:40:58.562986+0000","last_fullsized":"2024-09-10T20:40:58.562986+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8510,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31020,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":151490560,"data_stored":151465269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182551,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517212,"kb_used_data":57988,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854628,"statfs":{"total":96636764160,"available":96107139072,"internally_reserved":0,"allocated":59379712,"data_stored":59058784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986323,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":456408,"kb_used_data":91528,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93915432,"statfs":{"total":96636764160,"available":96169402368,"internally_reserved":0,"allocated":93724672,"data_stored":93407821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659270,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497304,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874536,"statfs":{"total":96636764160,"available":96127524864,"internally_reserved":0,"allocated":532480,"data_stored":197100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":92602368,"data_stored":92598755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:41:13.738 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:41:13.738 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20471", 'reported_seq': 20749, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:58.558601+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:40:58.558601+0000', 'last_peered': '2024-09-10T20:40:58.558601+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:40:58.558601+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17335, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3561}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:41:13.738 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:41:13.738 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'20608", 'reported_seq': 25317, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:41:10.235262+0000', 'last_change': '2024-09-10T20:40:57.471908+0000', 'last_active': '2024-09-10T20:41:10.235262+0000', 'last_peered': '2024-09-10T20:41:10.235262+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:41:10.235262+0000', 'last_undegraded': '2024-09-10T20:30:05.392406+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1290, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17203, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2267, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1290}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:41:13.738 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:41:13.738 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "51'20986", 'reported_seq': 21269, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:58.558883+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:40:58.558883+0000', 'last_peered': '2024-09-10T20:40:58.558883+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:40:58.558883+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17813, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3655}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:41:13.738 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:41:14.148 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:14.148 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 8510/6 objects degraded (141833.333%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":785,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1506226176,"bytes_avail":288404066304,"bytes_total":289910292480,"degraded_objects":8510,"degraded_total":6,"degraded_ratio":1418.333333333333,"recovering_objects_per_sec":140,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1689,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:41:14.161 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:41:14.162 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:14.467 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:14.467 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:14.480 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":447,"stamp":"2024-09-10T20:41:13.075302+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8510,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31022,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1470924,"kb_used_data":150036,"kb_used_omap":12,"kb_used_meta":1320819,"kb_avail":281644596,"statfs":{"total":289910292480,"available":288404066304,"internally_reserved":0,"allocated":153636864,"data_stored":152663705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1352518658},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1689,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1689,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002058"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":27996,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.617469+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:40:58.617469+0000","last_peered":"2024-09-10T20:40:58.617469+0000","last_clean":"2024-09-10T20:40:58.617469+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:40:58.617469+0000","last_undegraded":"2024-09-10T20:40:58.617469+0000","last_fullsized":"2024-09-10T20:40:58.617469+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21088,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707761+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:40:58.707761+0000","last_peered":"2024-09-10T20:40:58.707761+0000","last_clean":"2024-09-10T20:40:58.707761+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:40:58.707761+0000","last_undegraded":"2024-09-10T20:40:58.707761+0000","last_fullsized":"2024-09-10T20:40:58.707761+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28099,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562751+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:40:58.562751+0000","last_peered":"2024-09-10T20:40:58.562751+0000","last_clean":"2024-09-10T20:40:58.562751+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:40:58.562751+0000","last_undegraded":"2024-09-10T20:40:58.562751+0000","last_fullsized":"2024-09-10T20:40:58.562751+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20720,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707714+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:40:58.707714+0000","last_peered":"2024-09-10T20:40:58.707714+0000","last_clean":"2024-09-10T20:40:58.707714+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:40:58.707714+0000","last_undegraded":"2024-09-10T20:40:58.707714+0000","last_fullsized":"2024-09-10T20:40:58.707714+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21231,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558744+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:40:58.558744+0000","last_peered":"2024-09-10T20:40:58.558744+0000","last_clean":"2024-09-10T20:40:58.558744+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:40:58.558744+0000","last_undegraded":"2024-09-10T20:40:58.558744+0000","last_fullsized":"2024-09-10T20:40:58.558744+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20749,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:58.558601+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:40:58.558601+0000","last_peered":"2024-09-10T20:40:58.558601+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:40:58.558601+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":25317,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:10.235262+0000","last_change":"2024-09-10T20:40:57.471908+0000","last_active":"2024-09-10T20:41:10.235262+0000","last_peered":"2024-09-10T20:41:10.235262+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:41:10.235262+0000","last_undegraded":"2024-09-10T20:30:05.392406+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1290,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2267,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1290}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20896,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558180+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:40:58.558180+0000","last_peered":"2024-09-10T20:40:58.558180+0000","last_clean":"2024-09-10T20:40:58.558180+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:40:58.558180+0000","last_undegraded":"2024-09-10T20:40:58.558180+0000","last_fullsized":"2024-09-10T20:40:58.558180+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562632+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:40:58.562632+0000","last_peered":"2024-09-10T20:40:58.562632+0000","last_clean":"2024-09-10T20:40:58.562632+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:40:58.562632+0000","last_undegraded":"2024-09-10T20:40:58.562632+0000","last_fullsized":"2024-09-10T20:40:58.562632+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22138,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562862+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:40:58.562862+0000","last_peered":"2024-09-10T20:40:58.562862+0000","last_clean":"2024-09-10T20:40:58.562862+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:40:58.562862+0000","last_undegraded":"2024-09-10T20:40:58.562862+0000","last_fullsized":"2024-09-10T20:40:58.562862+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":129,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707515+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:40:58.707515+0000","last_peered":"2024-09-10T20:40:58.707515+0000","last_clean":"2024-09-10T20:40:58.707515+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:40:58.707515+0000","last_undegraded":"2024-09-10T20:40:58.707515+0000","last_fullsized":"2024-09-10T20:40:58.707515+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28124,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558664+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:40:58.558664+0000","last_peered":"2024-09-10T20:40:58.558664+0000","last_clean":"2024-09-10T20:40:58.558664+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:40:58.558664+0000","last_undegraded":"2024-09-10T20:40:58.558664+0000","last_fullsized":"2024-09-10T20:40:58.558664+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558561+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:40:58.558561+0000","last_peered":"2024-09-10T20:40:58.558561+0000","last_clean":"2024-09-10T20:40:58.558561+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:40:58.558561+0000","last_undegraded":"2024-09-10T20:40:58.558561+0000","last_fullsized":"2024-09-10T20:40:58.558561+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28007,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707523+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:40:58.707523+0000","last_peered":"2024-09-10T20:40:58.707523+0000","last_clean":"2024-09-10T20:40:58.707523+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:40:58.707523+0000","last_undegraded":"2024-09-10T20:40:58.707523+0000","last_fullsized":"2024-09-10T20:40:58.707523+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562762+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:40:58.562762+0000","last_peered":"2024-09-10T20:40:58.562762+0000","last_clean":"2024-09-10T20:40:58.562762+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:40:58.562762+0000","last_undegraded":"2024-09-10T20:40:58.562762+0000","last_fullsized":"2024-09-10T20:40:58.562762+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28060,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707442+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:40:58.707442+0000","last_peered":"2024-09-10T20:40:58.707442+0000","last_clean":"2024-09-10T20:40:58.707442+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:40:58.707442+0000","last_undegraded":"2024-09-10T20:40:58.707442+0000","last_fullsized":"2024-09-10T20:40:58.707442+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21117,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.563002+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:40:58.563002+0000","last_peered":"2024-09-10T20:40:58.563002+0000","last_clean":"2024-09-10T20:40:58.563002+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:40:58.563002+0000","last_undegraded":"2024-09-10T20:40:58.563002+0000","last_fullsized":"2024-09-10T20:40:58.563002+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562986+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:40:58.562986+0000","last_peered":"2024-09-10T20:40:58.562986+0000","last_clean":"2024-09-10T20:40:58.562986+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:40:58.562986+0000","last_undegraded":"2024-09-10T20:40:58.562986+0000","last_fullsized":"2024-09-10T20:40:58.562986+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27839,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562924+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:40:58.562924+0000","last_peered":"2024-09-10T20:40:58.562924+0000","last_clean":"2024-09-10T20:40:58.562924+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:40:58.562924+0000","last_undegraded":"2024-09-10T20:40:58.562924+0000","last_fullsized":"2024-09-10T20:40:58.562924+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":129,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707683+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:40:58.707683+0000","last_peered":"2024-09-10T20:40:58.707683+0000","last_clean":"2024-09-10T20:40:58.707683+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:40:58.707683+0000","last_undegraded":"2024-09-10T20:40:58.707683+0000","last_fullsized":"2024-09-10T20:40:58.707683+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21269,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:58.558883+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:40:58.558883+0000","last_peered":"2024-09-10T20:40:58.558883+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:40:58.558883+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562868+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:40:58.562868+0000","last_peered":"2024-09-10T20:40:58.562868+0000","last_clean":"2024-09-10T20:40:58.562868+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:40:58.562868+0000","last_undegraded":"2024-09-10T20:40:58.562868+0000","last_fullsized":"2024-09-10T20:40:58.562868+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562953+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:40:58.562953+0000","last_peered":"2024-09-10T20:40:58.562953+0000","last_clean":"2024-09-10T20:40:58.562953+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:40:58.562953+0000","last_undegraded":"2024-09-10T20:40:58.562953+0000","last_fullsized":"2024-09-10T20:40:58.562953+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27669,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.562986+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:40:58.562986+0000","last_peered":"2024-09-10T20:40:58.562986+0000","last_clean":"2024-09-10T20:40:58.562986+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:40:58.562986+0000","last_undegraded":"2024-09-10T20:40:58.562986+0000","last_fullsized":"2024-09-10T20:40:58.562986+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8510,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31020,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":151490560,"data_stored":151465269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182551,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517212,"kb_used_data":57988,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854628,"statfs":{"total":96636764160,"available":96107139072,"internally_reserved":0,"allocated":59379712,"data_stored":59058784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986323,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":456408,"kb_used_data":91528,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93915432,"statfs":{"total":96636764160,"available":96169402368,"internally_reserved":0,"allocated":93724672,"data_stored":93407821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659270,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497304,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874536,"statfs":{"total":96636764160,"available":96127524864,"internally_reserved":0,"allocated":532480,"data_stored":197100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":92602368,"data_stored":92598755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:41:17.482 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:17.851 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:17.852 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:17.865 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":451,"stamp":"2024-09-10T20:41:17.658039+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32312,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1476248,"kb_used_data":147036,"kb_used_omap":12,"kb_used_meta":1329139,"kb_avail":281639272,"statfs":{"total":289910292480,"available":288398614528,"internally_reserved":0,"allocated":150564864,"data_stored":149593939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1361038338},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2109,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2109,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.583379"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":27996,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.617469+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:40:58.617469+0000","last_peered":"2024-09-10T20:40:58.617469+0000","last_clean":"2024-09-10T20:40:58.617469+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:40:58.617469+0000","last_undegraded":"2024-09-10T20:40:58.617469+0000","last_fullsized":"2024-09-10T20:40:58.617469+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21088,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707761+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:40:58.707761+0000","last_peered":"2024-09-10T20:40:58.707761+0000","last_clean":"2024-09-10T20:40:58.707761+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:40:58.707761+0000","last_undegraded":"2024-09-10T20:40:58.707761+0000","last_fullsized":"2024-09-10T20:40:58.707761+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28101,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.651801+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:41:16.651801+0000","last_peered":"2024-09-10T20:41:16.651801+0000","last_clean":"2024-09-10T20:41:16.651801+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:41:16.651801+0000","last_undegraded":"2024-09-10T20:41:16.651801+0000","last_fullsized":"2024-09-10T20:41:16.651801+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20720,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707714+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:40:58.707714+0000","last_peered":"2024-09-10T20:40:58.707714+0000","last_clean":"2024-09-10T20:40:58.707714+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:40:58.707714+0000","last_undegraded":"2024-09-10T20:40:58.707714+0000","last_fullsized":"2024-09-10T20:40:58.707714+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21231,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558744+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:40:58.558744+0000","last_peered":"2024-09-10T20:40:58.558744+0000","last_clean":"2024-09-10T20:40:58.558744+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:40:58.558744+0000","last_undegraded":"2024-09-10T20:40:58.558744+0000","last_fullsized":"2024-09-10T20:40:58.558744+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20749,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:58.558601+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:40:58.558601+0000","last_peered":"2024-09-10T20:40:58.558601+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:40:58.558601+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20902,"reported_epoch":62,"state":"peering","last_fresh":"2024-09-10T20:41:15.931130+0000","last_change":"2024-09-10T20:41:17.283132+0000","last_active":"2024-09-10T20:41:17.283132+0000","last_peered":"2024-09-10T20:41:15.931130+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:41:15.931130+0000","last_undegraded":"2024-09-10T20:41:15.931130+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":62,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20896,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558180+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:40:58.558180+0000","last_peered":"2024-09-10T20:40:58.558180+0000","last_clean":"2024-09-10T20:40:58.558180+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:40:58.558180+0000","last_undegraded":"2024-09-10T20:40:58.558180+0000","last_fullsized":"2024-09-10T20:40:58.558180+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":107,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.651455+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:41:16.651455+0000","last_peered":"2024-09-10T20:41:16.651455+0000","last_clean":"2024-09-10T20:41:16.651455+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:41:16.651455+0000","last_undegraded":"2024-09-10T20:41:16.651455+0000","last_fullsized":"2024-09-10T20:41:16.651455+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22140,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.651892+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:41:16.651892+0000","last_peered":"2024-09-10T20:41:16.651892+0000","last_clean":"2024-09-10T20:41:16.651892+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:41:16.651892+0000","last_undegraded":"2024-09-10T20:41:16.651892+0000","last_fullsized":"2024-09-10T20:41:16.651892+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":129,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707515+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:40:58.707515+0000","last_peered":"2024-09-10T20:40:58.707515+0000","last_clean":"2024-09-10T20:40:58.707515+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:40:58.707515+0000","last_undegraded":"2024-09-10T20:40:58.707515+0000","last_fullsized":"2024-09-10T20:40:58.707515+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28124,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558664+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:40:58.558664+0000","last_peered":"2024-09-10T20:40:58.558664+0000","last_clean":"2024-09-10T20:40:58.558664+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:40:58.558664+0000","last_undegraded":"2024-09-10T20:40:58.558664+0000","last_fullsized":"2024-09-10T20:40:58.558664+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558561+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:40:58.558561+0000","last_peered":"2024-09-10T20:40:58.558561+0000","last_clean":"2024-09-10T20:40:58.558561+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:40:58.558561+0000","last_undegraded":"2024-09-10T20:40:58.558561+0000","last_fullsized":"2024-09-10T20:40:58.558561+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28007,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707523+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:40:58.707523+0000","last_peered":"2024-09-10T20:40:58.707523+0000","last_clean":"2024-09-10T20:40:58.707523+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:40:58.707523+0000","last_undegraded":"2024-09-10T20:40:58.707523+0000","last_fullsized":"2024-09-10T20:40:58.707523+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":117,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.651796+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:41:16.651796+0000","last_peered":"2024-09-10T20:41:16.651796+0000","last_clean":"2024-09-10T20:41:16.651796+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:41:16.651796+0000","last_undegraded":"2024-09-10T20:41:16.651796+0000","last_fullsized":"2024-09-10T20:41:16.651796+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28061,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707491+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:40:58.707491+0000","last_peered":"2024-09-10T20:40:58.707491+0000","last_clean":"2024-09-10T20:40:58.707491+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:40:58.707491+0000","last_undegraded":"2024-09-10T20:40:58.707491+0000","last_fullsized":"2024-09-10T20:40:58.707491+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21119,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.653426+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:41:16.653426+0000","last_peered":"2024-09-10T20:41:16.653426+0000","last_clean":"2024-09-10T20:41:16.653426+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:41:16.653426+0000","last_undegraded":"2024-09-10T20:41:16.653426+0000","last_fullsized":"2024-09-10T20:41:16.653426+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":107,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.652077+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:41:16.652077+0000","last_peered":"2024-09-10T20:41:16.652077+0000","last_clean":"2024-09-10T20:41:16.652077+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:41:16.652077+0000","last_undegraded":"2024-09-10T20:41:16.652077+0000","last_fullsized":"2024-09-10T20:41:16.652077+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27841,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.653381+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:41:16.653381+0000","last_peered":"2024-09-10T20:41:16.653381+0000","last_clean":"2024-09-10T20:41:16.653381+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:41:16.653381+0000","last_undegraded":"2024-09-10T20:41:16.653381+0000","last_fullsized":"2024-09-10T20:41:16.653381+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":129,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707683+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:40:58.707683+0000","last_peered":"2024-09-10T20:40:58.707683+0000","last_clean":"2024-09-10T20:40:58.707683+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:40:58.707683+0000","last_undegraded":"2024-09-10T20:40:58.707683+0000","last_fullsized":"2024-09-10T20:40:58.707683+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21269,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:58.558883+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:40:58.558883+0000","last_peered":"2024-09-10T20:40:58.558883+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:40:58.558883+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":107,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.653164+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:41:16.653164+0000","last_peered":"2024-09-10T20:41:16.653164+0000","last_clean":"2024-09-10T20:41:16.653164+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:41:16.653164+0000","last_undegraded":"2024-09-10T20:41:16.653164+0000","last_fullsized":"2024-09-10T20:41:16.653164+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":112,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.653404+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:41:16.653404+0000","last_peered":"2024-09-10T20:41:16.653404+0000","last_clean":"2024-09-10T20:41:16.653404+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:41:16.653404+0000","last_undegraded":"2024-09-10T20:41:16.653404+0000","last_fullsized":"2024-09-10T20:41:16.653404+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27671,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.651990+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:41:16.651990+0000","last_peered":"2024-09-10T20:41:16.651990+0000","last_clean":"2024-09-10T20:41:16.651990+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:41:16.651990+0000","last_undegraded":"2024-09-10T20:41:16.651990+0000","last_fullsized":"2024-09-10T20:41:16.651990+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32310,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":148410368,"data_stored":148392303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182552,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517212,"kb_used_data":57988,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854628,"statfs":{"total":96636764160,"available":96107139072,"internally_reserved":0,"allocated":59379712,"data_stored":59058784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986324,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":453408,"kb_used_data":88528,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93918432,"statfs":{"total":96636764160,"available":96172474368,"internally_reserved":0,"allocated":90652672,"data_stored":90335821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659271,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505628,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866212,"statfs":{"total":96636764160,"available":96119001088,"internally_reserved":0,"allocated":532480,"data_stored":199334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":517205678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":89530368,"data_stored":89526755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:41:17.867 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:18.269 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:18.269 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:18.286 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":451,"stamp":"2024-09-10T20:41:17.658039+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32312,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1476248,"kb_used_data":147036,"kb_used_omap":12,"kb_used_meta":1329139,"kb_avail":281639272,"statfs":{"total":289910292480,"available":288398614528,"internally_reserved":0,"allocated":150564864,"data_stored":149593939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1361038338},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2109,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2109,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.583379"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":27996,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.617469+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:40:58.617469+0000","last_peered":"2024-09-10T20:40:58.617469+0000","last_clean":"2024-09-10T20:40:58.617469+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:40:58.617469+0000","last_undegraded":"2024-09-10T20:40:58.617469+0000","last_fullsized":"2024-09-10T20:40:58.617469+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21088,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707761+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:40:58.707761+0000","last_peered":"2024-09-10T20:40:58.707761+0000","last_clean":"2024-09-10T20:40:58.707761+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:40:58.707761+0000","last_undegraded":"2024-09-10T20:40:58.707761+0000","last_fullsized":"2024-09-10T20:40:58.707761+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28101,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.651801+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:41:16.651801+0000","last_peered":"2024-09-10T20:41:16.651801+0000","last_clean":"2024-09-10T20:41:16.651801+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:41:16.651801+0000","last_undegraded":"2024-09-10T20:41:16.651801+0000","last_fullsized":"2024-09-10T20:41:16.651801+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20720,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707714+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:40:58.707714+0000","last_peered":"2024-09-10T20:40:58.707714+0000","last_clean":"2024-09-10T20:40:58.707714+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:40:58.707714+0000","last_undegraded":"2024-09-10T20:40:58.707714+0000","last_fullsized":"2024-09-10T20:40:58.707714+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21231,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558744+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:40:58.558744+0000","last_peered":"2024-09-10T20:40:58.558744+0000","last_clean":"2024-09-10T20:40:58.558744+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:40:58.558744+0000","last_undegraded":"2024-09-10T20:40:58.558744+0000","last_fullsized":"2024-09-10T20:40:58.558744+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20749,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:58.558601+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:40:58.558601+0000","last_peered":"2024-09-10T20:40:58.558601+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:40:58.558601+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20902,"reported_epoch":62,"state":"peering","last_fresh":"2024-09-10T20:41:15.931130+0000","last_change":"2024-09-10T20:41:17.283132+0000","last_active":"2024-09-10T20:41:17.283132+0000","last_peered":"2024-09-10T20:41:15.931130+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:41:15.931130+0000","last_undegraded":"2024-09-10T20:41:15.931130+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":62,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20896,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558180+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:40:58.558180+0000","last_peered":"2024-09-10T20:40:58.558180+0000","last_clean":"2024-09-10T20:40:58.558180+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:40:58.558180+0000","last_undegraded":"2024-09-10T20:40:58.558180+0000","last_fullsized":"2024-09-10T20:40:58.558180+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":107,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.651455+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:41:16.651455+0000","last_peered":"2024-09-10T20:41:16.651455+0000","last_clean":"2024-09-10T20:41:16.651455+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:41:16.651455+0000","last_undegraded":"2024-09-10T20:41:16.651455+0000","last_fullsized":"2024-09-10T20:41:16.651455+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22140,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.651892+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:41:16.651892+0000","last_peered":"2024-09-10T20:41:16.651892+0000","last_clean":"2024-09-10T20:41:16.651892+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:41:16.651892+0000","last_undegraded":"2024-09-10T20:41:16.651892+0000","last_fullsized":"2024-09-10T20:41:16.651892+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":129,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707515+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:40:58.707515+0000","last_peered":"2024-09-10T20:40:58.707515+0000","last_clean":"2024-09-10T20:40:58.707515+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:40:58.707515+0000","last_undegraded":"2024-09-10T20:40:58.707515+0000","last_fullsized":"2024-09-10T20:40:58.707515+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28124,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558664+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:40:58.558664+0000","last_peered":"2024-09-10T20:40:58.558664+0000","last_clean":"2024-09-10T20:40:58.558664+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:40:58.558664+0000","last_undegraded":"2024-09-10T20:40:58.558664+0000","last_fullsized":"2024-09-10T20:40:58.558664+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558561+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:40:58.558561+0000","last_peered":"2024-09-10T20:40:58.558561+0000","last_clean":"2024-09-10T20:40:58.558561+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:40:58.558561+0000","last_undegraded":"2024-09-10T20:40:58.558561+0000","last_fullsized":"2024-09-10T20:40:58.558561+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28007,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707523+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:40:58.707523+0000","last_peered":"2024-09-10T20:40:58.707523+0000","last_clean":"2024-09-10T20:40:58.707523+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:40:58.707523+0000","last_undegraded":"2024-09-10T20:40:58.707523+0000","last_fullsized":"2024-09-10T20:40:58.707523+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":117,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.651796+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:41:16.651796+0000","last_peered":"2024-09-10T20:41:16.651796+0000","last_clean":"2024-09-10T20:41:16.651796+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:41:16.651796+0000","last_undegraded":"2024-09-10T20:41:16.651796+0000","last_fullsized":"2024-09-10T20:41:16.651796+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28061,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707491+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:40:58.707491+0000","last_peered":"2024-09-10T20:40:58.707491+0000","last_clean":"2024-09-10T20:40:58.707491+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:40:58.707491+0000","last_undegraded":"2024-09-10T20:40:58.707491+0000","last_fullsized":"2024-09-10T20:40:58.707491+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21119,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.653426+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:41:16.653426+0000","last_peered":"2024-09-10T20:41:16.653426+0000","last_clean":"2024-09-10T20:41:16.653426+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:41:16.653426+0000","last_undegraded":"2024-09-10T20:41:16.653426+0000","last_fullsized":"2024-09-10T20:41:16.653426+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":107,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.652077+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:41:16.652077+0000","last_peered":"2024-09-10T20:41:16.652077+0000","last_clean":"2024-09-10T20:41:16.652077+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:41:16.652077+0000","last_undegraded":"2024-09-10T20:41:16.652077+0000","last_fullsized":"2024-09-10T20:41:16.652077+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27841,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.653381+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:41:16.653381+0000","last_peered":"2024-09-10T20:41:16.653381+0000","last_clean":"2024-09-10T20:41:16.653381+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:41:16.653381+0000","last_undegraded":"2024-09-10T20:41:16.653381+0000","last_fullsized":"2024-09-10T20:41:16.653381+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":129,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707683+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:40:58.707683+0000","last_peered":"2024-09-10T20:40:58.707683+0000","last_clean":"2024-09-10T20:40:58.707683+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:40:58.707683+0000","last_undegraded":"2024-09-10T20:40:58.707683+0000","last_fullsized":"2024-09-10T20:40:58.707683+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21269,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:58.558883+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:40:58.558883+0000","last_peered":"2024-09-10T20:40:58.558883+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:40:58.558883+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":107,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.653164+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:41:16.653164+0000","last_peered":"2024-09-10T20:41:16.653164+0000","last_clean":"2024-09-10T20:41:16.653164+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:41:16.653164+0000","last_undegraded":"2024-09-10T20:41:16.653164+0000","last_fullsized":"2024-09-10T20:41:16.653164+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":112,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.653404+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:41:16.653404+0000","last_peered":"2024-09-10T20:41:16.653404+0000","last_clean":"2024-09-10T20:41:16.653404+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:41:16.653404+0000","last_undegraded":"2024-09-10T20:41:16.653404+0000","last_fullsized":"2024-09-10T20:41:16.653404+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27671,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.651990+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:41:16.651990+0000","last_peered":"2024-09-10T20:41:16.651990+0000","last_clean":"2024-09-10T20:41:16.651990+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:41:16.651990+0000","last_undegraded":"2024-09-10T20:41:16.651990+0000","last_fullsized":"2024-09-10T20:41:16.651990+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32310,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":148410368,"data_stored":148392303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182552,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517212,"kb_used_data":57988,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854628,"statfs":{"total":96636764160,"available":96107139072,"internally_reserved":0,"allocated":59379712,"data_stored":59058784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986324,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":453408,"kb_used_data":88528,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93918432,"statfs":{"total":96636764160,"available":96172474368,"internally_reserved":0,"allocated":90652672,"data_stored":90335821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659271,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505628,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866212,"statfs":{"total":96636764160,"available":96119001088,"internally_reserved":0,"allocated":532480,"data_stored":199334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":517205678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":89530368,"data_stored":89526755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:41:18.287 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:41:18.287 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20471", 'reported_seq': 20749, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:58.558601+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:40:58.558601+0000', 'last_peered': '2024-09-10T20:40:58.558601+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:40:58.558601+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17335, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3561}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:41:18.288 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:41:18.288 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'20608", 'reported_seq': 20902, 'reported_epoch': 62, 'state': 'peering', 'last_fresh': '2024-09-10T20:41:15.931130+0000', 'last_change': '2024-09-10T20:41:17.283132+0000', 'last_active': '2024-09-10T20:41:17.283132+0000', 'last_peered': '2024-09-10T20:41:15.931130+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:41:15.931130+0000', 'last_undegraded': '2024-09-10T20:41:15.931130+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 62, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17203, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3557, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T20:41:18.288 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:41:18.288 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "51'20986", 'reported_seq': 21269, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:40:58.558883+0000', 'last_change': '2024-09-10T20:30:05.789175+0000', 'last_active': '2024-09-10T20:40:58.558883+0000', 'last_peered': '2024-09-10T20:40:58.558883+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:40:58.558883+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3657, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17813, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3655}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:41:18.289 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:41:18.778 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:18.779 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 7409/6 objects degraded (123483.333%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":790,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1503154176,"bytes_avail":288407138304,"bytes_total":289910292480,"degraded_objects":7409,"degraded_total":6,"degraded_ratio":1234.8333333333333,"recovering_objects_per_sec":191,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1920,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:41:18.795 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:41:18.795 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:19.176 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:19.176 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:19.189 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":452,"stamp":"2024-09-10T20:41:19.076234+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32312,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1476248,"kb_used_data":147036,"kb_used_omap":12,"kb_used_meta":1329139,"kb_avail":281639272,"statfs":{"total":289910292480,"available":288398614528,"internally_reserved":0,"allocated":150564864,"data_stored":149593939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1361038338},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1290,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1290,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001255"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":27996,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.617469+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:40:58.617469+0000","last_peered":"2024-09-10T20:40:58.617469+0000","last_clean":"2024-09-10T20:40:58.617469+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:40:58.617469+0000","last_undegraded":"2024-09-10T20:40:58.617469+0000","last_fullsized":"2024-09-10T20:40:58.617469+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21088,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707761+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:40:58.707761+0000","last_peered":"2024-09-10T20:40:58.707761+0000","last_clean":"2024-09-10T20:40:58.707761+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:40:58.707761+0000","last_undegraded":"2024-09-10T20:40:58.707761+0000","last_fullsized":"2024-09-10T20:40:58.707761+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28101,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.651801+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:41:16.651801+0000","last_peered":"2024-09-10T20:41:16.651801+0000","last_clean":"2024-09-10T20:41:16.651801+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:41:16.651801+0000","last_undegraded":"2024-09-10T20:41:16.651801+0000","last_fullsized":"2024-09-10T20:41:16.651801+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20720,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707714+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:40:58.707714+0000","last_peered":"2024-09-10T20:40:58.707714+0000","last_clean":"2024-09-10T20:40:58.707714+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:40:58.707714+0000","last_undegraded":"2024-09-10T20:40:58.707714+0000","last_fullsized":"2024-09-10T20:40:58.707714+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21231,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558744+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:40:58.558744+0000","last_peered":"2024-09-10T20:40:58.558744+0000","last_clean":"2024-09-10T20:40:58.558744+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:40:58.558744+0000","last_undegraded":"2024-09-10T20:40:58.558744+0000","last_fullsized":"2024-09-10T20:40:58.558744+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20749,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:58.558601+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:40:58.558601+0000","last_peered":"2024-09-10T20:40:58.558601+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:40:58.558601+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20902,"reported_epoch":62,"state":"peering","last_fresh":"2024-09-10T20:41:15.931130+0000","last_change":"2024-09-10T20:41:17.283132+0000","last_active":"2024-09-10T20:41:17.283132+0000","last_peered":"2024-09-10T20:41:15.931130+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:41:15.931130+0000","last_undegraded":"2024-09-10T20:41:15.931130+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":62,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20896,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558180+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:40:58.558180+0000","last_peered":"2024-09-10T20:40:58.558180+0000","last_clean":"2024-09-10T20:40:58.558180+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:40:58.558180+0000","last_undegraded":"2024-09-10T20:40:58.558180+0000","last_fullsized":"2024-09-10T20:40:58.558180+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":107,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.651455+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:41:16.651455+0000","last_peered":"2024-09-10T20:41:16.651455+0000","last_clean":"2024-09-10T20:41:16.651455+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:41:16.651455+0000","last_undegraded":"2024-09-10T20:41:16.651455+0000","last_fullsized":"2024-09-10T20:41:16.651455+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22140,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.651892+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:41:16.651892+0000","last_peered":"2024-09-10T20:41:16.651892+0000","last_clean":"2024-09-10T20:41:16.651892+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:41:16.651892+0000","last_undegraded":"2024-09-10T20:41:16.651892+0000","last_fullsized":"2024-09-10T20:41:16.651892+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":129,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707515+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:40:58.707515+0000","last_peered":"2024-09-10T20:40:58.707515+0000","last_clean":"2024-09-10T20:40:58.707515+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:40:58.707515+0000","last_undegraded":"2024-09-10T20:40:58.707515+0000","last_fullsized":"2024-09-10T20:40:58.707515+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28124,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558664+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:40:58.558664+0000","last_peered":"2024-09-10T20:40:58.558664+0000","last_clean":"2024-09-10T20:40:58.558664+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:40:58.558664+0000","last_undegraded":"2024-09-10T20:40:58.558664+0000","last_fullsized":"2024-09-10T20:40:58.558664+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.558561+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:40:58.558561+0000","last_peered":"2024-09-10T20:40:58.558561+0000","last_clean":"2024-09-10T20:40:58.558561+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:40:58.558561+0000","last_undegraded":"2024-09-10T20:40:58.558561+0000","last_fullsized":"2024-09-10T20:40:58.558561+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28007,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707523+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:40:58.707523+0000","last_peered":"2024-09-10T20:40:58.707523+0000","last_clean":"2024-09-10T20:40:58.707523+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:40:58.707523+0000","last_undegraded":"2024-09-10T20:40:58.707523+0000","last_fullsized":"2024-09-10T20:40:58.707523+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":117,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.651796+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:41:16.651796+0000","last_peered":"2024-09-10T20:41:16.651796+0000","last_clean":"2024-09-10T20:41:16.651796+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:41:16.651796+0000","last_undegraded":"2024-09-10T20:41:16.651796+0000","last_fullsized":"2024-09-10T20:41:16.651796+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28061,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707491+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:40:58.707491+0000","last_peered":"2024-09-10T20:40:58.707491+0000","last_clean":"2024-09-10T20:40:58.707491+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:40:58.707491+0000","last_undegraded":"2024-09-10T20:40:58.707491+0000","last_fullsized":"2024-09-10T20:40:58.707491+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21119,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.653426+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:41:16.653426+0000","last_peered":"2024-09-10T20:41:16.653426+0000","last_clean":"2024-09-10T20:41:16.653426+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:41:16.653426+0000","last_undegraded":"2024-09-10T20:41:16.653426+0000","last_fullsized":"2024-09-10T20:41:16.653426+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":107,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.652077+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:41:16.652077+0000","last_peered":"2024-09-10T20:41:16.652077+0000","last_clean":"2024-09-10T20:41:16.652077+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:41:16.652077+0000","last_undegraded":"2024-09-10T20:41:16.652077+0000","last_fullsized":"2024-09-10T20:41:16.652077+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27841,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.653381+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:41:16.653381+0000","last_peered":"2024-09-10T20:41:16.653381+0000","last_clean":"2024-09-10T20:41:16.653381+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:41:16.653381+0000","last_undegraded":"2024-09-10T20:41:16.653381+0000","last_fullsized":"2024-09-10T20:41:16.653381+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":129,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T20:40:58.707683+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:40:58.707683+0000","last_peered":"2024-09-10T20:40:58.707683+0000","last_clean":"2024-09-10T20:40:58.707683+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:40:58.707683+0000","last_undegraded":"2024-09-10T20:40:58.707683+0000","last_fullsized":"2024-09-10T20:40:58.707683+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21269,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:40:58.558883+0000","last_change":"2024-09-10T20:30:05.789175+0000","last_active":"2024-09-10T20:40:58.558883+0000","last_peered":"2024-09-10T20:40:58.558883+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:40:58.558883+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":107,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.653164+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:41:16.653164+0000","last_peered":"2024-09-10T20:41:16.653164+0000","last_clean":"2024-09-10T20:41:16.653164+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:41:16.653164+0000","last_undegraded":"2024-09-10T20:41:16.653164+0000","last_fullsized":"2024-09-10T20:41:16.653164+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":112,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.653404+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:41:16.653404+0000","last_peered":"2024-09-10T20:41:16.653404+0000","last_clean":"2024-09-10T20:41:16.653404+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:41:16.653404+0000","last_undegraded":"2024-09-10T20:41:16.653404+0000","last_fullsized":"2024-09-10T20:41:16.653404+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27671,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.651990+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:41:16.651990+0000","last_peered":"2024-09-10T20:41:16.651990+0000","last_clean":"2024-09-10T20:41:16.651990+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:41:16.651990+0000","last_undegraded":"2024-09-10T20:41:16.651990+0000","last_fullsized":"2024-09-10T20:41:16.651990+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32310,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":148410368,"data_stored":148392303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182552,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517212,"kb_used_data":57988,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854628,"statfs":{"total":96636764160,"available":96107139072,"internally_reserved":0,"allocated":59379712,"data_stored":59058784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986324,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":453408,"kb_used_data":88528,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93918432,"statfs":{"total":96636764160,"available":96172474368,"internally_reserved":0,"allocated":90652672,"data_stored":90335821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659271,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505628,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866212,"statfs":{"total":96636764160,"available":96119001088,"internally_reserved":0,"allocated":532480,"data_stored":199334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":517205678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":89530368,"data_stored":89526755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:41:22.191 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:22.576 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:22.576 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:22.589 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":453,"stamp":"2024-09-10T20:41:21.076517+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6480,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33052,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1473052,"kb_used_data":143832,"kb_used_omap":12,"kb_used_meta":1329139,"kb_avail":281642468,"statfs":{"total":289910292480,"available":288401887232,"internally_reserved":0,"allocated":147283968,"data_stored":146293033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1361038338},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2030,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2030,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001214"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":28000,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663983+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:41:17.663983+0000","last_peered":"2024-09-10T20:41:17.663983+0000","last_clean":"2024-09-10T20:41:17.663983+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:41:17.663983+0000","last_undegraded":"2024-09-10T20:41:17.663983+0000","last_fullsized":"2024-09-10T20:41:17.663983+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21092,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663678+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:41:17.663678+0000","last_peered":"2024-09-10T20:41:17.663678+0000","last_clean":"2024-09-10T20:41:17.663678+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:41:17.663678+0000","last_undegraded":"2024-09-10T20:41:17.663678+0000","last_fullsized":"2024-09-10T20:41:17.663678+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28101,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.651801+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:41:16.651801+0000","last_peered":"2024-09-10T20:41:16.651801+0000","last_clean":"2024-09-10T20:41:16.651801+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:41:16.651801+0000","last_undegraded":"2024-09-10T20:41:16.651801+0000","last_fullsized":"2024-09-10T20:41:16.651801+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20724,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.664035+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:41:17.664035+0000","last_peered":"2024-09-10T20:41:17.664035+0000","last_clean":"2024-09-10T20:41:17.664035+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:41:17.664035+0000","last_undegraded":"2024-09-10T20:41:17.664035+0000","last_fullsized":"2024-09-10T20:41:17.664035+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21235,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663037+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:41:17.663037+0000","last_peered":"2024-09-10T20:41:17.663037+0000","last_clean":"2024-09-10T20:41:17.663037+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:41:17.663037+0000","last_undegraded":"2024-09-10T20:41:17.663037+0000","last_fullsized":"2024-09-10T20:41:17.663037+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20753,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:17.662327+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:41:17.662327+0000","last_peered":"2024-09-10T20:41:17.662327+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:41:17.662327+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20902,"reported_epoch":62,"state":"peering","last_fresh":"2024-09-10T20:41:15.931130+0000","last_change":"2024-09-10T20:41:17.283132+0000","last_active":"2024-09-10T20:41:17.283132+0000","last_peered":"2024-09-10T20:41:15.931130+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:41:15.931130+0000","last_undegraded":"2024-09-10T20:41:15.931130+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":62,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20900,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662099+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:41:17.662099+0000","last_peered":"2024-09-10T20:41:17.662099+0000","last_clean":"2024-09-10T20:41:17.662099+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:41:17.662099+0000","last_undegraded":"2024-09-10T20:41:17.662099+0000","last_fullsized":"2024-09-10T20:41:17.662099+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":107,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.651455+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:41:16.651455+0000","last_peered":"2024-09-10T20:41:16.651455+0000","last_clean":"2024-09-10T20:41:16.651455+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:41:16.651455+0000","last_undegraded":"2024-09-10T20:41:16.651455+0000","last_fullsized":"2024-09-10T20:41:16.651455+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22140,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.651892+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:41:16.651892+0000","last_peered":"2024-09-10T20:41:16.651892+0000","last_clean":"2024-09-10T20:41:16.651892+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:41:16.651892+0000","last_undegraded":"2024-09-10T20:41:16.651892+0000","last_fullsized":"2024-09-10T20:41:16.651892+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":133,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663417+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:41:17.663417+0000","last_peered":"2024-09-10T20:41:17.663417+0000","last_clean":"2024-09-10T20:41:17.663417+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:41:17.663417+0000","last_undegraded":"2024-09-10T20:41:17.663417+0000","last_fullsized":"2024-09-10T20:41:17.663417+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28128,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662796+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:41:17.662796+0000","last_peered":"2024-09-10T20:41:17.662796+0000","last_clean":"2024-09-10T20:41:17.662796+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:41:17.662796+0000","last_undegraded":"2024-09-10T20:41:17.662796+0000","last_fullsized":"2024-09-10T20:41:17.662796+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":140,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662225+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:41:17.662225+0000","last_peered":"2024-09-10T20:41:17.662225+0000","last_clean":"2024-09-10T20:41:17.662225+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:41:17.662225+0000","last_undegraded":"2024-09-10T20:41:17.662225+0000","last_fullsized":"2024-09-10T20:41:17.662225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28011,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663503+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:41:17.663503+0000","last_peered":"2024-09-10T20:41:17.663503+0000","last_clean":"2024-09-10T20:41:17.663503+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:41:17.663503+0000","last_undegraded":"2024-09-10T20:41:17.663503+0000","last_fullsized":"2024-09-10T20:41:17.663503+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":117,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.651796+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:41:16.651796+0000","last_peered":"2024-09-10T20:41:16.651796+0000","last_clean":"2024-09-10T20:41:16.651796+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:41:16.651796+0000","last_undegraded":"2024-09-10T20:41:16.651796+0000","last_fullsized":"2024-09-10T20:41:16.651796+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28065,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662536+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:41:17.662536+0000","last_peered":"2024-09-10T20:41:17.662536+0000","last_clean":"2024-09-10T20:41:17.662536+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:41:17.662536+0000","last_undegraded":"2024-09-10T20:41:17.662536+0000","last_fullsized":"2024-09-10T20:41:17.662536+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21119,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.653426+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:41:16.653426+0000","last_peered":"2024-09-10T20:41:16.653426+0000","last_clean":"2024-09-10T20:41:16.653426+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:41:16.653426+0000","last_undegraded":"2024-09-10T20:41:16.653426+0000","last_fullsized":"2024-09-10T20:41:16.653426+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":107,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.652077+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:41:16.652077+0000","last_peered":"2024-09-10T20:41:16.652077+0000","last_clean":"2024-09-10T20:41:16.652077+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:41:16.652077+0000","last_undegraded":"2024-09-10T20:41:16.652077+0000","last_fullsized":"2024-09-10T20:41:16.652077+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27841,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.653381+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:41:16.653381+0000","last_peered":"2024-09-10T20:41:16.653381+0000","last_clean":"2024-09-10T20:41:16.653381+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:41:16.653381+0000","last_undegraded":"2024-09-10T20:41:16.653381+0000","last_fullsized":"2024-09-10T20:41:16.653381+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":133,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663305+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:41:17.663305+0000","last_peered":"2024-09-10T20:41:17.663305+0000","last_clean":"2024-09-10T20:41:17.663305+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:41:17.663305+0000","last_undegraded":"2024-09-10T20:41:17.663305+0000","last_fullsized":"2024-09-10T20:41:17.663305+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":22754,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:20.244611+0000","last_change":"2024-09-10T20:41:15.931451+0000","last_active":"2024-09-10T20:41:20.244611+0000","last_peered":"2024-09-10T20:41:20.244611+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:41:20.244611+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2917,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":798,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":107,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.653164+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:41:16.653164+0000","last_peered":"2024-09-10T20:41:16.653164+0000","last_clean":"2024-09-10T20:41:16.653164+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:41:16.653164+0000","last_undegraded":"2024-09-10T20:41:16.653164+0000","last_fullsized":"2024-09-10T20:41:16.653164+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":112,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.653404+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:41:16.653404+0000","last_peered":"2024-09-10T20:41:16.653404+0000","last_clean":"2024-09-10T20:41:16.653404+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:41:16.653404+0000","last_undegraded":"2024-09-10T20:41:16.653404+0000","last_fullsized":"2024-09-10T20:41:16.653404+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27671,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.651990+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:41:16.651990+0000","last_peered":"2024-09-10T20:41:16.651990+0000","last_clean":"2024-09-10T20:41:16.651990+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:41:16.651990+0000","last_undegraded":"2024-09-10T20:41:16.651990+0000","last_fullsized":"2024-09-10T20:41:16.651990+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6480,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33050,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":144973824,"data_stored":144955759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182553,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517228,"kb_used_data":58004,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854612,"statfs":{"total":96636764160,"available":96107122688,"internally_reserved":0,"allocated":59396096,"data_stored":59065163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986325,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":450196,"kb_used_data":85308,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93921644,"statfs":{"total":96636764160,"available":96175763456,"internally_reserved":0,"allocated":87355392,"data_stored":87028536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659271,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505628,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866212,"statfs":{"total":96636764160,"available":96119001088,"internally_reserved":0,"allocated":532480,"data_stored":199334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":517205678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86093824,"data_stored":86090211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:41:22.590 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:22.928 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:22.929 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:22.942 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":453,"stamp":"2024-09-10T20:41:21.076517+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6480,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33052,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1473052,"kb_used_data":143832,"kb_used_omap":12,"kb_used_meta":1329139,"kb_avail":281642468,"statfs":{"total":289910292480,"available":288401887232,"internally_reserved":0,"allocated":147283968,"data_stored":146293033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1361038338},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2030,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2030,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001214"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":28000,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663983+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:41:17.663983+0000","last_peered":"2024-09-10T20:41:17.663983+0000","last_clean":"2024-09-10T20:41:17.663983+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:41:17.663983+0000","last_undegraded":"2024-09-10T20:41:17.663983+0000","last_fullsized":"2024-09-10T20:41:17.663983+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21092,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663678+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:41:17.663678+0000","last_peered":"2024-09-10T20:41:17.663678+0000","last_clean":"2024-09-10T20:41:17.663678+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:41:17.663678+0000","last_undegraded":"2024-09-10T20:41:17.663678+0000","last_fullsized":"2024-09-10T20:41:17.663678+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28101,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.651801+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:41:16.651801+0000","last_peered":"2024-09-10T20:41:16.651801+0000","last_clean":"2024-09-10T20:41:16.651801+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:41:16.651801+0000","last_undegraded":"2024-09-10T20:41:16.651801+0000","last_fullsized":"2024-09-10T20:41:16.651801+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20724,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.664035+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:41:17.664035+0000","last_peered":"2024-09-10T20:41:17.664035+0000","last_clean":"2024-09-10T20:41:17.664035+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:41:17.664035+0000","last_undegraded":"2024-09-10T20:41:17.664035+0000","last_fullsized":"2024-09-10T20:41:17.664035+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21235,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663037+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:41:17.663037+0000","last_peered":"2024-09-10T20:41:17.663037+0000","last_clean":"2024-09-10T20:41:17.663037+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:41:17.663037+0000","last_undegraded":"2024-09-10T20:41:17.663037+0000","last_fullsized":"2024-09-10T20:41:17.663037+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20753,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:17.662327+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:41:17.662327+0000","last_peered":"2024-09-10T20:41:17.662327+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:41:17.662327+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20902,"reported_epoch":62,"state":"peering","last_fresh":"2024-09-10T20:41:15.931130+0000","last_change":"2024-09-10T20:41:17.283132+0000","last_active":"2024-09-10T20:41:17.283132+0000","last_peered":"2024-09-10T20:41:15.931130+0000","last_clean":"2024-09-10T20:30:03.280382+0000","last_became_active":"2024-09-10T20:30:05.781208+0000","last_became_peered":"2024-09-10T20:30:05.781208+0000","last_unstale":"2024-09-10T20:41:15.931130+0000","last_undegraded":"2024-09-10T20:41:15.931130+0000","last_fullsized":"2024-09-10T20:30:05.305831+0000","mapping_epoch":62,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20900,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662099+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:41:17.662099+0000","last_peered":"2024-09-10T20:41:17.662099+0000","last_clean":"2024-09-10T20:41:17.662099+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:41:17.662099+0000","last_undegraded":"2024-09-10T20:41:17.662099+0000","last_fullsized":"2024-09-10T20:41:17.662099+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":107,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.651455+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:41:16.651455+0000","last_peered":"2024-09-10T20:41:16.651455+0000","last_clean":"2024-09-10T20:41:16.651455+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:41:16.651455+0000","last_undegraded":"2024-09-10T20:41:16.651455+0000","last_fullsized":"2024-09-10T20:41:16.651455+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22140,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.651892+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:41:16.651892+0000","last_peered":"2024-09-10T20:41:16.651892+0000","last_clean":"2024-09-10T20:41:16.651892+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:41:16.651892+0000","last_undegraded":"2024-09-10T20:41:16.651892+0000","last_fullsized":"2024-09-10T20:41:16.651892+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":133,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663417+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:41:17.663417+0000","last_peered":"2024-09-10T20:41:17.663417+0000","last_clean":"2024-09-10T20:41:17.663417+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:41:17.663417+0000","last_undegraded":"2024-09-10T20:41:17.663417+0000","last_fullsized":"2024-09-10T20:41:17.663417+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28128,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662796+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:41:17.662796+0000","last_peered":"2024-09-10T20:41:17.662796+0000","last_clean":"2024-09-10T20:41:17.662796+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:41:17.662796+0000","last_undegraded":"2024-09-10T20:41:17.662796+0000","last_fullsized":"2024-09-10T20:41:17.662796+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":140,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662225+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:41:17.662225+0000","last_peered":"2024-09-10T20:41:17.662225+0000","last_clean":"2024-09-10T20:41:17.662225+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:41:17.662225+0000","last_undegraded":"2024-09-10T20:41:17.662225+0000","last_fullsized":"2024-09-10T20:41:17.662225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28011,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663503+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:41:17.663503+0000","last_peered":"2024-09-10T20:41:17.663503+0000","last_clean":"2024-09-10T20:41:17.663503+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:41:17.663503+0000","last_undegraded":"2024-09-10T20:41:17.663503+0000","last_fullsized":"2024-09-10T20:41:17.663503+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":117,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.651796+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:41:16.651796+0000","last_peered":"2024-09-10T20:41:16.651796+0000","last_clean":"2024-09-10T20:41:16.651796+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:41:16.651796+0000","last_undegraded":"2024-09-10T20:41:16.651796+0000","last_fullsized":"2024-09-10T20:41:16.651796+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28065,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662536+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:41:17.662536+0000","last_peered":"2024-09-10T20:41:17.662536+0000","last_clean":"2024-09-10T20:41:17.662536+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:41:17.662536+0000","last_undegraded":"2024-09-10T20:41:17.662536+0000","last_fullsized":"2024-09-10T20:41:17.662536+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21119,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.653426+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:41:16.653426+0000","last_peered":"2024-09-10T20:41:16.653426+0000","last_clean":"2024-09-10T20:41:16.653426+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:41:16.653426+0000","last_undegraded":"2024-09-10T20:41:16.653426+0000","last_fullsized":"2024-09-10T20:41:16.653426+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":107,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.652077+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:41:16.652077+0000","last_peered":"2024-09-10T20:41:16.652077+0000","last_clean":"2024-09-10T20:41:16.652077+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:41:16.652077+0000","last_undegraded":"2024-09-10T20:41:16.652077+0000","last_fullsized":"2024-09-10T20:41:16.652077+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27841,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.653381+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:41:16.653381+0000","last_peered":"2024-09-10T20:41:16.653381+0000","last_clean":"2024-09-10T20:41:16.653381+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:41:16.653381+0000","last_undegraded":"2024-09-10T20:41:16.653381+0000","last_fullsized":"2024-09-10T20:41:16.653381+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":133,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663305+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:41:17.663305+0000","last_peered":"2024-09-10T20:41:17.663305+0000","last_clean":"2024-09-10T20:41:17.663305+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:41:17.663305+0000","last_undegraded":"2024-09-10T20:41:17.663305+0000","last_fullsized":"2024-09-10T20:41:17.663305+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":22754,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:20.244611+0000","last_change":"2024-09-10T20:41:15.931451+0000","last_active":"2024-09-10T20:41:20.244611+0000","last_peered":"2024-09-10T20:41:20.244611+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:41:20.244611+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2917,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":798,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":107,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.653164+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:41:16.653164+0000","last_peered":"2024-09-10T20:41:16.653164+0000","last_clean":"2024-09-10T20:41:16.653164+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:41:16.653164+0000","last_undegraded":"2024-09-10T20:41:16.653164+0000","last_fullsized":"2024-09-10T20:41:16.653164+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":112,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.653404+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:41:16.653404+0000","last_peered":"2024-09-10T20:41:16.653404+0000","last_clean":"2024-09-10T20:41:16.653404+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:41:16.653404+0000","last_undegraded":"2024-09-10T20:41:16.653404+0000","last_fullsized":"2024-09-10T20:41:16.653404+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27671,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T20:41:16.651990+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:41:16.651990+0000","last_peered":"2024-09-10T20:41:16.651990+0000","last_clean":"2024-09-10T20:41:16.651990+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:41:16.651990+0000","last_undegraded":"2024-09-10T20:41:16.651990+0000","last_fullsized":"2024-09-10T20:41:16.651990+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6480,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33050,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":144973824,"data_stored":144955759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182553,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517228,"kb_used_data":58004,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854612,"statfs":{"total":96636764160,"available":96107122688,"internally_reserved":0,"allocated":59396096,"data_stored":59065163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986325,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":450196,"kb_used_data":85308,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93921644,"statfs":{"total":96636764160,"available":96175763456,"internally_reserved":0,"allocated":87355392,"data_stored":87028536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659271,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505628,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866212,"statfs":{"total":96636764160,"available":96119001088,"internally_reserved":0,"allocated":532480,"data_stored":199334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":517205678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86093824,"data_stored":86090211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:41:22.943 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:41:22.944 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20471", 'reported_seq': 20753, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:41:17.662327+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:41:17.662327+0000', 'last_peered': '2024-09-10T20:41:17.662327+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:41:17.662327+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17335, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3561}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:41:22.944 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T20:41:22.944 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'20608", 'reported_seq': 20902, 'reported_epoch': 62, 'state': 'peering', 'last_fresh': '2024-09-10T20:41:15.931130+0000', 'last_change': '2024-09-10T20:41:17.283132+0000', 'last_active': '2024-09-10T20:41:17.283132+0000', 'last_peered': '2024-09-10T20:41:15.931130+0000', 'last_clean': '2024-09-10T20:30:03.280382+0000', 'last_became_active': '2024-09-10T20:30:05.781208+0000', 'last_became_peered': '2024-09-10T20:30:05.781208+0000', 'last_unstale': '2024-09-10T20:41:15.931130+0000', 'last_undegraded': '2024-09-10T20:41:15.931130+0000', 'last_fullsized': '2024-09-10T20:30:05.305831+0000', 'mapping_epoch': 62, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3448", 'last_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:30:01.859333+0000', 'objects_scrubbed': 3399, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17203, 'num_write_kb': 27608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3557, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T20:41:22.944 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:41:22.944 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "51'20986", 'reported_seq': 22754, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:41:20.244611+0000', 'last_change': '2024-09-10T20:41:15.931451+0000', 'last_active': '2024-09-10T20:41:20.244611+0000', 'last_peered': '2024-09-10T20:41:20.244611+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:41:20.244611+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2917, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17813, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 798, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:41:22.944 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:41:23.350 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:23.350 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 6480/6 objects degraded (108000.000%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":794,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"peering","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1508405248,"bytes_avail":288401887232,"bytes_total":289910292480,"inactive_pgs_ratio":0.041666667908430099,"degraded_objects":6480,"degraded_total":6,"degraded_ratio":1080,"recovering_objects_per_sec":253,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2030,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:41:23.364 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:41:23.364 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:23.668 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:23.669 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:23.683 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":454,"stamp":"2024-09-10T20:41:23.076852+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6480,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33052,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1464736,"kb_used_data":143840,"kb_used_omap":12,"kb_used_meta":1320819,"kb_avail":281650784,"statfs":{"total":289910292480,"available":288410402816,"internally_reserved":0,"allocated":147292160,"data_stored":146296212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1352518658},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2030,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2030,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001221"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":28000,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663983+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:41:17.663983+0000","last_peered":"2024-09-10T20:41:17.663983+0000","last_clean":"2024-09-10T20:41:17.663983+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:41:17.663983+0000","last_undegraded":"2024-09-10T20:41:17.663983+0000","last_fullsized":"2024-09-10T20:41:17.663983+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21092,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663678+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:41:17.663678+0000","last_peered":"2024-09-10T20:41:17.663678+0000","last_clean":"2024-09-10T20:41:17.663678+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:41:17.663678+0000","last_undegraded":"2024-09-10T20:41:17.663678+0000","last_fullsized":"2024-09-10T20:41:17.663678+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28104,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665777+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:41:17.665777+0000","last_peered":"2024-09-10T20:41:17.665777+0000","last_clean":"2024-09-10T20:41:17.665777+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:41:17.665777+0000","last_undegraded":"2024-09-10T20:41:17.665777+0000","last_fullsized":"2024-09-10T20:41:17.665777+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20724,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.664035+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:41:17.664035+0000","last_peered":"2024-09-10T20:41:17.664035+0000","last_clean":"2024-09-10T20:41:17.664035+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:41:17.664035+0000","last_undegraded":"2024-09-10T20:41:17.664035+0000","last_fullsized":"2024-09-10T20:41:17.664035+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21235,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663037+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:41:17.663037+0000","last_peered":"2024-09-10T20:41:17.663037+0000","last_clean":"2024-09-10T20:41:17.663037+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:41:17.663037+0000","last_undegraded":"2024-09-10T20:41:17.663037+0000","last_fullsized":"2024-09-10T20:41:17.663037+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20753,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:17.662327+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:41:17.662327+0000","last_peered":"2024-09-10T20:41:17.662327+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:41:17.662327+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20911,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.750111+0000","last_change":"2024-09-10T20:41:17.750026+0000","last_active":"2024-09-10T20:41:17.750111+0000","last_peered":"2024-09-10T20:41:17.750111+0000","last_clean":"2024-09-10T20:41:17.750111+0000","last_became_active":"2024-09-10T20:41:17.668013+0000","last_became_peered":"2024-09-10T20:41:17.668013+0000","last_unstale":"2024-09-10T20:41:17.750111+0000","last_undegraded":"2024-09-10T20:41:17.750111+0000","last_fullsized":"2024-09-10T20:41:17.750111+0000","mapping_epoch":62,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:55.885282+0000","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20900,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662099+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:41:17.662099+0000","last_peered":"2024-09-10T20:41:17.662099+0000","last_clean":"2024-09-10T20:41:17.662099+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:41:17.662099+0000","last_undegraded":"2024-09-10T20:41:17.662099+0000","last_fullsized":"2024-09-10T20:41:17.662099+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665321+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:41:17.665321+0000","last_peered":"2024-09-10T20:41:17.665321+0000","last_clean":"2024-09-10T20:41:17.665321+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:41:17.665321+0000","last_undegraded":"2024-09-10T20:41:17.665321+0000","last_fullsized":"2024-09-10T20:41:17.665321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22142,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665632+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:41:17.665632+0000","last_peered":"2024-09-10T20:41:17.665632+0000","last_clean":"2024-09-10T20:41:17.665632+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:41:17.665632+0000","last_undegraded":"2024-09-10T20:41:17.665632+0000","last_fullsized":"2024-09-10T20:41:17.665632+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":133,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663417+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:41:17.663417+0000","last_peered":"2024-09-10T20:41:17.663417+0000","last_clean":"2024-09-10T20:41:17.663417+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:41:17.663417+0000","last_undegraded":"2024-09-10T20:41:17.663417+0000","last_fullsized":"2024-09-10T20:41:17.663417+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28128,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662796+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:41:17.662796+0000","last_peered":"2024-09-10T20:41:17.662796+0000","last_clean":"2024-09-10T20:41:17.662796+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:41:17.662796+0000","last_undegraded":"2024-09-10T20:41:17.662796+0000","last_fullsized":"2024-09-10T20:41:17.662796+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":140,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662225+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:41:17.662225+0000","last_peered":"2024-09-10T20:41:17.662225+0000","last_clean":"2024-09-10T20:41:17.662225+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:41:17.662225+0000","last_undegraded":"2024-09-10T20:41:17.662225+0000","last_fullsized":"2024-09-10T20:41:17.662225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28011,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663503+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:41:17.663503+0000","last_peered":"2024-09-10T20:41:17.663503+0000","last_clean":"2024-09-10T20:41:17.663503+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:41:17.663503+0000","last_undegraded":"2024-09-10T20:41:17.663503+0000","last_fullsized":"2024-09-10T20:41:17.663503+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665485+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:41:17.665485+0000","last_peered":"2024-09-10T20:41:17.665485+0000","last_clean":"2024-09-10T20:41:17.665485+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:41:17.665485+0000","last_undegraded":"2024-09-10T20:41:17.665485+0000","last_fullsized":"2024-09-10T20:41:17.665485+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28065,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662536+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:41:17.662536+0000","last_peered":"2024-09-10T20:41:17.662536+0000","last_clean":"2024-09-10T20:41:17.662536+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:41:17.662536+0000","last_undegraded":"2024-09-10T20:41:17.662536+0000","last_fullsized":"2024-09-10T20:41:17.662536+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665900+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:41:17.665900+0000","last_peered":"2024-09-10T20:41:17.665900+0000","last_clean":"2024-09-10T20:41:17.665900+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:41:17.665900+0000","last_undegraded":"2024-09-10T20:41:17.665900+0000","last_fullsized":"2024-09-10T20:41:17.665900+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665840+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:41:17.665840+0000","last_peered":"2024-09-10T20:41:17.665840+0000","last_clean":"2024-09-10T20:41:17.665840+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:41:17.665840+0000","last_undegraded":"2024-09-10T20:41:17.665840+0000","last_fullsized":"2024-09-10T20:41:17.665840+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27843,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665757+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:41:17.665757+0000","last_peered":"2024-09-10T20:41:17.665757+0000","last_clean":"2024-09-10T20:41:17.665757+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:41:17.665757+0000","last_undegraded":"2024-09-10T20:41:17.665757+0000","last_fullsized":"2024-09-10T20:41:17.665757+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":133,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663305+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:41:17.663305+0000","last_peered":"2024-09-10T20:41:17.663305+0000","last_clean":"2024-09-10T20:41:17.663305+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:41:17.663305+0000","last_undegraded":"2024-09-10T20:41:17.663305+0000","last_fullsized":"2024-09-10T20:41:17.663305+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":22754,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:20.244611+0000","last_change":"2024-09-10T20:41:15.931451+0000","last_active":"2024-09-10T20:41:20.244611+0000","last_peered":"2024-09-10T20:41:20.244611+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:41:20.244611+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2917,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":798,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665623+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:41:17.665623+0000","last_peered":"2024-09-10T20:41:17.665623+0000","last_clean":"2024-09-10T20:41:17.665623+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:41:17.665623+0000","last_undegraded":"2024-09-10T20:41:17.665623+0000","last_fullsized":"2024-09-10T20:41:17.665623+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665817+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:41:17.665817+0000","last_peered":"2024-09-10T20:41:17.665817+0000","last_clean":"2024-09-10T20:41:17.665817+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:41:17.665817+0000","last_undegraded":"2024-09-10T20:41:17.665817+0000","last_fullsized":"2024-09-10T20:41:17.665817+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27673,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665728+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:41:17.665728+0000","last_peered":"2024-09-10T20:41:17.665728+0000","last_clean":"2024-09-10T20:41:17.665728+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:41:17.665728+0000","last_undegraded":"2024-09-10T20:41:17.665728+0000","last_fullsized":"2024-09-10T20:41:17.665728+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6480,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33050,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":144973824,"data_stored":144955759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182553,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517228,"kb_used_data":58004,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854612,"statfs":{"total":96636764160,"available":96107122688,"internally_reserved":0,"allocated":59396096,"data_stored":59065163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986325,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":450196,"kb_used_data":85308,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93921644,"statfs":{"total":96636764160,"available":96175763456,"internally_reserved":0,"allocated":87355392,"data_stored":87028536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659272,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497312,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874528,"statfs":{"total":96636764160,"available":96127516672,"internally_reserved":0,"allocated":540672,"data_stored":202513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86093824,"data_stored":86090211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:41:26.685 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:26.999 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:26.999 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:27.012 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":455,"stamp":"2024-09-10T20:41:25.077252+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6480,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33052,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1461128,"kb_used_data":140240,"kb_used_omap":12,"kb_used_meta":1320819,"kb_avail":281654392,"statfs":{"total":289910292480,"available":288414097408,"internally_reserved":0,"allocated":143605760,"data_stored":142609812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1352518658},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-929,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":929,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.427035"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":28000,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663983+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:41:17.663983+0000","last_peered":"2024-09-10T20:41:17.663983+0000","last_clean":"2024-09-10T20:41:17.663983+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:41:17.663983+0000","last_undegraded":"2024-09-10T20:41:17.663983+0000","last_fullsized":"2024-09-10T20:41:17.663983+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21092,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663678+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:41:17.663678+0000","last_peered":"2024-09-10T20:41:17.663678+0000","last_clean":"2024-09-10T20:41:17.663678+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:41:17.663678+0000","last_undegraded":"2024-09-10T20:41:17.663678+0000","last_fullsized":"2024-09-10T20:41:17.663678+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28104,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665777+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:41:17.665777+0000","last_peered":"2024-09-10T20:41:17.665777+0000","last_clean":"2024-09-10T20:41:17.665777+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:41:17.665777+0000","last_undegraded":"2024-09-10T20:41:17.665777+0000","last_fullsized":"2024-09-10T20:41:17.665777+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20724,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.664035+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:41:17.664035+0000","last_peered":"2024-09-10T20:41:17.664035+0000","last_clean":"2024-09-10T20:41:17.664035+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:41:17.664035+0000","last_undegraded":"2024-09-10T20:41:17.664035+0000","last_fullsized":"2024-09-10T20:41:17.664035+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21235,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663037+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:41:17.663037+0000","last_peered":"2024-09-10T20:41:17.663037+0000","last_clean":"2024-09-10T20:41:17.663037+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:41:17.663037+0000","last_undegraded":"2024-09-10T20:41:17.663037+0000","last_fullsized":"2024-09-10T20:41:17.663037+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20753,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:17.662327+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:41:17.662327+0000","last_peered":"2024-09-10T20:41:17.662327+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:41:17.662327+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20911,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.750111+0000","last_change":"2024-09-10T20:41:17.750026+0000","last_active":"2024-09-10T20:41:17.750111+0000","last_peered":"2024-09-10T20:41:17.750111+0000","last_clean":"2024-09-10T20:41:17.750111+0000","last_became_active":"2024-09-10T20:41:17.668013+0000","last_became_peered":"2024-09-10T20:41:17.668013+0000","last_unstale":"2024-09-10T20:41:17.750111+0000","last_undegraded":"2024-09-10T20:41:17.750111+0000","last_fullsized":"2024-09-10T20:41:17.750111+0000","mapping_epoch":62,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:55.885282+0000","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20900,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662099+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:41:17.662099+0000","last_peered":"2024-09-10T20:41:17.662099+0000","last_clean":"2024-09-10T20:41:17.662099+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:41:17.662099+0000","last_undegraded":"2024-09-10T20:41:17.662099+0000","last_fullsized":"2024-09-10T20:41:17.662099+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665321+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:41:17.665321+0000","last_peered":"2024-09-10T20:41:17.665321+0000","last_clean":"2024-09-10T20:41:17.665321+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:41:17.665321+0000","last_undegraded":"2024-09-10T20:41:17.665321+0000","last_fullsized":"2024-09-10T20:41:17.665321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22142,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665632+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:41:17.665632+0000","last_peered":"2024-09-10T20:41:17.665632+0000","last_clean":"2024-09-10T20:41:17.665632+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:41:17.665632+0000","last_undegraded":"2024-09-10T20:41:17.665632+0000","last_fullsized":"2024-09-10T20:41:17.665632+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":133,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663417+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:41:17.663417+0000","last_peered":"2024-09-10T20:41:17.663417+0000","last_clean":"2024-09-10T20:41:17.663417+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:41:17.663417+0000","last_undegraded":"2024-09-10T20:41:17.663417+0000","last_fullsized":"2024-09-10T20:41:17.663417+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28128,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662796+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:41:17.662796+0000","last_peered":"2024-09-10T20:41:17.662796+0000","last_clean":"2024-09-10T20:41:17.662796+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:41:17.662796+0000","last_undegraded":"2024-09-10T20:41:17.662796+0000","last_fullsized":"2024-09-10T20:41:17.662796+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":140,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662225+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:41:17.662225+0000","last_peered":"2024-09-10T20:41:17.662225+0000","last_clean":"2024-09-10T20:41:17.662225+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:41:17.662225+0000","last_undegraded":"2024-09-10T20:41:17.662225+0000","last_fullsized":"2024-09-10T20:41:17.662225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28011,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663503+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:41:17.663503+0000","last_peered":"2024-09-10T20:41:17.663503+0000","last_clean":"2024-09-10T20:41:17.663503+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:41:17.663503+0000","last_undegraded":"2024-09-10T20:41:17.663503+0000","last_fullsized":"2024-09-10T20:41:17.663503+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665485+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:41:17.665485+0000","last_peered":"2024-09-10T20:41:17.665485+0000","last_clean":"2024-09-10T20:41:17.665485+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:41:17.665485+0000","last_undegraded":"2024-09-10T20:41:17.665485+0000","last_fullsized":"2024-09-10T20:41:17.665485+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28065,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662536+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:41:17.662536+0000","last_peered":"2024-09-10T20:41:17.662536+0000","last_clean":"2024-09-10T20:41:17.662536+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:41:17.662536+0000","last_undegraded":"2024-09-10T20:41:17.662536+0000","last_fullsized":"2024-09-10T20:41:17.662536+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665900+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:41:17.665900+0000","last_peered":"2024-09-10T20:41:17.665900+0000","last_clean":"2024-09-10T20:41:17.665900+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:41:17.665900+0000","last_undegraded":"2024-09-10T20:41:17.665900+0000","last_fullsized":"2024-09-10T20:41:17.665900+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665840+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:41:17.665840+0000","last_peered":"2024-09-10T20:41:17.665840+0000","last_clean":"2024-09-10T20:41:17.665840+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:41:17.665840+0000","last_undegraded":"2024-09-10T20:41:17.665840+0000","last_fullsized":"2024-09-10T20:41:17.665840+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27843,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665757+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:41:17.665757+0000","last_peered":"2024-09-10T20:41:17.665757+0000","last_clean":"2024-09-10T20:41:17.665757+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:41:17.665757+0000","last_undegraded":"2024-09-10T20:41:17.665757+0000","last_fullsized":"2024-09-10T20:41:17.665757+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":133,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663305+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:41:17.663305+0000","last_peered":"2024-09-10T20:41:17.663305+0000","last_clean":"2024-09-10T20:41:17.663305+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:41:17.663305+0000","last_undegraded":"2024-09-10T20:41:17.663305+0000","last_fullsized":"2024-09-10T20:41:17.663305+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":22754,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:20.244611+0000","last_change":"2024-09-10T20:41:15.931451+0000","last_active":"2024-09-10T20:41:20.244611+0000","last_peered":"2024-09-10T20:41:20.244611+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:41:20.244611+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2917,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":798,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665623+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:41:17.665623+0000","last_peered":"2024-09-10T20:41:17.665623+0000","last_clean":"2024-09-10T20:41:17.665623+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:41:17.665623+0000","last_undegraded":"2024-09-10T20:41:17.665623+0000","last_fullsized":"2024-09-10T20:41:17.665623+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665817+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:41:17.665817+0000","last_peered":"2024-09-10T20:41:17.665817+0000","last_clean":"2024-09-10T20:41:17.665817+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:41:17.665817+0000","last_undegraded":"2024-09-10T20:41:17.665817+0000","last_fullsized":"2024-09-10T20:41:17.665817+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27673,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665728+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:41:17.665728+0000","last_peered":"2024-09-10T20:41:17.665728+0000","last_clean":"2024-09-10T20:41:17.665728+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:41:17.665728+0000","last_undegraded":"2024-09-10T20:41:17.665728+0000","last_fullsized":"2024-09-10T20:41:17.665728+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6480,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33050,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":141287424,"data_stored":141269359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182553,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517228,"kb_used_data":58004,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854612,"statfs":{"total":96636764160,"available":96107122688,"internally_reserved":0,"allocated":59396096,"data_stored":59065163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986326,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":446588,"kb_used_data":81708,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93925252,"statfs":{"total":96636764160,"available":96179458048,"internally_reserved":0,"allocated":83668992,"data_stored":83342136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659272,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497312,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874528,"statfs":{"total":96636764160,"available":96127516672,"internally_reserved":0,"allocated":540672,"data_stored":202513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":82407424,"data_stored":82403811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:41:27.014 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:27.365 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:27.366 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:27.379 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":456,"stamp":"2024-09-10T20:41:27.077560+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5583,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33949,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1461128,"kb_used_data":140240,"kb_used_omap":12,"kb_used_meta":1320819,"kb_avail":281654392,"statfs":{"total":289910292480,"available":288414097408,"internally_reserved":0,"allocated":143605760,"data_stored":142609812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1352518658},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1826,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1826,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001630"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":28000,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663983+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:41:17.663983+0000","last_peered":"2024-09-10T20:41:17.663983+0000","last_clean":"2024-09-10T20:41:17.663983+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:41:17.663983+0000","last_undegraded":"2024-09-10T20:41:17.663983+0000","last_fullsized":"2024-09-10T20:41:17.663983+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21092,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663678+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:41:17.663678+0000","last_peered":"2024-09-10T20:41:17.663678+0000","last_clean":"2024-09-10T20:41:17.663678+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:41:17.663678+0000","last_undegraded":"2024-09-10T20:41:17.663678+0000","last_fullsized":"2024-09-10T20:41:17.663678+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28104,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665777+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:41:17.665777+0000","last_peered":"2024-09-10T20:41:17.665777+0000","last_clean":"2024-09-10T20:41:17.665777+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:41:17.665777+0000","last_undegraded":"2024-09-10T20:41:17.665777+0000","last_fullsized":"2024-09-10T20:41:17.665777+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20724,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.664035+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:41:17.664035+0000","last_peered":"2024-09-10T20:41:17.664035+0000","last_clean":"2024-09-10T20:41:17.664035+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:41:17.664035+0000","last_undegraded":"2024-09-10T20:41:17.664035+0000","last_fullsized":"2024-09-10T20:41:17.664035+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21235,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663037+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:41:17.663037+0000","last_peered":"2024-09-10T20:41:17.663037+0000","last_clean":"2024-09-10T20:41:17.663037+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:41:17.663037+0000","last_undegraded":"2024-09-10T20:41:17.663037+0000","last_fullsized":"2024-09-10T20:41:17.663037+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20753,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:17.662327+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:41:17.662327+0000","last_peered":"2024-09-10T20:41:17.662327+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:41:17.662327+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20911,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.750111+0000","last_change":"2024-09-10T20:41:17.750026+0000","last_active":"2024-09-10T20:41:17.750111+0000","last_peered":"2024-09-10T20:41:17.750111+0000","last_clean":"2024-09-10T20:41:17.750111+0000","last_became_active":"2024-09-10T20:41:17.668013+0000","last_became_peered":"2024-09-10T20:41:17.668013+0000","last_unstale":"2024-09-10T20:41:17.750111+0000","last_undegraded":"2024-09-10T20:41:17.750111+0000","last_fullsized":"2024-09-10T20:41:17.750111+0000","mapping_epoch":62,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:55.885282+0000","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20900,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662099+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:41:17.662099+0000","last_peered":"2024-09-10T20:41:17.662099+0000","last_clean":"2024-09-10T20:41:17.662099+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:41:17.662099+0000","last_undegraded":"2024-09-10T20:41:17.662099+0000","last_fullsized":"2024-09-10T20:41:17.662099+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665321+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:41:17.665321+0000","last_peered":"2024-09-10T20:41:17.665321+0000","last_clean":"2024-09-10T20:41:17.665321+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:41:17.665321+0000","last_undegraded":"2024-09-10T20:41:17.665321+0000","last_fullsized":"2024-09-10T20:41:17.665321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22142,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665632+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:41:17.665632+0000","last_peered":"2024-09-10T20:41:17.665632+0000","last_clean":"2024-09-10T20:41:17.665632+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:41:17.665632+0000","last_undegraded":"2024-09-10T20:41:17.665632+0000","last_fullsized":"2024-09-10T20:41:17.665632+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":133,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663417+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:41:17.663417+0000","last_peered":"2024-09-10T20:41:17.663417+0000","last_clean":"2024-09-10T20:41:17.663417+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:41:17.663417+0000","last_undegraded":"2024-09-10T20:41:17.663417+0000","last_fullsized":"2024-09-10T20:41:17.663417+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28128,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662796+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:41:17.662796+0000","last_peered":"2024-09-10T20:41:17.662796+0000","last_clean":"2024-09-10T20:41:17.662796+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:41:17.662796+0000","last_undegraded":"2024-09-10T20:41:17.662796+0000","last_fullsized":"2024-09-10T20:41:17.662796+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":140,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662225+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:41:17.662225+0000","last_peered":"2024-09-10T20:41:17.662225+0000","last_clean":"2024-09-10T20:41:17.662225+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:41:17.662225+0000","last_undegraded":"2024-09-10T20:41:17.662225+0000","last_fullsized":"2024-09-10T20:41:17.662225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28011,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663503+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:41:17.663503+0000","last_peered":"2024-09-10T20:41:17.663503+0000","last_clean":"2024-09-10T20:41:17.663503+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:41:17.663503+0000","last_undegraded":"2024-09-10T20:41:17.663503+0000","last_fullsized":"2024-09-10T20:41:17.663503+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665485+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:41:17.665485+0000","last_peered":"2024-09-10T20:41:17.665485+0000","last_clean":"2024-09-10T20:41:17.665485+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:41:17.665485+0000","last_undegraded":"2024-09-10T20:41:17.665485+0000","last_fullsized":"2024-09-10T20:41:17.665485+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28065,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662536+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:41:17.662536+0000","last_peered":"2024-09-10T20:41:17.662536+0000","last_clean":"2024-09-10T20:41:17.662536+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:41:17.662536+0000","last_undegraded":"2024-09-10T20:41:17.662536+0000","last_fullsized":"2024-09-10T20:41:17.662536+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665900+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:41:17.665900+0000","last_peered":"2024-09-10T20:41:17.665900+0000","last_clean":"2024-09-10T20:41:17.665900+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:41:17.665900+0000","last_undegraded":"2024-09-10T20:41:17.665900+0000","last_fullsized":"2024-09-10T20:41:17.665900+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665840+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:41:17.665840+0000","last_peered":"2024-09-10T20:41:17.665840+0000","last_clean":"2024-09-10T20:41:17.665840+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:41:17.665840+0000","last_undegraded":"2024-09-10T20:41:17.665840+0000","last_fullsized":"2024-09-10T20:41:17.665840+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27843,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665757+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:41:17.665757+0000","last_peered":"2024-09-10T20:41:17.665757+0000","last_clean":"2024-09-10T20:41:17.665757+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:41:17.665757+0000","last_undegraded":"2024-09-10T20:41:17.665757+0000","last_fullsized":"2024-09-10T20:41:17.665757+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":133,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663305+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:41:17.663305+0000","last_peered":"2024-09-10T20:41:17.663305+0000","last_clean":"2024-09-10T20:41:17.663305+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:41:17.663305+0000","last_undegraded":"2024-09-10T20:41:17.663305+0000","last_fullsized":"2024-09-10T20:41:17.663305+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":24548,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:25.244488+0000","last_change":"2024-09-10T20:41:15.931451+0000","last_active":"2024-09-10T20:41:25.244488+0000","last_peered":"2024-09-10T20:41:25.244488+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:41:25.244488+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2020,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1695,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2020}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665623+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:41:17.665623+0000","last_peered":"2024-09-10T20:41:17.665623+0000","last_clean":"2024-09-10T20:41:17.665623+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:41:17.665623+0000","last_undegraded":"2024-09-10T20:41:17.665623+0000","last_fullsized":"2024-09-10T20:41:17.665623+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665817+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:41:17.665817+0000","last_peered":"2024-09-10T20:41:17.665817+0000","last_clean":"2024-09-10T20:41:17.665817+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:41:17.665817+0000","last_undegraded":"2024-09-10T20:41:17.665817+0000","last_fullsized":"2024-09-10T20:41:17.665817+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27673,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665728+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:41:17.665728+0000","last_peered":"2024-09-10T20:41:17.665728+0000","last_clean":"2024-09-10T20:41:17.665728+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:41:17.665728+0000","last_undegraded":"2024-09-10T20:41:17.665728+0000","last_fullsized":"2024-09-10T20:41:17.665728+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5583,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33947,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":141287424,"data_stored":141269359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182554,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517228,"kb_used_data":58004,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854612,"statfs":{"total":96636764160,"available":96107122688,"internally_reserved":0,"allocated":59396096,"data_stored":59065163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986326,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":446588,"kb_used_data":81708,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93925252,"statfs":{"total":96636764160,"available":96179458048,"internally_reserved":0,"allocated":83668992,"data_stored":83342136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659272,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497312,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874528,"statfs":{"total":96636764160,"available":96127516672,"internally_reserved":0,"allocated":540672,"data_stored":202513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":82407424,"data_stored":82403811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:41:27.380 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:41:27.380 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20471", 'reported_seq': 20753, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:41:17.662327+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:41:17.662327+0000', 'last_peered': '2024-09-10T20:41:17.662327+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:41:17.662327+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17335, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3561}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:41:27.381 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:41:27.381 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "51'20986", 'reported_seq': 24548, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:41:25.244488+0000', 'last_change': '2024-09-10T20:41:15.931451+0000', 'last_active': '2024-09-10T20:41:25.244488+0000', 'last_peered': '2024-09-10T20:41:25.244488+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:41:25.244488+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2020, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17813, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1695, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2020}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:41:27.381 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:41:27.759 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:27.760 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 5583/6 objects degraded (93050.000%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":799,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1496195072,"bytes_avail":288414097408,"bytes_total":289910292480,"degraded_objects":5583,"degraded_total":6,"degraded_ratio":930.5,"recovering_objects_per_sec":182,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1826,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:41:27.773 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:41:27.773 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:28.082 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:28.082 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:28.095 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":456,"stamp":"2024-09-10T20:41:27.077560+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5583,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33949,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1461128,"kb_used_data":140240,"kb_used_omap":12,"kb_used_meta":1320819,"kb_avail":281654392,"statfs":{"total":289910292480,"available":288414097408,"internally_reserved":0,"allocated":143605760,"data_stored":142609812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1352518658},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1826,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1826,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001630"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":28000,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663983+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:41:17.663983+0000","last_peered":"2024-09-10T20:41:17.663983+0000","last_clean":"2024-09-10T20:41:17.663983+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:41:17.663983+0000","last_undegraded":"2024-09-10T20:41:17.663983+0000","last_fullsized":"2024-09-10T20:41:17.663983+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21092,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663678+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:41:17.663678+0000","last_peered":"2024-09-10T20:41:17.663678+0000","last_clean":"2024-09-10T20:41:17.663678+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:41:17.663678+0000","last_undegraded":"2024-09-10T20:41:17.663678+0000","last_fullsized":"2024-09-10T20:41:17.663678+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28104,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665777+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:41:17.665777+0000","last_peered":"2024-09-10T20:41:17.665777+0000","last_clean":"2024-09-10T20:41:17.665777+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:41:17.665777+0000","last_undegraded":"2024-09-10T20:41:17.665777+0000","last_fullsized":"2024-09-10T20:41:17.665777+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20724,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.664035+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:41:17.664035+0000","last_peered":"2024-09-10T20:41:17.664035+0000","last_clean":"2024-09-10T20:41:17.664035+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:41:17.664035+0000","last_undegraded":"2024-09-10T20:41:17.664035+0000","last_fullsized":"2024-09-10T20:41:17.664035+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21235,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663037+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:41:17.663037+0000","last_peered":"2024-09-10T20:41:17.663037+0000","last_clean":"2024-09-10T20:41:17.663037+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:41:17.663037+0000","last_undegraded":"2024-09-10T20:41:17.663037+0000","last_fullsized":"2024-09-10T20:41:17.663037+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20753,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:17.662327+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:41:17.662327+0000","last_peered":"2024-09-10T20:41:17.662327+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:41:17.662327+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20911,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.750111+0000","last_change":"2024-09-10T20:41:17.750026+0000","last_active":"2024-09-10T20:41:17.750111+0000","last_peered":"2024-09-10T20:41:17.750111+0000","last_clean":"2024-09-10T20:41:17.750111+0000","last_became_active":"2024-09-10T20:41:17.668013+0000","last_became_peered":"2024-09-10T20:41:17.668013+0000","last_unstale":"2024-09-10T20:41:17.750111+0000","last_undegraded":"2024-09-10T20:41:17.750111+0000","last_fullsized":"2024-09-10T20:41:17.750111+0000","mapping_epoch":62,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:55.885282+0000","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20900,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662099+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:41:17.662099+0000","last_peered":"2024-09-10T20:41:17.662099+0000","last_clean":"2024-09-10T20:41:17.662099+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:41:17.662099+0000","last_undegraded":"2024-09-10T20:41:17.662099+0000","last_fullsized":"2024-09-10T20:41:17.662099+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665321+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:41:17.665321+0000","last_peered":"2024-09-10T20:41:17.665321+0000","last_clean":"2024-09-10T20:41:17.665321+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:41:17.665321+0000","last_undegraded":"2024-09-10T20:41:17.665321+0000","last_fullsized":"2024-09-10T20:41:17.665321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22142,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665632+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:41:17.665632+0000","last_peered":"2024-09-10T20:41:17.665632+0000","last_clean":"2024-09-10T20:41:17.665632+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:41:17.665632+0000","last_undegraded":"2024-09-10T20:41:17.665632+0000","last_fullsized":"2024-09-10T20:41:17.665632+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":133,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663417+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:41:17.663417+0000","last_peered":"2024-09-10T20:41:17.663417+0000","last_clean":"2024-09-10T20:41:17.663417+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:41:17.663417+0000","last_undegraded":"2024-09-10T20:41:17.663417+0000","last_fullsized":"2024-09-10T20:41:17.663417+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28128,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662796+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:41:17.662796+0000","last_peered":"2024-09-10T20:41:17.662796+0000","last_clean":"2024-09-10T20:41:17.662796+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:41:17.662796+0000","last_undegraded":"2024-09-10T20:41:17.662796+0000","last_fullsized":"2024-09-10T20:41:17.662796+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":140,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662225+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:41:17.662225+0000","last_peered":"2024-09-10T20:41:17.662225+0000","last_clean":"2024-09-10T20:41:17.662225+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:41:17.662225+0000","last_undegraded":"2024-09-10T20:41:17.662225+0000","last_fullsized":"2024-09-10T20:41:17.662225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28011,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663503+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:41:17.663503+0000","last_peered":"2024-09-10T20:41:17.663503+0000","last_clean":"2024-09-10T20:41:17.663503+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:41:17.663503+0000","last_undegraded":"2024-09-10T20:41:17.663503+0000","last_fullsized":"2024-09-10T20:41:17.663503+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665485+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:41:17.665485+0000","last_peered":"2024-09-10T20:41:17.665485+0000","last_clean":"2024-09-10T20:41:17.665485+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:41:17.665485+0000","last_undegraded":"2024-09-10T20:41:17.665485+0000","last_fullsized":"2024-09-10T20:41:17.665485+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28065,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662536+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:41:17.662536+0000","last_peered":"2024-09-10T20:41:17.662536+0000","last_clean":"2024-09-10T20:41:17.662536+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:41:17.662536+0000","last_undegraded":"2024-09-10T20:41:17.662536+0000","last_fullsized":"2024-09-10T20:41:17.662536+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665900+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:41:17.665900+0000","last_peered":"2024-09-10T20:41:17.665900+0000","last_clean":"2024-09-10T20:41:17.665900+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:41:17.665900+0000","last_undegraded":"2024-09-10T20:41:17.665900+0000","last_fullsized":"2024-09-10T20:41:17.665900+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665840+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:41:17.665840+0000","last_peered":"2024-09-10T20:41:17.665840+0000","last_clean":"2024-09-10T20:41:17.665840+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:41:17.665840+0000","last_undegraded":"2024-09-10T20:41:17.665840+0000","last_fullsized":"2024-09-10T20:41:17.665840+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27843,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665757+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:41:17.665757+0000","last_peered":"2024-09-10T20:41:17.665757+0000","last_clean":"2024-09-10T20:41:17.665757+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:41:17.665757+0000","last_undegraded":"2024-09-10T20:41:17.665757+0000","last_fullsized":"2024-09-10T20:41:17.665757+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":133,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663305+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:41:17.663305+0000","last_peered":"2024-09-10T20:41:17.663305+0000","last_clean":"2024-09-10T20:41:17.663305+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:41:17.663305+0000","last_undegraded":"2024-09-10T20:41:17.663305+0000","last_fullsized":"2024-09-10T20:41:17.663305+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":24548,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:25.244488+0000","last_change":"2024-09-10T20:41:15.931451+0000","last_active":"2024-09-10T20:41:25.244488+0000","last_peered":"2024-09-10T20:41:25.244488+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:41:25.244488+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2020,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1695,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2020}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665623+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:41:17.665623+0000","last_peered":"2024-09-10T20:41:17.665623+0000","last_clean":"2024-09-10T20:41:17.665623+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:41:17.665623+0000","last_undegraded":"2024-09-10T20:41:17.665623+0000","last_fullsized":"2024-09-10T20:41:17.665623+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665817+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:41:17.665817+0000","last_peered":"2024-09-10T20:41:17.665817+0000","last_clean":"2024-09-10T20:41:17.665817+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:41:17.665817+0000","last_undegraded":"2024-09-10T20:41:17.665817+0000","last_fullsized":"2024-09-10T20:41:17.665817+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27673,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665728+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:41:17.665728+0000","last_peered":"2024-09-10T20:41:17.665728+0000","last_clean":"2024-09-10T20:41:17.665728+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:41:17.665728+0000","last_undegraded":"2024-09-10T20:41:17.665728+0000","last_fullsized":"2024-09-10T20:41:17.665728+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5583,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33947,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":141287424,"data_stored":141269359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182554,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517228,"kb_used_data":58004,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854612,"statfs":{"total":96636764160,"available":96107122688,"internally_reserved":0,"allocated":59396096,"data_stored":59065163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986326,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":446588,"kb_used_data":81708,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93925252,"statfs":{"total":96636764160,"available":96179458048,"internally_reserved":0,"allocated":83668992,"data_stored":83342136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659272,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497312,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874528,"statfs":{"total":96636764160,"available":96127516672,"internally_reserved":0,"allocated":540672,"data_stored":202513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":82407424,"data_stored":82403811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:41:31.097 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:31.397 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:31.397 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:31.409 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":458,"stamp":"2024-09-10T20:41:31.078155+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4732,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34800,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1465848,"kb_used_data":136640,"kb_used_omap":12,"kb_used_meta":1329139,"kb_avail":281649672,"statfs":{"total":289910292480,"available":288409264128,"internally_reserved":0,"allocated":139919360,"data_stored":138927025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1361038338},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2488,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001921"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":28000,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663983+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:41:17.663983+0000","last_peered":"2024-09-10T20:41:17.663983+0000","last_clean":"2024-09-10T20:41:17.663983+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:41:17.663983+0000","last_undegraded":"2024-09-10T20:41:17.663983+0000","last_fullsized":"2024-09-10T20:41:17.663983+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21092,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663678+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:41:17.663678+0000","last_peered":"2024-09-10T20:41:17.663678+0000","last_clean":"2024-09-10T20:41:17.663678+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:41:17.663678+0000","last_undegraded":"2024-09-10T20:41:17.663678+0000","last_fullsized":"2024-09-10T20:41:17.663678+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28104,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665777+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:41:17.665777+0000","last_peered":"2024-09-10T20:41:17.665777+0000","last_clean":"2024-09-10T20:41:17.665777+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:41:17.665777+0000","last_undegraded":"2024-09-10T20:41:17.665777+0000","last_fullsized":"2024-09-10T20:41:17.665777+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20724,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.664035+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:41:17.664035+0000","last_peered":"2024-09-10T20:41:17.664035+0000","last_clean":"2024-09-10T20:41:17.664035+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:41:17.664035+0000","last_undegraded":"2024-09-10T20:41:17.664035+0000","last_fullsized":"2024-09-10T20:41:17.664035+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21235,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663037+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:41:17.663037+0000","last_peered":"2024-09-10T20:41:17.663037+0000","last_clean":"2024-09-10T20:41:17.663037+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:41:17.663037+0000","last_undegraded":"2024-09-10T20:41:17.663037+0000","last_fullsized":"2024-09-10T20:41:17.663037+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20753,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:17.662327+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:41:17.662327+0000","last_peered":"2024-09-10T20:41:17.662327+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:41:17.662327+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20911,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.750111+0000","last_change":"2024-09-10T20:41:17.750026+0000","last_active":"2024-09-10T20:41:17.750111+0000","last_peered":"2024-09-10T20:41:17.750111+0000","last_clean":"2024-09-10T20:41:17.750111+0000","last_became_active":"2024-09-10T20:41:17.668013+0000","last_became_peered":"2024-09-10T20:41:17.668013+0000","last_unstale":"2024-09-10T20:41:17.750111+0000","last_undegraded":"2024-09-10T20:41:17.750111+0000","last_fullsized":"2024-09-10T20:41:17.750111+0000","mapping_epoch":62,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:55.885282+0000","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20900,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662099+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:41:17.662099+0000","last_peered":"2024-09-10T20:41:17.662099+0000","last_clean":"2024-09-10T20:41:17.662099+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:41:17.662099+0000","last_undegraded":"2024-09-10T20:41:17.662099+0000","last_fullsized":"2024-09-10T20:41:17.662099+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665321+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:41:17.665321+0000","last_peered":"2024-09-10T20:41:17.665321+0000","last_clean":"2024-09-10T20:41:17.665321+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:41:17.665321+0000","last_undegraded":"2024-09-10T20:41:17.665321+0000","last_fullsized":"2024-09-10T20:41:17.665321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22142,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665632+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:41:17.665632+0000","last_peered":"2024-09-10T20:41:17.665632+0000","last_clean":"2024-09-10T20:41:17.665632+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:41:17.665632+0000","last_undegraded":"2024-09-10T20:41:17.665632+0000","last_fullsized":"2024-09-10T20:41:17.665632+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":133,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663417+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:41:17.663417+0000","last_peered":"2024-09-10T20:41:17.663417+0000","last_clean":"2024-09-10T20:41:17.663417+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:41:17.663417+0000","last_undegraded":"2024-09-10T20:41:17.663417+0000","last_fullsized":"2024-09-10T20:41:17.663417+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28129,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662845+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:41:17.662845+0000","last_peered":"2024-09-10T20:41:17.662845+0000","last_clean":"2024-09-10T20:41:17.662845+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:41:17.662845+0000","last_undegraded":"2024-09-10T20:41:17.662845+0000","last_fullsized":"2024-09-10T20:41:17.662845+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662267+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:41:17.662267+0000","last_peered":"2024-09-10T20:41:17.662267+0000","last_clean":"2024-09-10T20:41:17.662267+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:41:17.662267+0000","last_undegraded":"2024-09-10T20:41:17.662267+0000","last_fullsized":"2024-09-10T20:41:17.662267+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28011,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663503+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:41:17.663503+0000","last_peered":"2024-09-10T20:41:17.663503+0000","last_clean":"2024-09-10T20:41:17.663503+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:41:17.663503+0000","last_undegraded":"2024-09-10T20:41:17.663503+0000","last_fullsized":"2024-09-10T20:41:17.663503+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665485+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:41:17.665485+0000","last_peered":"2024-09-10T20:41:17.665485+0000","last_clean":"2024-09-10T20:41:17.665485+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:41:17.665485+0000","last_undegraded":"2024-09-10T20:41:17.665485+0000","last_fullsized":"2024-09-10T20:41:17.665485+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28065,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662536+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:41:17.662536+0000","last_peered":"2024-09-10T20:41:17.662536+0000","last_clean":"2024-09-10T20:41:17.662536+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:41:17.662536+0000","last_undegraded":"2024-09-10T20:41:17.662536+0000","last_fullsized":"2024-09-10T20:41:17.662536+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665900+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:41:17.665900+0000","last_peered":"2024-09-10T20:41:17.665900+0000","last_clean":"2024-09-10T20:41:17.665900+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:41:17.665900+0000","last_undegraded":"2024-09-10T20:41:17.665900+0000","last_fullsized":"2024-09-10T20:41:17.665900+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665840+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:41:17.665840+0000","last_peered":"2024-09-10T20:41:17.665840+0000","last_clean":"2024-09-10T20:41:17.665840+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:41:17.665840+0000","last_undegraded":"2024-09-10T20:41:17.665840+0000","last_fullsized":"2024-09-10T20:41:17.665840+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27843,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665757+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:41:17.665757+0000","last_peered":"2024-09-10T20:41:17.665757+0000","last_clean":"2024-09-10T20:41:17.665757+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:41:17.665757+0000","last_undegraded":"2024-09-10T20:41:17.665757+0000","last_fullsized":"2024-09-10T20:41:17.665757+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":133,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663305+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:41:17.663305+0000","last_peered":"2024-09-10T20:41:17.663305+0000","last_clean":"2024-09-10T20:41:17.663305+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:41:17.663305+0000","last_undegraded":"2024-09-10T20:41:17.663305+0000","last_fullsized":"2024-09-10T20:41:17.663305+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":26250,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:30.247049+0000","last_change":"2024-09-10T20:41:15.931451+0000","last_active":"2024-09-10T20:41:30.247049+0000","last_peered":"2024-09-10T20:41:30.247049+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:41:30.247049+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1169}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665623+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:41:17.665623+0000","last_peered":"2024-09-10T20:41:17.665623+0000","last_clean":"2024-09-10T20:41:17.665623+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:41:17.665623+0000","last_undegraded":"2024-09-10T20:41:17.665623+0000","last_fullsized":"2024-09-10T20:41:17.665623+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665817+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:41:17.665817+0000","last_peered":"2024-09-10T20:41:17.665817+0000","last_clean":"2024-09-10T20:41:17.665817+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:41:17.665817+0000","last_undegraded":"2024-09-10T20:41:17.665817+0000","last_fullsized":"2024-09-10T20:41:17.665817+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27673,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665728+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:41:17.665728+0000","last_peered":"2024-09-10T20:41:17.665728+0000","last_clean":"2024-09-10T20:41:17.665728+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:41:17.665728+0000","last_undegraded":"2024-09-10T20:41:17.665728+0000","last_fullsized":"2024-09-10T20:41:17.665728+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4732,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34798,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":137601024,"data_stored":137586572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182555,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517228,"kb_used_data":58004,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854612,"statfs":{"total":96636764160,"available":96107122688,"internally_reserved":0,"allocated":59396096,"data_stored":59065163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986327,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442988,"kb_used_data":78108,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93928852,"statfs":{"total":96636764160,"available":96183144448,"internally_reserved":0,"allocated":79982592,"data_stored":79659349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659273,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505632,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866208,"statfs":{"total":96636764160,"available":96118996992,"internally_reserved":0,"allocated":540672,"data_stored":202513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":517205678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":78721024,"data_stored":78721024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:41:31.411 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:31.708 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:31.708 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:31.720 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":458,"stamp":"2024-09-10T20:41:31.078155+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4732,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34800,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1465848,"kb_used_data":136640,"kb_used_omap":12,"kb_used_meta":1329139,"kb_avail":281649672,"statfs":{"total":289910292480,"available":288409264128,"internally_reserved":0,"allocated":139919360,"data_stored":138927025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1361038338},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2488,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001921"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":28000,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663983+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:41:17.663983+0000","last_peered":"2024-09-10T20:41:17.663983+0000","last_clean":"2024-09-10T20:41:17.663983+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:41:17.663983+0000","last_undegraded":"2024-09-10T20:41:17.663983+0000","last_fullsized":"2024-09-10T20:41:17.663983+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21092,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663678+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:41:17.663678+0000","last_peered":"2024-09-10T20:41:17.663678+0000","last_clean":"2024-09-10T20:41:17.663678+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:41:17.663678+0000","last_undegraded":"2024-09-10T20:41:17.663678+0000","last_fullsized":"2024-09-10T20:41:17.663678+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28104,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665777+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:41:17.665777+0000","last_peered":"2024-09-10T20:41:17.665777+0000","last_clean":"2024-09-10T20:41:17.665777+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:41:17.665777+0000","last_undegraded":"2024-09-10T20:41:17.665777+0000","last_fullsized":"2024-09-10T20:41:17.665777+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20724,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.664035+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:41:17.664035+0000","last_peered":"2024-09-10T20:41:17.664035+0000","last_clean":"2024-09-10T20:41:17.664035+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:41:17.664035+0000","last_undegraded":"2024-09-10T20:41:17.664035+0000","last_fullsized":"2024-09-10T20:41:17.664035+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21235,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663037+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:41:17.663037+0000","last_peered":"2024-09-10T20:41:17.663037+0000","last_clean":"2024-09-10T20:41:17.663037+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:41:17.663037+0000","last_undegraded":"2024-09-10T20:41:17.663037+0000","last_fullsized":"2024-09-10T20:41:17.663037+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20753,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:17.662327+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:41:17.662327+0000","last_peered":"2024-09-10T20:41:17.662327+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:41:17.662327+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20911,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.750111+0000","last_change":"2024-09-10T20:41:17.750026+0000","last_active":"2024-09-10T20:41:17.750111+0000","last_peered":"2024-09-10T20:41:17.750111+0000","last_clean":"2024-09-10T20:41:17.750111+0000","last_became_active":"2024-09-10T20:41:17.668013+0000","last_became_peered":"2024-09-10T20:41:17.668013+0000","last_unstale":"2024-09-10T20:41:17.750111+0000","last_undegraded":"2024-09-10T20:41:17.750111+0000","last_fullsized":"2024-09-10T20:41:17.750111+0000","mapping_epoch":62,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:55.885282+0000","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20900,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662099+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:41:17.662099+0000","last_peered":"2024-09-10T20:41:17.662099+0000","last_clean":"2024-09-10T20:41:17.662099+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:41:17.662099+0000","last_undegraded":"2024-09-10T20:41:17.662099+0000","last_fullsized":"2024-09-10T20:41:17.662099+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665321+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:41:17.665321+0000","last_peered":"2024-09-10T20:41:17.665321+0000","last_clean":"2024-09-10T20:41:17.665321+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:41:17.665321+0000","last_undegraded":"2024-09-10T20:41:17.665321+0000","last_fullsized":"2024-09-10T20:41:17.665321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22142,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665632+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:41:17.665632+0000","last_peered":"2024-09-10T20:41:17.665632+0000","last_clean":"2024-09-10T20:41:17.665632+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:41:17.665632+0000","last_undegraded":"2024-09-10T20:41:17.665632+0000","last_fullsized":"2024-09-10T20:41:17.665632+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":133,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663417+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:41:17.663417+0000","last_peered":"2024-09-10T20:41:17.663417+0000","last_clean":"2024-09-10T20:41:17.663417+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:41:17.663417+0000","last_undegraded":"2024-09-10T20:41:17.663417+0000","last_fullsized":"2024-09-10T20:41:17.663417+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28129,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662845+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:41:17.662845+0000","last_peered":"2024-09-10T20:41:17.662845+0000","last_clean":"2024-09-10T20:41:17.662845+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:41:17.662845+0000","last_undegraded":"2024-09-10T20:41:17.662845+0000","last_fullsized":"2024-09-10T20:41:17.662845+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662267+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:41:17.662267+0000","last_peered":"2024-09-10T20:41:17.662267+0000","last_clean":"2024-09-10T20:41:17.662267+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:41:17.662267+0000","last_undegraded":"2024-09-10T20:41:17.662267+0000","last_fullsized":"2024-09-10T20:41:17.662267+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28011,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663503+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:41:17.663503+0000","last_peered":"2024-09-10T20:41:17.663503+0000","last_clean":"2024-09-10T20:41:17.663503+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:41:17.663503+0000","last_undegraded":"2024-09-10T20:41:17.663503+0000","last_fullsized":"2024-09-10T20:41:17.663503+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665485+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:41:17.665485+0000","last_peered":"2024-09-10T20:41:17.665485+0000","last_clean":"2024-09-10T20:41:17.665485+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:41:17.665485+0000","last_undegraded":"2024-09-10T20:41:17.665485+0000","last_fullsized":"2024-09-10T20:41:17.665485+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28065,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662536+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:41:17.662536+0000","last_peered":"2024-09-10T20:41:17.662536+0000","last_clean":"2024-09-10T20:41:17.662536+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:41:17.662536+0000","last_undegraded":"2024-09-10T20:41:17.662536+0000","last_fullsized":"2024-09-10T20:41:17.662536+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665900+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:41:17.665900+0000","last_peered":"2024-09-10T20:41:17.665900+0000","last_clean":"2024-09-10T20:41:17.665900+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:41:17.665900+0000","last_undegraded":"2024-09-10T20:41:17.665900+0000","last_fullsized":"2024-09-10T20:41:17.665900+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665840+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:41:17.665840+0000","last_peered":"2024-09-10T20:41:17.665840+0000","last_clean":"2024-09-10T20:41:17.665840+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:41:17.665840+0000","last_undegraded":"2024-09-10T20:41:17.665840+0000","last_fullsized":"2024-09-10T20:41:17.665840+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27843,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665757+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:41:17.665757+0000","last_peered":"2024-09-10T20:41:17.665757+0000","last_clean":"2024-09-10T20:41:17.665757+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:41:17.665757+0000","last_undegraded":"2024-09-10T20:41:17.665757+0000","last_fullsized":"2024-09-10T20:41:17.665757+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":133,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663305+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:41:17.663305+0000","last_peered":"2024-09-10T20:41:17.663305+0000","last_clean":"2024-09-10T20:41:17.663305+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:41:17.663305+0000","last_undegraded":"2024-09-10T20:41:17.663305+0000","last_fullsized":"2024-09-10T20:41:17.663305+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":26250,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:30.247049+0000","last_change":"2024-09-10T20:41:15.931451+0000","last_active":"2024-09-10T20:41:30.247049+0000","last_peered":"2024-09-10T20:41:30.247049+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:41:30.247049+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1169}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665623+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:41:17.665623+0000","last_peered":"2024-09-10T20:41:17.665623+0000","last_clean":"2024-09-10T20:41:17.665623+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:41:17.665623+0000","last_undegraded":"2024-09-10T20:41:17.665623+0000","last_fullsized":"2024-09-10T20:41:17.665623+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665817+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:41:17.665817+0000","last_peered":"2024-09-10T20:41:17.665817+0000","last_clean":"2024-09-10T20:41:17.665817+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:41:17.665817+0000","last_undegraded":"2024-09-10T20:41:17.665817+0000","last_fullsized":"2024-09-10T20:41:17.665817+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27673,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665728+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:41:17.665728+0000","last_peered":"2024-09-10T20:41:17.665728+0000","last_clean":"2024-09-10T20:41:17.665728+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:41:17.665728+0000","last_undegraded":"2024-09-10T20:41:17.665728+0000","last_fullsized":"2024-09-10T20:41:17.665728+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4732,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34798,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":137601024,"data_stored":137586572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182555,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517228,"kb_used_data":58004,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854612,"statfs":{"total":96636764160,"available":96107122688,"internally_reserved":0,"allocated":59396096,"data_stored":59065163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986327,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442988,"kb_used_data":78108,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93928852,"statfs":{"total":96636764160,"available":96183144448,"internally_reserved":0,"allocated":79982592,"data_stored":79659349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659273,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505632,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866208,"statfs":{"total":96636764160,"available":96118996992,"internally_reserved":0,"allocated":540672,"data_stored":202513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":517205678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":78721024,"data_stored":78721024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:41:31.722 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:41:31.722 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20471", 'reported_seq': 20753, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:41:17.662327+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:41:17.662327+0000', 'last_peered': '2024-09-10T20:41:17.662327+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:41:17.662327+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17335, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3561}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:41:31.722 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:41:31.722 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "51'20986", 'reported_seq': 26250, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:41:30.247049+0000', 'last_change': '2024-09-10T20:41:15.931451+0000', 'last_active': '2024-09-10T20:41:30.247049+0000', 'last_peered': '2024-09-10T20:41:30.247049+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:41:30.247049+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1169, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17813, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2546, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1169}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:41:31.722 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:41:32.097 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:32.098 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 4732/6 objects degraded (78866.667%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":803,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1501028352,"bytes_avail":288409264128,"bytes_total":289910292480,"degraded_objects":4732,"degraded_total":6,"degraded_ratio":788.66666666666652,"recovering_objects_per_sec":207,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2488,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:41:32.110 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:41:32.111 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:32.422 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:32.423 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:32.435 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":458,"stamp":"2024-09-10T20:41:31.078155+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4732,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34800,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1465848,"kb_used_data":136640,"kb_used_omap":12,"kb_used_meta":1329139,"kb_avail":281649672,"statfs":{"total":289910292480,"available":288409264128,"internally_reserved":0,"allocated":139919360,"data_stored":138927025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1361038338},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2488,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001921"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":28000,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663983+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:41:17.663983+0000","last_peered":"2024-09-10T20:41:17.663983+0000","last_clean":"2024-09-10T20:41:17.663983+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:41:17.663983+0000","last_undegraded":"2024-09-10T20:41:17.663983+0000","last_fullsized":"2024-09-10T20:41:17.663983+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21092,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663678+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:41:17.663678+0000","last_peered":"2024-09-10T20:41:17.663678+0000","last_clean":"2024-09-10T20:41:17.663678+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:41:17.663678+0000","last_undegraded":"2024-09-10T20:41:17.663678+0000","last_fullsized":"2024-09-10T20:41:17.663678+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28104,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665777+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:41:17.665777+0000","last_peered":"2024-09-10T20:41:17.665777+0000","last_clean":"2024-09-10T20:41:17.665777+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:41:17.665777+0000","last_undegraded":"2024-09-10T20:41:17.665777+0000","last_fullsized":"2024-09-10T20:41:17.665777+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20724,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.664035+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:41:17.664035+0000","last_peered":"2024-09-10T20:41:17.664035+0000","last_clean":"2024-09-10T20:41:17.664035+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:41:17.664035+0000","last_undegraded":"2024-09-10T20:41:17.664035+0000","last_fullsized":"2024-09-10T20:41:17.664035+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21235,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663037+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:41:17.663037+0000","last_peered":"2024-09-10T20:41:17.663037+0000","last_clean":"2024-09-10T20:41:17.663037+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:41:17.663037+0000","last_undegraded":"2024-09-10T20:41:17.663037+0000","last_fullsized":"2024-09-10T20:41:17.663037+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20753,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:17.662327+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:41:17.662327+0000","last_peered":"2024-09-10T20:41:17.662327+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:41:17.662327+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20911,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.750111+0000","last_change":"2024-09-10T20:41:17.750026+0000","last_active":"2024-09-10T20:41:17.750111+0000","last_peered":"2024-09-10T20:41:17.750111+0000","last_clean":"2024-09-10T20:41:17.750111+0000","last_became_active":"2024-09-10T20:41:17.668013+0000","last_became_peered":"2024-09-10T20:41:17.668013+0000","last_unstale":"2024-09-10T20:41:17.750111+0000","last_undegraded":"2024-09-10T20:41:17.750111+0000","last_fullsized":"2024-09-10T20:41:17.750111+0000","mapping_epoch":62,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:55.885282+0000","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20900,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662099+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:41:17.662099+0000","last_peered":"2024-09-10T20:41:17.662099+0000","last_clean":"2024-09-10T20:41:17.662099+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:41:17.662099+0000","last_undegraded":"2024-09-10T20:41:17.662099+0000","last_fullsized":"2024-09-10T20:41:17.662099+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665321+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:41:17.665321+0000","last_peered":"2024-09-10T20:41:17.665321+0000","last_clean":"2024-09-10T20:41:17.665321+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:41:17.665321+0000","last_undegraded":"2024-09-10T20:41:17.665321+0000","last_fullsized":"2024-09-10T20:41:17.665321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22142,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665632+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:41:17.665632+0000","last_peered":"2024-09-10T20:41:17.665632+0000","last_clean":"2024-09-10T20:41:17.665632+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:41:17.665632+0000","last_undegraded":"2024-09-10T20:41:17.665632+0000","last_fullsized":"2024-09-10T20:41:17.665632+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":133,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663417+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:41:17.663417+0000","last_peered":"2024-09-10T20:41:17.663417+0000","last_clean":"2024-09-10T20:41:17.663417+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:41:17.663417+0000","last_undegraded":"2024-09-10T20:41:17.663417+0000","last_fullsized":"2024-09-10T20:41:17.663417+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28129,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662845+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:41:17.662845+0000","last_peered":"2024-09-10T20:41:17.662845+0000","last_clean":"2024-09-10T20:41:17.662845+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:41:17.662845+0000","last_undegraded":"2024-09-10T20:41:17.662845+0000","last_fullsized":"2024-09-10T20:41:17.662845+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662267+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:41:17.662267+0000","last_peered":"2024-09-10T20:41:17.662267+0000","last_clean":"2024-09-10T20:41:17.662267+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:41:17.662267+0000","last_undegraded":"2024-09-10T20:41:17.662267+0000","last_fullsized":"2024-09-10T20:41:17.662267+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28011,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663503+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:41:17.663503+0000","last_peered":"2024-09-10T20:41:17.663503+0000","last_clean":"2024-09-10T20:41:17.663503+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:41:17.663503+0000","last_undegraded":"2024-09-10T20:41:17.663503+0000","last_fullsized":"2024-09-10T20:41:17.663503+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665485+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:41:17.665485+0000","last_peered":"2024-09-10T20:41:17.665485+0000","last_clean":"2024-09-10T20:41:17.665485+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:41:17.665485+0000","last_undegraded":"2024-09-10T20:41:17.665485+0000","last_fullsized":"2024-09-10T20:41:17.665485+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28065,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662536+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:41:17.662536+0000","last_peered":"2024-09-10T20:41:17.662536+0000","last_clean":"2024-09-10T20:41:17.662536+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:41:17.662536+0000","last_undegraded":"2024-09-10T20:41:17.662536+0000","last_fullsized":"2024-09-10T20:41:17.662536+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665900+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:41:17.665900+0000","last_peered":"2024-09-10T20:41:17.665900+0000","last_clean":"2024-09-10T20:41:17.665900+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:41:17.665900+0000","last_undegraded":"2024-09-10T20:41:17.665900+0000","last_fullsized":"2024-09-10T20:41:17.665900+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665840+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:41:17.665840+0000","last_peered":"2024-09-10T20:41:17.665840+0000","last_clean":"2024-09-10T20:41:17.665840+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:41:17.665840+0000","last_undegraded":"2024-09-10T20:41:17.665840+0000","last_fullsized":"2024-09-10T20:41:17.665840+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27843,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665757+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:41:17.665757+0000","last_peered":"2024-09-10T20:41:17.665757+0000","last_clean":"2024-09-10T20:41:17.665757+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:41:17.665757+0000","last_undegraded":"2024-09-10T20:41:17.665757+0000","last_fullsized":"2024-09-10T20:41:17.665757+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":133,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663305+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:41:17.663305+0000","last_peered":"2024-09-10T20:41:17.663305+0000","last_clean":"2024-09-10T20:41:17.663305+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:41:17.663305+0000","last_undegraded":"2024-09-10T20:41:17.663305+0000","last_fullsized":"2024-09-10T20:41:17.663305+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":26250,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:30.247049+0000","last_change":"2024-09-10T20:41:15.931451+0000","last_active":"2024-09-10T20:41:30.247049+0000","last_peered":"2024-09-10T20:41:30.247049+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:41:30.247049+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1169}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665623+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:41:17.665623+0000","last_peered":"2024-09-10T20:41:17.665623+0000","last_clean":"2024-09-10T20:41:17.665623+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:41:17.665623+0000","last_undegraded":"2024-09-10T20:41:17.665623+0000","last_fullsized":"2024-09-10T20:41:17.665623+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665817+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:41:17.665817+0000","last_peered":"2024-09-10T20:41:17.665817+0000","last_clean":"2024-09-10T20:41:17.665817+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:41:17.665817+0000","last_undegraded":"2024-09-10T20:41:17.665817+0000","last_fullsized":"2024-09-10T20:41:17.665817+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27673,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665728+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:41:17.665728+0000","last_peered":"2024-09-10T20:41:17.665728+0000","last_clean":"2024-09-10T20:41:17.665728+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:41:17.665728+0000","last_undegraded":"2024-09-10T20:41:17.665728+0000","last_fullsized":"2024-09-10T20:41:17.665728+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4732,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34798,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":137601024,"data_stored":137586572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182555,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517228,"kb_used_data":58004,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854612,"statfs":{"total":96636764160,"available":96107122688,"internally_reserved":0,"allocated":59396096,"data_stored":59065163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986327,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442988,"kb_used_data":78108,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93928852,"statfs":{"total":96636764160,"available":96183144448,"internally_reserved":0,"allocated":79982592,"data_stored":79659349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659273,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505632,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866208,"statfs":{"total":96636764160,"available":96118996992,"internally_reserved":0,"allocated":540672,"data_stored":202513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":517205678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":78721024,"data_stored":78721024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:41:35.438 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:35.803 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:35.804 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:35.819 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2024-09-10T20:41:35.746988+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3605,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35927,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1466408,"kb_used_data":133040,"kb_used_omap":12,"kb_used_meta":1333299,"kb_avail":281649112,"statfs":{"total":289910292480,"available":288408690688,"internally_reserved":0,"allocated":136232960,"data_stored":135240625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1365298178},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2875,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2875,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.669736"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":28000,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663983+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:41:17.663983+0000","last_peered":"2024-09-10T20:41:17.663983+0000","last_clean":"2024-09-10T20:41:17.663983+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:41:17.663983+0000","last_undegraded":"2024-09-10T20:41:17.663983+0000","last_fullsized":"2024-09-10T20:41:17.663983+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21092,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663678+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:41:17.663678+0000","last_peered":"2024-09-10T20:41:17.663678+0000","last_clean":"2024-09-10T20:41:17.663678+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:41:17.663678+0000","last_undegraded":"2024-09-10T20:41:17.663678+0000","last_fullsized":"2024-09-10T20:41:17.663678+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28104,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665777+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:41:17.665777+0000","last_peered":"2024-09-10T20:41:17.665777+0000","last_clean":"2024-09-10T20:41:17.665777+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:41:17.665777+0000","last_undegraded":"2024-09-10T20:41:17.665777+0000","last_fullsized":"2024-09-10T20:41:17.665777+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20724,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.664035+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:41:17.664035+0000","last_peered":"2024-09-10T20:41:17.664035+0000","last_clean":"2024-09-10T20:41:17.664035+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:41:17.664035+0000","last_undegraded":"2024-09-10T20:41:17.664035+0000","last_fullsized":"2024-09-10T20:41:17.664035+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21235,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663037+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:41:17.663037+0000","last_peered":"2024-09-10T20:41:17.663037+0000","last_clean":"2024-09-10T20:41:17.663037+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:41:17.663037+0000","last_undegraded":"2024-09-10T20:41:17.663037+0000","last_fullsized":"2024-09-10T20:41:17.663037+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20753,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:17.662327+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:41:17.662327+0000","last_peered":"2024-09-10T20:41:17.662327+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:41:17.662327+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20911,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.750111+0000","last_change":"2024-09-10T20:41:17.750026+0000","last_active":"2024-09-10T20:41:17.750111+0000","last_peered":"2024-09-10T20:41:17.750111+0000","last_clean":"2024-09-10T20:41:17.750111+0000","last_became_active":"2024-09-10T20:41:17.668013+0000","last_became_peered":"2024-09-10T20:41:17.668013+0000","last_unstale":"2024-09-10T20:41:17.750111+0000","last_undegraded":"2024-09-10T20:41:17.750111+0000","last_fullsized":"2024-09-10T20:41:17.750111+0000","mapping_epoch":62,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:55.885282+0000","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20900,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662099+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:41:17.662099+0000","last_peered":"2024-09-10T20:41:17.662099+0000","last_clean":"2024-09-10T20:41:17.662099+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:41:17.662099+0000","last_undegraded":"2024-09-10T20:41:17.662099+0000","last_fullsized":"2024-09-10T20:41:17.662099+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665321+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:41:17.665321+0000","last_peered":"2024-09-10T20:41:17.665321+0000","last_clean":"2024-09-10T20:41:17.665321+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:41:17.665321+0000","last_undegraded":"2024-09-10T20:41:17.665321+0000","last_fullsized":"2024-09-10T20:41:17.665321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22142,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665632+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:41:17.665632+0000","last_peered":"2024-09-10T20:41:17.665632+0000","last_clean":"2024-09-10T20:41:17.665632+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:41:17.665632+0000","last_undegraded":"2024-09-10T20:41:17.665632+0000","last_fullsized":"2024-09-10T20:41:17.665632+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":133,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663417+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:41:17.663417+0000","last_peered":"2024-09-10T20:41:17.663417+0000","last_clean":"2024-09-10T20:41:17.663417+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:41:17.663417+0000","last_undegraded":"2024-09-10T20:41:17.663417+0000","last_fullsized":"2024-09-10T20:41:17.663417+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28129,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662845+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:41:17.662845+0000","last_peered":"2024-09-10T20:41:17.662845+0000","last_clean":"2024-09-10T20:41:17.662845+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:41:17.662845+0000","last_undegraded":"2024-09-10T20:41:17.662845+0000","last_fullsized":"2024-09-10T20:41:17.662845+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662267+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:41:17.662267+0000","last_peered":"2024-09-10T20:41:17.662267+0000","last_clean":"2024-09-10T20:41:17.662267+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:41:17.662267+0000","last_undegraded":"2024-09-10T20:41:17.662267+0000","last_fullsized":"2024-09-10T20:41:17.662267+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28011,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663503+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:41:17.663503+0000","last_peered":"2024-09-10T20:41:17.663503+0000","last_clean":"2024-09-10T20:41:17.663503+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:41:17.663503+0000","last_undegraded":"2024-09-10T20:41:17.663503+0000","last_fullsized":"2024-09-10T20:41:17.663503+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665485+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:41:17.665485+0000","last_peered":"2024-09-10T20:41:17.665485+0000","last_clean":"2024-09-10T20:41:17.665485+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:41:17.665485+0000","last_undegraded":"2024-09-10T20:41:17.665485+0000","last_fullsized":"2024-09-10T20:41:17.665485+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28065,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662536+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:41:17.662536+0000","last_peered":"2024-09-10T20:41:17.662536+0000","last_clean":"2024-09-10T20:41:17.662536+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:41:17.662536+0000","last_undegraded":"2024-09-10T20:41:17.662536+0000","last_fullsized":"2024-09-10T20:41:17.662536+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665900+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:41:17.665900+0000","last_peered":"2024-09-10T20:41:17.665900+0000","last_clean":"2024-09-10T20:41:17.665900+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:41:17.665900+0000","last_undegraded":"2024-09-10T20:41:17.665900+0000","last_fullsized":"2024-09-10T20:41:17.665900+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665840+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:41:17.665840+0000","last_peered":"2024-09-10T20:41:17.665840+0000","last_clean":"2024-09-10T20:41:17.665840+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:41:17.665840+0000","last_undegraded":"2024-09-10T20:41:17.665840+0000","last_fullsized":"2024-09-10T20:41:17.665840+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27843,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665757+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:41:17.665757+0000","last_peered":"2024-09-10T20:41:17.665757+0000","last_clean":"2024-09-10T20:41:17.665757+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:41:17.665757+0000","last_undegraded":"2024-09-10T20:41:17.665757+0000","last_fullsized":"2024-09-10T20:41:17.665757+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":133,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663305+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:41:17.663305+0000","last_peered":"2024-09-10T20:41:17.663305+0000","last_clean":"2024-09-10T20:41:17.663305+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:41:17.663305+0000","last_undegraded":"2024-09-10T20:41:17.663305+0000","last_fullsized":"2024-09-10T20:41:17.663305+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":28504,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:35.247757+0000","last_change":"2024-09-10T20:41:15.931451+0000","last_active":"2024-09-10T20:41:35.247757+0000","last_peered":"2024-09-10T20:41:35.247757+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:41:35.247757+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3673,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-42}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665623+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:41:17.665623+0000","last_peered":"2024-09-10T20:41:17.665623+0000","last_clean":"2024-09-10T20:41:17.665623+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:41:17.665623+0000","last_undegraded":"2024-09-10T20:41:17.665623+0000","last_fullsized":"2024-09-10T20:41:17.665623+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665817+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:41:17.665817+0000","last_peered":"2024-09-10T20:41:17.665817+0000","last_clean":"2024-09-10T20:41:17.665817+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:41:17.665817+0000","last_undegraded":"2024-09-10T20:41:17.665817+0000","last_fullsized":"2024-09-10T20:41:17.665817+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27673,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665728+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:41:17.665728+0000","last_peered":"2024-09-10T20:41:17.665728+0000","last_clean":"2024-09-10T20:41:17.665728+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:41:17.665728+0000","last_undegraded":"2024-09-10T20:41:17.665728+0000","last_fullsized":"2024-09-10T20:41:17.665728+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3605,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35925,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":133914624,"data_stored":133900172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182556,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517228,"kb_used_data":58004,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854612,"statfs":{"total":96636764160,"available":96107122688,"internally_reserved":0,"allocated":59396096,"data_stored":59065163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986328,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":439388,"kb_used_data":74508,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93932452,"statfs":{"total":96636764160,"available":96186830848,"internally_reserved":0,"allocated":76296192,"data_stored":75972949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659274,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509792,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862048,"statfs":{"total":96636764160,"available":96114737152,"internally_reserved":0,"allocated":540672,"data_stored":202513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521465518},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":75034624,"data_stored":75034624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:41:35.821 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:36.148 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:36.149 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:36.161 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2024-09-10T20:41:35.746988+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3605,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35927,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1466408,"kb_used_data":133040,"kb_used_omap":12,"kb_used_meta":1333299,"kb_avail":281649112,"statfs":{"total":289910292480,"available":288408690688,"internally_reserved":0,"allocated":136232960,"data_stored":135240625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1365298178},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2875,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2875,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.669736"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":28000,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663983+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:41:17.663983+0000","last_peered":"2024-09-10T20:41:17.663983+0000","last_clean":"2024-09-10T20:41:17.663983+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:41:17.663983+0000","last_undegraded":"2024-09-10T20:41:17.663983+0000","last_fullsized":"2024-09-10T20:41:17.663983+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21092,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663678+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:41:17.663678+0000","last_peered":"2024-09-10T20:41:17.663678+0000","last_clean":"2024-09-10T20:41:17.663678+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:41:17.663678+0000","last_undegraded":"2024-09-10T20:41:17.663678+0000","last_fullsized":"2024-09-10T20:41:17.663678+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28104,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665777+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:41:17.665777+0000","last_peered":"2024-09-10T20:41:17.665777+0000","last_clean":"2024-09-10T20:41:17.665777+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:41:17.665777+0000","last_undegraded":"2024-09-10T20:41:17.665777+0000","last_fullsized":"2024-09-10T20:41:17.665777+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20724,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.664035+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:41:17.664035+0000","last_peered":"2024-09-10T20:41:17.664035+0000","last_clean":"2024-09-10T20:41:17.664035+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:41:17.664035+0000","last_undegraded":"2024-09-10T20:41:17.664035+0000","last_fullsized":"2024-09-10T20:41:17.664035+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21235,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663037+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:41:17.663037+0000","last_peered":"2024-09-10T20:41:17.663037+0000","last_clean":"2024-09-10T20:41:17.663037+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:41:17.663037+0000","last_undegraded":"2024-09-10T20:41:17.663037+0000","last_fullsized":"2024-09-10T20:41:17.663037+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20753,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:17.662327+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:41:17.662327+0000","last_peered":"2024-09-10T20:41:17.662327+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:41:17.662327+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20911,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.750111+0000","last_change":"2024-09-10T20:41:17.750026+0000","last_active":"2024-09-10T20:41:17.750111+0000","last_peered":"2024-09-10T20:41:17.750111+0000","last_clean":"2024-09-10T20:41:17.750111+0000","last_became_active":"2024-09-10T20:41:17.668013+0000","last_became_peered":"2024-09-10T20:41:17.668013+0000","last_unstale":"2024-09-10T20:41:17.750111+0000","last_undegraded":"2024-09-10T20:41:17.750111+0000","last_fullsized":"2024-09-10T20:41:17.750111+0000","mapping_epoch":62,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:55.885282+0000","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20900,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662099+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:41:17.662099+0000","last_peered":"2024-09-10T20:41:17.662099+0000","last_clean":"2024-09-10T20:41:17.662099+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:41:17.662099+0000","last_undegraded":"2024-09-10T20:41:17.662099+0000","last_fullsized":"2024-09-10T20:41:17.662099+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665321+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:41:17.665321+0000","last_peered":"2024-09-10T20:41:17.665321+0000","last_clean":"2024-09-10T20:41:17.665321+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:41:17.665321+0000","last_undegraded":"2024-09-10T20:41:17.665321+0000","last_fullsized":"2024-09-10T20:41:17.665321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22142,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665632+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:41:17.665632+0000","last_peered":"2024-09-10T20:41:17.665632+0000","last_clean":"2024-09-10T20:41:17.665632+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:41:17.665632+0000","last_undegraded":"2024-09-10T20:41:17.665632+0000","last_fullsized":"2024-09-10T20:41:17.665632+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":133,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663417+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:41:17.663417+0000","last_peered":"2024-09-10T20:41:17.663417+0000","last_clean":"2024-09-10T20:41:17.663417+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:41:17.663417+0000","last_undegraded":"2024-09-10T20:41:17.663417+0000","last_fullsized":"2024-09-10T20:41:17.663417+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28129,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662845+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:41:17.662845+0000","last_peered":"2024-09-10T20:41:17.662845+0000","last_clean":"2024-09-10T20:41:17.662845+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:41:17.662845+0000","last_undegraded":"2024-09-10T20:41:17.662845+0000","last_fullsized":"2024-09-10T20:41:17.662845+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662267+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:41:17.662267+0000","last_peered":"2024-09-10T20:41:17.662267+0000","last_clean":"2024-09-10T20:41:17.662267+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:41:17.662267+0000","last_undegraded":"2024-09-10T20:41:17.662267+0000","last_fullsized":"2024-09-10T20:41:17.662267+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28011,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663503+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:41:17.663503+0000","last_peered":"2024-09-10T20:41:17.663503+0000","last_clean":"2024-09-10T20:41:17.663503+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:41:17.663503+0000","last_undegraded":"2024-09-10T20:41:17.663503+0000","last_fullsized":"2024-09-10T20:41:17.663503+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665485+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:41:17.665485+0000","last_peered":"2024-09-10T20:41:17.665485+0000","last_clean":"2024-09-10T20:41:17.665485+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:41:17.665485+0000","last_undegraded":"2024-09-10T20:41:17.665485+0000","last_fullsized":"2024-09-10T20:41:17.665485+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28065,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662536+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:41:17.662536+0000","last_peered":"2024-09-10T20:41:17.662536+0000","last_clean":"2024-09-10T20:41:17.662536+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:41:17.662536+0000","last_undegraded":"2024-09-10T20:41:17.662536+0000","last_fullsized":"2024-09-10T20:41:17.662536+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665900+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:41:17.665900+0000","last_peered":"2024-09-10T20:41:17.665900+0000","last_clean":"2024-09-10T20:41:17.665900+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:41:17.665900+0000","last_undegraded":"2024-09-10T20:41:17.665900+0000","last_fullsized":"2024-09-10T20:41:17.665900+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665840+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:41:17.665840+0000","last_peered":"2024-09-10T20:41:17.665840+0000","last_clean":"2024-09-10T20:41:17.665840+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:41:17.665840+0000","last_undegraded":"2024-09-10T20:41:17.665840+0000","last_fullsized":"2024-09-10T20:41:17.665840+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27843,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665757+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:41:17.665757+0000","last_peered":"2024-09-10T20:41:17.665757+0000","last_clean":"2024-09-10T20:41:17.665757+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:41:17.665757+0000","last_undegraded":"2024-09-10T20:41:17.665757+0000","last_fullsized":"2024-09-10T20:41:17.665757+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":133,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663305+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:41:17.663305+0000","last_peered":"2024-09-10T20:41:17.663305+0000","last_clean":"2024-09-10T20:41:17.663305+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:41:17.663305+0000","last_undegraded":"2024-09-10T20:41:17.663305+0000","last_fullsized":"2024-09-10T20:41:17.663305+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":28504,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:35.247757+0000","last_change":"2024-09-10T20:41:15.931451+0000","last_active":"2024-09-10T20:41:35.247757+0000","last_peered":"2024-09-10T20:41:35.247757+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:41:35.247757+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3673,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-42}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665623+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:41:17.665623+0000","last_peered":"2024-09-10T20:41:17.665623+0000","last_clean":"2024-09-10T20:41:17.665623+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:41:17.665623+0000","last_undegraded":"2024-09-10T20:41:17.665623+0000","last_fullsized":"2024-09-10T20:41:17.665623+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665817+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:41:17.665817+0000","last_peered":"2024-09-10T20:41:17.665817+0000","last_clean":"2024-09-10T20:41:17.665817+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:41:17.665817+0000","last_undegraded":"2024-09-10T20:41:17.665817+0000","last_fullsized":"2024-09-10T20:41:17.665817+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27673,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665728+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:41:17.665728+0000","last_peered":"2024-09-10T20:41:17.665728+0000","last_clean":"2024-09-10T20:41:17.665728+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:41:17.665728+0000","last_undegraded":"2024-09-10T20:41:17.665728+0000","last_fullsized":"2024-09-10T20:41:17.665728+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3605,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35925,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":133914624,"data_stored":133900172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182556,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517228,"kb_used_data":58004,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854612,"statfs":{"total":96636764160,"available":96107122688,"internally_reserved":0,"allocated":59396096,"data_stored":59065163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986328,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":439388,"kb_used_data":74508,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93932452,"statfs":{"total":96636764160,"available":96186830848,"internally_reserved":0,"allocated":76296192,"data_stored":75972949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659274,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509792,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862048,"statfs":{"total":96636764160,"available":96114737152,"internally_reserved":0,"allocated":540672,"data_stored":202513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521465518},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":75034624,"data_stored":75034624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:41:36.163 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:41:36.163 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20471", 'reported_seq': 20753, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:41:17.662327+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:41:17.662327+0000', 'last_peered': '2024-09-10T20:41:17.662327+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:41:17.662327+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17335, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3561}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:41:36.163 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T20:41:36.163 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "51'20986", 'reported_seq': 28504, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:41:35.247757+0000', 'last_change': '2024-09-10T20:41:15.931451+0000', 'last_active': '2024-09-10T20:41:35.247757+0000', 'last_peered': '2024-09-10T20:41:35.247757+0000', 'last_clean': '2024-09-10T20:30:03.280949+0000', 'last_became_active': '2024-09-10T20:30:05.781305+0000', 'last_became_peered': '2024-09-10T20:30:05.781305+0000', 'last_unstale': '2024-09-10T20:41:35.247757+0000', 'last_undegraded': '2024-09-10T20:30:05.390091+0000', 'last_fullsized': '2024-09-10T20:30:05.308903+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3233", 'last_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:54.754780+0000', 'objects_scrubbed': 3169, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 42, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17813, 'num_write_kb': 29288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3673, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -42}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:41:36.163 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:41:36.579 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:36.579 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 4732/6 objects degraded (78866.667%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":808,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":64,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1501601792,"bytes_avail":288408690688,"bytes_total":289910292480,"degraded_objects":4732,"degraded_total":6,"degraded_ratio":788.66666666666652,"recovering_objects_per_sec":145,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1748,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:41:36.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:41:36.595 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:36.999 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:37.000 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:37.015 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":462,"stamp":"2024-09-10T20:41:36.836742+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3605,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35927,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1466408,"kb_used_data":133040,"kb_used_omap":12,"kb_used_meta":1333299,"kb_avail":281649112,"statfs":{"total":289910292480,"available":288408690688,"internally_reserved":0,"allocated":136232960,"data_stored":135240625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1365298178},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1978,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1978,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.759182"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":28000,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663983+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:41:17.663983+0000","last_peered":"2024-09-10T20:41:17.663983+0000","last_clean":"2024-09-10T20:41:17.663983+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:41:17.663983+0000","last_undegraded":"2024-09-10T20:41:17.663983+0000","last_fullsized":"2024-09-10T20:41:17.663983+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21092,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663678+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:41:17.663678+0000","last_peered":"2024-09-10T20:41:17.663678+0000","last_clean":"2024-09-10T20:41:17.663678+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:41:17.663678+0000","last_undegraded":"2024-09-10T20:41:17.663678+0000","last_fullsized":"2024-09-10T20:41:17.663678+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28104,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665777+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:41:17.665777+0000","last_peered":"2024-09-10T20:41:17.665777+0000","last_clean":"2024-09-10T20:41:17.665777+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:41:17.665777+0000","last_undegraded":"2024-09-10T20:41:17.665777+0000","last_fullsized":"2024-09-10T20:41:17.665777+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20724,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.664035+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:41:17.664035+0000","last_peered":"2024-09-10T20:41:17.664035+0000","last_clean":"2024-09-10T20:41:17.664035+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:41:17.664035+0000","last_undegraded":"2024-09-10T20:41:17.664035+0000","last_fullsized":"2024-09-10T20:41:17.664035+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21235,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663037+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:41:17.663037+0000","last_peered":"2024-09-10T20:41:17.663037+0000","last_clean":"2024-09-10T20:41:17.663037+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:41:17.663037+0000","last_undegraded":"2024-09-10T20:41:17.663037+0000","last_fullsized":"2024-09-10T20:41:17.663037+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20753,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:17.662327+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:41:17.662327+0000","last_peered":"2024-09-10T20:41:17.662327+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:41:17.662327+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20911,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.750111+0000","last_change":"2024-09-10T20:41:17.750026+0000","last_active":"2024-09-10T20:41:17.750111+0000","last_peered":"2024-09-10T20:41:17.750111+0000","last_clean":"2024-09-10T20:41:17.750111+0000","last_became_active":"2024-09-10T20:41:17.668013+0000","last_became_peered":"2024-09-10T20:41:17.668013+0000","last_unstale":"2024-09-10T20:41:17.750111+0000","last_undegraded":"2024-09-10T20:41:17.750111+0000","last_fullsized":"2024-09-10T20:41:17.750111+0000","mapping_epoch":62,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:55.885282+0000","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20900,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662099+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:41:17.662099+0000","last_peered":"2024-09-10T20:41:17.662099+0000","last_clean":"2024-09-10T20:41:17.662099+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:41:17.662099+0000","last_undegraded":"2024-09-10T20:41:17.662099+0000","last_fullsized":"2024-09-10T20:41:17.662099+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665321+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:41:17.665321+0000","last_peered":"2024-09-10T20:41:17.665321+0000","last_clean":"2024-09-10T20:41:17.665321+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:41:17.665321+0000","last_undegraded":"2024-09-10T20:41:17.665321+0000","last_fullsized":"2024-09-10T20:41:17.665321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22142,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665632+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:41:17.665632+0000","last_peered":"2024-09-10T20:41:17.665632+0000","last_clean":"2024-09-10T20:41:17.665632+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:41:17.665632+0000","last_undegraded":"2024-09-10T20:41:17.665632+0000","last_fullsized":"2024-09-10T20:41:17.665632+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":133,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663417+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:41:17.663417+0000","last_peered":"2024-09-10T20:41:17.663417+0000","last_clean":"2024-09-10T20:41:17.663417+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:41:17.663417+0000","last_undegraded":"2024-09-10T20:41:17.663417+0000","last_fullsized":"2024-09-10T20:41:17.663417+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28129,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662845+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:41:17.662845+0000","last_peered":"2024-09-10T20:41:17.662845+0000","last_clean":"2024-09-10T20:41:17.662845+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:41:17.662845+0000","last_undegraded":"2024-09-10T20:41:17.662845+0000","last_fullsized":"2024-09-10T20:41:17.662845+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662267+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:41:17.662267+0000","last_peered":"2024-09-10T20:41:17.662267+0000","last_clean":"2024-09-10T20:41:17.662267+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:41:17.662267+0000","last_undegraded":"2024-09-10T20:41:17.662267+0000","last_fullsized":"2024-09-10T20:41:17.662267+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28011,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663503+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:41:17.663503+0000","last_peered":"2024-09-10T20:41:17.663503+0000","last_clean":"2024-09-10T20:41:17.663503+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:41:17.663503+0000","last_undegraded":"2024-09-10T20:41:17.663503+0000","last_fullsized":"2024-09-10T20:41:17.663503+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665485+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:41:17.665485+0000","last_peered":"2024-09-10T20:41:17.665485+0000","last_clean":"2024-09-10T20:41:17.665485+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:41:17.665485+0000","last_undegraded":"2024-09-10T20:41:17.665485+0000","last_fullsized":"2024-09-10T20:41:17.665485+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28065,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662536+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:41:17.662536+0000","last_peered":"2024-09-10T20:41:17.662536+0000","last_clean":"2024-09-10T20:41:17.662536+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:41:17.662536+0000","last_undegraded":"2024-09-10T20:41:17.662536+0000","last_fullsized":"2024-09-10T20:41:17.662536+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665900+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:41:17.665900+0000","last_peered":"2024-09-10T20:41:17.665900+0000","last_clean":"2024-09-10T20:41:17.665900+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:41:17.665900+0000","last_undegraded":"2024-09-10T20:41:17.665900+0000","last_fullsized":"2024-09-10T20:41:17.665900+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665840+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:41:17.665840+0000","last_peered":"2024-09-10T20:41:17.665840+0000","last_clean":"2024-09-10T20:41:17.665840+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:41:17.665840+0000","last_undegraded":"2024-09-10T20:41:17.665840+0000","last_fullsized":"2024-09-10T20:41:17.665840+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27843,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665757+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:41:17.665757+0000","last_peered":"2024-09-10T20:41:17.665757+0000","last_clean":"2024-09-10T20:41:17.665757+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:41:17.665757+0000","last_undegraded":"2024-09-10T20:41:17.665757+0000","last_fullsized":"2024-09-10T20:41:17.665757+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":133,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663305+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:41:17.663305+0000","last_peered":"2024-09-10T20:41:17.663305+0000","last_clean":"2024-09-10T20:41:17.663305+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:41:17.663305+0000","last_undegraded":"2024-09-10T20:41:17.663305+0000","last_fullsized":"2024-09-10T20:41:17.663305+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":28504,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:35.247757+0000","last_change":"2024-09-10T20:41:15.931451+0000","last_active":"2024-09-10T20:41:35.247757+0000","last_peered":"2024-09-10T20:41:35.247757+0000","last_clean":"2024-09-10T20:30:03.280949+0000","last_became_active":"2024-09-10T20:30:05.781305+0000","last_became_peered":"2024-09-10T20:30:05.781305+0000","last_unstale":"2024-09-10T20:41:35.247757+0000","last_undegraded":"2024-09-10T20:30:05.390091+0000","last_fullsized":"2024-09-10T20:30:05.308903+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3673,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-42}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665623+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:41:17.665623+0000","last_peered":"2024-09-10T20:41:17.665623+0000","last_clean":"2024-09-10T20:41:17.665623+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:41:17.665623+0000","last_undegraded":"2024-09-10T20:41:17.665623+0000","last_fullsized":"2024-09-10T20:41:17.665623+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665817+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:41:17.665817+0000","last_peered":"2024-09-10T20:41:17.665817+0000","last_clean":"2024-09-10T20:41:17.665817+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:41:17.665817+0000","last_undegraded":"2024-09-10T20:41:17.665817+0000","last_fullsized":"2024-09-10T20:41:17.665817+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27673,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.665728+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:41:17.665728+0000","last_peered":"2024-09-10T20:41:17.665728+0000","last_clean":"2024-09-10T20:41:17.665728+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:41:17.665728+0000","last_undegraded":"2024-09-10T20:41:17.665728+0000","last_fullsized":"2024-09-10T20:41:17.665728+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3605,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35925,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":133914624,"data_stored":133900172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182556,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517228,"kb_used_data":58004,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854612,"statfs":{"total":96636764160,"available":96107122688,"internally_reserved":0,"allocated":59396096,"data_stored":59065163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986328,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":439388,"kb_used_data":74508,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93932452,"statfs":{"total":96636764160,"available":96186830848,"internally_reserved":0,"allocated":76296192,"data_stored":75972949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659274,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509792,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862048,"statfs":{"total":96636764160,"available":96114737152,"internally_reserved":0,"allocated":540672,"data_stored":202513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521465518},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":75034624,"data_stored":75034624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:41:40.017 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:40.351 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:40.352 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:40.364 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":464,"stamp":"2024-09-10T20:41:39.079462+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35969,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1458092,"kb_used_data":133040,"kb_used_omap":12,"kb_used_meta":1324979,"kb_avail":281657428,"statfs":{"total":289910292480,"available":288417206272,"internally_reserved":0,"allocated":136232960,"data_stored":135242834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1356778498},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001306"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":28000,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663983+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:41:17.663983+0000","last_peered":"2024-09-10T20:41:17.663983+0000","last_clean":"2024-09-10T20:41:17.663983+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:41:17.663983+0000","last_undegraded":"2024-09-10T20:41:17.663983+0000","last_fullsized":"2024-09-10T20:41:17.663983+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21092,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663678+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:41:17.663678+0000","last_peered":"2024-09-10T20:41:17.663678+0000","last_clean":"2024-09-10T20:41:17.663678+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:41:17.663678+0000","last_undegraded":"2024-09-10T20:41:17.663678+0000","last_fullsized":"2024-09-10T20:41:17.663678+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28108,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841579+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:41:36.841579+0000","last_peered":"2024-09-10T20:41:36.841579+0000","last_clean":"2024-09-10T20:41:36.841579+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:41:36.841579+0000","last_undegraded":"2024-09-10T20:41:36.841579+0000","last_fullsized":"2024-09-10T20:41:36.841579+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20724,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.664035+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:41:17.664035+0000","last_peered":"2024-09-10T20:41:17.664035+0000","last_clean":"2024-09-10T20:41:17.664035+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:41:17.664035+0000","last_undegraded":"2024-09-10T20:41:17.664035+0000","last_fullsized":"2024-09-10T20:41:17.664035+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21235,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663037+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:41:17.663037+0000","last_peered":"2024-09-10T20:41:17.663037+0000","last_clean":"2024-09-10T20:41:17.663037+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:41:17.663037+0000","last_undegraded":"2024-09-10T20:41:17.663037+0000","last_fullsized":"2024-09-10T20:41:17.663037+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20753,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:17.662327+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:41:17.662327+0000","last_peered":"2024-09-10T20:41:17.662327+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:41:17.662327+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20915,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841297+0000","last_change":"2024-09-10T20:41:17.750026+0000","last_active":"2024-09-10T20:41:36.841297+0000","last_peered":"2024-09-10T20:41:36.841297+0000","last_clean":"2024-09-10T20:41:36.841297+0000","last_became_active":"2024-09-10T20:41:17.668013+0000","last_became_peered":"2024-09-10T20:41:17.668013+0000","last_unstale":"2024-09-10T20:41:36.841297+0000","last_undegraded":"2024-09-10T20:41:36.841297+0000","last_fullsized":"2024-09-10T20:41:36.841297+0000","mapping_epoch":62,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:55.885282+0000","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20900,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662099+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:41:17.662099+0000","last_peered":"2024-09-10T20:41:17.662099+0000","last_clean":"2024-09-10T20:41:17.662099+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:41:17.662099+0000","last_undegraded":"2024-09-10T20:41:17.662099+0000","last_fullsized":"2024-09-10T20:41:17.662099+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841162+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:41:36.841162+0000","last_peered":"2024-09-10T20:41:36.841162+0000","last_clean":"2024-09-10T20:41:36.841162+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:41:36.841162+0000","last_undegraded":"2024-09-10T20:41:36.841162+0000","last_fullsized":"2024-09-10T20:41:36.841162+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22146,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841354+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:41:36.841354+0000","last_peered":"2024-09-10T20:41:36.841354+0000","last_clean":"2024-09-10T20:41:36.841354+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:41:36.841354+0000","last_undegraded":"2024-09-10T20:41:36.841354+0000","last_fullsized":"2024-09-10T20:41:36.841354+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":133,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663417+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:41:17.663417+0000","last_peered":"2024-09-10T20:41:17.663417+0000","last_clean":"2024-09-10T20:41:17.663417+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:41:17.663417+0000","last_undegraded":"2024-09-10T20:41:17.663417+0000","last_fullsized":"2024-09-10T20:41:17.663417+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28129,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662845+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:41:17.662845+0000","last_peered":"2024-09-10T20:41:17.662845+0000","last_clean":"2024-09-10T20:41:17.662845+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:41:17.662845+0000","last_undegraded":"2024-09-10T20:41:17.662845+0000","last_fullsized":"2024-09-10T20:41:17.662845+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662267+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:41:17.662267+0000","last_peered":"2024-09-10T20:41:17.662267+0000","last_clean":"2024-09-10T20:41:17.662267+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:41:17.662267+0000","last_undegraded":"2024-09-10T20:41:17.662267+0000","last_fullsized":"2024-09-10T20:41:17.662267+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28011,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663503+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:41:17.663503+0000","last_peered":"2024-09-10T20:41:17.663503+0000","last_clean":"2024-09-10T20:41:17.663503+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:41:17.663503+0000","last_undegraded":"2024-09-10T20:41:17.663503+0000","last_fullsized":"2024-09-10T20:41:17.663503+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":123,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841261+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:41:36.841261+0000","last_peered":"2024-09-10T20:41:36.841261+0000","last_clean":"2024-09-10T20:41:36.841261+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:41:36.841261+0000","last_undegraded":"2024-09-10T20:41:36.841261+0000","last_fullsized":"2024-09-10T20:41:36.841261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28065,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662536+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:41:17.662536+0000","last_peered":"2024-09-10T20:41:17.662536+0000","last_clean":"2024-09-10T20:41:17.662536+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:41:17.662536+0000","last_undegraded":"2024-09-10T20:41:17.662536+0000","last_fullsized":"2024-09-10T20:41:17.662536+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21125,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841491+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:41:36.841491+0000","last_peered":"2024-09-10T20:41:36.841491+0000","last_clean":"2024-09-10T20:41:36.841491+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:41:36.841491+0000","last_undegraded":"2024-09-10T20:41:36.841491+0000","last_fullsized":"2024-09-10T20:41:36.841491+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841466+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:41:36.841466+0000","last_peered":"2024-09-10T20:41:36.841466+0000","last_clean":"2024-09-10T20:41:36.841466+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:41:36.841466+0000","last_undegraded":"2024-09-10T20:41:36.841466+0000","last_fullsized":"2024-09-10T20:41:36.841466+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27847,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841464+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:41:36.841464+0000","last_peered":"2024-09-10T20:41:36.841464+0000","last_clean":"2024-09-10T20:41:36.841464+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:41:36.841464+0000","last_undegraded":"2024-09-10T20:41:36.841464+0000","last_fullsized":"2024-09-10T20:41:36.841464+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":133,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663305+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:41:17.663305+0000","last_peered":"2024-09-10T20:41:17.663305+0000","last_clean":"2024-09-10T20:41:17.663305+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:41:17.663305+0000","last_undegraded":"2024-09-10T20:41:17.663305+0000","last_fullsized":"2024-09-10T20:41:17.663305+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21261,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.929378+0000","last_change":"2024-09-10T20:41:36.928383+0000","last_active":"2024-09-10T20:41:36.929378+0000","last_peered":"2024-09-10T20:41:36.929378+0000","last_clean":"2024-09-10T20:41:36.929378+0000","last_became_active":"2024-09-10T20:41:36.842440+0000","last_became_peered":"2024-09-10T20:41:36.842440+0000","last_unstale":"2024-09-10T20:41:36.929378+0000","last_undegraded":"2024-09-10T20:41:36.929378+0000","last_fullsized":"2024-09-10T20:41:36.929378+0000","mapping_epoch":64,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.940798+0000","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3715,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841381+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:41:36.841381+0000","last_peered":"2024-09-10T20:41:36.841381+0000","last_clean":"2024-09-10T20:41:36.841381+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:41:36.841381+0000","last_undegraded":"2024-09-10T20:41:36.841381+0000","last_fullsized":"2024-09-10T20:41:36.841381+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841415+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:41:36.841415+0000","last_peered":"2024-09-10T20:41:36.841415+0000","last_clean":"2024-09-10T20:41:36.841415+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:41:36.841415+0000","last_undegraded":"2024-09-10T20:41:36.841415+0000","last_fullsized":"2024-09-10T20:41:36.841415+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27677,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841380+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:41:36.841380+0000","last_peered":"2024-09-10T20:41:36.841380+0000","last_clean":"2024-09-10T20:41:36.841380+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:41:36.841380+0000","last_undegraded":"2024-09-10T20:41:36.841380+0000","last_fullsized":"2024-09-10T20:41:36.841380+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35967,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":133906432,"data_stored":133899206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182556,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517228,"kb_used_data":58004,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854612,"statfs":{"total":96636764160,"available":96107122688,"internally_reserved":0,"allocated":59396096,"data_stored":59065163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986328,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":439388,"kb_used_data":74508,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93932452,"statfs":{"total":96636764160,"available":96186830848,"internally_reserved":0,"allocated":76296192,"data_stored":75972949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659275,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501476,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870364,"statfs":{"total":96636764160,"available":96123252736,"internally_reserved":0,"allocated":540672,"data_stored":204722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512945838},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":75034624,"data_stored":75034624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:41:40.365 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:40.682 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:40.682 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:40.698 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":464,"stamp":"2024-09-10T20:41:39.079462+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35969,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1458092,"kb_used_data":133040,"kb_used_omap":12,"kb_used_meta":1324979,"kb_avail":281657428,"statfs":{"total":289910292480,"available":288417206272,"internally_reserved":0,"allocated":136232960,"data_stored":135242834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1356778498},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001306"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":28000,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663983+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:41:17.663983+0000","last_peered":"2024-09-10T20:41:17.663983+0000","last_clean":"2024-09-10T20:41:17.663983+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:41:17.663983+0000","last_undegraded":"2024-09-10T20:41:17.663983+0000","last_fullsized":"2024-09-10T20:41:17.663983+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21092,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663678+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:41:17.663678+0000","last_peered":"2024-09-10T20:41:17.663678+0000","last_clean":"2024-09-10T20:41:17.663678+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:41:17.663678+0000","last_undegraded":"2024-09-10T20:41:17.663678+0000","last_fullsized":"2024-09-10T20:41:17.663678+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28108,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841579+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:41:36.841579+0000","last_peered":"2024-09-10T20:41:36.841579+0000","last_clean":"2024-09-10T20:41:36.841579+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:41:36.841579+0000","last_undegraded":"2024-09-10T20:41:36.841579+0000","last_fullsized":"2024-09-10T20:41:36.841579+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20724,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.664035+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:41:17.664035+0000","last_peered":"2024-09-10T20:41:17.664035+0000","last_clean":"2024-09-10T20:41:17.664035+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:41:17.664035+0000","last_undegraded":"2024-09-10T20:41:17.664035+0000","last_fullsized":"2024-09-10T20:41:17.664035+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21235,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663037+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:41:17.663037+0000","last_peered":"2024-09-10T20:41:17.663037+0000","last_clean":"2024-09-10T20:41:17.663037+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:41:17.663037+0000","last_undegraded":"2024-09-10T20:41:17.663037+0000","last_fullsized":"2024-09-10T20:41:17.663037+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20753,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:17.662327+0000","last_change":"2024-09-10T20:30:05.805563+0000","last_active":"2024-09-10T20:41:17.662327+0000","last_peered":"2024-09-10T20:41:17.662327+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:41:17.662327+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20915,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841297+0000","last_change":"2024-09-10T20:41:17.750026+0000","last_active":"2024-09-10T20:41:36.841297+0000","last_peered":"2024-09-10T20:41:36.841297+0000","last_clean":"2024-09-10T20:41:36.841297+0000","last_became_active":"2024-09-10T20:41:17.668013+0000","last_became_peered":"2024-09-10T20:41:17.668013+0000","last_unstale":"2024-09-10T20:41:36.841297+0000","last_undegraded":"2024-09-10T20:41:36.841297+0000","last_fullsized":"2024-09-10T20:41:36.841297+0000","mapping_epoch":62,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:55.885282+0000","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20900,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662099+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:41:17.662099+0000","last_peered":"2024-09-10T20:41:17.662099+0000","last_clean":"2024-09-10T20:41:17.662099+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:41:17.662099+0000","last_undegraded":"2024-09-10T20:41:17.662099+0000","last_fullsized":"2024-09-10T20:41:17.662099+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841162+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:41:36.841162+0000","last_peered":"2024-09-10T20:41:36.841162+0000","last_clean":"2024-09-10T20:41:36.841162+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:41:36.841162+0000","last_undegraded":"2024-09-10T20:41:36.841162+0000","last_fullsized":"2024-09-10T20:41:36.841162+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22146,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841354+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:41:36.841354+0000","last_peered":"2024-09-10T20:41:36.841354+0000","last_clean":"2024-09-10T20:41:36.841354+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:41:36.841354+0000","last_undegraded":"2024-09-10T20:41:36.841354+0000","last_fullsized":"2024-09-10T20:41:36.841354+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":133,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663417+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:41:17.663417+0000","last_peered":"2024-09-10T20:41:17.663417+0000","last_clean":"2024-09-10T20:41:17.663417+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:41:17.663417+0000","last_undegraded":"2024-09-10T20:41:17.663417+0000","last_fullsized":"2024-09-10T20:41:17.663417+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28129,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662845+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:41:17.662845+0000","last_peered":"2024-09-10T20:41:17.662845+0000","last_clean":"2024-09-10T20:41:17.662845+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:41:17.662845+0000","last_undegraded":"2024-09-10T20:41:17.662845+0000","last_fullsized":"2024-09-10T20:41:17.662845+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662267+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:41:17.662267+0000","last_peered":"2024-09-10T20:41:17.662267+0000","last_clean":"2024-09-10T20:41:17.662267+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:41:17.662267+0000","last_undegraded":"2024-09-10T20:41:17.662267+0000","last_fullsized":"2024-09-10T20:41:17.662267+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28011,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663503+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:41:17.663503+0000","last_peered":"2024-09-10T20:41:17.663503+0000","last_clean":"2024-09-10T20:41:17.663503+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:41:17.663503+0000","last_undegraded":"2024-09-10T20:41:17.663503+0000","last_fullsized":"2024-09-10T20:41:17.663503+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":123,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841261+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:41:36.841261+0000","last_peered":"2024-09-10T20:41:36.841261+0000","last_clean":"2024-09-10T20:41:36.841261+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:41:36.841261+0000","last_undegraded":"2024-09-10T20:41:36.841261+0000","last_fullsized":"2024-09-10T20:41:36.841261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28065,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.662536+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:41:17.662536+0000","last_peered":"2024-09-10T20:41:17.662536+0000","last_clean":"2024-09-10T20:41:17.662536+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:41:17.662536+0000","last_undegraded":"2024-09-10T20:41:17.662536+0000","last_fullsized":"2024-09-10T20:41:17.662536+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21125,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841491+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:41:36.841491+0000","last_peered":"2024-09-10T20:41:36.841491+0000","last_clean":"2024-09-10T20:41:36.841491+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:41:36.841491+0000","last_undegraded":"2024-09-10T20:41:36.841491+0000","last_fullsized":"2024-09-10T20:41:36.841491+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841466+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:41:36.841466+0000","last_peered":"2024-09-10T20:41:36.841466+0000","last_clean":"2024-09-10T20:41:36.841466+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:41:36.841466+0000","last_undegraded":"2024-09-10T20:41:36.841466+0000","last_fullsized":"2024-09-10T20:41:36.841466+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27847,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841464+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:41:36.841464+0000","last_peered":"2024-09-10T20:41:36.841464+0000","last_clean":"2024-09-10T20:41:36.841464+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:41:36.841464+0000","last_undegraded":"2024-09-10T20:41:36.841464+0000","last_fullsized":"2024-09-10T20:41:36.841464+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":133,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T20:41:17.663305+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:41:17.663305+0000","last_peered":"2024-09-10T20:41:17.663305+0000","last_clean":"2024-09-10T20:41:17.663305+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:41:17.663305+0000","last_undegraded":"2024-09-10T20:41:17.663305+0000","last_fullsized":"2024-09-10T20:41:17.663305+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21261,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.929378+0000","last_change":"2024-09-10T20:41:36.928383+0000","last_active":"2024-09-10T20:41:36.929378+0000","last_peered":"2024-09-10T20:41:36.929378+0000","last_clean":"2024-09-10T20:41:36.929378+0000","last_became_active":"2024-09-10T20:41:36.842440+0000","last_became_peered":"2024-09-10T20:41:36.842440+0000","last_unstale":"2024-09-10T20:41:36.929378+0000","last_undegraded":"2024-09-10T20:41:36.929378+0000","last_fullsized":"2024-09-10T20:41:36.929378+0000","mapping_epoch":64,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.940798+0000","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3715,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841381+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:41:36.841381+0000","last_peered":"2024-09-10T20:41:36.841381+0000","last_clean":"2024-09-10T20:41:36.841381+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:41:36.841381+0000","last_undegraded":"2024-09-10T20:41:36.841381+0000","last_fullsized":"2024-09-10T20:41:36.841381+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841415+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:41:36.841415+0000","last_peered":"2024-09-10T20:41:36.841415+0000","last_clean":"2024-09-10T20:41:36.841415+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:41:36.841415+0000","last_undegraded":"2024-09-10T20:41:36.841415+0000","last_fullsized":"2024-09-10T20:41:36.841415+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27677,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841380+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:41:36.841380+0000","last_peered":"2024-09-10T20:41:36.841380+0000","last_clean":"2024-09-10T20:41:36.841380+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:41:36.841380+0000","last_undegraded":"2024-09-10T20:41:36.841380+0000","last_fullsized":"2024-09-10T20:41:36.841380+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35967,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":133906432,"data_stored":133899206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182556,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517228,"kb_used_data":58004,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854612,"statfs":{"total":96636764160,"available":96107122688,"internally_reserved":0,"allocated":59396096,"data_stored":59065163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986328,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":439388,"kb_used_data":74508,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93932452,"statfs":{"total":96636764160,"available":96186830848,"internally_reserved":0,"allocated":76296192,"data_stored":75972949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659275,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501476,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870364,"statfs":{"total":96636764160,"available":96123252736,"internally_reserved":0,"allocated":540672,"data_stored":204722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512945838},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":75034624,"data_stored":75034624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:41:40.699 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:41:40.699 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20471", 'reported_seq': 20753, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:41:17.662327+0000', 'last_change': '2024-09-10T20:30:05.805563+0000', 'last_active': '2024-09-10T20:41:17.662327+0000', 'last_peered': '2024-09-10T20:41:17.662327+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:41:17.662327+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17335, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3561}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:41:40.699 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:41:41.091 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:41.092 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 3563/6 objects degraded (59383.333%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":812,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1493086208,"bytes_avail":288417206272,"bytes_total":289910292480,"degraded_objects":3563,"degraded_total":6,"degraded_ratio":593.83333333333337,"recovering_objects_per_sec":146,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1169,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:41:41.105 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:41:41.106 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:41.411 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:41.411 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:41.424 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":465,"stamp":"2024-09-10T20:41:41.079814+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36729,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1454164,"kb_used_data":129116,"kb_used_omap":12,"kb_used_meta":1324979,"kb_avail":281661356,"statfs":{"total":289910292480,"available":288421228544,"internally_reserved":0,"allocated":132214784,"data_stored":131204548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1356778498},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1929,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1929,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001313"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":28004,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840754+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:41:36.840754+0000","last_peered":"2024-09-10T20:41:36.840754+0000","last_clean":"2024-09-10T20:41:36.840754+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:41:36.840754+0000","last_undegraded":"2024-09-10T20:41:36.840754+0000","last_fullsized":"2024-09-10T20:41:36.840754+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21096,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840133+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:41:36.840133+0000","last_peered":"2024-09-10T20:41:36.840133+0000","last_clean":"2024-09-10T20:41:36.840133+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:41:36.840133+0000","last_undegraded":"2024-09-10T20:41:36.840133+0000","last_fullsized":"2024-09-10T20:41:36.840133+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28108,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841579+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:41:36.841579+0000","last_peered":"2024-09-10T20:41:36.841579+0000","last_clean":"2024-09-10T20:41:36.841579+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:41:36.841579+0000","last_undegraded":"2024-09-10T20:41:36.841579+0000","last_fullsized":"2024-09-10T20:41:36.841579+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20728,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839605+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:41:36.839605+0000","last_peered":"2024-09-10T20:41:36.839605+0000","last_clean":"2024-09-10T20:41:36.839605+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:41:36.839605+0000","last_undegraded":"2024-09-10T20:41:36.839605+0000","last_fullsized":"2024-09-10T20:41:36.839605+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21239,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839929+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:41:36.839929+0000","last_peered":"2024-09-10T20:41:36.839929+0000","last_clean":"2024-09-10T20:41:36.839929+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:41:36.839929+0000","last_undegraded":"2024-09-10T20:41:36.839929+0000","last_fullsized":"2024-09-10T20:41:36.839929+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":22278,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:40.242224+0000","last_change":"2024-09-10T20:41:35.396233+0000","last_active":"2024-09-10T20:41:40.242224+0000","last_peered":"2024-09-10T20:41:40.242224+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:41:40.242224+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":813,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2803}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20915,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841297+0000","last_change":"2024-09-10T20:41:17.750026+0000","last_active":"2024-09-10T20:41:36.841297+0000","last_peered":"2024-09-10T20:41:36.841297+0000","last_clean":"2024-09-10T20:41:36.841297+0000","last_became_active":"2024-09-10T20:41:17.668013+0000","last_became_peered":"2024-09-10T20:41:17.668013+0000","last_unstale":"2024-09-10T20:41:36.841297+0000","last_undegraded":"2024-09-10T20:41:36.841297+0000","last_fullsized":"2024-09-10T20:41:36.841297+0000","mapping_epoch":62,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:55.885282+0000","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20904,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839732+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:41:36.839732+0000","last_peered":"2024-09-10T20:41:36.839732+0000","last_clean":"2024-09-10T20:41:36.839732+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:41:36.839732+0000","last_undegraded":"2024-09-10T20:41:36.839732+0000","last_fullsized":"2024-09-10T20:41:36.839732+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841162+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:41:36.841162+0000","last_peered":"2024-09-10T20:41:36.841162+0000","last_clean":"2024-09-10T20:41:36.841162+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:41:36.841162+0000","last_undegraded":"2024-09-10T20:41:36.841162+0000","last_fullsized":"2024-09-10T20:41:36.841162+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22146,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841354+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:41:36.841354+0000","last_peered":"2024-09-10T20:41:36.841354+0000","last_clean":"2024-09-10T20:41:36.841354+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:41:36.841354+0000","last_undegraded":"2024-09-10T20:41:36.841354+0000","last_fullsized":"2024-09-10T20:41:36.841354+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":137,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840381+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:41:36.840381+0000","last_peered":"2024-09-10T20:41:36.840381+0000","last_clean":"2024-09-10T20:41:36.840381+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:41:36.840381+0000","last_undegraded":"2024-09-10T20:41:36.840381+0000","last_fullsized":"2024-09-10T20:41:36.840381+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28133,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839783+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:41:36.839783+0000","last_peered":"2024-09-10T20:41:36.839783+0000","last_clean":"2024-09-10T20:41:36.839783+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:41:36.839783+0000","last_undegraded":"2024-09-10T20:41:36.839783+0000","last_fullsized":"2024-09-10T20:41:36.839783+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839670+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:41:36.839670+0000","last_peered":"2024-09-10T20:41:36.839670+0000","last_clean":"2024-09-10T20:41:36.839670+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:41:36.839670+0000","last_undegraded":"2024-09-10T20:41:36.839670+0000","last_fullsized":"2024-09-10T20:41:36.839670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28015,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840380+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:41:36.840380+0000","last_peered":"2024-09-10T20:41:36.840380+0000","last_clean":"2024-09-10T20:41:36.840380+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:41:36.840380+0000","last_undegraded":"2024-09-10T20:41:36.840380+0000","last_fullsized":"2024-09-10T20:41:36.840380+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":123,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841261+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:41:36.841261+0000","last_peered":"2024-09-10T20:41:36.841261+0000","last_clean":"2024-09-10T20:41:36.841261+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:41:36.841261+0000","last_undegraded":"2024-09-10T20:41:36.841261+0000","last_fullsized":"2024-09-10T20:41:36.841261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28069,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839259+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:41:36.839259+0000","last_peered":"2024-09-10T20:41:36.839259+0000","last_clean":"2024-09-10T20:41:36.839259+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:41:36.839259+0000","last_undegraded":"2024-09-10T20:41:36.839259+0000","last_fullsized":"2024-09-10T20:41:36.839259+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21125,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841491+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:41:36.841491+0000","last_peered":"2024-09-10T20:41:36.841491+0000","last_clean":"2024-09-10T20:41:36.841491+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:41:36.841491+0000","last_undegraded":"2024-09-10T20:41:36.841491+0000","last_fullsized":"2024-09-10T20:41:36.841491+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841466+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:41:36.841466+0000","last_peered":"2024-09-10T20:41:36.841466+0000","last_clean":"2024-09-10T20:41:36.841466+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:41:36.841466+0000","last_undegraded":"2024-09-10T20:41:36.841466+0000","last_fullsized":"2024-09-10T20:41:36.841466+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27847,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841464+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:41:36.841464+0000","last_peered":"2024-09-10T20:41:36.841464+0000","last_clean":"2024-09-10T20:41:36.841464+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:41:36.841464+0000","last_undegraded":"2024-09-10T20:41:36.841464+0000","last_fullsized":"2024-09-10T20:41:36.841464+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":137,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839923+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:41:36.839923+0000","last_peered":"2024-09-10T20:41:36.839923+0000","last_clean":"2024-09-10T20:41:36.839923+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:41:36.839923+0000","last_undegraded":"2024-09-10T20:41:36.839923+0000","last_fullsized":"2024-09-10T20:41:36.839923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21261,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.929378+0000","last_change":"2024-09-10T20:41:36.928383+0000","last_active":"2024-09-10T20:41:36.929378+0000","last_peered":"2024-09-10T20:41:36.929378+0000","last_clean":"2024-09-10T20:41:36.929378+0000","last_became_active":"2024-09-10T20:41:36.842440+0000","last_became_peered":"2024-09-10T20:41:36.842440+0000","last_unstale":"2024-09-10T20:41:36.929378+0000","last_undegraded":"2024-09-10T20:41:36.929378+0000","last_fullsized":"2024-09-10T20:41:36.929378+0000","mapping_epoch":64,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.940798+0000","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3715,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841381+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:41:36.841381+0000","last_peered":"2024-09-10T20:41:36.841381+0000","last_clean":"2024-09-10T20:41:36.841381+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:41:36.841381+0000","last_undegraded":"2024-09-10T20:41:36.841381+0000","last_fullsized":"2024-09-10T20:41:36.841381+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841415+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:41:36.841415+0000","last_peered":"2024-09-10T20:41:36.841415+0000","last_clean":"2024-09-10T20:41:36.841415+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:41:36.841415+0000","last_undegraded":"2024-09-10T20:41:36.841415+0000","last_fullsized":"2024-09-10T20:41:36.841415+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27677,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841380+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:41:36.841380+0000","last_peered":"2024-09-10T20:41:36.841380+0000","last_clean":"2024-09-10T20:41:36.841380+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:41:36.841380+0000","last_undegraded":"2024-09-10T20:41:36.841380+0000","last_fullsized":"2024-09-10T20:41:36.841380+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36727,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":129855488,"data_stored":129848262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182557,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517240,"kb_used_data":58020,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854600,"statfs":{"total":96636764160,"available":96107110400,"internally_reserved":0,"allocated":59412480,"data_stored":59071492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986329,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":435448,"kb_used_data":70568,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93936392,"statfs":{"total":96636764160,"available":96190865408,"internally_reserved":0,"allocated":72261632,"data_stored":71928334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659275,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501476,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870364,"statfs":{"total":96636764160,"available":96123252736,"internally_reserved":0,"allocated":540672,"data_stored":204722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512945838},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":70983680,"data_stored":70983680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:41:44.426 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:44.735 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:44.736 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:44.751 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":466,"stamp":"2024-09-10T20:41:43.080121+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36729,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1462488,"kb_used_data":129124,"kb_used_omap":12,"kb_used_meta":1333299,"kb_avail":281653032,"statfs":{"total":289910292480,"available":288412704768,"internally_reserved":0,"allocated":132222976,"data_stored":131207702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1365298178},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1929,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1929,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001269"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":28004,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840754+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:41:36.840754+0000","last_peered":"2024-09-10T20:41:36.840754+0000","last_clean":"2024-09-10T20:41:36.840754+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:41:36.840754+0000","last_undegraded":"2024-09-10T20:41:36.840754+0000","last_fullsized":"2024-09-10T20:41:36.840754+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21096,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840133+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:41:36.840133+0000","last_peered":"2024-09-10T20:41:36.840133+0000","last_clean":"2024-09-10T20:41:36.840133+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:41:36.840133+0000","last_undegraded":"2024-09-10T20:41:36.840133+0000","last_fullsized":"2024-09-10T20:41:36.840133+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28108,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841579+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:41:36.841579+0000","last_peered":"2024-09-10T20:41:36.841579+0000","last_clean":"2024-09-10T20:41:36.841579+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:41:36.841579+0000","last_undegraded":"2024-09-10T20:41:36.841579+0000","last_fullsized":"2024-09-10T20:41:36.841579+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20728,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839605+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:41:36.839605+0000","last_peered":"2024-09-10T20:41:36.839605+0000","last_clean":"2024-09-10T20:41:36.839605+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:41:36.839605+0000","last_undegraded":"2024-09-10T20:41:36.839605+0000","last_fullsized":"2024-09-10T20:41:36.839605+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21239,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839929+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:41:36.839929+0000","last_peered":"2024-09-10T20:41:36.839929+0000","last_clean":"2024-09-10T20:41:36.839929+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:41:36.839929+0000","last_undegraded":"2024-09-10T20:41:36.839929+0000","last_fullsized":"2024-09-10T20:41:36.839929+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":22278,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:40.242224+0000","last_change":"2024-09-10T20:41:35.396233+0000","last_active":"2024-09-10T20:41:40.242224+0000","last_peered":"2024-09-10T20:41:40.242224+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:41:40.242224+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":813,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2803}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20915,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841297+0000","last_change":"2024-09-10T20:41:17.750026+0000","last_active":"2024-09-10T20:41:36.841297+0000","last_peered":"2024-09-10T20:41:36.841297+0000","last_clean":"2024-09-10T20:41:36.841297+0000","last_became_active":"2024-09-10T20:41:17.668013+0000","last_became_peered":"2024-09-10T20:41:17.668013+0000","last_unstale":"2024-09-10T20:41:36.841297+0000","last_undegraded":"2024-09-10T20:41:36.841297+0000","last_fullsized":"2024-09-10T20:41:36.841297+0000","mapping_epoch":62,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:55.885282+0000","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20904,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839732+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:41:36.839732+0000","last_peered":"2024-09-10T20:41:36.839732+0000","last_clean":"2024-09-10T20:41:36.839732+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:41:36.839732+0000","last_undegraded":"2024-09-10T20:41:36.839732+0000","last_fullsized":"2024-09-10T20:41:36.839732+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841162+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:41:36.841162+0000","last_peered":"2024-09-10T20:41:36.841162+0000","last_clean":"2024-09-10T20:41:36.841162+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:41:36.841162+0000","last_undegraded":"2024-09-10T20:41:36.841162+0000","last_fullsized":"2024-09-10T20:41:36.841162+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22147,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841390+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:41:36.841390+0000","last_peered":"2024-09-10T20:41:36.841390+0000","last_clean":"2024-09-10T20:41:36.841390+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:41:36.841390+0000","last_undegraded":"2024-09-10T20:41:36.841390+0000","last_fullsized":"2024-09-10T20:41:36.841390+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":137,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840381+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:41:36.840381+0000","last_peered":"2024-09-10T20:41:36.840381+0000","last_clean":"2024-09-10T20:41:36.840381+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:41:36.840381+0000","last_undegraded":"2024-09-10T20:41:36.840381+0000","last_fullsized":"2024-09-10T20:41:36.840381+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28133,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839783+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:41:36.839783+0000","last_peered":"2024-09-10T20:41:36.839783+0000","last_clean":"2024-09-10T20:41:36.839783+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:41:36.839783+0000","last_undegraded":"2024-09-10T20:41:36.839783+0000","last_fullsized":"2024-09-10T20:41:36.839783+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839670+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:41:36.839670+0000","last_peered":"2024-09-10T20:41:36.839670+0000","last_clean":"2024-09-10T20:41:36.839670+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:41:36.839670+0000","last_undegraded":"2024-09-10T20:41:36.839670+0000","last_fullsized":"2024-09-10T20:41:36.839670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28015,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840380+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:41:36.840380+0000","last_peered":"2024-09-10T20:41:36.840380+0000","last_clean":"2024-09-10T20:41:36.840380+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:41:36.840380+0000","last_undegraded":"2024-09-10T20:41:36.840380+0000","last_fullsized":"2024-09-10T20:41:36.840380+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":123,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841261+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:41:36.841261+0000","last_peered":"2024-09-10T20:41:36.841261+0000","last_clean":"2024-09-10T20:41:36.841261+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:41:36.841261+0000","last_undegraded":"2024-09-10T20:41:36.841261+0000","last_fullsized":"2024-09-10T20:41:36.841261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28069,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839259+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:41:36.839259+0000","last_peered":"2024-09-10T20:41:36.839259+0000","last_clean":"2024-09-10T20:41:36.839259+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:41:36.839259+0000","last_undegraded":"2024-09-10T20:41:36.839259+0000","last_fullsized":"2024-09-10T20:41:36.839259+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21125,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841491+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:41:36.841491+0000","last_peered":"2024-09-10T20:41:36.841491+0000","last_clean":"2024-09-10T20:41:36.841491+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:41:36.841491+0000","last_undegraded":"2024-09-10T20:41:36.841491+0000","last_fullsized":"2024-09-10T20:41:36.841491+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841466+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:41:36.841466+0000","last_peered":"2024-09-10T20:41:36.841466+0000","last_clean":"2024-09-10T20:41:36.841466+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:41:36.841466+0000","last_undegraded":"2024-09-10T20:41:36.841466+0000","last_fullsized":"2024-09-10T20:41:36.841466+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27847,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841464+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:41:36.841464+0000","last_peered":"2024-09-10T20:41:36.841464+0000","last_clean":"2024-09-10T20:41:36.841464+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:41:36.841464+0000","last_undegraded":"2024-09-10T20:41:36.841464+0000","last_fullsized":"2024-09-10T20:41:36.841464+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":137,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839923+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:41:36.839923+0000","last_peered":"2024-09-10T20:41:36.839923+0000","last_clean":"2024-09-10T20:41:36.839923+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:41:36.839923+0000","last_undegraded":"2024-09-10T20:41:36.839923+0000","last_fullsized":"2024-09-10T20:41:36.839923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21261,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.929378+0000","last_change":"2024-09-10T20:41:36.928383+0000","last_active":"2024-09-10T20:41:36.929378+0000","last_peered":"2024-09-10T20:41:36.929378+0000","last_clean":"2024-09-10T20:41:36.929378+0000","last_became_active":"2024-09-10T20:41:36.842440+0000","last_became_peered":"2024-09-10T20:41:36.842440+0000","last_unstale":"2024-09-10T20:41:36.929378+0000","last_undegraded":"2024-09-10T20:41:36.929378+0000","last_fullsized":"2024-09-10T20:41:36.929378+0000","mapping_epoch":64,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.940798+0000","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3715,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841381+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:41:36.841381+0000","last_peered":"2024-09-10T20:41:36.841381+0000","last_clean":"2024-09-10T20:41:36.841381+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:41:36.841381+0000","last_undegraded":"2024-09-10T20:41:36.841381+0000","last_fullsized":"2024-09-10T20:41:36.841381+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841415+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:41:36.841415+0000","last_peered":"2024-09-10T20:41:36.841415+0000","last_clean":"2024-09-10T20:41:36.841415+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:41:36.841415+0000","last_undegraded":"2024-09-10T20:41:36.841415+0000","last_fullsized":"2024-09-10T20:41:36.841415+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27677,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841380+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:41:36.841380+0000","last_peered":"2024-09-10T20:41:36.841380+0000","last_clean":"2024-09-10T20:41:36.841380+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:41:36.841380+0000","last_undegraded":"2024-09-10T20:41:36.841380+0000","last_fullsized":"2024-09-10T20:41:36.841380+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36727,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":129855488,"data_stored":129848262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182557,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517240,"kb_used_data":58020,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854600,"statfs":{"total":96636764160,"available":96107110400,"internally_reserved":0,"allocated":59412480,"data_stored":59071492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986329,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":435448,"kb_used_data":70568,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93936392,"statfs":{"total":96636764160,"available":96190865408,"internally_reserved":0,"allocated":72261632,"data_stored":71928334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659276,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509800,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862040,"statfs":{"total":96636764160,"available":96114728960,"internally_reserved":0,"allocated":548864,"data_stored":207876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521465518},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":70983680,"data_stored":70983680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:41:44.752 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:45.084 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:45.084 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:45.096 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":466,"stamp":"2024-09-10T20:41:43.080121+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36729,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1462488,"kb_used_data":129124,"kb_used_omap":12,"kb_used_meta":1333299,"kb_avail":281653032,"statfs":{"total":289910292480,"available":288412704768,"internally_reserved":0,"allocated":132222976,"data_stored":131207702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1365298178},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1929,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1929,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001269"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":28004,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840754+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:41:36.840754+0000","last_peered":"2024-09-10T20:41:36.840754+0000","last_clean":"2024-09-10T20:41:36.840754+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:41:36.840754+0000","last_undegraded":"2024-09-10T20:41:36.840754+0000","last_fullsized":"2024-09-10T20:41:36.840754+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21096,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840133+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:41:36.840133+0000","last_peered":"2024-09-10T20:41:36.840133+0000","last_clean":"2024-09-10T20:41:36.840133+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:41:36.840133+0000","last_undegraded":"2024-09-10T20:41:36.840133+0000","last_fullsized":"2024-09-10T20:41:36.840133+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28108,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841579+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:41:36.841579+0000","last_peered":"2024-09-10T20:41:36.841579+0000","last_clean":"2024-09-10T20:41:36.841579+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:41:36.841579+0000","last_undegraded":"2024-09-10T20:41:36.841579+0000","last_fullsized":"2024-09-10T20:41:36.841579+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20728,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839605+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:41:36.839605+0000","last_peered":"2024-09-10T20:41:36.839605+0000","last_clean":"2024-09-10T20:41:36.839605+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:41:36.839605+0000","last_undegraded":"2024-09-10T20:41:36.839605+0000","last_fullsized":"2024-09-10T20:41:36.839605+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21239,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839929+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:41:36.839929+0000","last_peered":"2024-09-10T20:41:36.839929+0000","last_clean":"2024-09-10T20:41:36.839929+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:41:36.839929+0000","last_undegraded":"2024-09-10T20:41:36.839929+0000","last_fullsized":"2024-09-10T20:41:36.839929+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":22278,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:40.242224+0000","last_change":"2024-09-10T20:41:35.396233+0000","last_active":"2024-09-10T20:41:40.242224+0000","last_peered":"2024-09-10T20:41:40.242224+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:41:40.242224+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":813,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2803}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20915,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841297+0000","last_change":"2024-09-10T20:41:17.750026+0000","last_active":"2024-09-10T20:41:36.841297+0000","last_peered":"2024-09-10T20:41:36.841297+0000","last_clean":"2024-09-10T20:41:36.841297+0000","last_became_active":"2024-09-10T20:41:17.668013+0000","last_became_peered":"2024-09-10T20:41:17.668013+0000","last_unstale":"2024-09-10T20:41:36.841297+0000","last_undegraded":"2024-09-10T20:41:36.841297+0000","last_fullsized":"2024-09-10T20:41:36.841297+0000","mapping_epoch":62,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:55.885282+0000","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20904,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839732+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:41:36.839732+0000","last_peered":"2024-09-10T20:41:36.839732+0000","last_clean":"2024-09-10T20:41:36.839732+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:41:36.839732+0000","last_undegraded":"2024-09-10T20:41:36.839732+0000","last_fullsized":"2024-09-10T20:41:36.839732+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841162+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:41:36.841162+0000","last_peered":"2024-09-10T20:41:36.841162+0000","last_clean":"2024-09-10T20:41:36.841162+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:41:36.841162+0000","last_undegraded":"2024-09-10T20:41:36.841162+0000","last_fullsized":"2024-09-10T20:41:36.841162+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22147,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841390+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:41:36.841390+0000","last_peered":"2024-09-10T20:41:36.841390+0000","last_clean":"2024-09-10T20:41:36.841390+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:41:36.841390+0000","last_undegraded":"2024-09-10T20:41:36.841390+0000","last_fullsized":"2024-09-10T20:41:36.841390+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":137,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840381+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:41:36.840381+0000","last_peered":"2024-09-10T20:41:36.840381+0000","last_clean":"2024-09-10T20:41:36.840381+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:41:36.840381+0000","last_undegraded":"2024-09-10T20:41:36.840381+0000","last_fullsized":"2024-09-10T20:41:36.840381+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28133,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839783+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:41:36.839783+0000","last_peered":"2024-09-10T20:41:36.839783+0000","last_clean":"2024-09-10T20:41:36.839783+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:41:36.839783+0000","last_undegraded":"2024-09-10T20:41:36.839783+0000","last_fullsized":"2024-09-10T20:41:36.839783+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839670+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:41:36.839670+0000","last_peered":"2024-09-10T20:41:36.839670+0000","last_clean":"2024-09-10T20:41:36.839670+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:41:36.839670+0000","last_undegraded":"2024-09-10T20:41:36.839670+0000","last_fullsized":"2024-09-10T20:41:36.839670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28015,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840380+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:41:36.840380+0000","last_peered":"2024-09-10T20:41:36.840380+0000","last_clean":"2024-09-10T20:41:36.840380+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:41:36.840380+0000","last_undegraded":"2024-09-10T20:41:36.840380+0000","last_fullsized":"2024-09-10T20:41:36.840380+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":123,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841261+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:41:36.841261+0000","last_peered":"2024-09-10T20:41:36.841261+0000","last_clean":"2024-09-10T20:41:36.841261+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:41:36.841261+0000","last_undegraded":"2024-09-10T20:41:36.841261+0000","last_fullsized":"2024-09-10T20:41:36.841261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28069,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839259+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:41:36.839259+0000","last_peered":"2024-09-10T20:41:36.839259+0000","last_clean":"2024-09-10T20:41:36.839259+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:41:36.839259+0000","last_undegraded":"2024-09-10T20:41:36.839259+0000","last_fullsized":"2024-09-10T20:41:36.839259+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21125,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841491+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:41:36.841491+0000","last_peered":"2024-09-10T20:41:36.841491+0000","last_clean":"2024-09-10T20:41:36.841491+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:41:36.841491+0000","last_undegraded":"2024-09-10T20:41:36.841491+0000","last_fullsized":"2024-09-10T20:41:36.841491+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841466+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:41:36.841466+0000","last_peered":"2024-09-10T20:41:36.841466+0000","last_clean":"2024-09-10T20:41:36.841466+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:41:36.841466+0000","last_undegraded":"2024-09-10T20:41:36.841466+0000","last_fullsized":"2024-09-10T20:41:36.841466+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27847,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841464+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:41:36.841464+0000","last_peered":"2024-09-10T20:41:36.841464+0000","last_clean":"2024-09-10T20:41:36.841464+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:41:36.841464+0000","last_undegraded":"2024-09-10T20:41:36.841464+0000","last_fullsized":"2024-09-10T20:41:36.841464+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":137,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839923+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:41:36.839923+0000","last_peered":"2024-09-10T20:41:36.839923+0000","last_clean":"2024-09-10T20:41:36.839923+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:41:36.839923+0000","last_undegraded":"2024-09-10T20:41:36.839923+0000","last_fullsized":"2024-09-10T20:41:36.839923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21261,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.929378+0000","last_change":"2024-09-10T20:41:36.928383+0000","last_active":"2024-09-10T20:41:36.929378+0000","last_peered":"2024-09-10T20:41:36.929378+0000","last_clean":"2024-09-10T20:41:36.929378+0000","last_became_active":"2024-09-10T20:41:36.842440+0000","last_became_peered":"2024-09-10T20:41:36.842440+0000","last_unstale":"2024-09-10T20:41:36.929378+0000","last_undegraded":"2024-09-10T20:41:36.929378+0000","last_fullsized":"2024-09-10T20:41:36.929378+0000","mapping_epoch":64,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.940798+0000","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3715,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841381+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:41:36.841381+0000","last_peered":"2024-09-10T20:41:36.841381+0000","last_clean":"2024-09-10T20:41:36.841381+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:41:36.841381+0000","last_undegraded":"2024-09-10T20:41:36.841381+0000","last_fullsized":"2024-09-10T20:41:36.841381+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841415+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:41:36.841415+0000","last_peered":"2024-09-10T20:41:36.841415+0000","last_clean":"2024-09-10T20:41:36.841415+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:41:36.841415+0000","last_undegraded":"2024-09-10T20:41:36.841415+0000","last_fullsized":"2024-09-10T20:41:36.841415+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27677,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841380+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:41:36.841380+0000","last_peered":"2024-09-10T20:41:36.841380+0000","last_clean":"2024-09-10T20:41:36.841380+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:41:36.841380+0000","last_undegraded":"2024-09-10T20:41:36.841380+0000","last_fullsized":"2024-09-10T20:41:36.841380+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36727,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":129855488,"data_stored":129848262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182557,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517240,"kb_used_data":58020,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854600,"statfs":{"total":96636764160,"available":96107110400,"internally_reserved":0,"allocated":59412480,"data_stored":59071492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986329,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":435448,"kb_used_data":70568,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93936392,"statfs":{"total":96636764160,"available":96190865408,"internally_reserved":0,"allocated":72261632,"data_stored":71928334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659276,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509800,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862040,"statfs":{"total":96636764160,"available":96114728960,"internally_reserved":0,"allocated":548864,"data_stored":207876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521465518},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":70983680,"data_stored":70983680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:41:45.098 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:41:45.098 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20471", 'reported_seq': 22278, 'reported_epoch': 65, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:41:40.242224+0000', 'last_change': '2024-09-10T20:41:35.396233+0000', 'last_active': '2024-09-10T20:41:40.242224+0000', 'last_peered': '2024-09-10T20:41:40.242224+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:41:40.242224+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2803, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17335, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 813, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2803}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:41:45.098 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:41:45.484 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:45.484 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2803/6 objects degraded (46716.667%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":817,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1493286912,"bytes_avail":288417005568,"bytes_total":289910292480,"degraded_objects":2803,"degraded_total":6,"degraded_ratio":467.16666666666674,"recovering_objects_per_sec":85,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":802,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:41:45.498 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:41:45.499 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:45.859 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:45.860 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:45.873 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":467,"stamp":"2024-09-10T20:41:45.080409+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36729,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1458288,"kb_used_data":124924,"kb_used_omap":12,"kb_used_meta":1333299,"kb_avail":281657232,"statfs":{"total":289910292480,"available":288417005568,"internally_reserved":0,"allocated":127922176,"data_stored":126906902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1365298178},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-802,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":802,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.333421"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":28004,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840754+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:41:36.840754+0000","last_peered":"2024-09-10T20:41:36.840754+0000","last_clean":"2024-09-10T20:41:36.840754+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:41:36.840754+0000","last_undegraded":"2024-09-10T20:41:36.840754+0000","last_fullsized":"2024-09-10T20:41:36.840754+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21096,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840133+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:41:36.840133+0000","last_peered":"2024-09-10T20:41:36.840133+0000","last_clean":"2024-09-10T20:41:36.840133+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:41:36.840133+0000","last_undegraded":"2024-09-10T20:41:36.840133+0000","last_fullsized":"2024-09-10T20:41:36.840133+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28108,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841579+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:41:36.841579+0000","last_peered":"2024-09-10T20:41:36.841579+0000","last_clean":"2024-09-10T20:41:36.841579+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:41:36.841579+0000","last_undegraded":"2024-09-10T20:41:36.841579+0000","last_fullsized":"2024-09-10T20:41:36.841579+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20728,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839605+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:41:36.839605+0000","last_peered":"2024-09-10T20:41:36.839605+0000","last_clean":"2024-09-10T20:41:36.839605+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:41:36.839605+0000","last_undegraded":"2024-09-10T20:41:36.839605+0000","last_fullsized":"2024-09-10T20:41:36.839605+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21239,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839929+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:41:36.839929+0000","last_peered":"2024-09-10T20:41:36.839929+0000","last_clean":"2024-09-10T20:41:36.839929+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:41:36.839929+0000","last_undegraded":"2024-09-10T20:41:36.839929+0000","last_fullsized":"2024-09-10T20:41:36.839929+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":22278,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:40.242224+0000","last_change":"2024-09-10T20:41:35.396233+0000","last_active":"2024-09-10T20:41:40.242224+0000","last_peered":"2024-09-10T20:41:40.242224+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:41:40.242224+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":813,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2803}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20915,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841297+0000","last_change":"2024-09-10T20:41:17.750026+0000","last_active":"2024-09-10T20:41:36.841297+0000","last_peered":"2024-09-10T20:41:36.841297+0000","last_clean":"2024-09-10T20:41:36.841297+0000","last_became_active":"2024-09-10T20:41:17.668013+0000","last_became_peered":"2024-09-10T20:41:17.668013+0000","last_unstale":"2024-09-10T20:41:36.841297+0000","last_undegraded":"2024-09-10T20:41:36.841297+0000","last_fullsized":"2024-09-10T20:41:36.841297+0000","mapping_epoch":62,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:55.885282+0000","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20904,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839732+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:41:36.839732+0000","last_peered":"2024-09-10T20:41:36.839732+0000","last_clean":"2024-09-10T20:41:36.839732+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:41:36.839732+0000","last_undegraded":"2024-09-10T20:41:36.839732+0000","last_fullsized":"2024-09-10T20:41:36.839732+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841162+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:41:36.841162+0000","last_peered":"2024-09-10T20:41:36.841162+0000","last_clean":"2024-09-10T20:41:36.841162+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:41:36.841162+0000","last_undegraded":"2024-09-10T20:41:36.841162+0000","last_fullsized":"2024-09-10T20:41:36.841162+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22147,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841390+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:41:36.841390+0000","last_peered":"2024-09-10T20:41:36.841390+0000","last_clean":"2024-09-10T20:41:36.841390+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:41:36.841390+0000","last_undegraded":"2024-09-10T20:41:36.841390+0000","last_fullsized":"2024-09-10T20:41:36.841390+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840435+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:41:36.840435+0000","last_peered":"2024-09-10T20:41:36.840435+0000","last_clean":"2024-09-10T20:41:36.840435+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:41:36.840435+0000","last_undegraded":"2024-09-10T20:41:36.840435+0000","last_fullsized":"2024-09-10T20:41:36.840435+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28133,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839783+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:41:36.839783+0000","last_peered":"2024-09-10T20:41:36.839783+0000","last_clean":"2024-09-10T20:41:36.839783+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:41:36.839783+0000","last_undegraded":"2024-09-10T20:41:36.839783+0000","last_fullsized":"2024-09-10T20:41:36.839783+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839670+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:41:36.839670+0000","last_peered":"2024-09-10T20:41:36.839670+0000","last_clean":"2024-09-10T20:41:36.839670+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:41:36.839670+0000","last_undegraded":"2024-09-10T20:41:36.839670+0000","last_fullsized":"2024-09-10T20:41:36.839670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28016,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840428+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:41:36.840428+0000","last_peered":"2024-09-10T20:41:36.840428+0000","last_clean":"2024-09-10T20:41:36.840428+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:41:36.840428+0000","last_undegraded":"2024-09-10T20:41:36.840428+0000","last_fullsized":"2024-09-10T20:41:36.840428+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":123,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841261+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:41:36.841261+0000","last_peered":"2024-09-10T20:41:36.841261+0000","last_clean":"2024-09-10T20:41:36.841261+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:41:36.841261+0000","last_undegraded":"2024-09-10T20:41:36.841261+0000","last_fullsized":"2024-09-10T20:41:36.841261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28069,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839259+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:41:36.839259+0000","last_peered":"2024-09-10T20:41:36.839259+0000","last_clean":"2024-09-10T20:41:36.839259+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:41:36.839259+0000","last_undegraded":"2024-09-10T20:41:36.839259+0000","last_fullsized":"2024-09-10T20:41:36.839259+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21125,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841491+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:41:36.841491+0000","last_peered":"2024-09-10T20:41:36.841491+0000","last_clean":"2024-09-10T20:41:36.841491+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:41:36.841491+0000","last_undegraded":"2024-09-10T20:41:36.841491+0000","last_fullsized":"2024-09-10T20:41:36.841491+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841466+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:41:36.841466+0000","last_peered":"2024-09-10T20:41:36.841466+0000","last_clean":"2024-09-10T20:41:36.841466+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:41:36.841466+0000","last_undegraded":"2024-09-10T20:41:36.841466+0000","last_fullsized":"2024-09-10T20:41:36.841466+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27847,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841464+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:41:36.841464+0000","last_peered":"2024-09-10T20:41:36.841464+0000","last_clean":"2024-09-10T20:41:36.841464+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:41:36.841464+0000","last_undegraded":"2024-09-10T20:41:36.841464+0000","last_fullsized":"2024-09-10T20:41:36.841464+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839959+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:41:36.839959+0000","last_peered":"2024-09-10T20:41:36.839959+0000","last_clean":"2024-09-10T20:41:36.839959+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:41:36.839959+0000","last_undegraded":"2024-09-10T20:41:36.839959+0000","last_fullsized":"2024-09-10T20:41:36.839959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21261,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.929378+0000","last_change":"2024-09-10T20:41:36.928383+0000","last_active":"2024-09-10T20:41:36.929378+0000","last_peered":"2024-09-10T20:41:36.929378+0000","last_clean":"2024-09-10T20:41:36.929378+0000","last_became_active":"2024-09-10T20:41:36.842440+0000","last_became_peered":"2024-09-10T20:41:36.842440+0000","last_unstale":"2024-09-10T20:41:36.929378+0000","last_undegraded":"2024-09-10T20:41:36.929378+0000","last_fullsized":"2024-09-10T20:41:36.929378+0000","mapping_epoch":64,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.940798+0000","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3715,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841381+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:41:36.841381+0000","last_peered":"2024-09-10T20:41:36.841381+0000","last_clean":"2024-09-10T20:41:36.841381+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:41:36.841381+0000","last_undegraded":"2024-09-10T20:41:36.841381+0000","last_fullsized":"2024-09-10T20:41:36.841381+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841415+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:41:36.841415+0000","last_peered":"2024-09-10T20:41:36.841415+0000","last_clean":"2024-09-10T20:41:36.841415+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:41:36.841415+0000","last_undegraded":"2024-09-10T20:41:36.841415+0000","last_fullsized":"2024-09-10T20:41:36.841415+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27677,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841380+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:41:36.841380+0000","last_peered":"2024-09-10T20:41:36.841380+0000","last_clean":"2024-09-10T20:41:36.841380+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:41:36.841380+0000","last_undegraded":"2024-09-10T20:41:36.841380+0000","last_fullsized":"2024-09-10T20:41:36.841380+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36727,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":125554688,"data_stored":125547462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182557,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517240,"kb_used_data":58020,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854600,"statfs":{"total":96636764160,"available":96107110400,"internally_reserved":0,"allocated":59412480,"data_stored":59071492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986330,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":431248,"kb_used_data":66368,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93940592,"statfs":{"total":96636764160,"available":96195166208,"internally_reserved":0,"allocated":67960832,"data_stored":67627534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659276,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509800,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862040,"statfs":{"total":96636764160,"available":96114728960,"internally_reserved":0,"allocated":548864,"data_stored":207876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521465518},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":66682880,"data_stored":66682880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:41:48.876 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:49.180 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:49.180 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:49.194 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":469,"stamp":"2024-09-10T20:41:49.081036+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2144,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37388,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1445808,"kb_used_data":124924,"kb_used_omap":12,"kb_used_meta":1320819,"kb_avail":281669712,"statfs":{"total":289910292480,"available":288429785088,"internally_reserved":0,"allocated":127922176,"data_stored":126906902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1352518658},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1461,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1461,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001856"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":28004,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840754+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:41:36.840754+0000","last_peered":"2024-09-10T20:41:36.840754+0000","last_clean":"2024-09-10T20:41:36.840754+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:41:36.840754+0000","last_undegraded":"2024-09-10T20:41:36.840754+0000","last_fullsized":"2024-09-10T20:41:36.840754+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21096,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840133+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:41:36.840133+0000","last_peered":"2024-09-10T20:41:36.840133+0000","last_clean":"2024-09-10T20:41:36.840133+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:41:36.840133+0000","last_undegraded":"2024-09-10T20:41:36.840133+0000","last_fullsized":"2024-09-10T20:41:36.840133+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28108,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841579+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:41:36.841579+0000","last_peered":"2024-09-10T20:41:36.841579+0000","last_clean":"2024-09-10T20:41:36.841579+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:41:36.841579+0000","last_undegraded":"2024-09-10T20:41:36.841579+0000","last_fullsized":"2024-09-10T20:41:36.841579+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20728,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839605+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:41:36.839605+0000","last_peered":"2024-09-10T20:41:36.839605+0000","last_clean":"2024-09-10T20:41:36.839605+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:41:36.839605+0000","last_undegraded":"2024-09-10T20:41:36.839605+0000","last_fullsized":"2024-09-10T20:41:36.839605+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21239,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839929+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:41:36.839929+0000","last_peered":"2024-09-10T20:41:36.839929+0000","last_clean":"2024-09-10T20:41:36.839929+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:41:36.839929+0000","last_undegraded":"2024-09-10T20:41:36.839929+0000","last_fullsized":"2024-09-10T20:41:36.839929+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":23596,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:45.248517+0000","last_change":"2024-09-10T20:41:35.396233+0000","last_active":"2024-09-10T20:41:45.248517+0000","last_peered":"2024-09-10T20:41:45.248517+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:41:45.248517+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2144,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1472,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2144}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20915,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841297+0000","last_change":"2024-09-10T20:41:17.750026+0000","last_active":"2024-09-10T20:41:36.841297+0000","last_peered":"2024-09-10T20:41:36.841297+0000","last_clean":"2024-09-10T20:41:36.841297+0000","last_became_active":"2024-09-10T20:41:17.668013+0000","last_became_peered":"2024-09-10T20:41:17.668013+0000","last_unstale":"2024-09-10T20:41:36.841297+0000","last_undegraded":"2024-09-10T20:41:36.841297+0000","last_fullsized":"2024-09-10T20:41:36.841297+0000","mapping_epoch":62,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:55.885282+0000","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20904,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839732+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:41:36.839732+0000","last_peered":"2024-09-10T20:41:36.839732+0000","last_clean":"2024-09-10T20:41:36.839732+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:41:36.839732+0000","last_undegraded":"2024-09-10T20:41:36.839732+0000","last_fullsized":"2024-09-10T20:41:36.839732+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841162+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:41:36.841162+0000","last_peered":"2024-09-10T20:41:36.841162+0000","last_clean":"2024-09-10T20:41:36.841162+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:41:36.841162+0000","last_undegraded":"2024-09-10T20:41:36.841162+0000","last_fullsized":"2024-09-10T20:41:36.841162+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22147,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841390+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:41:36.841390+0000","last_peered":"2024-09-10T20:41:36.841390+0000","last_clean":"2024-09-10T20:41:36.841390+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:41:36.841390+0000","last_undegraded":"2024-09-10T20:41:36.841390+0000","last_fullsized":"2024-09-10T20:41:36.841390+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840435+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:41:36.840435+0000","last_peered":"2024-09-10T20:41:36.840435+0000","last_clean":"2024-09-10T20:41:36.840435+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:41:36.840435+0000","last_undegraded":"2024-09-10T20:41:36.840435+0000","last_fullsized":"2024-09-10T20:41:36.840435+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28133,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839783+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:41:36.839783+0000","last_peered":"2024-09-10T20:41:36.839783+0000","last_clean":"2024-09-10T20:41:36.839783+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:41:36.839783+0000","last_undegraded":"2024-09-10T20:41:36.839783+0000","last_fullsized":"2024-09-10T20:41:36.839783+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839670+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:41:36.839670+0000","last_peered":"2024-09-10T20:41:36.839670+0000","last_clean":"2024-09-10T20:41:36.839670+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:41:36.839670+0000","last_undegraded":"2024-09-10T20:41:36.839670+0000","last_fullsized":"2024-09-10T20:41:36.839670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28016,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840428+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:41:36.840428+0000","last_peered":"2024-09-10T20:41:36.840428+0000","last_clean":"2024-09-10T20:41:36.840428+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:41:36.840428+0000","last_undegraded":"2024-09-10T20:41:36.840428+0000","last_fullsized":"2024-09-10T20:41:36.840428+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":123,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841261+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:41:36.841261+0000","last_peered":"2024-09-10T20:41:36.841261+0000","last_clean":"2024-09-10T20:41:36.841261+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:41:36.841261+0000","last_undegraded":"2024-09-10T20:41:36.841261+0000","last_fullsized":"2024-09-10T20:41:36.841261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28069,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839259+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:41:36.839259+0000","last_peered":"2024-09-10T20:41:36.839259+0000","last_clean":"2024-09-10T20:41:36.839259+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:41:36.839259+0000","last_undegraded":"2024-09-10T20:41:36.839259+0000","last_fullsized":"2024-09-10T20:41:36.839259+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21125,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841491+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:41:36.841491+0000","last_peered":"2024-09-10T20:41:36.841491+0000","last_clean":"2024-09-10T20:41:36.841491+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:41:36.841491+0000","last_undegraded":"2024-09-10T20:41:36.841491+0000","last_fullsized":"2024-09-10T20:41:36.841491+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841466+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:41:36.841466+0000","last_peered":"2024-09-10T20:41:36.841466+0000","last_clean":"2024-09-10T20:41:36.841466+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:41:36.841466+0000","last_undegraded":"2024-09-10T20:41:36.841466+0000","last_fullsized":"2024-09-10T20:41:36.841466+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27847,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841464+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:41:36.841464+0000","last_peered":"2024-09-10T20:41:36.841464+0000","last_clean":"2024-09-10T20:41:36.841464+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:41:36.841464+0000","last_undegraded":"2024-09-10T20:41:36.841464+0000","last_fullsized":"2024-09-10T20:41:36.841464+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839959+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:41:36.839959+0000","last_peered":"2024-09-10T20:41:36.839959+0000","last_clean":"2024-09-10T20:41:36.839959+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:41:36.839959+0000","last_undegraded":"2024-09-10T20:41:36.839959+0000","last_fullsized":"2024-09-10T20:41:36.839959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21261,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.929378+0000","last_change":"2024-09-10T20:41:36.928383+0000","last_active":"2024-09-10T20:41:36.929378+0000","last_peered":"2024-09-10T20:41:36.929378+0000","last_clean":"2024-09-10T20:41:36.929378+0000","last_became_active":"2024-09-10T20:41:36.842440+0000","last_became_peered":"2024-09-10T20:41:36.842440+0000","last_unstale":"2024-09-10T20:41:36.929378+0000","last_undegraded":"2024-09-10T20:41:36.929378+0000","last_fullsized":"2024-09-10T20:41:36.929378+0000","mapping_epoch":64,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.940798+0000","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3715,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841381+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:41:36.841381+0000","last_peered":"2024-09-10T20:41:36.841381+0000","last_clean":"2024-09-10T20:41:36.841381+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:41:36.841381+0000","last_undegraded":"2024-09-10T20:41:36.841381+0000","last_fullsized":"2024-09-10T20:41:36.841381+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841415+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:41:36.841415+0000","last_peered":"2024-09-10T20:41:36.841415+0000","last_clean":"2024-09-10T20:41:36.841415+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:41:36.841415+0000","last_undegraded":"2024-09-10T20:41:36.841415+0000","last_fullsized":"2024-09-10T20:41:36.841415+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27677,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841380+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:41:36.841380+0000","last_peered":"2024-09-10T20:41:36.841380+0000","last_clean":"2024-09-10T20:41:36.841380+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:41:36.841380+0000","last_undegraded":"2024-09-10T20:41:36.841380+0000","last_fullsized":"2024-09-10T20:41:36.841380+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2144,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37386,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":125554688,"data_stored":125547462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182558,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517240,"kb_used_data":58020,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854600,"statfs":{"total":96636764160,"available":96107110400,"internally_reserved":0,"allocated":59412480,"data_stored":59071492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986330,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":431248,"kb_used_data":66368,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93940592,"statfs":{"total":96636764160,"available":96195166208,"internally_reserved":0,"allocated":67960832,"data_stored":67627534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659277,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497320,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874520,"statfs":{"total":96636764160,"available":96127508480,"internally_reserved":0,"allocated":548864,"data_stored":207876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":66682880,"data_stored":66682880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:41:49.195 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:49.492 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:49.492 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:49.505 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":469,"stamp":"2024-09-10T20:41:49.081036+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2144,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37388,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1445808,"kb_used_data":124924,"kb_used_omap":12,"kb_used_meta":1320819,"kb_avail":281669712,"statfs":{"total":289910292480,"available":288429785088,"internally_reserved":0,"allocated":127922176,"data_stored":126906902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1352518658},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1461,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1461,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001856"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":28004,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840754+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:41:36.840754+0000","last_peered":"2024-09-10T20:41:36.840754+0000","last_clean":"2024-09-10T20:41:36.840754+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:41:36.840754+0000","last_undegraded":"2024-09-10T20:41:36.840754+0000","last_fullsized":"2024-09-10T20:41:36.840754+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21096,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840133+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:41:36.840133+0000","last_peered":"2024-09-10T20:41:36.840133+0000","last_clean":"2024-09-10T20:41:36.840133+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:41:36.840133+0000","last_undegraded":"2024-09-10T20:41:36.840133+0000","last_fullsized":"2024-09-10T20:41:36.840133+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28108,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841579+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:41:36.841579+0000","last_peered":"2024-09-10T20:41:36.841579+0000","last_clean":"2024-09-10T20:41:36.841579+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:41:36.841579+0000","last_undegraded":"2024-09-10T20:41:36.841579+0000","last_fullsized":"2024-09-10T20:41:36.841579+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20728,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839605+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:41:36.839605+0000","last_peered":"2024-09-10T20:41:36.839605+0000","last_clean":"2024-09-10T20:41:36.839605+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:41:36.839605+0000","last_undegraded":"2024-09-10T20:41:36.839605+0000","last_fullsized":"2024-09-10T20:41:36.839605+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21239,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839929+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:41:36.839929+0000","last_peered":"2024-09-10T20:41:36.839929+0000","last_clean":"2024-09-10T20:41:36.839929+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:41:36.839929+0000","last_undegraded":"2024-09-10T20:41:36.839929+0000","last_fullsized":"2024-09-10T20:41:36.839929+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":23596,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:45.248517+0000","last_change":"2024-09-10T20:41:35.396233+0000","last_active":"2024-09-10T20:41:45.248517+0000","last_peered":"2024-09-10T20:41:45.248517+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:41:45.248517+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2144,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1472,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2144}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20915,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841297+0000","last_change":"2024-09-10T20:41:17.750026+0000","last_active":"2024-09-10T20:41:36.841297+0000","last_peered":"2024-09-10T20:41:36.841297+0000","last_clean":"2024-09-10T20:41:36.841297+0000","last_became_active":"2024-09-10T20:41:17.668013+0000","last_became_peered":"2024-09-10T20:41:17.668013+0000","last_unstale":"2024-09-10T20:41:36.841297+0000","last_undegraded":"2024-09-10T20:41:36.841297+0000","last_fullsized":"2024-09-10T20:41:36.841297+0000","mapping_epoch":62,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:55.885282+0000","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20904,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839732+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:41:36.839732+0000","last_peered":"2024-09-10T20:41:36.839732+0000","last_clean":"2024-09-10T20:41:36.839732+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:41:36.839732+0000","last_undegraded":"2024-09-10T20:41:36.839732+0000","last_fullsized":"2024-09-10T20:41:36.839732+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841162+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:41:36.841162+0000","last_peered":"2024-09-10T20:41:36.841162+0000","last_clean":"2024-09-10T20:41:36.841162+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:41:36.841162+0000","last_undegraded":"2024-09-10T20:41:36.841162+0000","last_fullsized":"2024-09-10T20:41:36.841162+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22147,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841390+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:41:36.841390+0000","last_peered":"2024-09-10T20:41:36.841390+0000","last_clean":"2024-09-10T20:41:36.841390+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:41:36.841390+0000","last_undegraded":"2024-09-10T20:41:36.841390+0000","last_fullsized":"2024-09-10T20:41:36.841390+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840435+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:41:36.840435+0000","last_peered":"2024-09-10T20:41:36.840435+0000","last_clean":"2024-09-10T20:41:36.840435+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:41:36.840435+0000","last_undegraded":"2024-09-10T20:41:36.840435+0000","last_fullsized":"2024-09-10T20:41:36.840435+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28133,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839783+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:41:36.839783+0000","last_peered":"2024-09-10T20:41:36.839783+0000","last_clean":"2024-09-10T20:41:36.839783+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:41:36.839783+0000","last_undegraded":"2024-09-10T20:41:36.839783+0000","last_fullsized":"2024-09-10T20:41:36.839783+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839670+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:41:36.839670+0000","last_peered":"2024-09-10T20:41:36.839670+0000","last_clean":"2024-09-10T20:41:36.839670+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:41:36.839670+0000","last_undegraded":"2024-09-10T20:41:36.839670+0000","last_fullsized":"2024-09-10T20:41:36.839670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28016,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840428+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:41:36.840428+0000","last_peered":"2024-09-10T20:41:36.840428+0000","last_clean":"2024-09-10T20:41:36.840428+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:41:36.840428+0000","last_undegraded":"2024-09-10T20:41:36.840428+0000","last_fullsized":"2024-09-10T20:41:36.840428+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":123,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841261+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:41:36.841261+0000","last_peered":"2024-09-10T20:41:36.841261+0000","last_clean":"2024-09-10T20:41:36.841261+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:41:36.841261+0000","last_undegraded":"2024-09-10T20:41:36.841261+0000","last_fullsized":"2024-09-10T20:41:36.841261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28069,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839259+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:41:36.839259+0000","last_peered":"2024-09-10T20:41:36.839259+0000","last_clean":"2024-09-10T20:41:36.839259+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:41:36.839259+0000","last_undegraded":"2024-09-10T20:41:36.839259+0000","last_fullsized":"2024-09-10T20:41:36.839259+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21125,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841491+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:41:36.841491+0000","last_peered":"2024-09-10T20:41:36.841491+0000","last_clean":"2024-09-10T20:41:36.841491+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:41:36.841491+0000","last_undegraded":"2024-09-10T20:41:36.841491+0000","last_fullsized":"2024-09-10T20:41:36.841491+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841466+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:41:36.841466+0000","last_peered":"2024-09-10T20:41:36.841466+0000","last_clean":"2024-09-10T20:41:36.841466+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:41:36.841466+0000","last_undegraded":"2024-09-10T20:41:36.841466+0000","last_fullsized":"2024-09-10T20:41:36.841466+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27847,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841464+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:41:36.841464+0000","last_peered":"2024-09-10T20:41:36.841464+0000","last_clean":"2024-09-10T20:41:36.841464+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:41:36.841464+0000","last_undegraded":"2024-09-10T20:41:36.841464+0000","last_fullsized":"2024-09-10T20:41:36.841464+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839959+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:41:36.839959+0000","last_peered":"2024-09-10T20:41:36.839959+0000","last_clean":"2024-09-10T20:41:36.839959+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:41:36.839959+0000","last_undegraded":"2024-09-10T20:41:36.839959+0000","last_fullsized":"2024-09-10T20:41:36.839959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21261,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.929378+0000","last_change":"2024-09-10T20:41:36.928383+0000","last_active":"2024-09-10T20:41:36.929378+0000","last_peered":"2024-09-10T20:41:36.929378+0000","last_clean":"2024-09-10T20:41:36.929378+0000","last_became_active":"2024-09-10T20:41:36.842440+0000","last_became_peered":"2024-09-10T20:41:36.842440+0000","last_unstale":"2024-09-10T20:41:36.929378+0000","last_undegraded":"2024-09-10T20:41:36.929378+0000","last_fullsized":"2024-09-10T20:41:36.929378+0000","mapping_epoch":64,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.940798+0000","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3715,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841381+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:41:36.841381+0000","last_peered":"2024-09-10T20:41:36.841381+0000","last_clean":"2024-09-10T20:41:36.841381+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:41:36.841381+0000","last_undegraded":"2024-09-10T20:41:36.841381+0000","last_fullsized":"2024-09-10T20:41:36.841381+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841415+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:41:36.841415+0000","last_peered":"2024-09-10T20:41:36.841415+0000","last_clean":"2024-09-10T20:41:36.841415+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:41:36.841415+0000","last_undegraded":"2024-09-10T20:41:36.841415+0000","last_fullsized":"2024-09-10T20:41:36.841415+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27677,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841380+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:41:36.841380+0000","last_peered":"2024-09-10T20:41:36.841380+0000","last_clean":"2024-09-10T20:41:36.841380+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:41:36.841380+0000","last_undegraded":"2024-09-10T20:41:36.841380+0000","last_fullsized":"2024-09-10T20:41:36.841380+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2144,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37386,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":125554688,"data_stored":125547462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182558,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517240,"kb_used_data":58020,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854600,"statfs":{"total":96636764160,"available":96107110400,"internally_reserved":0,"allocated":59412480,"data_stored":59071492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986330,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":431248,"kb_used_data":66368,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93940592,"statfs":{"total":96636764160,"available":96195166208,"internally_reserved":0,"allocated":67960832,"data_stored":67627534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659277,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497320,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874520,"statfs":{"total":96636764160,"available":96127508480,"internally_reserved":0,"allocated":548864,"data_stored":207876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":66682880,"data_stored":66682880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:41:49.507 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:41:49.507 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20471", 'reported_seq': 23596, 'reported_epoch': 65, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:41:45.248517+0000', 'last_change': '2024-09-10T20:41:35.396233+0000', 'last_active': '2024-09-10T20:41:45.248517+0000', 'last_peered': '2024-09-10T20:41:45.248517+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:41:45.248517+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2144, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17335, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1472, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2144}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:41:49.507 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:41:49.916 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:49.916 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2144/6 objects degraded (35733.333%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":821,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1480507392,"bytes_avail":288429785088,"bytes_total":289910292480,"degraded_objects":2144,"degraded_total":6,"degraded_ratio":357.33333333333326,"recovering_objects_per_sec":121,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1461,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:41:49.932 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:41:49.932 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:50.282 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:50.283 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:50.296 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":469,"stamp":"2024-09-10T20:41:49.081036+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2144,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37388,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1445808,"kb_used_data":124924,"kb_used_omap":12,"kb_used_meta":1320819,"kb_avail":281669712,"statfs":{"total":289910292480,"available":288429785088,"internally_reserved":0,"allocated":127922176,"data_stored":126906902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1352518658},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1461,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1461,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001856"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":28004,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840754+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:41:36.840754+0000","last_peered":"2024-09-10T20:41:36.840754+0000","last_clean":"2024-09-10T20:41:36.840754+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:41:36.840754+0000","last_undegraded":"2024-09-10T20:41:36.840754+0000","last_fullsized":"2024-09-10T20:41:36.840754+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21096,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840133+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:41:36.840133+0000","last_peered":"2024-09-10T20:41:36.840133+0000","last_clean":"2024-09-10T20:41:36.840133+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:41:36.840133+0000","last_undegraded":"2024-09-10T20:41:36.840133+0000","last_fullsized":"2024-09-10T20:41:36.840133+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28108,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841579+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:41:36.841579+0000","last_peered":"2024-09-10T20:41:36.841579+0000","last_clean":"2024-09-10T20:41:36.841579+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:41:36.841579+0000","last_undegraded":"2024-09-10T20:41:36.841579+0000","last_fullsized":"2024-09-10T20:41:36.841579+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20728,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839605+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:41:36.839605+0000","last_peered":"2024-09-10T20:41:36.839605+0000","last_clean":"2024-09-10T20:41:36.839605+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:41:36.839605+0000","last_undegraded":"2024-09-10T20:41:36.839605+0000","last_fullsized":"2024-09-10T20:41:36.839605+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21239,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839929+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:41:36.839929+0000","last_peered":"2024-09-10T20:41:36.839929+0000","last_clean":"2024-09-10T20:41:36.839929+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:41:36.839929+0000","last_undegraded":"2024-09-10T20:41:36.839929+0000","last_fullsized":"2024-09-10T20:41:36.839929+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":23596,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:45.248517+0000","last_change":"2024-09-10T20:41:35.396233+0000","last_active":"2024-09-10T20:41:45.248517+0000","last_peered":"2024-09-10T20:41:45.248517+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:41:45.248517+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2144,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1472,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2144}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20915,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841297+0000","last_change":"2024-09-10T20:41:17.750026+0000","last_active":"2024-09-10T20:41:36.841297+0000","last_peered":"2024-09-10T20:41:36.841297+0000","last_clean":"2024-09-10T20:41:36.841297+0000","last_became_active":"2024-09-10T20:41:17.668013+0000","last_became_peered":"2024-09-10T20:41:17.668013+0000","last_unstale":"2024-09-10T20:41:36.841297+0000","last_undegraded":"2024-09-10T20:41:36.841297+0000","last_fullsized":"2024-09-10T20:41:36.841297+0000","mapping_epoch":62,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:55.885282+0000","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20904,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839732+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:41:36.839732+0000","last_peered":"2024-09-10T20:41:36.839732+0000","last_clean":"2024-09-10T20:41:36.839732+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:41:36.839732+0000","last_undegraded":"2024-09-10T20:41:36.839732+0000","last_fullsized":"2024-09-10T20:41:36.839732+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841162+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:41:36.841162+0000","last_peered":"2024-09-10T20:41:36.841162+0000","last_clean":"2024-09-10T20:41:36.841162+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:41:36.841162+0000","last_undegraded":"2024-09-10T20:41:36.841162+0000","last_fullsized":"2024-09-10T20:41:36.841162+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22147,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841390+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:41:36.841390+0000","last_peered":"2024-09-10T20:41:36.841390+0000","last_clean":"2024-09-10T20:41:36.841390+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:41:36.841390+0000","last_undegraded":"2024-09-10T20:41:36.841390+0000","last_fullsized":"2024-09-10T20:41:36.841390+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840435+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:41:36.840435+0000","last_peered":"2024-09-10T20:41:36.840435+0000","last_clean":"2024-09-10T20:41:36.840435+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:41:36.840435+0000","last_undegraded":"2024-09-10T20:41:36.840435+0000","last_fullsized":"2024-09-10T20:41:36.840435+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28133,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839783+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:41:36.839783+0000","last_peered":"2024-09-10T20:41:36.839783+0000","last_clean":"2024-09-10T20:41:36.839783+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:41:36.839783+0000","last_undegraded":"2024-09-10T20:41:36.839783+0000","last_fullsized":"2024-09-10T20:41:36.839783+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839670+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:41:36.839670+0000","last_peered":"2024-09-10T20:41:36.839670+0000","last_clean":"2024-09-10T20:41:36.839670+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:41:36.839670+0000","last_undegraded":"2024-09-10T20:41:36.839670+0000","last_fullsized":"2024-09-10T20:41:36.839670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28016,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840428+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:41:36.840428+0000","last_peered":"2024-09-10T20:41:36.840428+0000","last_clean":"2024-09-10T20:41:36.840428+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:41:36.840428+0000","last_undegraded":"2024-09-10T20:41:36.840428+0000","last_fullsized":"2024-09-10T20:41:36.840428+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":123,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841261+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:41:36.841261+0000","last_peered":"2024-09-10T20:41:36.841261+0000","last_clean":"2024-09-10T20:41:36.841261+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:41:36.841261+0000","last_undegraded":"2024-09-10T20:41:36.841261+0000","last_fullsized":"2024-09-10T20:41:36.841261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28069,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839259+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:41:36.839259+0000","last_peered":"2024-09-10T20:41:36.839259+0000","last_clean":"2024-09-10T20:41:36.839259+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:41:36.839259+0000","last_undegraded":"2024-09-10T20:41:36.839259+0000","last_fullsized":"2024-09-10T20:41:36.839259+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21125,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841491+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:41:36.841491+0000","last_peered":"2024-09-10T20:41:36.841491+0000","last_clean":"2024-09-10T20:41:36.841491+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:41:36.841491+0000","last_undegraded":"2024-09-10T20:41:36.841491+0000","last_fullsized":"2024-09-10T20:41:36.841491+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841466+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:41:36.841466+0000","last_peered":"2024-09-10T20:41:36.841466+0000","last_clean":"2024-09-10T20:41:36.841466+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:41:36.841466+0000","last_undegraded":"2024-09-10T20:41:36.841466+0000","last_fullsized":"2024-09-10T20:41:36.841466+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27847,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841464+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:41:36.841464+0000","last_peered":"2024-09-10T20:41:36.841464+0000","last_clean":"2024-09-10T20:41:36.841464+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:41:36.841464+0000","last_undegraded":"2024-09-10T20:41:36.841464+0000","last_fullsized":"2024-09-10T20:41:36.841464+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839959+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:41:36.839959+0000","last_peered":"2024-09-10T20:41:36.839959+0000","last_clean":"2024-09-10T20:41:36.839959+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:41:36.839959+0000","last_undegraded":"2024-09-10T20:41:36.839959+0000","last_fullsized":"2024-09-10T20:41:36.839959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21261,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.929378+0000","last_change":"2024-09-10T20:41:36.928383+0000","last_active":"2024-09-10T20:41:36.929378+0000","last_peered":"2024-09-10T20:41:36.929378+0000","last_clean":"2024-09-10T20:41:36.929378+0000","last_became_active":"2024-09-10T20:41:36.842440+0000","last_became_peered":"2024-09-10T20:41:36.842440+0000","last_unstale":"2024-09-10T20:41:36.929378+0000","last_undegraded":"2024-09-10T20:41:36.929378+0000","last_fullsized":"2024-09-10T20:41:36.929378+0000","mapping_epoch":64,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.940798+0000","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3715,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841381+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:41:36.841381+0000","last_peered":"2024-09-10T20:41:36.841381+0000","last_clean":"2024-09-10T20:41:36.841381+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:41:36.841381+0000","last_undegraded":"2024-09-10T20:41:36.841381+0000","last_fullsized":"2024-09-10T20:41:36.841381+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841415+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:41:36.841415+0000","last_peered":"2024-09-10T20:41:36.841415+0000","last_clean":"2024-09-10T20:41:36.841415+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:41:36.841415+0000","last_undegraded":"2024-09-10T20:41:36.841415+0000","last_fullsized":"2024-09-10T20:41:36.841415+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27677,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841380+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:41:36.841380+0000","last_peered":"2024-09-10T20:41:36.841380+0000","last_clean":"2024-09-10T20:41:36.841380+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:41:36.841380+0000","last_undegraded":"2024-09-10T20:41:36.841380+0000","last_fullsized":"2024-09-10T20:41:36.841380+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2144,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37386,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":125554688,"data_stored":125547462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182558,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517240,"kb_used_data":58020,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854600,"statfs":{"total":96636764160,"available":96107110400,"internally_reserved":0,"allocated":59412480,"data_stored":59071492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986330,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":431248,"kb_used_data":66368,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93940592,"statfs":{"total":96636764160,"available":96195166208,"internally_reserved":0,"allocated":67960832,"data_stored":67627534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659277,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497320,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874520,"statfs":{"total":96636764160,"available":96127508480,"internally_reserved":0,"allocated":548864,"data_stored":207876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":66682880,"data_stored":66682880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:41:53.299 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:53.599 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:53.599 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:53.612 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":471,"stamp":"2024-09-10T20:41:53.081717+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1362,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38170,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1449928,"kb_used_data":120724,"kb_used_omap":12,"kb_used_meta":1329139,"kb_avail":281665592,"statfs":{"total":289910292480,"available":288425566208,"internally_reserved":0,"allocated":123621376,"data_stored":122606102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1361038338},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1441,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1441,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001902"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":28004,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840754+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:41:36.840754+0000","last_peered":"2024-09-10T20:41:36.840754+0000","last_clean":"2024-09-10T20:41:36.840754+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:41:36.840754+0000","last_undegraded":"2024-09-10T20:41:36.840754+0000","last_fullsized":"2024-09-10T20:41:36.840754+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21096,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840133+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:41:36.840133+0000","last_peered":"2024-09-10T20:41:36.840133+0000","last_clean":"2024-09-10T20:41:36.840133+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:41:36.840133+0000","last_undegraded":"2024-09-10T20:41:36.840133+0000","last_fullsized":"2024-09-10T20:41:36.840133+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28108,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841579+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:41:36.841579+0000","last_peered":"2024-09-10T20:41:36.841579+0000","last_clean":"2024-09-10T20:41:36.841579+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:41:36.841579+0000","last_undegraded":"2024-09-10T20:41:36.841579+0000","last_fullsized":"2024-09-10T20:41:36.841579+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20728,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839605+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:41:36.839605+0000","last_peered":"2024-09-10T20:41:36.839605+0000","last_clean":"2024-09-10T20:41:36.839605+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:41:36.839605+0000","last_undegraded":"2024-09-10T20:41:36.839605+0000","last_fullsized":"2024-09-10T20:41:36.839605+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21239,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839929+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:41:36.839929+0000","last_peered":"2024-09-10T20:41:36.839929+0000","last_clean":"2024-09-10T20:41:36.839929+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:41:36.839929+0000","last_undegraded":"2024-09-10T20:41:36.839929+0000","last_fullsized":"2024-09-10T20:41:36.839929+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":25160,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:50.251446+0000","last_change":"2024-09-10T20:41:35.396233+0000","last_active":"2024-09-10T20:41:50.251446+0000","last_peered":"2024-09-10T20:41:50.251446+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:41:50.251446+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1362,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2254,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1362}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20915,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841297+0000","last_change":"2024-09-10T20:41:17.750026+0000","last_active":"2024-09-10T20:41:36.841297+0000","last_peered":"2024-09-10T20:41:36.841297+0000","last_clean":"2024-09-10T20:41:36.841297+0000","last_became_active":"2024-09-10T20:41:17.668013+0000","last_became_peered":"2024-09-10T20:41:17.668013+0000","last_unstale":"2024-09-10T20:41:36.841297+0000","last_undegraded":"2024-09-10T20:41:36.841297+0000","last_fullsized":"2024-09-10T20:41:36.841297+0000","mapping_epoch":62,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:55.885282+0000","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20904,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839732+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:41:36.839732+0000","last_peered":"2024-09-10T20:41:36.839732+0000","last_clean":"2024-09-10T20:41:36.839732+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:41:36.839732+0000","last_undegraded":"2024-09-10T20:41:36.839732+0000","last_fullsized":"2024-09-10T20:41:36.839732+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841162+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:41:36.841162+0000","last_peered":"2024-09-10T20:41:36.841162+0000","last_clean":"2024-09-10T20:41:36.841162+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:41:36.841162+0000","last_undegraded":"2024-09-10T20:41:36.841162+0000","last_fullsized":"2024-09-10T20:41:36.841162+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22147,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841390+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:41:36.841390+0000","last_peered":"2024-09-10T20:41:36.841390+0000","last_clean":"2024-09-10T20:41:36.841390+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:41:36.841390+0000","last_undegraded":"2024-09-10T20:41:36.841390+0000","last_fullsized":"2024-09-10T20:41:36.841390+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840435+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:41:36.840435+0000","last_peered":"2024-09-10T20:41:36.840435+0000","last_clean":"2024-09-10T20:41:36.840435+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:41:36.840435+0000","last_undegraded":"2024-09-10T20:41:36.840435+0000","last_fullsized":"2024-09-10T20:41:36.840435+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28133,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839783+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:41:36.839783+0000","last_peered":"2024-09-10T20:41:36.839783+0000","last_clean":"2024-09-10T20:41:36.839783+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:41:36.839783+0000","last_undegraded":"2024-09-10T20:41:36.839783+0000","last_fullsized":"2024-09-10T20:41:36.839783+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839670+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:41:36.839670+0000","last_peered":"2024-09-10T20:41:36.839670+0000","last_clean":"2024-09-10T20:41:36.839670+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:41:36.839670+0000","last_undegraded":"2024-09-10T20:41:36.839670+0000","last_fullsized":"2024-09-10T20:41:36.839670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28016,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840428+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:41:36.840428+0000","last_peered":"2024-09-10T20:41:36.840428+0000","last_clean":"2024-09-10T20:41:36.840428+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:41:36.840428+0000","last_undegraded":"2024-09-10T20:41:36.840428+0000","last_fullsized":"2024-09-10T20:41:36.840428+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":123,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841261+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:41:36.841261+0000","last_peered":"2024-09-10T20:41:36.841261+0000","last_clean":"2024-09-10T20:41:36.841261+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:41:36.841261+0000","last_undegraded":"2024-09-10T20:41:36.841261+0000","last_fullsized":"2024-09-10T20:41:36.841261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28069,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839259+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:41:36.839259+0000","last_peered":"2024-09-10T20:41:36.839259+0000","last_clean":"2024-09-10T20:41:36.839259+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:41:36.839259+0000","last_undegraded":"2024-09-10T20:41:36.839259+0000","last_fullsized":"2024-09-10T20:41:36.839259+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21125,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841491+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:41:36.841491+0000","last_peered":"2024-09-10T20:41:36.841491+0000","last_clean":"2024-09-10T20:41:36.841491+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:41:36.841491+0000","last_undegraded":"2024-09-10T20:41:36.841491+0000","last_fullsized":"2024-09-10T20:41:36.841491+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841466+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:41:36.841466+0000","last_peered":"2024-09-10T20:41:36.841466+0000","last_clean":"2024-09-10T20:41:36.841466+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:41:36.841466+0000","last_undegraded":"2024-09-10T20:41:36.841466+0000","last_fullsized":"2024-09-10T20:41:36.841466+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27847,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841464+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:41:36.841464+0000","last_peered":"2024-09-10T20:41:36.841464+0000","last_clean":"2024-09-10T20:41:36.841464+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:41:36.841464+0000","last_undegraded":"2024-09-10T20:41:36.841464+0000","last_fullsized":"2024-09-10T20:41:36.841464+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839959+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:41:36.839959+0000","last_peered":"2024-09-10T20:41:36.839959+0000","last_clean":"2024-09-10T20:41:36.839959+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:41:36.839959+0000","last_undegraded":"2024-09-10T20:41:36.839959+0000","last_fullsized":"2024-09-10T20:41:36.839959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21261,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.929378+0000","last_change":"2024-09-10T20:41:36.928383+0000","last_active":"2024-09-10T20:41:36.929378+0000","last_peered":"2024-09-10T20:41:36.929378+0000","last_clean":"2024-09-10T20:41:36.929378+0000","last_became_active":"2024-09-10T20:41:36.842440+0000","last_became_peered":"2024-09-10T20:41:36.842440+0000","last_unstale":"2024-09-10T20:41:36.929378+0000","last_undegraded":"2024-09-10T20:41:36.929378+0000","last_fullsized":"2024-09-10T20:41:36.929378+0000","mapping_epoch":64,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.940798+0000","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3715,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841381+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:41:36.841381+0000","last_peered":"2024-09-10T20:41:36.841381+0000","last_clean":"2024-09-10T20:41:36.841381+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:41:36.841381+0000","last_undegraded":"2024-09-10T20:41:36.841381+0000","last_fullsized":"2024-09-10T20:41:36.841381+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841415+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:41:36.841415+0000","last_peered":"2024-09-10T20:41:36.841415+0000","last_clean":"2024-09-10T20:41:36.841415+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:41:36.841415+0000","last_undegraded":"2024-09-10T20:41:36.841415+0000","last_fullsized":"2024-09-10T20:41:36.841415+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27677,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841380+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:41:36.841380+0000","last_peered":"2024-09-10T20:41:36.841380+0000","last_clean":"2024-09-10T20:41:36.841380+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:41:36.841380+0000","last_undegraded":"2024-09-10T20:41:36.841380+0000","last_fullsized":"2024-09-10T20:41:36.841380+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1362,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38168,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":121253888,"data_stored":121246662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182559,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517240,"kb_used_data":58020,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854600,"statfs":{"total":96636764160,"available":96107110400,"internally_reserved":0,"allocated":59412480,"data_stored":59071492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986331,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":427048,"kb_used_data":62168,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93944792,"statfs":{"total":96636764160,"available":96199467008,"internally_reserved":0,"allocated":63660032,"data_stored":63326734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659278,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505640,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866200,"statfs":{"total":96636764160,"available":96118988800,"internally_reserved":0,"allocated":548864,"data_stored":207876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":517205678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":62382080,"data_stored":62382080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:41:53.613 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:53.933 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:53.934 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:53.946 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":471,"stamp":"2024-09-10T20:41:53.081717+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1362,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38170,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1449928,"kb_used_data":120724,"kb_used_omap":12,"kb_used_meta":1329139,"kb_avail":281665592,"statfs":{"total":289910292480,"available":288425566208,"internally_reserved":0,"allocated":123621376,"data_stored":122606102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1361038338},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1441,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1441,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001902"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":28004,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840754+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:41:36.840754+0000","last_peered":"2024-09-10T20:41:36.840754+0000","last_clean":"2024-09-10T20:41:36.840754+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:41:36.840754+0000","last_undegraded":"2024-09-10T20:41:36.840754+0000","last_fullsized":"2024-09-10T20:41:36.840754+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21096,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840133+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:41:36.840133+0000","last_peered":"2024-09-10T20:41:36.840133+0000","last_clean":"2024-09-10T20:41:36.840133+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:41:36.840133+0000","last_undegraded":"2024-09-10T20:41:36.840133+0000","last_fullsized":"2024-09-10T20:41:36.840133+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28108,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841579+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:41:36.841579+0000","last_peered":"2024-09-10T20:41:36.841579+0000","last_clean":"2024-09-10T20:41:36.841579+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:41:36.841579+0000","last_undegraded":"2024-09-10T20:41:36.841579+0000","last_fullsized":"2024-09-10T20:41:36.841579+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20728,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839605+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:41:36.839605+0000","last_peered":"2024-09-10T20:41:36.839605+0000","last_clean":"2024-09-10T20:41:36.839605+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:41:36.839605+0000","last_undegraded":"2024-09-10T20:41:36.839605+0000","last_fullsized":"2024-09-10T20:41:36.839605+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21239,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839929+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:41:36.839929+0000","last_peered":"2024-09-10T20:41:36.839929+0000","last_clean":"2024-09-10T20:41:36.839929+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:41:36.839929+0000","last_undegraded":"2024-09-10T20:41:36.839929+0000","last_fullsized":"2024-09-10T20:41:36.839929+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":25160,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:50.251446+0000","last_change":"2024-09-10T20:41:35.396233+0000","last_active":"2024-09-10T20:41:50.251446+0000","last_peered":"2024-09-10T20:41:50.251446+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:41:50.251446+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1362,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2254,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1362}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20915,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841297+0000","last_change":"2024-09-10T20:41:17.750026+0000","last_active":"2024-09-10T20:41:36.841297+0000","last_peered":"2024-09-10T20:41:36.841297+0000","last_clean":"2024-09-10T20:41:36.841297+0000","last_became_active":"2024-09-10T20:41:17.668013+0000","last_became_peered":"2024-09-10T20:41:17.668013+0000","last_unstale":"2024-09-10T20:41:36.841297+0000","last_undegraded":"2024-09-10T20:41:36.841297+0000","last_fullsized":"2024-09-10T20:41:36.841297+0000","mapping_epoch":62,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:55.885282+0000","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20904,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839732+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:41:36.839732+0000","last_peered":"2024-09-10T20:41:36.839732+0000","last_clean":"2024-09-10T20:41:36.839732+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:41:36.839732+0000","last_undegraded":"2024-09-10T20:41:36.839732+0000","last_fullsized":"2024-09-10T20:41:36.839732+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841162+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:41:36.841162+0000","last_peered":"2024-09-10T20:41:36.841162+0000","last_clean":"2024-09-10T20:41:36.841162+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:41:36.841162+0000","last_undegraded":"2024-09-10T20:41:36.841162+0000","last_fullsized":"2024-09-10T20:41:36.841162+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22147,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841390+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:41:36.841390+0000","last_peered":"2024-09-10T20:41:36.841390+0000","last_clean":"2024-09-10T20:41:36.841390+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:41:36.841390+0000","last_undegraded":"2024-09-10T20:41:36.841390+0000","last_fullsized":"2024-09-10T20:41:36.841390+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840435+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:41:36.840435+0000","last_peered":"2024-09-10T20:41:36.840435+0000","last_clean":"2024-09-10T20:41:36.840435+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:41:36.840435+0000","last_undegraded":"2024-09-10T20:41:36.840435+0000","last_fullsized":"2024-09-10T20:41:36.840435+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28133,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839783+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:41:36.839783+0000","last_peered":"2024-09-10T20:41:36.839783+0000","last_clean":"2024-09-10T20:41:36.839783+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:41:36.839783+0000","last_undegraded":"2024-09-10T20:41:36.839783+0000","last_fullsized":"2024-09-10T20:41:36.839783+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839670+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:41:36.839670+0000","last_peered":"2024-09-10T20:41:36.839670+0000","last_clean":"2024-09-10T20:41:36.839670+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:41:36.839670+0000","last_undegraded":"2024-09-10T20:41:36.839670+0000","last_fullsized":"2024-09-10T20:41:36.839670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28016,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840428+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:41:36.840428+0000","last_peered":"2024-09-10T20:41:36.840428+0000","last_clean":"2024-09-10T20:41:36.840428+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:41:36.840428+0000","last_undegraded":"2024-09-10T20:41:36.840428+0000","last_fullsized":"2024-09-10T20:41:36.840428+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":123,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841261+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:41:36.841261+0000","last_peered":"2024-09-10T20:41:36.841261+0000","last_clean":"2024-09-10T20:41:36.841261+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:41:36.841261+0000","last_undegraded":"2024-09-10T20:41:36.841261+0000","last_fullsized":"2024-09-10T20:41:36.841261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28069,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839259+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:41:36.839259+0000","last_peered":"2024-09-10T20:41:36.839259+0000","last_clean":"2024-09-10T20:41:36.839259+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:41:36.839259+0000","last_undegraded":"2024-09-10T20:41:36.839259+0000","last_fullsized":"2024-09-10T20:41:36.839259+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21125,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841491+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:41:36.841491+0000","last_peered":"2024-09-10T20:41:36.841491+0000","last_clean":"2024-09-10T20:41:36.841491+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:41:36.841491+0000","last_undegraded":"2024-09-10T20:41:36.841491+0000","last_fullsized":"2024-09-10T20:41:36.841491+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841466+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:41:36.841466+0000","last_peered":"2024-09-10T20:41:36.841466+0000","last_clean":"2024-09-10T20:41:36.841466+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:41:36.841466+0000","last_undegraded":"2024-09-10T20:41:36.841466+0000","last_fullsized":"2024-09-10T20:41:36.841466+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27847,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841464+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:41:36.841464+0000","last_peered":"2024-09-10T20:41:36.841464+0000","last_clean":"2024-09-10T20:41:36.841464+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:41:36.841464+0000","last_undegraded":"2024-09-10T20:41:36.841464+0000","last_fullsized":"2024-09-10T20:41:36.841464+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839959+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:41:36.839959+0000","last_peered":"2024-09-10T20:41:36.839959+0000","last_clean":"2024-09-10T20:41:36.839959+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:41:36.839959+0000","last_undegraded":"2024-09-10T20:41:36.839959+0000","last_fullsized":"2024-09-10T20:41:36.839959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21261,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.929378+0000","last_change":"2024-09-10T20:41:36.928383+0000","last_active":"2024-09-10T20:41:36.929378+0000","last_peered":"2024-09-10T20:41:36.929378+0000","last_clean":"2024-09-10T20:41:36.929378+0000","last_became_active":"2024-09-10T20:41:36.842440+0000","last_became_peered":"2024-09-10T20:41:36.842440+0000","last_unstale":"2024-09-10T20:41:36.929378+0000","last_undegraded":"2024-09-10T20:41:36.929378+0000","last_fullsized":"2024-09-10T20:41:36.929378+0000","mapping_epoch":64,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.940798+0000","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3715,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841381+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:41:36.841381+0000","last_peered":"2024-09-10T20:41:36.841381+0000","last_clean":"2024-09-10T20:41:36.841381+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:41:36.841381+0000","last_undegraded":"2024-09-10T20:41:36.841381+0000","last_fullsized":"2024-09-10T20:41:36.841381+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841415+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:41:36.841415+0000","last_peered":"2024-09-10T20:41:36.841415+0000","last_clean":"2024-09-10T20:41:36.841415+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:41:36.841415+0000","last_undegraded":"2024-09-10T20:41:36.841415+0000","last_fullsized":"2024-09-10T20:41:36.841415+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27677,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841380+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:41:36.841380+0000","last_peered":"2024-09-10T20:41:36.841380+0000","last_clean":"2024-09-10T20:41:36.841380+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:41:36.841380+0000","last_undegraded":"2024-09-10T20:41:36.841380+0000","last_fullsized":"2024-09-10T20:41:36.841380+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1362,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38168,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":121253888,"data_stored":121246662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182559,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517240,"kb_used_data":58020,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854600,"statfs":{"total":96636764160,"available":96107110400,"internally_reserved":0,"allocated":59412480,"data_stored":59071492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986331,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":427048,"kb_used_data":62168,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93944792,"statfs":{"total":96636764160,"available":96199467008,"internally_reserved":0,"allocated":63660032,"data_stored":63326734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659278,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505640,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866200,"statfs":{"total":96636764160,"available":96118988800,"internally_reserved":0,"allocated":548864,"data_stored":207876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":517205678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":62382080,"data_stored":62382080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:41:53.947 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:41:53.947 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20471", 'reported_seq': 25160, 'reported_epoch': 65, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T20:41:50.251446+0000', 'last_change': '2024-09-10T20:41:35.396233+0000', 'last_active': '2024-09-10T20:41:50.251446+0000', 'last_peered': '2024-09-10T20:41:50.251446+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:41:50.251446+0000', 'last_undegraded': '2024-09-10T20:30:05.410858+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1362, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17335, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2254, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1362}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T20:41:53.948 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:41:54.343 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:54.343 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 1362/6 objects degraded (22700.000%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":825,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1484726272,"bytes_avail":288425566208,"bytes_total":289910292480,"degraded_objects":1362,"degraded_total":6,"degraded_ratio":227,"recovering_objects_per_sec":120,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1441,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:41:54.356 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:41:54.357 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:54.670 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:54.670 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:54.682 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":471,"stamp":"2024-09-10T20:41:53.081717+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1362,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38170,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1449928,"kb_used_data":120724,"kb_used_omap":12,"kb_used_meta":1329139,"kb_avail":281665592,"statfs":{"total":289910292480,"available":288425566208,"internally_reserved":0,"allocated":123621376,"data_stored":122606102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1361038338},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1441,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1441,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001902"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":28004,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840754+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:41:36.840754+0000","last_peered":"2024-09-10T20:41:36.840754+0000","last_clean":"2024-09-10T20:41:36.840754+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:41:36.840754+0000","last_undegraded":"2024-09-10T20:41:36.840754+0000","last_fullsized":"2024-09-10T20:41:36.840754+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21096,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840133+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:41:36.840133+0000","last_peered":"2024-09-10T20:41:36.840133+0000","last_clean":"2024-09-10T20:41:36.840133+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:41:36.840133+0000","last_undegraded":"2024-09-10T20:41:36.840133+0000","last_fullsized":"2024-09-10T20:41:36.840133+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28108,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841579+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:41:36.841579+0000","last_peered":"2024-09-10T20:41:36.841579+0000","last_clean":"2024-09-10T20:41:36.841579+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:41:36.841579+0000","last_undegraded":"2024-09-10T20:41:36.841579+0000","last_fullsized":"2024-09-10T20:41:36.841579+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20728,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839605+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:41:36.839605+0000","last_peered":"2024-09-10T20:41:36.839605+0000","last_clean":"2024-09-10T20:41:36.839605+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:41:36.839605+0000","last_undegraded":"2024-09-10T20:41:36.839605+0000","last_fullsized":"2024-09-10T20:41:36.839605+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21239,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839929+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:41:36.839929+0000","last_peered":"2024-09-10T20:41:36.839929+0000","last_clean":"2024-09-10T20:41:36.839929+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:41:36.839929+0000","last_undegraded":"2024-09-10T20:41:36.839929+0000","last_fullsized":"2024-09-10T20:41:36.839929+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":25160,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:50.251446+0000","last_change":"2024-09-10T20:41:35.396233+0000","last_active":"2024-09-10T20:41:50.251446+0000","last_peered":"2024-09-10T20:41:50.251446+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:41:50.251446+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1362,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2254,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1362}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20915,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841297+0000","last_change":"2024-09-10T20:41:17.750026+0000","last_active":"2024-09-10T20:41:36.841297+0000","last_peered":"2024-09-10T20:41:36.841297+0000","last_clean":"2024-09-10T20:41:36.841297+0000","last_became_active":"2024-09-10T20:41:17.668013+0000","last_became_peered":"2024-09-10T20:41:17.668013+0000","last_unstale":"2024-09-10T20:41:36.841297+0000","last_undegraded":"2024-09-10T20:41:36.841297+0000","last_fullsized":"2024-09-10T20:41:36.841297+0000","mapping_epoch":62,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:55.885282+0000","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20904,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839732+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:41:36.839732+0000","last_peered":"2024-09-10T20:41:36.839732+0000","last_clean":"2024-09-10T20:41:36.839732+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:41:36.839732+0000","last_undegraded":"2024-09-10T20:41:36.839732+0000","last_fullsized":"2024-09-10T20:41:36.839732+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841162+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:41:36.841162+0000","last_peered":"2024-09-10T20:41:36.841162+0000","last_clean":"2024-09-10T20:41:36.841162+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:41:36.841162+0000","last_undegraded":"2024-09-10T20:41:36.841162+0000","last_fullsized":"2024-09-10T20:41:36.841162+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22147,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841390+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:41:36.841390+0000","last_peered":"2024-09-10T20:41:36.841390+0000","last_clean":"2024-09-10T20:41:36.841390+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:41:36.841390+0000","last_undegraded":"2024-09-10T20:41:36.841390+0000","last_fullsized":"2024-09-10T20:41:36.841390+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840435+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:41:36.840435+0000","last_peered":"2024-09-10T20:41:36.840435+0000","last_clean":"2024-09-10T20:41:36.840435+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:41:36.840435+0000","last_undegraded":"2024-09-10T20:41:36.840435+0000","last_fullsized":"2024-09-10T20:41:36.840435+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28133,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839783+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:41:36.839783+0000","last_peered":"2024-09-10T20:41:36.839783+0000","last_clean":"2024-09-10T20:41:36.839783+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:41:36.839783+0000","last_undegraded":"2024-09-10T20:41:36.839783+0000","last_fullsized":"2024-09-10T20:41:36.839783+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839670+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:41:36.839670+0000","last_peered":"2024-09-10T20:41:36.839670+0000","last_clean":"2024-09-10T20:41:36.839670+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:41:36.839670+0000","last_undegraded":"2024-09-10T20:41:36.839670+0000","last_fullsized":"2024-09-10T20:41:36.839670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28016,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840428+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:41:36.840428+0000","last_peered":"2024-09-10T20:41:36.840428+0000","last_clean":"2024-09-10T20:41:36.840428+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:41:36.840428+0000","last_undegraded":"2024-09-10T20:41:36.840428+0000","last_fullsized":"2024-09-10T20:41:36.840428+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":123,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841261+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:41:36.841261+0000","last_peered":"2024-09-10T20:41:36.841261+0000","last_clean":"2024-09-10T20:41:36.841261+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:41:36.841261+0000","last_undegraded":"2024-09-10T20:41:36.841261+0000","last_fullsized":"2024-09-10T20:41:36.841261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28069,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839259+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:41:36.839259+0000","last_peered":"2024-09-10T20:41:36.839259+0000","last_clean":"2024-09-10T20:41:36.839259+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:41:36.839259+0000","last_undegraded":"2024-09-10T20:41:36.839259+0000","last_fullsized":"2024-09-10T20:41:36.839259+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21125,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841491+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:41:36.841491+0000","last_peered":"2024-09-10T20:41:36.841491+0000","last_clean":"2024-09-10T20:41:36.841491+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:41:36.841491+0000","last_undegraded":"2024-09-10T20:41:36.841491+0000","last_fullsized":"2024-09-10T20:41:36.841491+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841466+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:41:36.841466+0000","last_peered":"2024-09-10T20:41:36.841466+0000","last_clean":"2024-09-10T20:41:36.841466+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:41:36.841466+0000","last_undegraded":"2024-09-10T20:41:36.841466+0000","last_fullsized":"2024-09-10T20:41:36.841466+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27847,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841464+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:41:36.841464+0000","last_peered":"2024-09-10T20:41:36.841464+0000","last_clean":"2024-09-10T20:41:36.841464+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:41:36.841464+0000","last_undegraded":"2024-09-10T20:41:36.841464+0000","last_fullsized":"2024-09-10T20:41:36.841464+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839959+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:41:36.839959+0000","last_peered":"2024-09-10T20:41:36.839959+0000","last_clean":"2024-09-10T20:41:36.839959+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:41:36.839959+0000","last_undegraded":"2024-09-10T20:41:36.839959+0000","last_fullsized":"2024-09-10T20:41:36.839959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21261,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.929378+0000","last_change":"2024-09-10T20:41:36.928383+0000","last_active":"2024-09-10T20:41:36.929378+0000","last_peered":"2024-09-10T20:41:36.929378+0000","last_clean":"2024-09-10T20:41:36.929378+0000","last_became_active":"2024-09-10T20:41:36.842440+0000","last_became_peered":"2024-09-10T20:41:36.842440+0000","last_unstale":"2024-09-10T20:41:36.929378+0000","last_undegraded":"2024-09-10T20:41:36.929378+0000","last_fullsized":"2024-09-10T20:41:36.929378+0000","mapping_epoch":64,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.940798+0000","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3715,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841381+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:41:36.841381+0000","last_peered":"2024-09-10T20:41:36.841381+0000","last_clean":"2024-09-10T20:41:36.841381+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:41:36.841381+0000","last_undegraded":"2024-09-10T20:41:36.841381+0000","last_fullsized":"2024-09-10T20:41:36.841381+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841415+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:41:36.841415+0000","last_peered":"2024-09-10T20:41:36.841415+0000","last_clean":"2024-09-10T20:41:36.841415+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:41:36.841415+0000","last_undegraded":"2024-09-10T20:41:36.841415+0000","last_fullsized":"2024-09-10T20:41:36.841415+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27677,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841380+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:41:36.841380+0000","last_peered":"2024-09-10T20:41:36.841380+0000","last_clean":"2024-09-10T20:41:36.841380+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:41:36.841380+0000","last_undegraded":"2024-09-10T20:41:36.841380+0000","last_fullsized":"2024-09-10T20:41:36.841380+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1362,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38168,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":121253888,"data_stored":121246662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182559,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517240,"kb_used_data":58020,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854600,"statfs":{"total":96636764160,"available":96107110400,"internally_reserved":0,"allocated":59412480,"data_stored":59071492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986331,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":427048,"kb_used_data":62168,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93944792,"statfs":{"total":96636764160,"available":96199467008,"internally_reserved":0,"allocated":63660032,"data_stored":63326734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659278,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505640,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866200,"statfs":{"total":96636764160,"available":96118988800,"internally_reserved":0,"allocated":548864,"data_stored":207876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":517205678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":62382080,"data_stored":62382080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:41:57.684 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:57.973 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:57.974 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:57.985 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":474,"stamp":"2024-09-10T20:41:57.196012+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":342,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39190,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1445728,"kb_used_data":116524,"kb_used_omap":12,"kb_used_meta":1329139,"kb_avail":281669792,"statfs":{"total":289910292480,"available":288429867008,"internally_reserved":0,"allocated":119320576,"data_stored":118305302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1361038338},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1802,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1802,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.115333"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":28004,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840754+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:41:36.840754+0000","last_peered":"2024-09-10T20:41:36.840754+0000","last_clean":"2024-09-10T20:41:36.840754+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:41:36.840754+0000","last_undegraded":"2024-09-10T20:41:36.840754+0000","last_fullsized":"2024-09-10T20:41:36.840754+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21096,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840133+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:41:36.840133+0000","last_peered":"2024-09-10T20:41:36.840133+0000","last_clean":"2024-09-10T20:41:36.840133+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:41:36.840133+0000","last_undegraded":"2024-09-10T20:41:36.840133+0000","last_fullsized":"2024-09-10T20:41:36.840133+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28108,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841579+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:41:36.841579+0000","last_peered":"2024-09-10T20:41:36.841579+0000","last_clean":"2024-09-10T20:41:36.841579+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:41:36.841579+0000","last_undegraded":"2024-09-10T20:41:36.841579+0000","last_fullsized":"2024-09-10T20:41:36.841579+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20728,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839605+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:41:36.839605+0000","last_peered":"2024-09-10T20:41:36.839605+0000","last_clean":"2024-09-10T20:41:36.839605+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:41:36.839605+0000","last_undegraded":"2024-09-10T20:41:36.839605+0000","last_fullsized":"2024-09-10T20:41:36.839605+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21239,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839929+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:41:36.839929+0000","last_peered":"2024-09-10T20:41:36.839929+0000","last_clean":"2024-09-10T20:41:36.839929+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:41:36.839929+0000","last_undegraded":"2024-09-10T20:41:36.839929+0000","last_fullsized":"2024-09-10T20:41:36.839929+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":27200,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T20:41:55.250441+0000","last_change":"2024-09-10T20:41:35.396233+0000","last_active":"2024-09-10T20:41:55.250441+0000","last_peered":"2024-09-10T20:41:55.250441+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:41:55.250441+0000","last_undegraded":"2024-09-10T20:30:05.410858+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":342,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3274,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-342}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20915,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841297+0000","last_change":"2024-09-10T20:41:17.750026+0000","last_active":"2024-09-10T20:41:36.841297+0000","last_peered":"2024-09-10T20:41:36.841297+0000","last_clean":"2024-09-10T20:41:36.841297+0000","last_became_active":"2024-09-10T20:41:17.668013+0000","last_became_peered":"2024-09-10T20:41:17.668013+0000","last_unstale":"2024-09-10T20:41:36.841297+0000","last_undegraded":"2024-09-10T20:41:36.841297+0000","last_fullsized":"2024-09-10T20:41:36.841297+0000","mapping_epoch":62,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:55.885282+0000","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20904,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839732+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:41:36.839732+0000","last_peered":"2024-09-10T20:41:36.839732+0000","last_clean":"2024-09-10T20:41:36.839732+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:41:36.839732+0000","last_undegraded":"2024-09-10T20:41:36.839732+0000","last_fullsized":"2024-09-10T20:41:36.839732+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841162+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:41:36.841162+0000","last_peered":"2024-09-10T20:41:36.841162+0000","last_clean":"2024-09-10T20:41:36.841162+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:41:36.841162+0000","last_undegraded":"2024-09-10T20:41:36.841162+0000","last_fullsized":"2024-09-10T20:41:36.841162+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22147,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841390+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:41:36.841390+0000","last_peered":"2024-09-10T20:41:36.841390+0000","last_clean":"2024-09-10T20:41:36.841390+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:41:36.841390+0000","last_undegraded":"2024-09-10T20:41:36.841390+0000","last_fullsized":"2024-09-10T20:41:36.841390+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840435+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:41:36.840435+0000","last_peered":"2024-09-10T20:41:36.840435+0000","last_clean":"2024-09-10T20:41:36.840435+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:41:36.840435+0000","last_undegraded":"2024-09-10T20:41:36.840435+0000","last_fullsized":"2024-09-10T20:41:36.840435+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28133,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839783+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:41:36.839783+0000","last_peered":"2024-09-10T20:41:36.839783+0000","last_clean":"2024-09-10T20:41:36.839783+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:41:36.839783+0000","last_undegraded":"2024-09-10T20:41:36.839783+0000","last_fullsized":"2024-09-10T20:41:36.839783+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839670+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:41:36.839670+0000","last_peered":"2024-09-10T20:41:36.839670+0000","last_clean":"2024-09-10T20:41:36.839670+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:41:36.839670+0000","last_undegraded":"2024-09-10T20:41:36.839670+0000","last_fullsized":"2024-09-10T20:41:36.839670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28016,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840428+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:41:36.840428+0000","last_peered":"2024-09-10T20:41:36.840428+0000","last_clean":"2024-09-10T20:41:36.840428+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:41:36.840428+0000","last_undegraded":"2024-09-10T20:41:36.840428+0000","last_fullsized":"2024-09-10T20:41:36.840428+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":123,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841261+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:41:36.841261+0000","last_peered":"2024-09-10T20:41:36.841261+0000","last_clean":"2024-09-10T20:41:36.841261+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:41:36.841261+0000","last_undegraded":"2024-09-10T20:41:36.841261+0000","last_fullsized":"2024-09-10T20:41:36.841261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28069,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839259+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:41:36.839259+0000","last_peered":"2024-09-10T20:41:36.839259+0000","last_clean":"2024-09-10T20:41:36.839259+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:41:36.839259+0000","last_undegraded":"2024-09-10T20:41:36.839259+0000","last_fullsized":"2024-09-10T20:41:36.839259+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21125,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841491+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:41:36.841491+0000","last_peered":"2024-09-10T20:41:36.841491+0000","last_clean":"2024-09-10T20:41:36.841491+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:41:36.841491+0000","last_undegraded":"2024-09-10T20:41:36.841491+0000","last_fullsized":"2024-09-10T20:41:36.841491+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841466+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:41:36.841466+0000","last_peered":"2024-09-10T20:41:36.841466+0000","last_clean":"2024-09-10T20:41:36.841466+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:41:36.841466+0000","last_undegraded":"2024-09-10T20:41:36.841466+0000","last_fullsized":"2024-09-10T20:41:36.841466+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27847,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841464+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:41:36.841464+0000","last_peered":"2024-09-10T20:41:36.841464+0000","last_clean":"2024-09-10T20:41:36.841464+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:41:36.841464+0000","last_undegraded":"2024-09-10T20:41:36.841464+0000","last_fullsized":"2024-09-10T20:41:36.841464+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839959+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:41:36.839959+0000","last_peered":"2024-09-10T20:41:36.839959+0000","last_clean":"2024-09-10T20:41:36.839959+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:41:36.839959+0000","last_undegraded":"2024-09-10T20:41:36.839959+0000","last_fullsized":"2024-09-10T20:41:36.839959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21261,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.929378+0000","last_change":"2024-09-10T20:41:36.928383+0000","last_active":"2024-09-10T20:41:36.929378+0000","last_peered":"2024-09-10T20:41:36.929378+0000","last_clean":"2024-09-10T20:41:36.929378+0000","last_became_active":"2024-09-10T20:41:36.842440+0000","last_became_peered":"2024-09-10T20:41:36.842440+0000","last_unstale":"2024-09-10T20:41:36.929378+0000","last_undegraded":"2024-09-10T20:41:36.929378+0000","last_fullsized":"2024-09-10T20:41:36.929378+0000","mapping_epoch":64,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.940798+0000","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3715,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841381+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:41:36.841381+0000","last_peered":"2024-09-10T20:41:36.841381+0000","last_clean":"2024-09-10T20:41:36.841381+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:41:36.841381+0000","last_undegraded":"2024-09-10T20:41:36.841381+0000","last_fullsized":"2024-09-10T20:41:36.841381+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841415+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:41:36.841415+0000","last_peered":"2024-09-10T20:41:36.841415+0000","last_clean":"2024-09-10T20:41:36.841415+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:41:36.841415+0000","last_undegraded":"2024-09-10T20:41:36.841415+0000","last_fullsized":"2024-09-10T20:41:36.841415+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27677,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.841380+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:41:36.841380+0000","last_peered":"2024-09-10T20:41:36.841380+0000","last_clean":"2024-09-10T20:41:36.841380+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:41:36.841380+0000","last_undegraded":"2024-09-10T20:41:36.841380+0000","last_fullsized":"2024-09-10T20:41:36.841380+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":342,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39188,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":116953088,"data_stored":116945862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182560,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517240,"kb_used_data":58020,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854600,"statfs":{"total":96636764160,"available":96107110400,"internally_reserved":0,"allocated":59412480,"data_stored":59071492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986332,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422848,"kb_used_data":57968,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93948992,"statfs":{"total":96636764160,"available":96203767808,"internally_reserved":0,"allocated":59359232,"data_stored":59025934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659278,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505640,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866200,"statfs":{"total":96636764160,"available":96118988800,"internally_reserved":0,"allocated":548864,"data_stored":207876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":517205678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":58081280,"data_stored":58081280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:41:57.987 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:58.272 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:58.272 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:58.284 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":475,"stamp":"2024-09-10T20:41:58.195591+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39532,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1437408,"kb_used_data":116524,"kb_used_omap":12,"kb_used_meta":1320819,"kb_avail":281678112,"statfs":{"total":289910292480,"available":288438386688,"internally_reserved":0,"allocated":119320576,"data_stored":118305302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1352518658},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2144,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.114555"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":28004,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840754+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:41:36.840754+0000","last_peered":"2024-09-10T20:41:36.840754+0000","last_clean":"2024-09-10T20:41:36.840754+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:41:36.840754+0000","last_undegraded":"2024-09-10T20:41:36.840754+0000","last_fullsized":"2024-09-10T20:41:36.840754+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21096,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840133+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:41:36.840133+0000","last_peered":"2024-09-10T20:41:36.840133+0000","last_clean":"2024-09-10T20:41:36.840133+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:41:36.840133+0000","last_undegraded":"2024-09-10T20:41:36.840133+0000","last_fullsized":"2024-09-10T20:41:36.840133+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28110,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197414+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:41:57.197414+0000","last_peered":"2024-09-10T20:41:57.197414+0000","last_clean":"2024-09-10T20:41:57.197414+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:41:57.197414+0000","last_undegraded":"2024-09-10T20:41:57.197414+0000","last_fullsized":"2024-09-10T20:41:57.197414+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20728,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839605+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:41:36.839605+0000","last_peered":"2024-09-10T20:41:36.839605+0000","last_clean":"2024-09-10T20:41:36.839605+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:41:36.839605+0000","last_undegraded":"2024-09-10T20:41:36.839605+0000","last_fullsized":"2024-09-10T20:41:36.839605+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21239,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839929+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:41:36.839929+0000","last_peered":"2024-09-10T20:41:36.839929+0000","last_clean":"2024-09-10T20:41:36.839929+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:41:36.839929+0000","last_undegraded":"2024-09-10T20:41:36.839929+0000","last_fullsized":"2024-09-10T20:41:36.839929+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20732,"reported_epoch":66,"state":"peering","last_fresh":"2024-09-10T20:41:56.475957+0000","last_change":"2024-09-10T20:41:57.296954+0000","last_active":"2024-09-10T20:41:57.296954+0000","last_peered":"2024-09-10T20:41:56.475957+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:41:56.475957+0000","last_undegraded":"2024-09-10T20:41:56.475957+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":66,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3616,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20917,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197229+0000","last_change":"2024-09-10T20:41:17.750026+0000","last_active":"2024-09-10T20:41:57.197229+0000","last_peered":"2024-09-10T20:41:57.197229+0000","last_clean":"2024-09-10T20:41:57.197229+0000","last_became_active":"2024-09-10T20:41:17.668013+0000","last_became_peered":"2024-09-10T20:41:17.668013+0000","last_unstale":"2024-09-10T20:41:57.197229+0000","last_undegraded":"2024-09-10T20:41:57.197229+0000","last_fullsized":"2024-09-10T20:41:57.197229+0000","mapping_epoch":62,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:55.885282+0000","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20904,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839732+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:41:36.839732+0000","last_peered":"2024-09-10T20:41:36.839732+0000","last_clean":"2024-09-10T20:41:36.839732+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:41:36.839732+0000","last_undegraded":"2024-09-10T20:41:36.839732+0000","last_fullsized":"2024-09-10T20:41:36.839732+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":115,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197272+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:41:57.197272+0000","last_peered":"2024-09-10T20:41:57.197272+0000","last_clean":"2024-09-10T20:41:57.197272+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:41:57.197272+0000","last_undegraded":"2024-09-10T20:41:57.197272+0000","last_fullsized":"2024-09-10T20:41:57.197272+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22149,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197414+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:41:57.197414+0000","last_peered":"2024-09-10T20:41:57.197414+0000","last_clean":"2024-09-10T20:41:57.197414+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:41:57.197414+0000","last_undegraded":"2024-09-10T20:41:57.197414+0000","last_fullsized":"2024-09-10T20:41:57.197414+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840435+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:41:36.840435+0000","last_peered":"2024-09-10T20:41:36.840435+0000","last_clean":"2024-09-10T20:41:36.840435+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:41:36.840435+0000","last_undegraded":"2024-09-10T20:41:36.840435+0000","last_fullsized":"2024-09-10T20:41:36.840435+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28133,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839783+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:41:36.839783+0000","last_peered":"2024-09-10T20:41:36.839783+0000","last_clean":"2024-09-10T20:41:36.839783+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:41:36.839783+0000","last_undegraded":"2024-09-10T20:41:36.839783+0000","last_fullsized":"2024-09-10T20:41:36.839783+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839670+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:41:36.839670+0000","last_peered":"2024-09-10T20:41:36.839670+0000","last_clean":"2024-09-10T20:41:36.839670+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:41:36.839670+0000","last_undegraded":"2024-09-10T20:41:36.839670+0000","last_fullsized":"2024-09-10T20:41:36.839670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28016,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840428+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:41:36.840428+0000","last_peered":"2024-09-10T20:41:36.840428+0000","last_clean":"2024-09-10T20:41:36.840428+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:41:36.840428+0000","last_undegraded":"2024-09-10T20:41:36.840428+0000","last_fullsized":"2024-09-10T20:41:36.840428+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":125,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197372+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:41:57.197372+0000","last_peered":"2024-09-10T20:41:57.197372+0000","last_clean":"2024-09-10T20:41:57.197372+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:41:57.197372+0000","last_undegraded":"2024-09-10T20:41:57.197372+0000","last_fullsized":"2024-09-10T20:41:57.197372+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28069,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839259+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:41:36.839259+0000","last_peered":"2024-09-10T20:41:36.839259+0000","last_clean":"2024-09-10T20:41:36.839259+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:41:36.839259+0000","last_undegraded":"2024-09-10T20:41:36.839259+0000","last_fullsized":"2024-09-10T20:41:36.839259+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21127,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197730+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:41:57.197730+0000","last_peered":"2024-09-10T20:41:57.197730+0000","last_clean":"2024-09-10T20:41:57.197730+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:41:57.197730+0000","last_undegraded":"2024-09-10T20:41:57.197730+0000","last_fullsized":"2024-09-10T20:41:57.197730+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":115,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197508+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:41:57.197508+0000","last_peered":"2024-09-10T20:41:57.197508+0000","last_clean":"2024-09-10T20:41:57.197508+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:41:57.197508+0000","last_undegraded":"2024-09-10T20:41:57.197508+0000","last_fullsized":"2024-09-10T20:41:57.197508+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27849,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197609+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:41:57.197609+0000","last_peered":"2024-09-10T20:41:57.197609+0000","last_clean":"2024-09-10T20:41:57.197609+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:41:57.197609+0000","last_undegraded":"2024-09-10T20:41:57.197609+0000","last_fullsized":"2024-09-10T20:41:57.197609+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839959+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:41:36.839959+0000","last_peered":"2024-09-10T20:41:36.839959+0000","last_clean":"2024-09-10T20:41:36.839959+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:41:36.839959+0000","last_undegraded":"2024-09-10T20:41:36.839959+0000","last_fullsized":"2024-09-10T20:41:36.839959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21263,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197457+0000","last_change":"2024-09-10T20:41:36.928383+0000","last_active":"2024-09-10T20:41:57.197457+0000","last_peered":"2024-09-10T20:41:57.197457+0000","last_clean":"2024-09-10T20:41:57.197457+0000","last_became_active":"2024-09-10T20:41:36.842440+0000","last_became_peered":"2024-09-10T20:41:36.842440+0000","last_unstale":"2024-09-10T20:41:57.197457+0000","last_undegraded":"2024-09-10T20:41:57.197457+0000","last_fullsized":"2024-09-10T20:41:57.197457+0000","mapping_epoch":64,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.940798+0000","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3715,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":115,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197491+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:41:57.197491+0000","last_peered":"2024-09-10T20:41:57.197491+0000","last_clean":"2024-09-10T20:41:57.197491+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:41:57.197491+0000","last_undegraded":"2024-09-10T20:41:57.197491+0000","last_fullsized":"2024-09-10T20:41:57.197491+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":120,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197644+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:41:57.197644+0000","last_peered":"2024-09-10T20:41:57.197644+0000","last_clean":"2024-09-10T20:41:57.197644+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:41:57.197644+0000","last_undegraded":"2024-09-10T20:41:57.197644+0000","last_fullsized":"2024-09-10T20:41:57.197644+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27679,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197402+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:41:57.197402+0000","last_peered":"2024-09-10T20:41:57.197402+0000","last_clean":"2024-09-10T20:41:57.197402+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:41:57.197402+0000","last_undegraded":"2024-09-10T20:41:57.197402+0000","last_fullsized":"2024-09-10T20:41:57.197402+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39530,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":116953088,"data_stored":116945862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182560,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517240,"kb_used_data":58020,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854600,"statfs":{"total":96636764160,"available":96107110400,"internally_reserved":0,"allocated":59412480,"data_stored":59071492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986332,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422848,"kb_used_data":57968,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93948992,"statfs":{"total":96636764160,"available":96203767808,"internally_reserved":0,"allocated":59359232,"data_stored":59025934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659279,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497320,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874520,"statfs":{"total":96636764160,"available":96127508480,"internally_reserved":0,"allocated":548864,"data_stored":207876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":58081280,"data_stored":58081280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:41:58.286 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:41:58.286 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20471", 'reported_seq': 20732, 'reported_epoch': 66, 'state': 'peering', 'last_fresh': '2024-09-10T20:41:56.475957+0000', 'last_change': '2024-09-10T20:41:57.296954+0000', 'last_active': '2024-09-10T20:41:57.296954+0000', 'last_peered': '2024-09-10T20:41:56.475957+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:41:56.475957+0000', 'last_undegraded': '2024-09-10T20:41:56.475957+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 66, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17335, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3616, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T20:41:58.286 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:41:58.636 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:58.636 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 342/6 objects degraded (5700.000%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":830,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":67,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1480425472,"bytes_avail":288429867008,"bytes_total":289910292480,"degraded_objects":342,"degraded_total":6,"degraded_ratio":57,"recovering_objects_per_sec":205,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2461,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:41:58.649 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:41:58.649 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:41:58.932 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:41:58.933 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:41:58.945 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":475,"stamp":"2024-09-10T20:41:58.195591+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39532,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1437408,"kb_used_data":116524,"kb_used_omap":12,"kb_used_meta":1320819,"kb_avail":281678112,"statfs":{"total":289910292480,"available":288438386688,"internally_reserved":0,"allocated":119320576,"data_stored":118305302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1352518658},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2144,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.114555"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":28004,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840754+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:41:36.840754+0000","last_peered":"2024-09-10T20:41:36.840754+0000","last_clean":"2024-09-10T20:41:36.840754+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:41:36.840754+0000","last_undegraded":"2024-09-10T20:41:36.840754+0000","last_fullsized":"2024-09-10T20:41:36.840754+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21096,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840133+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:41:36.840133+0000","last_peered":"2024-09-10T20:41:36.840133+0000","last_clean":"2024-09-10T20:41:36.840133+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:41:36.840133+0000","last_undegraded":"2024-09-10T20:41:36.840133+0000","last_fullsized":"2024-09-10T20:41:36.840133+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28110,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197414+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:41:57.197414+0000","last_peered":"2024-09-10T20:41:57.197414+0000","last_clean":"2024-09-10T20:41:57.197414+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:41:57.197414+0000","last_undegraded":"2024-09-10T20:41:57.197414+0000","last_fullsized":"2024-09-10T20:41:57.197414+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20728,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839605+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:41:36.839605+0000","last_peered":"2024-09-10T20:41:36.839605+0000","last_clean":"2024-09-10T20:41:36.839605+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:41:36.839605+0000","last_undegraded":"2024-09-10T20:41:36.839605+0000","last_fullsized":"2024-09-10T20:41:36.839605+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21239,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839929+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:41:36.839929+0000","last_peered":"2024-09-10T20:41:36.839929+0000","last_clean":"2024-09-10T20:41:36.839929+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:41:36.839929+0000","last_undegraded":"2024-09-10T20:41:36.839929+0000","last_fullsized":"2024-09-10T20:41:36.839929+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20732,"reported_epoch":66,"state":"peering","last_fresh":"2024-09-10T20:41:56.475957+0000","last_change":"2024-09-10T20:41:57.296954+0000","last_active":"2024-09-10T20:41:57.296954+0000","last_peered":"2024-09-10T20:41:56.475957+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:41:56.475957+0000","last_undegraded":"2024-09-10T20:41:56.475957+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":66,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3616,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20917,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197229+0000","last_change":"2024-09-10T20:41:17.750026+0000","last_active":"2024-09-10T20:41:57.197229+0000","last_peered":"2024-09-10T20:41:57.197229+0000","last_clean":"2024-09-10T20:41:57.197229+0000","last_became_active":"2024-09-10T20:41:17.668013+0000","last_became_peered":"2024-09-10T20:41:17.668013+0000","last_unstale":"2024-09-10T20:41:57.197229+0000","last_undegraded":"2024-09-10T20:41:57.197229+0000","last_fullsized":"2024-09-10T20:41:57.197229+0000","mapping_epoch":62,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:55.885282+0000","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20904,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839732+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:41:36.839732+0000","last_peered":"2024-09-10T20:41:36.839732+0000","last_clean":"2024-09-10T20:41:36.839732+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:41:36.839732+0000","last_undegraded":"2024-09-10T20:41:36.839732+0000","last_fullsized":"2024-09-10T20:41:36.839732+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":115,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197272+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:41:57.197272+0000","last_peered":"2024-09-10T20:41:57.197272+0000","last_clean":"2024-09-10T20:41:57.197272+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:41:57.197272+0000","last_undegraded":"2024-09-10T20:41:57.197272+0000","last_fullsized":"2024-09-10T20:41:57.197272+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22149,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197414+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:41:57.197414+0000","last_peered":"2024-09-10T20:41:57.197414+0000","last_clean":"2024-09-10T20:41:57.197414+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:41:57.197414+0000","last_undegraded":"2024-09-10T20:41:57.197414+0000","last_fullsized":"2024-09-10T20:41:57.197414+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840435+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:41:36.840435+0000","last_peered":"2024-09-10T20:41:36.840435+0000","last_clean":"2024-09-10T20:41:36.840435+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:41:36.840435+0000","last_undegraded":"2024-09-10T20:41:36.840435+0000","last_fullsized":"2024-09-10T20:41:36.840435+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28133,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839783+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:41:36.839783+0000","last_peered":"2024-09-10T20:41:36.839783+0000","last_clean":"2024-09-10T20:41:36.839783+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:41:36.839783+0000","last_undegraded":"2024-09-10T20:41:36.839783+0000","last_fullsized":"2024-09-10T20:41:36.839783+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839670+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:41:36.839670+0000","last_peered":"2024-09-10T20:41:36.839670+0000","last_clean":"2024-09-10T20:41:36.839670+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:41:36.839670+0000","last_undegraded":"2024-09-10T20:41:36.839670+0000","last_fullsized":"2024-09-10T20:41:36.839670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28016,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.840428+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:41:36.840428+0000","last_peered":"2024-09-10T20:41:36.840428+0000","last_clean":"2024-09-10T20:41:36.840428+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:41:36.840428+0000","last_undegraded":"2024-09-10T20:41:36.840428+0000","last_fullsized":"2024-09-10T20:41:36.840428+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":125,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197372+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:41:57.197372+0000","last_peered":"2024-09-10T20:41:57.197372+0000","last_clean":"2024-09-10T20:41:57.197372+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:41:57.197372+0000","last_undegraded":"2024-09-10T20:41:57.197372+0000","last_fullsized":"2024-09-10T20:41:57.197372+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28069,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839259+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:41:36.839259+0000","last_peered":"2024-09-10T20:41:36.839259+0000","last_clean":"2024-09-10T20:41:36.839259+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:41:36.839259+0000","last_undegraded":"2024-09-10T20:41:36.839259+0000","last_fullsized":"2024-09-10T20:41:36.839259+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21127,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197730+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:41:57.197730+0000","last_peered":"2024-09-10T20:41:57.197730+0000","last_clean":"2024-09-10T20:41:57.197730+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:41:57.197730+0000","last_undegraded":"2024-09-10T20:41:57.197730+0000","last_fullsized":"2024-09-10T20:41:57.197730+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":115,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197508+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:41:57.197508+0000","last_peered":"2024-09-10T20:41:57.197508+0000","last_clean":"2024-09-10T20:41:57.197508+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:41:57.197508+0000","last_undegraded":"2024-09-10T20:41:57.197508+0000","last_fullsized":"2024-09-10T20:41:57.197508+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27849,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197609+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:41:57.197609+0000","last_peered":"2024-09-10T20:41:57.197609+0000","last_clean":"2024-09-10T20:41:57.197609+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:41:57.197609+0000","last_undegraded":"2024-09-10T20:41:57.197609+0000","last_fullsized":"2024-09-10T20:41:57.197609+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T20:41:36.839959+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:41:36.839959+0000","last_peered":"2024-09-10T20:41:36.839959+0000","last_clean":"2024-09-10T20:41:36.839959+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:41:36.839959+0000","last_undegraded":"2024-09-10T20:41:36.839959+0000","last_fullsized":"2024-09-10T20:41:36.839959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21263,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197457+0000","last_change":"2024-09-10T20:41:36.928383+0000","last_active":"2024-09-10T20:41:57.197457+0000","last_peered":"2024-09-10T20:41:57.197457+0000","last_clean":"2024-09-10T20:41:57.197457+0000","last_became_active":"2024-09-10T20:41:36.842440+0000","last_became_peered":"2024-09-10T20:41:36.842440+0000","last_unstale":"2024-09-10T20:41:57.197457+0000","last_undegraded":"2024-09-10T20:41:57.197457+0000","last_fullsized":"2024-09-10T20:41:57.197457+0000","mapping_epoch":64,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.940798+0000","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3715,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":115,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197491+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:41:57.197491+0000","last_peered":"2024-09-10T20:41:57.197491+0000","last_clean":"2024-09-10T20:41:57.197491+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:41:57.197491+0000","last_undegraded":"2024-09-10T20:41:57.197491+0000","last_fullsized":"2024-09-10T20:41:57.197491+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":120,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197644+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:41:57.197644+0000","last_peered":"2024-09-10T20:41:57.197644+0000","last_clean":"2024-09-10T20:41:57.197644+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:41:57.197644+0000","last_undegraded":"2024-09-10T20:41:57.197644+0000","last_fullsized":"2024-09-10T20:41:57.197644+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27679,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197402+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:41:57.197402+0000","last_peered":"2024-09-10T20:41:57.197402+0000","last_clean":"2024-09-10T20:41:57.197402+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:41:57.197402+0000","last_undegraded":"2024-09-10T20:41:57.197402+0000","last_fullsized":"2024-09-10T20:41:57.197402+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39530,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":116953088,"data_stored":116945862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182560,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517240,"kb_used_data":58020,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93854600,"statfs":{"total":96636764160,"available":96107110400,"internally_reserved":0,"allocated":59412480,"data_stored":59071492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":470216362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986332,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422848,"kb_used_data":57968,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93948992,"statfs":{"total":96636764160,"available":96203767808,"internally_reserved":0,"allocated":59359232,"data_stored":59025934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659279,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497320,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874520,"statfs":{"total":96636764160,"available":96127508480,"internally_reserved":0,"allocated":548864,"data_stored":207876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58867712,"data_stored":58864099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":58081280,"data_stored":58081280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:42:01.947 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:42:02.242 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:42:02.243 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:42:02.255 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":477,"stamp":"2024-09-10T20:42:01.083344+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39532,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1359004,"kb_used_data":54756,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281756516,"statfs":{"total":289910292480,"available":288518672384,"internally_reserved":0,"allocated":56070144,"data_stored":55034660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1335479298},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1362,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1362,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001626"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":28008,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197950+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:41:58.197950+0000","last_peered":"2024-09-10T20:41:58.197950+0000","last_clean":"2024-09-10T20:41:58.197950+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:41:58.197950+0000","last_undegraded":"2024-09-10T20:41:58.197950+0000","last_fullsized":"2024-09-10T20:41:58.197950+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21100,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275553+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:41:58.275553+0000","last_peered":"2024-09-10T20:41:58.275553+0000","last_clean":"2024-09-10T20:41:58.275553+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:41:58.275553+0000","last_undegraded":"2024-09-10T20:41:58.275553+0000","last_fullsized":"2024-09-10T20:41:58.275553+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28110,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197414+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:41:57.197414+0000","last_peered":"2024-09-10T20:41:57.197414+0000","last_clean":"2024-09-10T20:41:57.197414+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:41:57.197414+0000","last_undegraded":"2024-09-10T20:41:57.197414+0000","last_fullsized":"2024-09-10T20:41:57.197414+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20732,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275478+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:41:58.275478+0000","last_peered":"2024-09-10T20:41:58.275478+0000","last_clean":"2024-09-10T20:41:58.275478+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:41:58.275478+0000","last_undegraded":"2024-09-10T20:41:58.275478+0000","last_fullsized":"2024-09-10T20:41:58.275478+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21243,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197816+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:41:58.197816+0000","last_peered":"2024-09-10T20:41:58.197816+0000","last_clean":"2024-09-10T20:41:58.197816+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:41:58.197816+0000","last_undegraded":"2024-09-10T20:41:58.197816+0000","last_fullsized":"2024-09-10T20:41:58.197816+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20732,"reported_epoch":66,"state":"peering","last_fresh":"2024-09-10T20:41:56.475957+0000","last_change":"2024-09-10T20:41:57.296954+0000","last_active":"2024-09-10T20:41:57.296954+0000","last_peered":"2024-09-10T20:41:56.475957+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:41:56.475957+0000","last_undegraded":"2024-09-10T20:41:56.475957+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":66,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3616,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20917,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197229+0000","last_change":"2024-09-10T20:41:17.750026+0000","last_active":"2024-09-10T20:41:57.197229+0000","last_peered":"2024-09-10T20:41:57.197229+0000","last_clean":"2024-09-10T20:41:57.197229+0000","last_became_active":"2024-09-10T20:41:17.668013+0000","last_became_peered":"2024-09-10T20:41:17.668013+0000","last_unstale":"2024-09-10T20:41:57.197229+0000","last_undegraded":"2024-09-10T20:41:57.197229+0000","last_fullsized":"2024-09-10T20:41:57.197229+0000","mapping_epoch":62,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:55.885282+0000","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20909,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197768+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:41:58.197768+0000","last_peered":"2024-09-10T20:41:58.197768+0000","last_clean":"2024-09-10T20:41:58.197768+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:41:58.197768+0000","last_undegraded":"2024-09-10T20:41:58.197768+0000","last_fullsized":"2024-09-10T20:41:58.197768+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":115,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197272+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:41:57.197272+0000","last_peered":"2024-09-10T20:41:57.197272+0000","last_clean":"2024-09-10T20:41:57.197272+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:41:57.197272+0000","last_undegraded":"2024-09-10T20:41:57.197272+0000","last_fullsized":"2024-09-10T20:41:57.197272+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22149,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197414+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:41:57.197414+0000","last_peered":"2024-09-10T20:41:57.197414+0000","last_clean":"2024-09-10T20:41:57.197414+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:41:57.197414+0000","last_undegraded":"2024-09-10T20:41:57.197414+0000","last_fullsized":"2024-09-10T20:41:57.197414+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":142,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275332+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:41:58.275332+0000","last_peered":"2024-09-10T20:41:58.275332+0000","last_clean":"2024-09-10T20:41:58.275332+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:41:58.275332+0000","last_undegraded":"2024-09-10T20:41:58.275332+0000","last_fullsized":"2024-09-10T20:41:58.275332+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28137,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197970+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:41:58.197970+0000","last_peered":"2024-09-10T20:41:58.197970+0000","last_clean":"2024-09-10T20:41:58.197970+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:41:58.197970+0000","last_undegraded":"2024-09-10T20:41:58.197970+0000","last_fullsized":"2024-09-10T20:41:58.197970+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":149,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197712+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:41:58.197712+0000","last_peered":"2024-09-10T20:41:58.197712+0000","last_clean":"2024-09-10T20:41:58.197712+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:41:58.197712+0000","last_undegraded":"2024-09-10T20:41:58.197712+0000","last_fullsized":"2024-09-10T20:41:58.197712+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28020,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275395+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:41:58.275395+0000","last_peered":"2024-09-10T20:41:58.275395+0000","last_clean":"2024-09-10T20:41:58.275395+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:41:58.275395+0000","last_undegraded":"2024-09-10T20:41:58.275395+0000","last_fullsized":"2024-09-10T20:41:58.275395+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":125,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197372+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:41:57.197372+0000","last_peered":"2024-09-10T20:41:57.197372+0000","last_clean":"2024-09-10T20:41:57.197372+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:41:57.197372+0000","last_undegraded":"2024-09-10T20:41:57.197372+0000","last_fullsized":"2024-09-10T20:41:57.197372+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28073,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275280+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:41:58.275280+0000","last_peered":"2024-09-10T20:41:58.275280+0000","last_clean":"2024-09-10T20:41:58.275280+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:41:58.275280+0000","last_undegraded":"2024-09-10T20:41:58.275280+0000","last_fullsized":"2024-09-10T20:41:58.275280+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21127,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197730+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:41:57.197730+0000","last_peered":"2024-09-10T20:41:57.197730+0000","last_clean":"2024-09-10T20:41:57.197730+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:41:57.197730+0000","last_undegraded":"2024-09-10T20:41:57.197730+0000","last_fullsized":"2024-09-10T20:41:57.197730+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":115,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197508+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:41:57.197508+0000","last_peered":"2024-09-10T20:41:57.197508+0000","last_clean":"2024-09-10T20:41:57.197508+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:41:57.197508+0000","last_undegraded":"2024-09-10T20:41:57.197508+0000","last_fullsized":"2024-09-10T20:41:57.197508+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27849,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197609+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:41:57.197609+0000","last_peered":"2024-09-10T20:41:57.197609+0000","last_clean":"2024-09-10T20:41:57.197609+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:41:57.197609+0000","last_undegraded":"2024-09-10T20:41:57.197609+0000","last_fullsized":"2024-09-10T20:41:57.197609+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":142,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275528+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:41:58.275528+0000","last_peered":"2024-09-10T20:41:58.275528+0000","last_clean":"2024-09-10T20:41:58.275528+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:41:58.275528+0000","last_undegraded":"2024-09-10T20:41:58.275528+0000","last_fullsized":"2024-09-10T20:41:58.275528+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21263,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197457+0000","last_change":"2024-09-10T20:41:36.928383+0000","last_active":"2024-09-10T20:41:57.197457+0000","last_peered":"2024-09-10T20:41:57.197457+0000","last_clean":"2024-09-10T20:41:57.197457+0000","last_became_active":"2024-09-10T20:41:36.842440+0000","last_became_peered":"2024-09-10T20:41:36.842440+0000","last_unstale":"2024-09-10T20:41:57.197457+0000","last_undegraded":"2024-09-10T20:41:57.197457+0000","last_fullsized":"2024-09-10T20:41:57.197457+0000","mapping_epoch":64,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.940798+0000","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3715,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":115,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197491+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:41:57.197491+0000","last_peered":"2024-09-10T20:41:57.197491+0000","last_clean":"2024-09-10T20:41:57.197491+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:41:57.197491+0000","last_undegraded":"2024-09-10T20:41:57.197491+0000","last_fullsized":"2024-09-10T20:41:57.197491+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":120,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197644+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:41:57.197644+0000","last_peered":"2024-09-10T20:41:57.197644+0000","last_clean":"2024-09-10T20:41:57.197644+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:41:57.197644+0000","last_undegraded":"2024-09-10T20:41:57.197644+0000","last_fullsized":"2024-09-10T20:41:57.197644+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27679,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197402+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:41:57.197402+0000","last_peered":"2024-09-10T20:41:57.197402+0000","last_clean":"2024-09-10T20:41:57.197402+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:41:57.197402+0000","last_undegraded":"2024-09-10T20:41:57.197402+0000","last_fullsized":"2024-09-10T20:41:57.197402+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39530,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53669888,"data_stored":53662662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182561,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443132,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93928708,"statfs":{"total":96636764160,"available":96182996992,"internally_reserved":0,"allocated":565248,"data_stored":214155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986333,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418552,"kb_used_data":53668,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93953288,"statfs":{"total":96636764160,"available":96208166912,"internally_reserved":0,"allocated":54956032,"data_stored":54612629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659279,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497320,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874520,"statfs":{"total":96636764160,"available":96127508480,"internally_reserved":0,"allocated":548864,"data_stored":207876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53661696,"data_stored":53661696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:42:02.256 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:42:02.537 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:42:02.537 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:42:02.549 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":477,"stamp":"2024-09-10T20:42:01.083344+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39532,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1359004,"kb_used_data":54756,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281756516,"statfs":{"total":289910292480,"available":288518672384,"internally_reserved":0,"allocated":56070144,"data_stored":55034660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1335479298},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1362,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1362,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001626"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":28008,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197950+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:41:58.197950+0000","last_peered":"2024-09-10T20:41:58.197950+0000","last_clean":"2024-09-10T20:41:58.197950+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:41:58.197950+0000","last_undegraded":"2024-09-10T20:41:58.197950+0000","last_fullsized":"2024-09-10T20:41:58.197950+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21100,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275553+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:41:58.275553+0000","last_peered":"2024-09-10T20:41:58.275553+0000","last_clean":"2024-09-10T20:41:58.275553+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:41:58.275553+0000","last_undegraded":"2024-09-10T20:41:58.275553+0000","last_fullsized":"2024-09-10T20:41:58.275553+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28110,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197414+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:41:57.197414+0000","last_peered":"2024-09-10T20:41:57.197414+0000","last_clean":"2024-09-10T20:41:57.197414+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:41:57.197414+0000","last_undegraded":"2024-09-10T20:41:57.197414+0000","last_fullsized":"2024-09-10T20:41:57.197414+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20732,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275478+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:41:58.275478+0000","last_peered":"2024-09-10T20:41:58.275478+0000","last_clean":"2024-09-10T20:41:58.275478+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:41:58.275478+0000","last_undegraded":"2024-09-10T20:41:58.275478+0000","last_fullsized":"2024-09-10T20:41:58.275478+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21243,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197816+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:41:58.197816+0000","last_peered":"2024-09-10T20:41:58.197816+0000","last_clean":"2024-09-10T20:41:58.197816+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:41:58.197816+0000","last_undegraded":"2024-09-10T20:41:58.197816+0000","last_fullsized":"2024-09-10T20:41:58.197816+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20732,"reported_epoch":66,"state":"peering","last_fresh":"2024-09-10T20:41:56.475957+0000","last_change":"2024-09-10T20:41:57.296954+0000","last_active":"2024-09-10T20:41:57.296954+0000","last_peered":"2024-09-10T20:41:56.475957+0000","last_clean":"2024-09-10T20:30:03.299683+0000","last_became_active":"2024-09-10T20:30:05.790193+0000","last_became_peered":"2024-09-10T20:30:05.790193+0000","last_unstale":"2024-09-10T20:41:56.475957+0000","last_undegraded":"2024-09-10T20:41:56.475957+0000","last_fullsized":"2024-09-10T20:30:05.306065+0000","mapping_epoch":66,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3616,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20917,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197229+0000","last_change":"2024-09-10T20:41:17.750026+0000","last_active":"2024-09-10T20:41:57.197229+0000","last_peered":"2024-09-10T20:41:57.197229+0000","last_clean":"2024-09-10T20:41:57.197229+0000","last_became_active":"2024-09-10T20:41:17.668013+0000","last_became_peered":"2024-09-10T20:41:17.668013+0000","last_unstale":"2024-09-10T20:41:57.197229+0000","last_undegraded":"2024-09-10T20:41:57.197229+0000","last_fullsized":"2024-09-10T20:41:57.197229+0000","mapping_epoch":62,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:55.885282+0000","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20909,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197768+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:41:58.197768+0000","last_peered":"2024-09-10T20:41:58.197768+0000","last_clean":"2024-09-10T20:41:58.197768+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:41:58.197768+0000","last_undegraded":"2024-09-10T20:41:58.197768+0000","last_fullsized":"2024-09-10T20:41:58.197768+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":115,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197272+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:41:57.197272+0000","last_peered":"2024-09-10T20:41:57.197272+0000","last_clean":"2024-09-10T20:41:57.197272+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:41:57.197272+0000","last_undegraded":"2024-09-10T20:41:57.197272+0000","last_fullsized":"2024-09-10T20:41:57.197272+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22149,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197414+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:41:57.197414+0000","last_peered":"2024-09-10T20:41:57.197414+0000","last_clean":"2024-09-10T20:41:57.197414+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:41:57.197414+0000","last_undegraded":"2024-09-10T20:41:57.197414+0000","last_fullsized":"2024-09-10T20:41:57.197414+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":142,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275332+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:41:58.275332+0000","last_peered":"2024-09-10T20:41:58.275332+0000","last_clean":"2024-09-10T20:41:58.275332+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:41:58.275332+0000","last_undegraded":"2024-09-10T20:41:58.275332+0000","last_fullsized":"2024-09-10T20:41:58.275332+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28137,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197970+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:41:58.197970+0000","last_peered":"2024-09-10T20:41:58.197970+0000","last_clean":"2024-09-10T20:41:58.197970+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:41:58.197970+0000","last_undegraded":"2024-09-10T20:41:58.197970+0000","last_fullsized":"2024-09-10T20:41:58.197970+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":149,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197712+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:41:58.197712+0000","last_peered":"2024-09-10T20:41:58.197712+0000","last_clean":"2024-09-10T20:41:58.197712+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:41:58.197712+0000","last_undegraded":"2024-09-10T20:41:58.197712+0000","last_fullsized":"2024-09-10T20:41:58.197712+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28020,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275395+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:41:58.275395+0000","last_peered":"2024-09-10T20:41:58.275395+0000","last_clean":"2024-09-10T20:41:58.275395+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:41:58.275395+0000","last_undegraded":"2024-09-10T20:41:58.275395+0000","last_fullsized":"2024-09-10T20:41:58.275395+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":125,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197372+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:41:57.197372+0000","last_peered":"2024-09-10T20:41:57.197372+0000","last_clean":"2024-09-10T20:41:57.197372+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:41:57.197372+0000","last_undegraded":"2024-09-10T20:41:57.197372+0000","last_fullsized":"2024-09-10T20:41:57.197372+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28073,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275280+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:41:58.275280+0000","last_peered":"2024-09-10T20:41:58.275280+0000","last_clean":"2024-09-10T20:41:58.275280+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:41:58.275280+0000","last_undegraded":"2024-09-10T20:41:58.275280+0000","last_fullsized":"2024-09-10T20:41:58.275280+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21127,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197730+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:41:57.197730+0000","last_peered":"2024-09-10T20:41:57.197730+0000","last_clean":"2024-09-10T20:41:57.197730+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:41:57.197730+0000","last_undegraded":"2024-09-10T20:41:57.197730+0000","last_fullsized":"2024-09-10T20:41:57.197730+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":115,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197508+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:41:57.197508+0000","last_peered":"2024-09-10T20:41:57.197508+0000","last_clean":"2024-09-10T20:41:57.197508+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:41:57.197508+0000","last_undegraded":"2024-09-10T20:41:57.197508+0000","last_fullsized":"2024-09-10T20:41:57.197508+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27849,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197609+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:41:57.197609+0000","last_peered":"2024-09-10T20:41:57.197609+0000","last_clean":"2024-09-10T20:41:57.197609+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:41:57.197609+0000","last_undegraded":"2024-09-10T20:41:57.197609+0000","last_fullsized":"2024-09-10T20:41:57.197609+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":142,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275528+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:41:58.275528+0000","last_peered":"2024-09-10T20:41:58.275528+0000","last_clean":"2024-09-10T20:41:58.275528+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:41:58.275528+0000","last_undegraded":"2024-09-10T20:41:58.275528+0000","last_fullsized":"2024-09-10T20:41:58.275528+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21263,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197457+0000","last_change":"2024-09-10T20:41:36.928383+0000","last_active":"2024-09-10T20:41:57.197457+0000","last_peered":"2024-09-10T20:41:57.197457+0000","last_clean":"2024-09-10T20:41:57.197457+0000","last_became_active":"2024-09-10T20:41:36.842440+0000","last_became_peered":"2024-09-10T20:41:36.842440+0000","last_unstale":"2024-09-10T20:41:57.197457+0000","last_undegraded":"2024-09-10T20:41:57.197457+0000","last_fullsized":"2024-09-10T20:41:57.197457+0000","mapping_epoch":64,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.940798+0000","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3715,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":115,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197491+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:41:57.197491+0000","last_peered":"2024-09-10T20:41:57.197491+0000","last_clean":"2024-09-10T20:41:57.197491+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:41:57.197491+0000","last_undegraded":"2024-09-10T20:41:57.197491+0000","last_fullsized":"2024-09-10T20:41:57.197491+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":120,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197644+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:41:57.197644+0000","last_peered":"2024-09-10T20:41:57.197644+0000","last_clean":"2024-09-10T20:41:57.197644+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:41:57.197644+0000","last_undegraded":"2024-09-10T20:41:57.197644+0000","last_fullsized":"2024-09-10T20:41:57.197644+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27679,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T20:41:57.197402+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:41:57.197402+0000","last_peered":"2024-09-10T20:41:57.197402+0000","last_clean":"2024-09-10T20:41:57.197402+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:41:57.197402+0000","last_undegraded":"2024-09-10T20:41:57.197402+0000","last_fullsized":"2024-09-10T20:41:57.197402+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39530,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53669888,"data_stored":53662662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182561,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443132,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93928708,"statfs":{"total":96636764160,"available":96182996992,"internally_reserved":0,"allocated":565248,"data_stored":214155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986333,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418552,"kb_used_data":53668,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93953288,"statfs":{"total":96636764160,"available":96208166912,"internally_reserved":0,"allocated":54956032,"data_stored":54612629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659279,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497320,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874520,"statfs":{"total":96636764160,"available":96127508480,"internally_reserved":0,"allocated":548864,"data_stored":207876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53661696,"data_stored":53661696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:42:02.550 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T20:42:02.550 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20471", 'reported_seq': 20732, 'reported_epoch': 66, 'state': 'peering', 'last_fresh': '2024-09-10T20:41:56.475957+0000', 'last_change': '2024-09-10T20:41:57.296954+0000', 'last_active': '2024-09-10T20:41:57.296954+0000', 'last_peered': '2024-09-10T20:41:56.475957+0000', 'last_clean': '2024-09-10T20:30:03.299683+0000', 'last_became_active': '2024-09-10T20:30:05.790193+0000', 'last_became_peered': '2024-09-10T20:30:05.790193+0000', 'last_unstale': '2024-09-10T20:41:56.475957+0000', 'last_undegraded': '2024-09-10T20:41:56.475957+0000', 'last_fullsized': '2024-09-10T20:30:05.306065+0000', 'mapping_epoch': 66, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3180", 'last_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:28:37.510222+0000', 'last_clean_scrub_stamp': '2024-09-10T20:29:55.781388+0000', 'objects_scrubbed': 3131, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1161, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17335, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3616, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T20:42:02.551 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:42:02.899 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:42:02.900 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":834,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":67,"num_osds":3,"num_up_osds":3,"osd_up_since":1726000203,"num_in_osds":3,"osd_in_since":1726000197,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"peering","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1391620096,"bytes_avail":288518672384,"bytes_total":289910292480,"inactive_pgs_ratio":0.041666667908430099,"recovering_objects_per_sec":170,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1362,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T20:28:08:371114+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T20:31:02.848164+0000","services":{}},"progress_events":{}} 2024-09-10T20:42:02.912 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T20:42:02.913 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:42:03.196 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:42:03.196 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:42:03.208 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":478,"stamp":"2024-09-10T20:42:03.083751+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39532,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1359024,"kb_used_data":54772,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281756496,"statfs":{"total":289910292480,"available":288518651904,"internally_reserved":0,"allocated":56086528,"data_stored":55040939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1335479298},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1362,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1362,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001702"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":28008,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197950+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:41:58.197950+0000","last_peered":"2024-09-10T20:41:58.197950+0000","last_clean":"2024-09-10T20:41:58.197950+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:41:58.197950+0000","last_undegraded":"2024-09-10T20:41:58.197950+0000","last_fullsized":"2024-09-10T20:41:58.197950+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21100,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275553+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:41:58.275553+0000","last_peered":"2024-09-10T20:41:58.275553+0000","last_clean":"2024-09-10T20:41:58.275553+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:41:58.275553+0000","last_undegraded":"2024-09-10T20:41:58.275553+0000","last_fullsized":"2024-09-10T20:41:58.275553+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28112,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197262+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:41:58.197262+0000","last_peered":"2024-09-10T20:41:58.197262+0000","last_clean":"2024-09-10T20:41:58.197262+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:41:58.197262+0000","last_undegraded":"2024-09-10T20:41:58.197262+0000","last_fullsized":"2024-09-10T20:41:58.197262+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20732,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275478+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:41:58.275478+0000","last_peered":"2024-09-10T20:41:58.275478+0000","last_clean":"2024-09-10T20:41:58.275478+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:41:58.275478+0000","last_undegraded":"2024-09-10T20:41:58.275478+0000","last_fullsized":"2024-09-10T20:41:58.275478+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21243,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197816+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:41:58.197816+0000","last_peered":"2024-09-10T20:41:58.197816+0000","last_clean":"2024-09-10T20:41:58.197816+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:41:58.197816+0000","last_undegraded":"2024-09-10T20:41:58.197816+0000","last_fullsized":"2024-09-10T20:41:58.197816+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20741,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.265870+0000","last_change":"2024-09-10T20:41:58.265777+0000","last_active":"2024-09-10T20:41:58.265870+0000","last_peered":"2024-09-10T20:41:58.265870+0000","last_clean":"2024-09-10T20:41:58.265870+0000","last_became_active":"2024-09-10T20:41:58.198887+0000","last_became_peered":"2024-09-10T20:41:58.198887+0000","last_unstale":"2024-09-10T20:41:58.265870+0000","last_undegraded":"2024-09-10T20:41:58.265870+0000","last_fullsized":"2024-09-10T20:41:58.265870+0000","mapping_epoch":66,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:51.015540+0000","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3616,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20919,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197002+0000","last_change":"2024-09-10T20:41:17.750026+0000","last_active":"2024-09-10T20:41:58.197002+0000","last_peered":"2024-09-10T20:41:58.197002+0000","last_clean":"2024-09-10T20:41:58.197002+0000","last_became_active":"2024-09-10T20:41:17.668013+0000","last_became_peered":"2024-09-10T20:41:17.668013+0000","last_unstale":"2024-09-10T20:41:58.197002+0000","last_undegraded":"2024-09-10T20:41:58.197002+0000","last_fullsized":"2024-09-10T20:41:58.197002+0000","mapping_epoch":62,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:55.885282+0000","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20909,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197768+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:41:58.197768+0000","last_peered":"2024-09-10T20:41:58.197768+0000","last_clean":"2024-09-10T20:41:58.197768+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:41:58.197768+0000","last_undegraded":"2024-09-10T20:41:58.197768+0000","last_fullsized":"2024-09-10T20:41:58.197768+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":117,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.196822+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:41:58.196822+0000","last_peered":"2024-09-10T20:41:58.196822+0000","last_clean":"2024-09-10T20:41:58.196822+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:41:58.196822+0000","last_undegraded":"2024-09-10T20:41:58.196822+0000","last_fullsized":"2024-09-10T20:41:58.196822+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22151,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197391+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:41:58.197391+0000","last_peered":"2024-09-10T20:41:58.197391+0000","last_clean":"2024-09-10T20:41:58.197391+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:41:58.197391+0000","last_undegraded":"2024-09-10T20:41:58.197391+0000","last_fullsized":"2024-09-10T20:41:58.197391+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":142,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275332+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:41:58.275332+0000","last_peered":"2024-09-10T20:41:58.275332+0000","last_clean":"2024-09-10T20:41:58.275332+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:41:58.275332+0000","last_undegraded":"2024-09-10T20:41:58.275332+0000","last_fullsized":"2024-09-10T20:41:58.275332+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28137,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197970+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:41:58.197970+0000","last_peered":"2024-09-10T20:41:58.197970+0000","last_clean":"2024-09-10T20:41:58.197970+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:41:58.197970+0000","last_undegraded":"2024-09-10T20:41:58.197970+0000","last_fullsized":"2024-09-10T20:41:58.197970+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":149,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197712+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:41:58.197712+0000","last_peered":"2024-09-10T20:41:58.197712+0000","last_clean":"2024-09-10T20:41:58.197712+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:41:58.197712+0000","last_undegraded":"2024-09-10T20:41:58.197712+0000","last_fullsized":"2024-09-10T20:41:58.197712+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28020,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275395+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:41:58.275395+0000","last_peered":"2024-09-10T20:41:58.275395+0000","last_clean":"2024-09-10T20:41:58.275395+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:41:58.275395+0000","last_undegraded":"2024-09-10T20:41:58.275395+0000","last_fullsized":"2024-09-10T20:41:58.275395+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":127,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197287+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:41:58.197287+0000","last_peered":"2024-09-10T20:41:58.197287+0000","last_clean":"2024-09-10T20:41:58.197287+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:41:58.197287+0000","last_undegraded":"2024-09-10T20:41:58.197287+0000","last_fullsized":"2024-09-10T20:41:58.197287+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28073,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275280+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:41:58.275280+0000","last_peered":"2024-09-10T20:41:58.275280+0000","last_clean":"2024-09-10T20:41:58.275280+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:41:58.275280+0000","last_undegraded":"2024-09-10T20:41:58.275280+0000","last_fullsized":"2024-09-10T20:41:58.275280+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21129,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197608+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:41:58.197608+0000","last_peered":"2024-09-10T20:41:58.197608+0000","last_clean":"2024-09-10T20:41:58.197608+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:41:58.197608+0000","last_undegraded":"2024-09-10T20:41:58.197608+0000","last_fullsized":"2024-09-10T20:41:58.197608+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":117,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197184+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:41:58.197184+0000","last_peered":"2024-09-10T20:41:58.197184+0000","last_clean":"2024-09-10T20:41:58.197184+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:41:58.197184+0000","last_undegraded":"2024-09-10T20:41:58.197184+0000","last_fullsized":"2024-09-10T20:41:58.197184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27851,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197520+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:41:58.197520+0000","last_peered":"2024-09-10T20:41:58.197520+0000","last_clean":"2024-09-10T20:41:58.197520+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:41:58.197520+0000","last_undegraded":"2024-09-10T20:41:58.197520+0000","last_fullsized":"2024-09-10T20:41:58.197520+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":142,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275528+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:41:58.275528+0000","last_peered":"2024-09-10T20:41:58.275528+0000","last_clean":"2024-09-10T20:41:58.275528+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:41:58.275528+0000","last_undegraded":"2024-09-10T20:41:58.275528+0000","last_fullsized":"2024-09-10T20:41:58.275528+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21265,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197345+0000","last_change":"2024-09-10T20:41:36.928383+0000","last_active":"2024-09-10T20:41:58.197345+0000","last_peered":"2024-09-10T20:41:58.197345+0000","last_clean":"2024-09-10T20:41:58.197345+0000","last_became_active":"2024-09-10T20:41:36.842440+0000","last_became_peered":"2024-09-10T20:41:36.842440+0000","last_unstale":"2024-09-10T20:41:58.197345+0000","last_undegraded":"2024-09-10T20:41:58.197345+0000","last_fullsized":"2024-09-10T20:41:58.197345+0000","mapping_epoch":64,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.940798+0000","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3715,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":117,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197437+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:41:58.197437+0000","last_peered":"2024-09-10T20:41:58.197437+0000","last_clean":"2024-09-10T20:41:58.197437+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:41:58.197437+0000","last_undegraded":"2024-09-10T20:41:58.197437+0000","last_fullsized":"2024-09-10T20:41:58.197437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":122,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197523+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:41:58.197523+0000","last_peered":"2024-09-10T20:41:58.197523+0000","last_clean":"2024-09-10T20:41:58.197523+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:41:58.197523+0000","last_undegraded":"2024-09-10T20:41:58.197523+0000","last_fullsized":"2024-09-10T20:41:58.197523+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27681,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197107+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:41:58.197107+0000","last_peered":"2024-09-10T20:41:58.197107+0000","last_clean":"2024-09-10T20:41:58.197107+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:41:58.197107+0000","last_undegraded":"2024-09-10T20:41:58.197107+0000","last_fullsized":"2024-09-10T20:41:58.197107+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39530,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53669888,"data_stored":53662662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182561,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443132,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93928708,"statfs":{"total":96636764160,"available":96182996992,"internally_reserved":0,"allocated":565248,"data_stored":214155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986333,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418552,"kb_used_data":53668,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93953288,"statfs":{"total":96636764160,"available":96208166912,"internally_reserved":0,"allocated":54956032,"data_stored":54612629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659280,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497340,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874500,"statfs":{"total":96636764160,"available":96127488000,"internally_reserved":0,"allocated":565248,"data_stored":214155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53661696,"data_stored":53661696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:42:06.210 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:42:06.508 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:42:06.509 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:42:06.521 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":479,"stamp":"2024-09-10T20:42:05.084153+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39532,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1354220,"kb_used_data":49972,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281761300,"statfs":{"total":289910292480,"available":288523571200,"internally_reserved":0,"allocated":51171328,"data_stored":50125739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1335479298},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-342,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001752"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":28008,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197950+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:41:58.197950+0000","last_peered":"2024-09-10T20:41:58.197950+0000","last_clean":"2024-09-10T20:41:58.197950+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:41:58.197950+0000","last_undegraded":"2024-09-10T20:41:58.197950+0000","last_fullsized":"2024-09-10T20:41:58.197950+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21100,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275553+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:41:58.275553+0000","last_peered":"2024-09-10T20:41:58.275553+0000","last_clean":"2024-09-10T20:41:58.275553+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:41:58.275553+0000","last_undegraded":"2024-09-10T20:41:58.275553+0000","last_fullsized":"2024-09-10T20:41:58.275553+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28112,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197262+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:41:58.197262+0000","last_peered":"2024-09-10T20:41:58.197262+0000","last_clean":"2024-09-10T20:41:58.197262+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:41:58.197262+0000","last_undegraded":"2024-09-10T20:41:58.197262+0000","last_fullsized":"2024-09-10T20:41:58.197262+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20732,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275478+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:41:58.275478+0000","last_peered":"2024-09-10T20:41:58.275478+0000","last_clean":"2024-09-10T20:41:58.275478+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:41:58.275478+0000","last_undegraded":"2024-09-10T20:41:58.275478+0000","last_fullsized":"2024-09-10T20:41:58.275478+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21243,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197816+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:41:58.197816+0000","last_peered":"2024-09-10T20:41:58.197816+0000","last_clean":"2024-09-10T20:41:58.197816+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:41:58.197816+0000","last_undegraded":"2024-09-10T20:41:58.197816+0000","last_fullsized":"2024-09-10T20:41:58.197816+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20741,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.265870+0000","last_change":"2024-09-10T20:41:58.265777+0000","last_active":"2024-09-10T20:41:58.265870+0000","last_peered":"2024-09-10T20:41:58.265870+0000","last_clean":"2024-09-10T20:41:58.265870+0000","last_became_active":"2024-09-10T20:41:58.198887+0000","last_became_peered":"2024-09-10T20:41:58.198887+0000","last_unstale":"2024-09-10T20:41:58.265870+0000","last_undegraded":"2024-09-10T20:41:58.265870+0000","last_fullsized":"2024-09-10T20:41:58.265870+0000","mapping_epoch":66,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:51.015540+0000","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3616,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20919,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197002+0000","last_change":"2024-09-10T20:41:17.750026+0000","last_active":"2024-09-10T20:41:58.197002+0000","last_peered":"2024-09-10T20:41:58.197002+0000","last_clean":"2024-09-10T20:41:58.197002+0000","last_became_active":"2024-09-10T20:41:17.668013+0000","last_became_peered":"2024-09-10T20:41:17.668013+0000","last_unstale":"2024-09-10T20:41:58.197002+0000","last_undegraded":"2024-09-10T20:41:58.197002+0000","last_fullsized":"2024-09-10T20:41:58.197002+0000","mapping_epoch":62,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:55.885282+0000","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20909,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197768+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:41:58.197768+0000","last_peered":"2024-09-10T20:41:58.197768+0000","last_clean":"2024-09-10T20:41:58.197768+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:41:58.197768+0000","last_undegraded":"2024-09-10T20:41:58.197768+0000","last_fullsized":"2024-09-10T20:41:58.197768+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":117,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.196822+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:41:58.196822+0000","last_peered":"2024-09-10T20:41:58.196822+0000","last_clean":"2024-09-10T20:41:58.196822+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:41:58.196822+0000","last_undegraded":"2024-09-10T20:41:58.196822+0000","last_fullsized":"2024-09-10T20:41:58.196822+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22151,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197391+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:41:58.197391+0000","last_peered":"2024-09-10T20:41:58.197391+0000","last_clean":"2024-09-10T20:41:58.197391+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:41:58.197391+0000","last_undegraded":"2024-09-10T20:41:58.197391+0000","last_fullsized":"2024-09-10T20:41:58.197391+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":142,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275332+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:41:58.275332+0000","last_peered":"2024-09-10T20:41:58.275332+0000","last_clean":"2024-09-10T20:41:58.275332+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:41:58.275332+0000","last_undegraded":"2024-09-10T20:41:58.275332+0000","last_fullsized":"2024-09-10T20:41:58.275332+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28137,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197970+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:41:58.197970+0000","last_peered":"2024-09-10T20:41:58.197970+0000","last_clean":"2024-09-10T20:41:58.197970+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:41:58.197970+0000","last_undegraded":"2024-09-10T20:41:58.197970+0000","last_fullsized":"2024-09-10T20:41:58.197970+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":149,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197712+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:41:58.197712+0000","last_peered":"2024-09-10T20:41:58.197712+0000","last_clean":"2024-09-10T20:41:58.197712+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:41:58.197712+0000","last_undegraded":"2024-09-10T20:41:58.197712+0000","last_fullsized":"2024-09-10T20:41:58.197712+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28020,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275395+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:41:58.275395+0000","last_peered":"2024-09-10T20:41:58.275395+0000","last_clean":"2024-09-10T20:41:58.275395+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:41:58.275395+0000","last_undegraded":"2024-09-10T20:41:58.275395+0000","last_fullsized":"2024-09-10T20:41:58.275395+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":127,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197287+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:41:58.197287+0000","last_peered":"2024-09-10T20:41:58.197287+0000","last_clean":"2024-09-10T20:41:58.197287+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:41:58.197287+0000","last_undegraded":"2024-09-10T20:41:58.197287+0000","last_fullsized":"2024-09-10T20:41:58.197287+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28073,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275280+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:41:58.275280+0000","last_peered":"2024-09-10T20:41:58.275280+0000","last_clean":"2024-09-10T20:41:58.275280+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:41:58.275280+0000","last_undegraded":"2024-09-10T20:41:58.275280+0000","last_fullsized":"2024-09-10T20:41:58.275280+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21129,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197608+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:41:58.197608+0000","last_peered":"2024-09-10T20:41:58.197608+0000","last_clean":"2024-09-10T20:41:58.197608+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:41:58.197608+0000","last_undegraded":"2024-09-10T20:41:58.197608+0000","last_fullsized":"2024-09-10T20:41:58.197608+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":117,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197184+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:41:58.197184+0000","last_peered":"2024-09-10T20:41:58.197184+0000","last_clean":"2024-09-10T20:41:58.197184+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:41:58.197184+0000","last_undegraded":"2024-09-10T20:41:58.197184+0000","last_fullsized":"2024-09-10T20:41:58.197184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27851,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197520+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:41:58.197520+0000","last_peered":"2024-09-10T20:41:58.197520+0000","last_clean":"2024-09-10T20:41:58.197520+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:41:58.197520+0000","last_undegraded":"2024-09-10T20:41:58.197520+0000","last_fullsized":"2024-09-10T20:41:58.197520+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":142,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275528+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:41:58.275528+0000","last_peered":"2024-09-10T20:41:58.275528+0000","last_clean":"2024-09-10T20:41:58.275528+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:41:58.275528+0000","last_undegraded":"2024-09-10T20:41:58.275528+0000","last_fullsized":"2024-09-10T20:41:58.275528+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21265,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197345+0000","last_change":"2024-09-10T20:41:36.928383+0000","last_active":"2024-09-10T20:41:58.197345+0000","last_peered":"2024-09-10T20:41:58.197345+0000","last_clean":"2024-09-10T20:41:58.197345+0000","last_became_active":"2024-09-10T20:41:36.842440+0000","last_became_peered":"2024-09-10T20:41:36.842440+0000","last_unstale":"2024-09-10T20:41:58.197345+0000","last_undegraded":"2024-09-10T20:41:58.197345+0000","last_fullsized":"2024-09-10T20:41:58.197345+0000","mapping_epoch":64,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.940798+0000","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3715,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":117,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197437+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:41:58.197437+0000","last_peered":"2024-09-10T20:41:58.197437+0000","last_clean":"2024-09-10T20:41:58.197437+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:41:58.197437+0000","last_undegraded":"2024-09-10T20:41:58.197437+0000","last_fullsized":"2024-09-10T20:41:58.197437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":122,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197523+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:41:58.197523+0000","last_peered":"2024-09-10T20:41:58.197523+0000","last_clean":"2024-09-10T20:41:58.197523+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:41:58.197523+0000","last_undegraded":"2024-09-10T20:41:58.197523+0000","last_fullsized":"2024-09-10T20:41:58.197523+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27681,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197107+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:41:58.197107+0000","last_peered":"2024-09-10T20:41:58.197107+0000","last_clean":"2024-09-10T20:41:58.197107+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:41:58.197107+0000","last_undegraded":"2024-09-10T20:41:58.197107+0000","last_fullsized":"2024-09-10T20:41:58.197107+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39530,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":48754688,"data_stored":48747462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182561,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443132,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93928708,"statfs":{"total":96636764160,"available":96182996992,"internally_reserved":0,"allocated":565248,"data_stored":214155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986334,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":413748,"kb_used_data":48868,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93958092,"statfs":{"total":96636764160,"available":96213086208,"internally_reserved":0,"allocated":50040832,"data_stored":49697429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659280,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497340,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874500,"statfs":{"total":96636764160,"available":96127488000,"internally_reserved":0,"allocated":565248,"data_stored":214155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48746496,"data_stored":48746496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:42:06.522 INFO:tasks.rep_lost_unfound_delete.ceph_manager:clean! 2024-09-10T20:42:06.522 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for clean 2024-09-10T20:42:06.522 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:42:06.804 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:42:06.805 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:42:06.817 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":479,"stamp":"2024-09-10T20:42:05.084153+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39532,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1354220,"kb_used_data":49972,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281761300,"statfs":{"total":289910292480,"available":288523571200,"internally_reserved":0,"allocated":51171328,"data_stored":50125739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1335479298},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-342,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001752"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":28008,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197950+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:41:58.197950+0000","last_peered":"2024-09-10T20:41:58.197950+0000","last_clean":"2024-09-10T20:41:58.197950+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:41:58.197950+0000","last_undegraded":"2024-09-10T20:41:58.197950+0000","last_fullsized":"2024-09-10T20:41:58.197950+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21100,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275553+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:41:58.275553+0000","last_peered":"2024-09-10T20:41:58.275553+0000","last_clean":"2024-09-10T20:41:58.275553+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:41:58.275553+0000","last_undegraded":"2024-09-10T20:41:58.275553+0000","last_fullsized":"2024-09-10T20:41:58.275553+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28112,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197262+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:41:58.197262+0000","last_peered":"2024-09-10T20:41:58.197262+0000","last_clean":"2024-09-10T20:41:58.197262+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:41:58.197262+0000","last_undegraded":"2024-09-10T20:41:58.197262+0000","last_fullsized":"2024-09-10T20:41:58.197262+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20732,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275478+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:41:58.275478+0000","last_peered":"2024-09-10T20:41:58.275478+0000","last_clean":"2024-09-10T20:41:58.275478+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:41:58.275478+0000","last_undegraded":"2024-09-10T20:41:58.275478+0000","last_fullsized":"2024-09-10T20:41:58.275478+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21243,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197816+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:41:58.197816+0000","last_peered":"2024-09-10T20:41:58.197816+0000","last_clean":"2024-09-10T20:41:58.197816+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:41:58.197816+0000","last_undegraded":"2024-09-10T20:41:58.197816+0000","last_fullsized":"2024-09-10T20:41:58.197816+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20741,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.265870+0000","last_change":"2024-09-10T20:41:58.265777+0000","last_active":"2024-09-10T20:41:58.265870+0000","last_peered":"2024-09-10T20:41:58.265870+0000","last_clean":"2024-09-10T20:41:58.265870+0000","last_became_active":"2024-09-10T20:41:58.198887+0000","last_became_peered":"2024-09-10T20:41:58.198887+0000","last_unstale":"2024-09-10T20:41:58.265870+0000","last_undegraded":"2024-09-10T20:41:58.265870+0000","last_fullsized":"2024-09-10T20:41:58.265870+0000","mapping_epoch":66,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:51.015540+0000","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3616,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20919,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197002+0000","last_change":"2024-09-10T20:41:17.750026+0000","last_active":"2024-09-10T20:41:58.197002+0000","last_peered":"2024-09-10T20:41:58.197002+0000","last_clean":"2024-09-10T20:41:58.197002+0000","last_became_active":"2024-09-10T20:41:17.668013+0000","last_became_peered":"2024-09-10T20:41:17.668013+0000","last_unstale":"2024-09-10T20:41:58.197002+0000","last_undegraded":"2024-09-10T20:41:58.197002+0000","last_fullsized":"2024-09-10T20:41:58.197002+0000","mapping_epoch":62,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:55.885282+0000","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20909,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197768+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:41:58.197768+0000","last_peered":"2024-09-10T20:41:58.197768+0000","last_clean":"2024-09-10T20:41:58.197768+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:41:58.197768+0000","last_undegraded":"2024-09-10T20:41:58.197768+0000","last_fullsized":"2024-09-10T20:41:58.197768+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":117,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.196822+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:41:58.196822+0000","last_peered":"2024-09-10T20:41:58.196822+0000","last_clean":"2024-09-10T20:41:58.196822+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:41:58.196822+0000","last_undegraded":"2024-09-10T20:41:58.196822+0000","last_fullsized":"2024-09-10T20:41:58.196822+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22151,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197391+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:41:58.197391+0000","last_peered":"2024-09-10T20:41:58.197391+0000","last_clean":"2024-09-10T20:41:58.197391+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:41:58.197391+0000","last_undegraded":"2024-09-10T20:41:58.197391+0000","last_fullsized":"2024-09-10T20:41:58.197391+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":142,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275332+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:41:58.275332+0000","last_peered":"2024-09-10T20:41:58.275332+0000","last_clean":"2024-09-10T20:41:58.275332+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:41:58.275332+0000","last_undegraded":"2024-09-10T20:41:58.275332+0000","last_fullsized":"2024-09-10T20:41:58.275332+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28137,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197970+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:41:58.197970+0000","last_peered":"2024-09-10T20:41:58.197970+0000","last_clean":"2024-09-10T20:41:58.197970+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:41:58.197970+0000","last_undegraded":"2024-09-10T20:41:58.197970+0000","last_fullsized":"2024-09-10T20:41:58.197970+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":149,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197712+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:41:58.197712+0000","last_peered":"2024-09-10T20:41:58.197712+0000","last_clean":"2024-09-10T20:41:58.197712+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:41:58.197712+0000","last_undegraded":"2024-09-10T20:41:58.197712+0000","last_fullsized":"2024-09-10T20:41:58.197712+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28020,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275395+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:41:58.275395+0000","last_peered":"2024-09-10T20:41:58.275395+0000","last_clean":"2024-09-10T20:41:58.275395+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:41:58.275395+0000","last_undegraded":"2024-09-10T20:41:58.275395+0000","last_fullsized":"2024-09-10T20:41:58.275395+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":127,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197287+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:41:58.197287+0000","last_peered":"2024-09-10T20:41:58.197287+0000","last_clean":"2024-09-10T20:41:58.197287+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:41:58.197287+0000","last_undegraded":"2024-09-10T20:41:58.197287+0000","last_fullsized":"2024-09-10T20:41:58.197287+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28073,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275280+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:41:58.275280+0000","last_peered":"2024-09-10T20:41:58.275280+0000","last_clean":"2024-09-10T20:41:58.275280+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:41:58.275280+0000","last_undegraded":"2024-09-10T20:41:58.275280+0000","last_fullsized":"2024-09-10T20:41:58.275280+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21129,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197608+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:41:58.197608+0000","last_peered":"2024-09-10T20:41:58.197608+0000","last_clean":"2024-09-10T20:41:58.197608+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:41:58.197608+0000","last_undegraded":"2024-09-10T20:41:58.197608+0000","last_fullsized":"2024-09-10T20:41:58.197608+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":117,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197184+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:41:58.197184+0000","last_peered":"2024-09-10T20:41:58.197184+0000","last_clean":"2024-09-10T20:41:58.197184+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:41:58.197184+0000","last_undegraded":"2024-09-10T20:41:58.197184+0000","last_fullsized":"2024-09-10T20:41:58.197184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27851,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197520+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:41:58.197520+0000","last_peered":"2024-09-10T20:41:58.197520+0000","last_clean":"2024-09-10T20:41:58.197520+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:41:58.197520+0000","last_undegraded":"2024-09-10T20:41:58.197520+0000","last_fullsized":"2024-09-10T20:41:58.197520+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":142,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275528+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:41:58.275528+0000","last_peered":"2024-09-10T20:41:58.275528+0000","last_clean":"2024-09-10T20:41:58.275528+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:41:58.275528+0000","last_undegraded":"2024-09-10T20:41:58.275528+0000","last_fullsized":"2024-09-10T20:41:58.275528+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21265,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197345+0000","last_change":"2024-09-10T20:41:36.928383+0000","last_active":"2024-09-10T20:41:58.197345+0000","last_peered":"2024-09-10T20:41:58.197345+0000","last_clean":"2024-09-10T20:41:58.197345+0000","last_became_active":"2024-09-10T20:41:36.842440+0000","last_became_peered":"2024-09-10T20:41:36.842440+0000","last_unstale":"2024-09-10T20:41:58.197345+0000","last_undegraded":"2024-09-10T20:41:58.197345+0000","last_fullsized":"2024-09-10T20:41:58.197345+0000","mapping_epoch":64,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.940798+0000","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3715,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":117,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197437+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:41:58.197437+0000","last_peered":"2024-09-10T20:41:58.197437+0000","last_clean":"2024-09-10T20:41:58.197437+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:41:58.197437+0000","last_undegraded":"2024-09-10T20:41:58.197437+0000","last_fullsized":"2024-09-10T20:41:58.197437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":122,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197523+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:41:58.197523+0000","last_peered":"2024-09-10T20:41:58.197523+0000","last_clean":"2024-09-10T20:41:58.197523+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:41:58.197523+0000","last_undegraded":"2024-09-10T20:41:58.197523+0000","last_fullsized":"2024-09-10T20:41:58.197523+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27681,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197107+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:41:58.197107+0000","last_peered":"2024-09-10T20:41:58.197107+0000","last_clean":"2024-09-10T20:41:58.197107+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:41:58.197107+0000","last_undegraded":"2024-09-10T20:41:58.197107+0000","last_fullsized":"2024-09-10T20:41:58.197107+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39530,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":48754688,"data_stored":48747462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182561,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443132,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93928708,"statfs":{"total":96636764160,"available":96182996992,"internally_reserved":0,"allocated":565248,"data_stored":214155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986334,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":413748,"kb_used_data":48868,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93958092,"statfs":{"total":96636764160,"available":96213086208,"internally_reserved":0,"allocated":50040832,"data_stored":49697429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659280,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497340,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874500,"statfs":{"total":96636764160,"available":96127488000,"internally_reserved":0,"allocated":565248,"data_stored":214155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48746496,"data_stored":48746496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:42:06.818 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:42:07.099 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:42:07.100 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:42:07.112 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":480,"stamp":"2024-09-10T20:42:07.084580+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39532,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1354220,"kb_used_data":49972,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281761300,"statfs":{"total":289910292480,"available":288523571200,"internally_reserved":0,"allocated":51171328,"data_stored":50125739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1335479298},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-342,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.888567"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":28008,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197950+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:41:58.197950+0000","last_peered":"2024-09-10T20:41:58.197950+0000","last_clean":"2024-09-10T20:41:58.197950+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:41:58.197950+0000","last_undegraded":"2024-09-10T20:41:58.197950+0000","last_fullsized":"2024-09-10T20:41:58.197950+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21100,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275553+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:41:58.275553+0000","last_peered":"2024-09-10T20:41:58.275553+0000","last_clean":"2024-09-10T20:41:58.275553+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:41:58.275553+0000","last_undegraded":"2024-09-10T20:41:58.275553+0000","last_fullsized":"2024-09-10T20:41:58.275553+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28112,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197262+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:41:58.197262+0000","last_peered":"2024-09-10T20:41:58.197262+0000","last_clean":"2024-09-10T20:41:58.197262+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:41:58.197262+0000","last_undegraded":"2024-09-10T20:41:58.197262+0000","last_fullsized":"2024-09-10T20:41:58.197262+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20732,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275478+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:41:58.275478+0000","last_peered":"2024-09-10T20:41:58.275478+0000","last_clean":"2024-09-10T20:41:58.275478+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:41:58.275478+0000","last_undegraded":"2024-09-10T20:41:58.275478+0000","last_fullsized":"2024-09-10T20:41:58.275478+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21243,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197816+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:41:58.197816+0000","last_peered":"2024-09-10T20:41:58.197816+0000","last_clean":"2024-09-10T20:41:58.197816+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:41:58.197816+0000","last_undegraded":"2024-09-10T20:41:58.197816+0000","last_fullsized":"2024-09-10T20:41:58.197816+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20741,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.265870+0000","last_change":"2024-09-10T20:41:58.265777+0000","last_active":"2024-09-10T20:41:58.265870+0000","last_peered":"2024-09-10T20:41:58.265870+0000","last_clean":"2024-09-10T20:41:58.265870+0000","last_became_active":"2024-09-10T20:41:58.198887+0000","last_became_peered":"2024-09-10T20:41:58.198887+0000","last_unstale":"2024-09-10T20:41:58.265870+0000","last_undegraded":"2024-09-10T20:41:58.265870+0000","last_fullsized":"2024-09-10T20:41:58.265870+0000","mapping_epoch":66,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:51.015540+0000","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3616,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20919,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197002+0000","last_change":"2024-09-10T20:41:17.750026+0000","last_active":"2024-09-10T20:41:58.197002+0000","last_peered":"2024-09-10T20:41:58.197002+0000","last_clean":"2024-09-10T20:41:58.197002+0000","last_became_active":"2024-09-10T20:41:17.668013+0000","last_became_peered":"2024-09-10T20:41:17.668013+0000","last_unstale":"2024-09-10T20:41:58.197002+0000","last_undegraded":"2024-09-10T20:41:58.197002+0000","last_fullsized":"2024-09-10T20:41:58.197002+0000","mapping_epoch":62,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:55.885282+0000","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20909,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197768+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:41:58.197768+0000","last_peered":"2024-09-10T20:41:58.197768+0000","last_clean":"2024-09-10T20:41:58.197768+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:41:58.197768+0000","last_undegraded":"2024-09-10T20:41:58.197768+0000","last_fullsized":"2024-09-10T20:41:58.197768+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":117,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.196822+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:41:58.196822+0000","last_peered":"2024-09-10T20:41:58.196822+0000","last_clean":"2024-09-10T20:41:58.196822+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:41:58.196822+0000","last_undegraded":"2024-09-10T20:41:58.196822+0000","last_fullsized":"2024-09-10T20:41:58.196822+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22151,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197391+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:41:58.197391+0000","last_peered":"2024-09-10T20:41:58.197391+0000","last_clean":"2024-09-10T20:41:58.197391+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:41:58.197391+0000","last_undegraded":"2024-09-10T20:41:58.197391+0000","last_fullsized":"2024-09-10T20:41:58.197391+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":142,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275332+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:41:58.275332+0000","last_peered":"2024-09-10T20:41:58.275332+0000","last_clean":"2024-09-10T20:41:58.275332+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:41:58.275332+0000","last_undegraded":"2024-09-10T20:41:58.275332+0000","last_fullsized":"2024-09-10T20:41:58.275332+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28137,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197970+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:41:58.197970+0000","last_peered":"2024-09-10T20:41:58.197970+0000","last_clean":"2024-09-10T20:41:58.197970+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:41:58.197970+0000","last_undegraded":"2024-09-10T20:41:58.197970+0000","last_fullsized":"2024-09-10T20:41:58.197970+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":149,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197712+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:41:58.197712+0000","last_peered":"2024-09-10T20:41:58.197712+0000","last_clean":"2024-09-10T20:41:58.197712+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:41:58.197712+0000","last_undegraded":"2024-09-10T20:41:58.197712+0000","last_fullsized":"2024-09-10T20:41:58.197712+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28020,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275395+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:41:58.275395+0000","last_peered":"2024-09-10T20:41:58.275395+0000","last_clean":"2024-09-10T20:41:58.275395+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:41:58.275395+0000","last_undegraded":"2024-09-10T20:41:58.275395+0000","last_fullsized":"2024-09-10T20:41:58.275395+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":127,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197287+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:41:58.197287+0000","last_peered":"2024-09-10T20:41:58.197287+0000","last_clean":"2024-09-10T20:41:58.197287+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:41:58.197287+0000","last_undegraded":"2024-09-10T20:41:58.197287+0000","last_fullsized":"2024-09-10T20:41:58.197287+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28073,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275280+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:41:58.275280+0000","last_peered":"2024-09-10T20:41:58.275280+0000","last_clean":"2024-09-10T20:41:58.275280+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:41:58.275280+0000","last_undegraded":"2024-09-10T20:41:58.275280+0000","last_fullsized":"2024-09-10T20:41:58.275280+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21129,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197608+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:41:58.197608+0000","last_peered":"2024-09-10T20:41:58.197608+0000","last_clean":"2024-09-10T20:41:58.197608+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:41:58.197608+0000","last_undegraded":"2024-09-10T20:41:58.197608+0000","last_fullsized":"2024-09-10T20:41:58.197608+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":117,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197184+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:41:58.197184+0000","last_peered":"2024-09-10T20:41:58.197184+0000","last_clean":"2024-09-10T20:41:58.197184+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:41:58.197184+0000","last_undegraded":"2024-09-10T20:41:58.197184+0000","last_fullsized":"2024-09-10T20:41:58.197184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27851,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197520+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:41:58.197520+0000","last_peered":"2024-09-10T20:41:58.197520+0000","last_clean":"2024-09-10T20:41:58.197520+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:41:58.197520+0000","last_undegraded":"2024-09-10T20:41:58.197520+0000","last_fullsized":"2024-09-10T20:41:58.197520+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":142,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275528+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:41:58.275528+0000","last_peered":"2024-09-10T20:41:58.275528+0000","last_clean":"2024-09-10T20:41:58.275528+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:41:58.275528+0000","last_undegraded":"2024-09-10T20:41:58.275528+0000","last_fullsized":"2024-09-10T20:41:58.275528+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21265,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197345+0000","last_change":"2024-09-10T20:41:36.928383+0000","last_active":"2024-09-10T20:41:58.197345+0000","last_peered":"2024-09-10T20:41:58.197345+0000","last_clean":"2024-09-10T20:41:58.197345+0000","last_became_active":"2024-09-10T20:41:36.842440+0000","last_became_peered":"2024-09-10T20:41:36.842440+0000","last_unstale":"2024-09-10T20:41:58.197345+0000","last_undegraded":"2024-09-10T20:41:58.197345+0000","last_fullsized":"2024-09-10T20:41:58.197345+0000","mapping_epoch":64,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.940798+0000","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3715,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":117,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197437+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:41:58.197437+0000","last_peered":"2024-09-10T20:41:58.197437+0000","last_clean":"2024-09-10T20:41:58.197437+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:41:58.197437+0000","last_undegraded":"2024-09-10T20:41:58.197437+0000","last_fullsized":"2024-09-10T20:41:58.197437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":122,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197523+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:41:58.197523+0000","last_peered":"2024-09-10T20:41:58.197523+0000","last_clean":"2024-09-10T20:41:58.197523+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:41:58.197523+0000","last_undegraded":"2024-09-10T20:41:58.197523+0000","last_fullsized":"2024-09-10T20:41:58.197523+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27681,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197107+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:41:58.197107+0000","last_peered":"2024-09-10T20:41:58.197107+0000","last_clean":"2024-09-10T20:41:58.197107+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:41:58.197107+0000","last_undegraded":"2024-09-10T20:41:58.197107+0000","last_fullsized":"2024-09-10T20:41:58.197107+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39530,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":48754688,"data_stored":48747462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182562,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443132,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93928708,"statfs":{"total":96636764160,"available":96182996992,"internally_reserved":0,"allocated":565248,"data_stored":214155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986334,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":413748,"kb_used_data":48868,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93958092,"statfs":{"total":96636764160,"available":96213086208,"internally_reserved":0,"allocated":50040832,"data_stored":49697429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659280,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497340,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874500,"statfs":{"total":96636764160,"available":96127488000,"internally_reserved":0,"allocated":565248,"data_stored":214155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48746496,"data_stored":48746496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:42:07.113 INFO:tasks.rep_lost_unfound_delete.ceph_manager:clean! 2024-09-10T20:42:07.114 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-10T20:42:07.123 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T20:42:07.123 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:42:07.408 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:42:07.408 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:42:07.421 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":480,"stamp":"2024-09-10T20:42:07.084580+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39532,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1354220,"kb_used_data":49972,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281761300,"statfs":{"total":289910292480,"available":288523571200,"internally_reserved":0,"allocated":51171328,"data_stored":50125739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1335479298},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-342,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.888567"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":28008,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197950+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:41:58.197950+0000","last_peered":"2024-09-10T20:41:58.197950+0000","last_clean":"2024-09-10T20:41:58.197950+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:41:58.197950+0000","last_undegraded":"2024-09-10T20:41:58.197950+0000","last_fullsized":"2024-09-10T20:41:58.197950+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21100,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275553+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:41:58.275553+0000","last_peered":"2024-09-10T20:41:58.275553+0000","last_clean":"2024-09-10T20:41:58.275553+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:41:58.275553+0000","last_undegraded":"2024-09-10T20:41:58.275553+0000","last_fullsized":"2024-09-10T20:41:58.275553+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28112,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197262+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:41:58.197262+0000","last_peered":"2024-09-10T20:41:58.197262+0000","last_clean":"2024-09-10T20:41:58.197262+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:41:58.197262+0000","last_undegraded":"2024-09-10T20:41:58.197262+0000","last_fullsized":"2024-09-10T20:41:58.197262+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20732,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275478+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:41:58.275478+0000","last_peered":"2024-09-10T20:41:58.275478+0000","last_clean":"2024-09-10T20:41:58.275478+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:41:58.275478+0000","last_undegraded":"2024-09-10T20:41:58.275478+0000","last_fullsized":"2024-09-10T20:41:58.275478+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21243,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197816+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:41:58.197816+0000","last_peered":"2024-09-10T20:41:58.197816+0000","last_clean":"2024-09-10T20:41:58.197816+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:41:58.197816+0000","last_undegraded":"2024-09-10T20:41:58.197816+0000","last_fullsized":"2024-09-10T20:41:58.197816+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20741,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.265870+0000","last_change":"2024-09-10T20:41:58.265777+0000","last_active":"2024-09-10T20:41:58.265870+0000","last_peered":"2024-09-10T20:41:58.265870+0000","last_clean":"2024-09-10T20:41:58.265870+0000","last_became_active":"2024-09-10T20:41:58.198887+0000","last_became_peered":"2024-09-10T20:41:58.198887+0000","last_unstale":"2024-09-10T20:41:58.265870+0000","last_undegraded":"2024-09-10T20:41:58.265870+0000","last_fullsized":"2024-09-10T20:41:58.265870+0000","mapping_epoch":66,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:51.015540+0000","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3616,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20919,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197002+0000","last_change":"2024-09-10T20:41:17.750026+0000","last_active":"2024-09-10T20:41:58.197002+0000","last_peered":"2024-09-10T20:41:58.197002+0000","last_clean":"2024-09-10T20:41:58.197002+0000","last_became_active":"2024-09-10T20:41:17.668013+0000","last_became_peered":"2024-09-10T20:41:17.668013+0000","last_unstale":"2024-09-10T20:41:58.197002+0000","last_undegraded":"2024-09-10T20:41:58.197002+0000","last_fullsized":"2024-09-10T20:41:58.197002+0000","mapping_epoch":62,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:55.885282+0000","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20909,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197768+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:41:58.197768+0000","last_peered":"2024-09-10T20:41:58.197768+0000","last_clean":"2024-09-10T20:41:58.197768+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:41:58.197768+0000","last_undegraded":"2024-09-10T20:41:58.197768+0000","last_fullsized":"2024-09-10T20:41:58.197768+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":117,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.196822+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:41:58.196822+0000","last_peered":"2024-09-10T20:41:58.196822+0000","last_clean":"2024-09-10T20:41:58.196822+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:41:58.196822+0000","last_undegraded":"2024-09-10T20:41:58.196822+0000","last_fullsized":"2024-09-10T20:41:58.196822+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22151,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197391+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:41:58.197391+0000","last_peered":"2024-09-10T20:41:58.197391+0000","last_clean":"2024-09-10T20:41:58.197391+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:41:58.197391+0000","last_undegraded":"2024-09-10T20:41:58.197391+0000","last_fullsized":"2024-09-10T20:41:58.197391+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":142,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275332+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:41:58.275332+0000","last_peered":"2024-09-10T20:41:58.275332+0000","last_clean":"2024-09-10T20:41:58.275332+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:41:58.275332+0000","last_undegraded":"2024-09-10T20:41:58.275332+0000","last_fullsized":"2024-09-10T20:41:58.275332+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28137,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197970+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:41:58.197970+0000","last_peered":"2024-09-10T20:41:58.197970+0000","last_clean":"2024-09-10T20:41:58.197970+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:41:58.197970+0000","last_undegraded":"2024-09-10T20:41:58.197970+0000","last_fullsized":"2024-09-10T20:41:58.197970+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":149,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197712+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:41:58.197712+0000","last_peered":"2024-09-10T20:41:58.197712+0000","last_clean":"2024-09-10T20:41:58.197712+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:41:58.197712+0000","last_undegraded":"2024-09-10T20:41:58.197712+0000","last_fullsized":"2024-09-10T20:41:58.197712+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28020,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275395+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:41:58.275395+0000","last_peered":"2024-09-10T20:41:58.275395+0000","last_clean":"2024-09-10T20:41:58.275395+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:41:58.275395+0000","last_undegraded":"2024-09-10T20:41:58.275395+0000","last_fullsized":"2024-09-10T20:41:58.275395+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":127,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197287+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:41:58.197287+0000","last_peered":"2024-09-10T20:41:58.197287+0000","last_clean":"2024-09-10T20:41:58.197287+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:41:58.197287+0000","last_undegraded":"2024-09-10T20:41:58.197287+0000","last_fullsized":"2024-09-10T20:41:58.197287+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28073,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275280+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:41:58.275280+0000","last_peered":"2024-09-10T20:41:58.275280+0000","last_clean":"2024-09-10T20:41:58.275280+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:41:58.275280+0000","last_undegraded":"2024-09-10T20:41:58.275280+0000","last_fullsized":"2024-09-10T20:41:58.275280+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21129,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197608+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:41:58.197608+0000","last_peered":"2024-09-10T20:41:58.197608+0000","last_clean":"2024-09-10T20:41:58.197608+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:41:58.197608+0000","last_undegraded":"2024-09-10T20:41:58.197608+0000","last_fullsized":"2024-09-10T20:41:58.197608+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":117,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197184+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:41:58.197184+0000","last_peered":"2024-09-10T20:41:58.197184+0000","last_clean":"2024-09-10T20:41:58.197184+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:41:58.197184+0000","last_undegraded":"2024-09-10T20:41:58.197184+0000","last_fullsized":"2024-09-10T20:41:58.197184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27851,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197520+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:41:58.197520+0000","last_peered":"2024-09-10T20:41:58.197520+0000","last_clean":"2024-09-10T20:41:58.197520+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:41:58.197520+0000","last_undegraded":"2024-09-10T20:41:58.197520+0000","last_fullsized":"2024-09-10T20:41:58.197520+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":142,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275528+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:41:58.275528+0000","last_peered":"2024-09-10T20:41:58.275528+0000","last_clean":"2024-09-10T20:41:58.275528+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:41:58.275528+0000","last_undegraded":"2024-09-10T20:41:58.275528+0000","last_fullsized":"2024-09-10T20:41:58.275528+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21265,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197345+0000","last_change":"2024-09-10T20:41:36.928383+0000","last_active":"2024-09-10T20:41:58.197345+0000","last_peered":"2024-09-10T20:41:58.197345+0000","last_clean":"2024-09-10T20:41:58.197345+0000","last_became_active":"2024-09-10T20:41:36.842440+0000","last_became_peered":"2024-09-10T20:41:36.842440+0000","last_unstale":"2024-09-10T20:41:58.197345+0000","last_undegraded":"2024-09-10T20:41:58.197345+0000","last_fullsized":"2024-09-10T20:41:58.197345+0000","mapping_epoch":64,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.940798+0000","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3715,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":117,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197437+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:41:58.197437+0000","last_peered":"2024-09-10T20:41:58.197437+0000","last_clean":"2024-09-10T20:41:58.197437+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:41:58.197437+0000","last_undegraded":"2024-09-10T20:41:58.197437+0000","last_fullsized":"2024-09-10T20:41:58.197437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":122,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197523+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:41:58.197523+0000","last_peered":"2024-09-10T20:41:58.197523+0000","last_clean":"2024-09-10T20:41:58.197523+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:41:58.197523+0000","last_undegraded":"2024-09-10T20:41:58.197523+0000","last_fullsized":"2024-09-10T20:41:58.197523+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27681,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197107+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:41:58.197107+0000","last_peered":"2024-09-10T20:41:58.197107+0000","last_clean":"2024-09-10T20:41:58.197107+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:41:58.197107+0000","last_undegraded":"2024-09-10T20:41:58.197107+0000","last_fullsized":"2024-09-10T20:41:58.197107+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39530,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":48754688,"data_stored":48747462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182562,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443132,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93928708,"statfs":{"total":96636764160,"available":96182996992,"internally_reserved":0,"allocated":565248,"data_stored":214155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986334,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":413748,"kb_used_data":48868,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93958092,"statfs":{"total":96636764160,"available":96213086208,"internally_reserved":0,"allocated":50040832,"data_stored":49697429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659280,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497340,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874500,"statfs":{"total":96636764160,"available":96127488000,"internally_reserved":0,"allocated":565248,"data_stored":214155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48746496,"data_stored":48746496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:42:07.422 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:42:07.703 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:42:07.704 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:42:07.716 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":480,"stamp":"2024-09-10T20:42:07.084580+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39532,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1354220,"kb_used_data":49972,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281761300,"statfs":{"total":289910292480,"available":288523571200,"internally_reserved":0,"allocated":51171328,"data_stored":50125739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1335479298},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-342,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.888567"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":28008,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197950+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:41:58.197950+0000","last_peered":"2024-09-10T20:41:58.197950+0000","last_clean":"2024-09-10T20:41:58.197950+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:41:58.197950+0000","last_undegraded":"2024-09-10T20:41:58.197950+0000","last_fullsized":"2024-09-10T20:41:58.197950+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21100,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275553+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:41:58.275553+0000","last_peered":"2024-09-10T20:41:58.275553+0000","last_clean":"2024-09-10T20:41:58.275553+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:41:58.275553+0000","last_undegraded":"2024-09-10T20:41:58.275553+0000","last_fullsized":"2024-09-10T20:41:58.275553+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28112,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197262+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:41:58.197262+0000","last_peered":"2024-09-10T20:41:58.197262+0000","last_clean":"2024-09-10T20:41:58.197262+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:41:58.197262+0000","last_undegraded":"2024-09-10T20:41:58.197262+0000","last_fullsized":"2024-09-10T20:41:58.197262+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20732,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275478+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:41:58.275478+0000","last_peered":"2024-09-10T20:41:58.275478+0000","last_clean":"2024-09-10T20:41:58.275478+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:41:58.275478+0000","last_undegraded":"2024-09-10T20:41:58.275478+0000","last_fullsized":"2024-09-10T20:41:58.275478+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21243,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197816+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:41:58.197816+0000","last_peered":"2024-09-10T20:41:58.197816+0000","last_clean":"2024-09-10T20:41:58.197816+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:41:58.197816+0000","last_undegraded":"2024-09-10T20:41:58.197816+0000","last_fullsized":"2024-09-10T20:41:58.197816+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20741,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.265870+0000","last_change":"2024-09-10T20:41:58.265777+0000","last_active":"2024-09-10T20:41:58.265870+0000","last_peered":"2024-09-10T20:41:58.265870+0000","last_clean":"2024-09-10T20:41:58.265870+0000","last_became_active":"2024-09-10T20:41:58.198887+0000","last_became_peered":"2024-09-10T20:41:58.198887+0000","last_unstale":"2024-09-10T20:41:58.265870+0000","last_undegraded":"2024-09-10T20:41:58.265870+0000","last_fullsized":"2024-09-10T20:41:58.265870+0000","mapping_epoch":66,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:51.015540+0000","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3616,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20919,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197002+0000","last_change":"2024-09-10T20:41:17.750026+0000","last_active":"2024-09-10T20:41:58.197002+0000","last_peered":"2024-09-10T20:41:58.197002+0000","last_clean":"2024-09-10T20:41:58.197002+0000","last_became_active":"2024-09-10T20:41:17.668013+0000","last_became_peered":"2024-09-10T20:41:17.668013+0000","last_unstale":"2024-09-10T20:41:58.197002+0000","last_undegraded":"2024-09-10T20:41:58.197002+0000","last_fullsized":"2024-09-10T20:41:58.197002+0000","mapping_epoch":62,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:55.885282+0000","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20909,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197768+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:41:58.197768+0000","last_peered":"2024-09-10T20:41:58.197768+0000","last_clean":"2024-09-10T20:41:58.197768+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:41:58.197768+0000","last_undegraded":"2024-09-10T20:41:58.197768+0000","last_fullsized":"2024-09-10T20:41:58.197768+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":117,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.196822+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:41:58.196822+0000","last_peered":"2024-09-10T20:41:58.196822+0000","last_clean":"2024-09-10T20:41:58.196822+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:41:58.196822+0000","last_undegraded":"2024-09-10T20:41:58.196822+0000","last_fullsized":"2024-09-10T20:41:58.196822+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22151,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197391+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:41:58.197391+0000","last_peered":"2024-09-10T20:41:58.197391+0000","last_clean":"2024-09-10T20:41:58.197391+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:41:58.197391+0000","last_undegraded":"2024-09-10T20:41:58.197391+0000","last_fullsized":"2024-09-10T20:41:58.197391+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":142,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275332+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:41:58.275332+0000","last_peered":"2024-09-10T20:41:58.275332+0000","last_clean":"2024-09-10T20:41:58.275332+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:41:58.275332+0000","last_undegraded":"2024-09-10T20:41:58.275332+0000","last_fullsized":"2024-09-10T20:41:58.275332+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28137,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197970+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:41:58.197970+0000","last_peered":"2024-09-10T20:41:58.197970+0000","last_clean":"2024-09-10T20:41:58.197970+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:41:58.197970+0000","last_undegraded":"2024-09-10T20:41:58.197970+0000","last_fullsized":"2024-09-10T20:41:58.197970+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":149,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197712+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:41:58.197712+0000","last_peered":"2024-09-10T20:41:58.197712+0000","last_clean":"2024-09-10T20:41:58.197712+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:41:58.197712+0000","last_undegraded":"2024-09-10T20:41:58.197712+0000","last_fullsized":"2024-09-10T20:41:58.197712+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28020,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275395+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:41:58.275395+0000","last_peered":"2024-09-10T20:41:58.275395+0000","last_clean":"2024-09-10T20:41:58.275395+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:41:58.275395+0000","last_undegraded":"2024-09-10T20:41:58.275395+0000","last_fullsized":"2024-09-10T20:41:58.275395+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":127,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197287+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:41:58.197287+0000","last_peered":"2024-09-10T20:41:58.197287+0000","last_clean":"2024-09-10T20:41:58.197287+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:41:58.197287+0000","last_undegraded":"2024-09-10T20:41:58.197287+0000","last_fullsized":"2024-09-10T20:41:58.197287+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28073,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275280+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:41:58.275280+0000","last_peered":"2024-09-10T20:41:58.275280+0000","last_clean":"2024-09-10T20:41:58.275280+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:41:58.275280+0000","last_undegraded":"2024-09-10T20:41:58.275280+0000","last_fullsized":"2024-09-10T20:41:58.275280+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21129,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197608+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:41:58.197608+0000","last_peered":"2024-09-10T20:41:58.197608+0000","last_clean":"2024-09-10T20:41:58.197608+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:41:58.197608+0000","last_undegraded":"2024-09-10T20:41:58.197608+0000","last_fullsized":"2024-09-10T20:41:58.197608+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":117,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197184+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:41:58.197184+0000","last_peered":"2024-09-10T20:41:58.197184+0000","last_clean":"2024-09-10T20:41:58.197184+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:41:58.197184+0000","last_undegraded":"2024-09-10T20:41:58.197184+0000","last_fullsized":"2024-09-10T20:41:58.197184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27851,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197520+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:41:58.197520+0000","last_peered":"2024-09-10T20:41:58.197520+0000","last_clean":"2024-09-10T20:41:58.197520+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:41:58.197520+0000","last_undegraded":"2024-09-10T20:41:58.197520+0000","last_fullsized":"2024-09-10T20:41:58.197520+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":142,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275528+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:41:58.275528+0000","last_peered":"2024-09-10T20:41:58.275528+0000","last_clean":"2024-09-10T20:41:58.275528+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:41:58.275528+0000","last_undegraded":"2024-09-10T20:41:58.275528+0000","last_fullsized":"2024-09-10T20:41:58.275528+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21265,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197345+0000","last_change":"2024-09-10T20:41:36.928383+0000","last_active":"2024-09-10T20:41:58.197345+0000","last_peered":"2024-09-10T20:41:58.197345+0000","last_clean":"2024-09-10T20:41:58.197345+0000","last_became_active":"2024-09-10T20:41:36.842440+0000","last_became_peered":"2024-09-10T20:41:36.842440+0000","last_unstale":"2024-09-10T20:41:58.197345+0000","last_undegraded":"2024-09-10T20:41:58.197345+0000","last_fullsized":"2024-09-10T20:41:58.197345+0000","mapping_epoch":64,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.940798+0000","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3715,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":117,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197437+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:41:58.197437+0000","last_peered":"2024-09-10T20:41:58.197437+0000","last_clean":"2024-09-10T20:41:58.197437+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:41:58.197437+0000","last_undegraded":"2024-09-10T20:41:58.197437+0000","last_fullsized":"2024-09-10T20:41:58.197437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":122,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197523+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:41:58.197523+0000","last_peered":"2024-09-10T20:41:58.197523+0000","last_clean":"2024-09-10T20:41:58.197523+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:41:58.197523+0000","last_undegraded":"2024-09-10T20:41:58.197523+0000","last_fullsized":"2024-09-10T20:41:58.197523+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27681,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197107+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:41:58.197107+0000","last_peered":"2024-09-10T20:41:58.197107+0000","last_clean":"2024-09-10T20:41:58.197107+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:41:58.197107+0000","last_undegraded":"2024-09-10T20:41:58.197107+0000","last_fullsized":"2024-09-10T20:41:58.197107+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39530,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":48754688,"data_stored":48747462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182562,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443132,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93928708,"statfs":{"total":96636764160,"available":96182996992,"internally_reserved":0,"allocated":565248,"data_stored":214155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986334,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":413748,"kb_used_data":48868,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93958092,"statfs":{"total":96636764160,"available":96213086208,"internally_reserved":0,"allocated":50040832,"data_stored":49697429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659280,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497340,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874500,"statfs":{"total":96636764160,"available":96127488000,"internally_reserved":0,"allocated":565248,"data_stored":214155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48746496,"data_stored":48746496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:42:07.717 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T20:42:07.717 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:42:07.998 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:42:07.999 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:42:08.013 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":480,"stamp":"2024-09-10T20:42:07.084580+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39532,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1354220,"kb_used_data":49972,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281761300,"statfs":{"total":289910292480,"available":288523571200,"internally_reserved":0,"allocated":51171328,"data_stored":50125739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1335479298},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-342,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.888567"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":28008,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197950+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:41:58.197950+0000","last_peered":"2024-09-10T20:41:58.197950+0000","last_clean":"2024-09-10T20:41:58.197950+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:41:58.197950+0000","last_undegraded":"2024-09-10T20:41:58.197950+0000","last_fullsized":"2024-09-10T20:41:58.197950+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21100,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275553+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:41:58.275553+0000","last_peered":"2024-09-10T20:41:58.275553+0000","last_clean":"2024-09-10T20:41:58.275553+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:41:58.275553+0000","last_undegraded":"2024-09-10T20:41:58.275553+0000","last_fullsized":"2024-09-10T20:41:58.275553+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28112,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197262+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:41:58.197262+0000","last_peered":"2024-09-10T20:41:58.197262+0000","last_clean":"2024-09-10T20:41:58.197262+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:41:58.197262+0000","last_undegraded":"2024-09-10T20:41:58.197262+0000","last_fullsized":"2024-09-10T20:41:58.197262+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20732,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275478+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:41:58.275478+0000","last_peered":"2024-09-10T20:41:58.275478+0000","last_clean":"2024-09-10T20:41:58.275478+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:41:58.275478+0000","last_undegraded":"2024-09-10T20:41:58.275478+0000","last_fullsized":"2024-09-10T20:41:58.275478+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21243,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197816+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:41:58.197816+0000","last_peered":"2024-09-10T20:41:58.197816+0000","last_clean":"2024-09-10T20:41:58.197816+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:41:58.197816+0000","last_undegraded":"2024-09-10T20:41:58.197816+0000","last_fullsized":"2024-09-10T20:41:58.197816+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20741,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.265870+0000","last_change":"2024-09-10T20:41:58.265777+0000","last_active":"2024-09-10T20:41:58.265870+0000","last_peered":"2024-09-10T20:41:58.265870+0000","last_clean":"2024-09-10T20:41:58.265870+0000","last_became_active":"2024-09-10T20:41:58.198887+0000","last_became_peered":"2024-09-10T20:41:58.198887+0000","last_unstale":"2024-09-10T20:41:58.265870+0000","last_undegraded":"2024-09-10T20:41:58.265870+0000","last_fullsized":"2024-09-10T20:41:58.265870+0000","mapping_epoch":66,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:51.015540+0000","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3616,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20919,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197002+0000","last_change":"2024-09-10T20:41:17.750026+0000","last_active":"2024-09-10T20:41:58.197002+0000","last_peered":"2024-09-10T20:41:58.197002+0000","last_clean":"2024-09-10T20:41:58.197002+0000","last_became_active":"2024-09-10T20:41:17.668013+0000","last_became_peered":"2024-09-10T20:41:17.668013+0000","last_unstale":"2024-09-10T20:41:58.197002+0000","last_undegraded":"2024-09-10T20:41:58.197002+0000","last_fullsized":"2024-09-10T20:41:58.197002+0000","mapping_epoch":62,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:55.885282+0000","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20909,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197768+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:41:58.197768+0000","last_peered":"2024-09-10T20:41:58.197768+0000","last_clean":"2024-09-10T20:41:58.197768+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:41:58.197768+0000","last_undegraded":"2024-09-10T20:41:58.197768+0000","last_fullsized":"2024-09-10T20:41:58.197768+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":117,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.196822+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:41:58.196822+0000","last_peered":"2024-09-10T20:41:58.196822+0000","last_clean":"2024-09-10T20:41:58.196822+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:41:58.196822+0000","last_undegraded":"2024-09-10T20:41:58.196822+0000","last_fullsized":"2024-09-10T20:41:58.196822+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22151,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197391+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:41:58.197391+0000","last_peered":"2024-09-10T20:41:58.197391+0000","last_clean":"2024-09-10T20:41:58.197391+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:41:58.197391+0000","last_undegraded":"2024-09-10T20:41:58.197391+0000","last_fullsized":"2024-09-10T20:41:58.197391+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":142,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275332+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:41:58.275332+0000","last_peered":"2024-09-10T20:41:58.275332+0000","last_clean":"2024-09-10T20:41:58.275332+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:41:58.275332+0000","last_undegraded":"2024-09-10T20:41:58.275332+0000","last_fullsized":"2024-09-10T20:41:58.275332+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28137,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197970+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:41:58.197970+0000","last_peered":"2024-09-10T20:41:58.197970+0000","last_clean":"2024-09-10T20:41:58.197970+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:41:58.197970+0000","last_undegraded":"2024-09-10T20:41:58.197970+0000","last_fullsized":"2024-09-10T20:41:58.197970+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":149,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197712+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:41:58.197712+0000","last_peered":"2024-09-10T20:41:58.197712+0000","last_clean":"2024-09-10T20:41:58.197712+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:41:58.197712+0000","last_undegraded":"2024-09-10T20:41:58.197712+0000","last_fullsized":"2024-09-10T20:41:58.197712+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28020,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275395+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:41:58.275395+0000","last_peered":"2024-09-10T20:41:58.275395+0000","last_clean":"2024-09-10T20:41:58.275395+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:41:58.275395+0000","last_undegraded":"2024-09-10T20:41:58.275395+0000","last_fullsized":"2024-09-10T20:41:58.275395+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":127,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197287+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:41:58.197287+0000","last_peered":"2024-09-10T20:41:58.197287+0000","last_clean":"2024-09-10T20:41:58.197287+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:41:58.197287+0000","last_undegraded":"2024-09-10T20:41:58.197287+0000","last_fullsized":"2024-09-10T20:41:58.197287+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28073,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275280+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:41:58.275280+0000","last_peered":"2024-09-10T20:41:58.275280+0000","last_clean":"2024-09-10T20:41:58.275280+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:41:58.275280+0000","last_undegraded":"2024-09-10T20:41:58.275280+0000","last_fullsized":"2024-09-10T20:41:58.275280+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21129,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197608+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:41:58.197608+0000","last_peered":"2024-09-10T20:41:58.197608+0000","last_clean":"2024-09-10T20:41:58.197608+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:41:58.197608+0000","last_undegraded":"2024-09-10T20:41:58.197608+0000","last_fullsized":"2024-09-10T20:41:58.197608+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":117,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197184+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:41:58.197184+0000","last_peered":"2024-09-10T20:41:58.197184+0000","last_clean":"2024-09-10T20:41:58.197184+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:41:58.197184+0000","last_undegraded":"2024-09-10T20:41:58.197184+0000","last_fullsized":"2024-09-10T20:41:58.197184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27851,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197520+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:41:58.197520+0000","last_peered":"2024-09-10T20:41:58.197520+0000","last_clean":"2024-09-10T20:41:58.197520+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:41:58.197520+0000","last_undegraded":"2024-09-10T20:41:58.197520+0000","last_fullsized":"2024-09-10T20:41:58.197520+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":142,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275528+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:41:58.275528+0000","last_peered":"2024-09-10T20:41:58.275528+0000","last_clean":"2024-09-10T20:41:58.275528+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:41:58.275528+0000","last_undegraded":"2024-09-10T20:41:58.275528+0000","last_fullsized":"2024-09-10T20:41:58.275528+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21265,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197345+0000","last_change":"2024-09-10T20:41:36.928383+0000","last_active":"2024-09-10T20:41:58.197345+0000","last_peered":"2024-09-10T20:41:58.197345+0000","last_clean":"2024-09-10T20:41:58.197345+0000","last_became_active":"2024-09-10T20:41:36.842440+0000","last_became_peered":"2024-09-10T20:41:36.842440+0000","last_unstale":"2024-09-10T20:41:58.197345+0000","last_undegraded":"2024-09-10T20:41:58.197345+0000","last_fullsized":"2024-09-10T20:41:58.197345+0000","mapping_epoch":64,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.940798+0000","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3715,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":117,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197437+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:41:58.197437+0000","last_peered":"2024-09-10T20:41:58.197437+0000","last_clean":"2024-09-10T20:41:58.197437+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:41:58.197437+0000","last_undegraded":"2024-09-10T20:41:58.197437+0000","last_fullsized":"2024-09-10T20:41:58.197437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":122,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197523+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:41:58.197523+0000","last_peered":"2024-09-10T20:41:58.197523+0000","last_clean":"2024-09-10T20:41:58.197523+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:41:58.197523+0000","last_undegraded":"2024-09-10T20:41:58.197523+0000","last_fullsized":"2024-09-10T20:41:58.197523+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27681,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197107+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:41:58.197107+0000","last_peered":"2024-09-10T20:41:58.197107+0000","last_clean":"2024-09-10T20:41:58.197107+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:41:58.197107+0000","last_undegraded":"2024-09-10T20:41:58.197107+0000","last_fullsized":"2024-09-10T20:41:58.197107+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39530,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":48754688,"data_stored":48747462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182562,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443132,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93928708,"statfs":{"total":96636764160,"available":96182996992,"internally_reserved":0,"allocated":565248,"data_stored":214155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986334,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":413748,"kb_used_data":48868,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93958092,"statfs":{"total":96636764160,"available":96213086208,"internally_reserved":0,"allocated":50040832,"data_stored":49697429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659280,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497340,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874500,"statfs":{"total":96636764160,"available":96127488000,"internally_reserved":0,"allocated":565248,"data_stored":214155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48746496,"data_stored":48746496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:42:08.014 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T20:42:08.302 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:42:08.303 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":67,"fsid":"3053cb46-0512-45f5-9707-4eb48df5a665","created":"2024-09-10T20:28:08.371455+0000","modified":"2024-09-10T20:41:58.192934+0000","last_up_change":"2024-09-10T20:30:03.291576+0000","last_in_change":"2024-09-10T20:29:57.228193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-10T20:28:23.669739+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfounddel_pool","create_time":"2024-09-10T20:28:36.617409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d2261461-51c1-40ff-9ad5-8f4b10800e7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":21,"up_from":25,"up_thru":65,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":567435338},{"type":"v1","addr":"172.21.15.131:6801","nonce":567435338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":567435338},{"type":"v1","addr":"172.21.15.131:6803","nonce":567435338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":567435338},{"type":"v1","addr":"172.21.15.131:6807","nonce":567435338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":567435338},{"type":"v1","addr":"172.21.15.131:6805","nonce":567435338}]},"public_addr":"172.21.15.131:6801/567435338","cluster_addr":"172.21.15.131:6803/567435338","heartbeat_back_addr":"172.21.15.131:6807/567435338","heartbeat_front_addr":"172.21.15.131:6805/567435338","state":["exists","up"]},{"osd":1,"uuid":"9cb327f8-9ea4-489b-9cdf-1007a983d385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":26,"up_from":41,"up_thru":66,"down_at":27,"lost_at":27,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1855213743},{"type":"v1","addr":"172.21.15.131:6817","nonce":1855213743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1855213743},{"type":"v1","addr":"172.21.15.131:6819","nonce":1855213743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1855213743},{"type":"v1","addr":"172.21.15.131:6823","nonce":1855213743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1855213743},{"type":"v1","addr":"172.21.15.131:6821","nonce":1855213743}]},"public_addr":"172.21.15.131:6817/1855213743","cluster_addr":"172.21.15.131:6819/1855213743","heartbeat_back_addr":"172.21.15.131:6823/1855213743","heartbeat_front_addr":"172.21.15.131:6821/1855213743","state":["exists","up"]},{"osd":2,"uuid":"2a1df90e-2933-4052-a62a-ffa454708bb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":18,"up_from":31,"up_thru":56,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":2043655954},{"type":"v1","addr":"172.21.15.131:6809","nonce":2043655954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2043655954},{"type":"v1","addr":"172.21.15.131:6811","nonce":2043655954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2043655954},{"type":"v1","addr":"172.21.15.131:6815","nonce":2043655954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2043655954},{"type":"v1","addr":"172.21.15.131:6813","nonce":2043655954}]},"public_addr":"172.21.15.131:6809/2043655954","cluster_addr":"172.21.15.131:6811/2043655954","heartbeat_back_addr":"172.21.15.131:6815/2043655954","heartbeat_front_addr":"172.21.15.131:6813/2043655954","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:28:16.289792+0000","dead_epoch":22},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:28:15.574958+0000","dead_epoch":27},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:28:15.599924+0000","dead_epoch":19}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T20:42:09.316 INFO:tasks.ceph:Scrubbing osd.0 2024-09-10T20:42:09.317 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-10T20:42:09.445 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-10T20:42:09.445 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T20:42:09.445 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-10T20:42:09.456 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-10T20:42:09.751 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 0 to deep-scrub 2024-09-10T20:42:09.765 INFO:tasks.ceph:Scrubbing osd.1 2024-09-10T20:42:09.765 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-10T20:42:09.887 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-10T20:42:09.888 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T20:42:09.888 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-10T20:42:09.898 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-10T20:42:10.180 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 1 to deep-scrub 2024-09-10T20:42:10.194 INFO:tasks.ceph:Scrubbing osd.2 2024-09-10T20:42:10.194 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-10T20:42:10.317 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-10T20:42:10.317 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T20:42:10.317 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-10T20:42:10.327 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-10T20:42:10.614 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 2 to deep-scrub 2024-09-10T20:42:10.628 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:42:10.908 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:42:10.909 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:42:10.921 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":481,"stamp":"2024-09-10T20:42:09.084920+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39532,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1354220,"kb_used_data":49972,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281761300,"statfs":{"total":289910292480,"available":288523571200,"internally_reserved":0,"allocated":51171328,"data_stored":50125739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1335479298},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.889329"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":28008,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197950+0000","last_change":"2024-09-10T20:40:58.617385+0000","last_active":"2024-09-10T20:41:58.197950+0000","last_peered":"2024-09-10T20:41:58.197950+0000","last_clean":"2024-09-10T20:41:58.197950+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:41:58.197950+0000","last_undegraded":"2024-09-10T20:41:58.197950+0000","last_fullsized":"2024-09-10T20:41:58.197950+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.725072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21100,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275553+0000","last_change":"2024-09-10T20:29:40.357759+0000","last_active":"2024-09-10T20:41:58.275553+0000","last_peered":"2024-09-10T20:41:58.275553+0000","last_clean":"2024-09-10T20:41:58.275553+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:41:58.275553+0000","last_undegraded":"2024-09-10T20:41:58.275553+0000","last_fullsized":"2024-09-10T20:41:58.275553+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1877","last_scrub_stamp":"2024-09-10T20:29:40.357630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:40.357630+0000","objects_scrubbed":1808,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:18.817571+0000","scrub_duration":699,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28112,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197262+0000","last_change":"2024-09-10T20:32:18.204963+0000","last_active":"2024-09-10T20:41:58.197262+0000","last_peered":"2024-09-10T20:41:58.197262+0000","last_clean":"2024-09-10T20:41:58.197262+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:41:58.197262+0000","last_undegraded":"2024-09-10T20:41:58.197262+0000","last_fullsized":"2024-09-10T20:41:58.197262+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1910","last_scrub_stamp":"2024-09-10T20:29:41.318256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:41.318256+0000","objects_scrubbed":1868,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:05.742390+0000","scrub_duration":673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20732,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275478+0000","last_change":"2024-09-10T20:29:20.018893+0000","last_active":"2024-09-10T20:41:58.275478+0000","last_peered":"2024-09-10T20:41:58.275478+0000","last_clean":"2024-09-10T20:41:58.275478+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:41:58.275478+0000","last_undegraded":"2024-09-10T20:41:58.275478+0000","last_fullsized":"2024-09-10T20:41:58.275478+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:28:37.510222+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:53.511948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21243,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197816+0000","last_change":"2024-09-10T20:29:53.768508+0000","last_active":"2024-09-10T20:41:58.197816+0000","last_peered":"2024-09-10T20:41:58.197816+0000","last_clean":"2024-09-10T20:41:58.197816+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:41:58.197816+0000","last_undegraded":"2024-09-10T20:41:58.197816+0000","last_fullsized":"2024-09-10T20:41:58.197816+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3048","last_scrub_stamp":"2024-09-10T20:29:53.768396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:53.768396+0000","objects_scrubbed":2997,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.392632+0000","scrub_duration":1288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20741,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.265870+0000","last_change":"2024-09-10T20:41:58.265777+0000","last_active":"2024-09-10T20:41:58.265870+0000","last_peered":"2024-09-10T20:41:58.265870+0000","last_clean":"2024-09-10T20:41:58.265870+0000","last_became_active":"2024-09-10T20:41:58.198887+0000","last_became_peered":"2024-09-10T20:41:58.198887+0000","last_unstale":"2024-09-10T20:41:58.265870+0000","last_undegraded":"2024-09-10T20:41:58.265870+0000","last_fullsized":"2024-09-10T20:41:58.265870+0000","mapping_epoch":66,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3180","last_scrub_stamp":"2024-09-10T20:29:55.781388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:55.781388+0000","objects_scrubbed":3131,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:51.015540+0000","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3616,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20919,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197002+0000","last_change":"2024-09-10T20:41:17.750026+0000","last_active":"2024-09-10T20:41:58.197002+0000","last_peered":"2024-09-10T20:41:58.197002+0000","last_clean":"2024-09-10T20:41:58.197002+0000","last_became_active":"2024-09-10T20:41:17.668013+0000","last_became_peered":"2024-09-10T20:41:17.668013+0000","last_unstale":"2024-09-10T20:41:58.197002+0000","last_undegraded":"2024-09-10T20:41:58.197002+0000","last_fullsized":"2024-09-10T20:41:58.197002+0000","mapping_epoch":62,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3448","last_scrub_stamp":"2024-09-10T20:30:01.859333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:01.859333+0000","objects_scrubbed":3399,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:55.885282+0000","scrub_duration":1285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20909,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197768+0000","last_change":"2024-09-10T20:29:52.392407+0000","last_active":"2024-09-10T20:41:58.197768+0000","last_peered":"2024-09-10T20:41:58.197768+0000","last_clean":"2024-09-10T20:41:58.197768+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:41:58.197768+0000","last_undegraded":"2024-09-10T20:41:58.197768+0000","last_fullsized":"2024-09-10T20:41:58.197768+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2966","last_scrub_stamp":"2024-09-10T20:29:52.392191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:52.392191+0000","objects_scrubbed":2927,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:16.337823+0000","scrub_duration":926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.196872+0000","last_change":"2024-09-10T20:30:05.316339+0000","last_active":"2024-09-10T20:41:58.196872+0000","last_peered":"2024-09-10T20:41:58.196872+0000","last_clean":"2024-09-10T20:41:58.196872+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:41:58.196872+0000","last_undegraded":"2024-09-10T20:41:58.196872+0000","last_fullsized":"2024-09-10T20:41:58.196872+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:50.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22151,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197391+0000","last_change":"2024-09-10T20:39:38.340551+0000","last_active":"2024-09-10T20:41:58.197391+0000","last_peered":"2024-09-10T20:41:58.197391+0000","last_clean":"2024-09-10T20:41:58.197391+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:41:58.197391+0000","last_undegraded":"2024-09-10T20:41:58.197391+0000","last_fullsized":"2024-09-10T20:41:58.197391+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T20:29:43.225780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:43.225780+0000","objects_scrubbed":2000,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:33.469097+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":142,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275332+0000","last_change":"2024-09-10T20:29:20.019355+0000","last_active":"2024-09-10T20:41:58.275332+0000","last_peered":"2024-09-10T20:41:58.275332+0000","last_clean":"2024-09-10T20:41:58.275332+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:41:58.275332+0000","last_undegraded":"2024-09-10T20:41:58.275332+0000","last_fullsized":"2024-09-10T20:41:58.275332+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:29.705821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28137,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197970+0000","last_change":"2024-09-10T20:40:39.949385+0000","last_active":"2024-09-10T20:41:58.197970+0000","last_peered":"2024-09-10T20:41:58.197970+0000","last_clean":"2024-09-10T20:41:58.197970+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:41:58.197970+0000","last_undegraded":"2024-09-10T20:41:58.197970+0000","last_fullsized":"2024-09-10T20:41:58.197970+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:40:39.948835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:40:39.948835+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:40.100997+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":149,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197712+0000","last_change":"2024-09-10T20:30:05.413075+0000","last_active":"2024-09-10T20:41:58.197712+0000","last_peered":"2024-09-10T20:41:58.197712+0000","last_clean":"2024-09-10T20:41:58.197712+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:41:58.197712+0000","last_undegraded":"2024-09-10T20:41:58.197712+0000","last_fullsized":"2024-09-10T20:41:58.197712+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:28.182472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28020,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275395+0000","last_change":"2024-09-10T20:40:17.475435+0000","last_active":"2024-09-10T20:41:58.275395+0000","last_peered":"2024-09-10T20:41:58.275395+0000","last_clean":"2024-09-10T20:41:58.275395+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:41:58.275395+0000","last_undegraded":"2024-09-10T20:41:58.275395+0000","last_fullsized":"2024-09-10T20:41:58.275395+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2197","last_scrub_stamp":"2024-09-10T20:29:44.304225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:44.304225+0000","objects_scrubbed":2167,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:27.560997+0000","scrub_duration":689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":127,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197287+0000","last_change":"2024-09-10T20:30:05.384991+0000","last_active":"2024-09-10T20:41:58.197287+0000","last_peered":"2024-09-10T20:41:58.197287+0000","last_clean":"2024-09-10T20:41:58.197287+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:41:58.197287+0000","last_undegraded":"2024-09-10T20:41:58.197287+0000","last_fullsized":"2024-09-10T20:41:58.197287+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:16.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28073,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275280+0000","last_change":"2024-09-10T20:36:30.363975+0000","last_active":"2024-09-10T20:41:58.275280+0000","last_peered":"2024-09-10T20:41:58.275280+0000","last_clean":"2024-09-10T20:41:58.275280+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:41:58.275280+0000","last_undegraded":"2024-09-10T20:41:58.275280+0000","last_fullsized":"2024-09-10T20:41:58.275280+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2246","last_scrub_stamp":"2024-09-10T20:29:45.018072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:45.018072+0000","objects_scrubbed":2208,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:22.807076+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21130,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197648+0000","last_change":"2024-09-10T20:39:58.338348+0000","last_active":"2024-09-10T20:41:58.197648+0000","last_peered":"2024-09-10T20:41:58.197648+0000","last_clean":"2024-09-10T20:41:58.197648+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:41:58.197648+0000","last_undegraded":"2024-09-10T20:41:58.197648+0000","last_fullsized":"2024-09-10T20:41:58.197648+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3319","last_scrub_stamp":"2024-09-10T20:29:56.793691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:56.793691+0000","objects_scrubbed":3275,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:04.402497+0000","scrub_duration":1221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":117,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197184+0000","last_change":"2024-09-10T20:30:05.463311+0000","last_active":"2024-09-10T20:41:58.197184+0000","last_peered":"2024-09-10T20:41:58.197184+0000","last_clean":"2024-09-10T20:41:58.197184+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:41:58.197184+0000","last_undegraded":"2024-09-10T20:41:58.197184+0000","last_fullsized":"2024-09-10T20:41:58.197184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:26.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27851,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197520+0000","last_change":"2024-09-10T20:38:50.778338+0000","last_active":"2024-09-10T20:41:58.197520+0000","last_peered":"2024-09-10T20:41:58.197520+0000","last_clean":"2024-09-10T20:41:58.197520+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:41:58.197520+0000","last_undegraded":"2024-09-10T20:41:58.197520+0000","last_fullsized":"2024-09-10T20:41:58.197520+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3352","last_scrub_stamp":"2024-09-10T20:30:00.666616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:00.666616+0000","objects_scrubbed":3298,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:32.500551+0000","scrub_duration":1096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":142,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.275528+0000","last_change":"2024-09-10T20:29:20.020842+0000","last_active":"2024-09-10T20:41:58.275528+0000","last_peered":"2024-09-10T20:41:58.275528+0000","last_clean":"2024-09-10T20:41:58.275528+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:41:58.275528+0000","last_undegraded":"2024-09-10T20:41:58.275528+0000","last_fullsized":"2024-09-10T20:41:58.275528+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:44.992975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21265,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197345+0000","last_change":"2024-09-10T20:41:36.928383+0000","last_active":"2024-09-10T20:41:58.197345+0000","last_peered":"2024-09-10T20:41:58.197345+0000","last_clean":"2024-09-10T20:41:58.197345+0000","last_became_active":"2024-09-10T20:41:36.842440+0000","last_became_peered":"2024-09-10T20:41:36.842440+0000","last_unstale":"2024-09-10T20:41:58.197345+0000","last_undegraded":"2024-09-10T20:41:58.197345+0000","last_fullsized":"2024-09-10T20:41:58.197345+0000","mapping_epoch":64,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3233","last_scrub_stamp":"2024-09-10T20:29:54.754780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:29:54.754780+0000","objects_scrubbed":3169,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:09.940798+0000","scrub_duration":1161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3715,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":117,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197437+0000","last_change":"2024-09-10T20:30:05.445309+0000","last_active":"2024-09-10T20:41:58.197437+0000","last_peered":"2024-09-10T20:41:58.197437+0000","last_clean":"2024-09-10T20:41:58.197437+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:41:58.197437+0000","last_undegraded":"2024-09-10T20:41:58.197437+0000","last_fullsized":"2024-09-10T20:41:58.197437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:19.367252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":122,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197523+0000","last_change":"2024-09-10T20:30:05.314481+0000","last_active":"2024-09-10T20:41:58.197523+0000","last_peered":"2024-09-10T20:41:58.197523+0000","last_clean":"2024-09-10T20:41:58.197523+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:41:58.197523+0000","last_undegraded":"2024-09-10T20:41:58.197523+0000","last_fullsized":"2024-09-10T20:41:58.197523+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:24.477689+0000","last_clean_scrub_stamp":"2024-09-10T20:28:24.477689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:51.783820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20516","reported_seq":27681,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:41:58.197107+0000","last_change":"2024-09-10T20:34:27.111873+0000","last_active":"2024-09-10T20:41:58.197107+0000","last_peered":"2024-09-10T20:41:58.197107+0000","last_clean":"2024-09-10T20:41:58.197107+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:41:58.197107+0000","last_undegraded":"2024-09-10T20:41:58.197107+0000","last_fullsized":"2024-09-10T20:41:58.197107+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3446","last_scrub_stamp":"2024-09-10T20:30:02.819432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:28:37.510222+0000","last_clean_scrub_stamp":"2024-09-10T20:30:02.819432+0000","objects_scrubbed":3382,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:17.818624+0000","scrub_duration":1287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39530,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":48754688,"data_stored":48747462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182562,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443132,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93928708,"statfs":{"total":96636764160,"available":96182996992,"internally_reserved":0,"allocated":565248,"data_stored":214155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986334,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":413748,"kb_used_data":48868,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93958092,"statfs":{"total":96636764160,"available":96213086208,"internally_reserved":0,"allocated":50040832,"data_stored":49697429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659281,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497340,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874500,"statfs":{"total":96636764160,"available":96127488000,"internally_reserved":0,"allocated":565248,"data_stored":214155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48746496,"data_stored":48746496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:42:10.925 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-09-10T20:28:37.510222+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=28, tm_sec=37, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=42, tm_sec=8, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T20:42:10.925 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-09-10T20:29:40.357630+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=29, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=42, tm_sec=8, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T20:42:10.925 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-09-10T20:29:41.318256+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=29, tm_sec=41, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=42, tm_sec=8, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T20:42:10.925 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2024-09-10T20:28:37.510222+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=28, tm_sec=37, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=42, tm_sec=8, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T20:42:10.925 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-09-10T20:29:53.768396+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=29, tm_sec=53, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=42, tm_sec=8, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T20:42:10.925 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-09-10T20:29:55.781388+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=29, tm_sec=55, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=42, tm_sec=8, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T20:42:10.926 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-09-10T20:30:01.859333+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=30, tm_sec=1, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=42, tm_sec=8, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T20:42:10.926 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-09-10T20:29:52.392191+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=29, tm_sec=52, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=42, tm_sec=8, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T20:42:10.926 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-10T20:28:24.477689+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=28, tm_sec=24, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=42, tm_sec=8, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T20:42:10.926 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-10T20:29:43.225780+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=29, tm_sec=43, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=42, tm_sec=8, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T20:42:10.926 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2024-09-10T20:28:24.477689+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=28, tm_sec=24, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=42, tm_sec=8, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T20:42:10.926 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-10T20:40:39.948835+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=40, tm_sec=39, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=42, tm_sec=8, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T20:42:10.926 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2024-09-10T20:28:24.477689+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=28, tm_sec=24, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=42, tm_sec=8, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T20:42:10.926 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-10T20:29:44.304225+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=29, tm_sec=44, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=42, tm_sec=8, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T20:42:10.927 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2024-09-10T20:28:24.477689+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=28, tm_sec=24, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=42, tm_sec=8, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T20:42:10.927 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-10T20:29:45.018072+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=29, tm_sec=45, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=42, tm_sec=8, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T20:42:10.927 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-10T20:29:56.793691+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=29, tm_sec=56, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=42, tm_sec=8, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T20:42:10.927 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2024-09-10T20:28:24.477689+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=28, tm_sec=24, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=42, tm_sec=8, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T20:42:10.927 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-10T20:30:00.666616+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=30, tm_sec=0, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=42, tm_sec=8, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T20:42:10.927 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2024-09-10T20:28:24.477689+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=28, tm_sec=24, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=42, tm_sec=8, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T20:42:10.927 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-10T20:29:54.754780+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=29, tm_sec=54, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=42, tm_sec=8, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T20:42:10.927 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2024-09-10T20:28:24.477689+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=28, tm_sec=24, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=42, tm_sec=8, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T20:42:10.927 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2024-09-10T20:28:24.477689+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=28, tm_sec=24, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=42, tm_sec=8, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T20:42:10.927 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-10T20:30:02.819432+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=30, tm_sec=2, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=42, tm_sec=8, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T20:42:10.928 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-10T20:42:30.928 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:42:31.229 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T20:42:31.229 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T20:42:31.242 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":492,"stamp":"2024-09-10T20:42:31.089532+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274688,"num_write_kb":478115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39532,"num_bytes_recovered":37052435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160724,"ondisk_log_size":160724,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1335484,"kb_used_data":31244,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281780036,"statfs":{"total":289910292480,"available":288542756864,"internally_reserved":0,"allocated":31993856,"data_stored":30948267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1335479298},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002603"},"pg_stats":[{"pgid":"2.f","version":"51'20658","reported_seq":28017,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:42:14.112873+0000","last_change":"2024-09-10T20:42:14.112804+0000","last_active":"2024-09-10T20:42:14.112873+0000","last_peered":"2024-09-10T20:42:14.112873+0000","last_clean":"2024-09-10T20:42:14.112873+0000","last_became_active":"2024-09-10T20:40:58.565539+0000","last_became_peered":"2024-09-10T20:40:58.565539+0000","last_unstale":"2024-09-10T20:42:14.112873+0000","last_undegraded":"2024-09-10T20:42:14.112873+0000","last_fullsized":"2024-09-10T20:42:14.112873+0000","mapping_epoch":60,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20658","last_scrub_stamp":"2024-09-10T20:42:14.112711+0000","last_deep_scrub":"51'20658","last_deep_scrub_stamp":"2024-09-10T20:42:14.112711+0000","last_clean_scrub_stamp":"2024-09-10T20:42:14.112711+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:36:57.018255+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20658,"num_write_kb":41316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3608,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'20942","reported_seq":21108,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:42:16.165537+0000","last_change":"2024-09-10T20:42:16.165537+0000","last_active":"2024-09-10T20:42:16.165537+0000","last_peered":"2024-09-10T20:42:16.165537+0000","last_clean":"2024-09-10T20:42:16.165537+0000","last_became_active":"2024-09-10T20:29:20.020969+0000","last_became_peered":"2024-09-10T20:29:20.020969+0000","last_unstale":"2024-09-10T20:42:16.165537+0000","last_undegraded":"2024-09-10T20:42:16.165537+0000","last_fullsized":"2024-09-10T20:42:16.165537+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20942","last_scrub_stamp":"2024-09-10T20:42:16.165203+0000","last_deep_scrub":"51'20942","last_deep_scrub_stamp":"2024-09-10T20:42:16.165203+0000","last_clean_scrub_stamp":"2024-09-10T20:42:16.165203+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:26:01.355664+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19128,"num_write_kb":34640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'20716","reported_seq":28120,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:42:21.518604+0000","last_change":"2024-09-10T20:42:21.518604+0000","last_active":"2024-09-10T20:42:21.518604+0000","last_peered":"2024-09-10T20:42:21.518604+0000","last_clean":"2024-09-10T20:42:21.518604+0000","last_became_active":"2024-09-10T20:32:18.193999+0000","last_became_peered":"2024-09-10T20:32:18.193999+0000","last_unstale":"2024-09-10T20:42:21.518604+0000","last_undegraded":"2024-09-10T20:42:21.518604+0000","last_fullsized":"2024-09-10T20:42:21.518604+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20716","last_scrub_stamp":"2024-09-10T20:42:21.518505+0000","last_deep_scrub":"51'20716","last_deep_scrub_stamp":"2024-09-10T20:42:21.518505+0000","last_clean_scrub_stamp":"2024-09-10T20:42:21.518505+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:35:45.038125+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18844,"num_write_kb":33949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3604,"num_bytes_recovered":14757888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20604","reported_seq":20740,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:42:15.174178+0000","last_change":"2024-09-10T20:42:15.174178+0000","last_active":"2024-09-10T20:42:15.174178+0000","last_peered":"2024-09-10T20:42:15.174178+0000","last_clean":"2024-09-10T20:42:15.174178+0000","last_became_active":"2024-09-10T20:29:20.018636+0000","last_became_peered":"2024-09-10T20:29:20.018636+0000","last_unstale":"2024-09-10T20:42:15.174178+0000","last_undegraded":"2024-09-10T20:42:15.174178+0000","last_fullsized":"2024-09-10T20:42:15.174178+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20604","last_scrub_stamp":"2024-09-10T20:42:15.174078+0000","last_deep_scrub":"51'20604","last_deep_scrub_stamp":"2024-09-10T20:42:15.174078+0000","last_clean_scrub_stamp":"2024-09-10T20:42:15.174078+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:35.034559+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20604,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'21069","reported_seq":21252,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:42:13.120990+0000","last_change":"2024-09-10T20:42:13.120890+0000","last_active":"2024-09-10T20:42:13.120990+0000","last_peered":"2024-09-10T20:42:13.120990+0000","last_clean":"2024-09-10T20:42:13.120990+0000","last_became_active":"2024-09-10T20:29:19.021893+0000","last_became_peered":"2024-09-10T20:29:19.021893+0000","last_unstale":"2024-09-10T20:42:13.120990+0000","last_undegraded":"2024-09-10T20:42:13.120990+0000","last_fullsized":"2024-09-10T20:42:13.120990+0000","mapping_epoch":34,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'21069","last_scrub_stamp":"2024-09-10T20:42:13.120764+0000","last_deep_scrub":"51'21069","last_deep_scrub_stamp":"2024-09-10T20:42:13.120764+0000","last_clean_scrub_stamp":"2024-09-10T20:42:13.120764+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:38:43.614147+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18065,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20471","reported_seq":20749,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:42:22.532318+0000","last_change":"2024-09-10T20:42:22.532318+0000","last_active":"2024-09-10T20:42:22.532318+0000","last_peered":"2024-09-10T20:42:22.532318+0000","last_clean":"2024-09-10T20:42:22.532318+0000","last_became_active":"2024-09-10T20:41:58.198887+0000","last_became_peered":"2024-09-10T20:41:58.198887+0000","last_unstale":"2024-09-10T20:42:22.532318+0000","last_undegraded":"2024-09-10T20:42:22.532318+0000","last_fullsized":"2024-09-10T20:42:22.532318+0000","mapping_epoch":66,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20471","last_scrub_stamp":"2024-09-10T20:42:22.532190+0000","last_deep_scrub":"51'20471","last_deep_scrub_stamp":"2024-09-10T20:42:22.532190+0000","last_clean_scrub_stamp":"2024-09-10T20:42:22.532190+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:39.164277+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17335,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3616,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"51'20608","reported_seq":20927,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:42:19.565517+0000","last_change":"2024-09-10T20:42:19.565517+0000","last_active":"2024-09-10T20:42:19.565517+0000","last_peered":"2024-09-10T20:42:19.565517+0000","last_clean":"2024-09-10T20:42:19.565517+0000","last_became_active":"2024-09-10T20:41:17.668013+0000","last_became_peered":"2024-09-10T20:41:17.668013+0000","last_unstale":"2024-09-10T20:42:19.565517+0000","last_undegraded":"2024-09-10T20:42:19.565517+0000","last_fullsized":"2024-09-10T20:42:19.565517+0000","mapping_epoch":62,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20608","last_scrub_stamp":"2024-09-10T20:42:19.565424+0000","last_deep_scrub":"51'20608","last_deep_scrub_stamp":"2024-09-10T20:42:19.565424+0000","last_clean_scrub_stamp":"2024-09-10T20:42:19.565424+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:17:58.573489+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20736","reported_seq":20917,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:42:12.159575+0000","last_change":"2024-09-10T20:42:12.159575+0000","last_active":"2024-09-10T20:42:12.159575+0000","last_peered":"2024-09-10T20:42:12.159575+0000","last_clean":"2024-09-10T20:42:12.159575+0000","last_became_active":"2024-09-10T20:29:19.021239+0000","last_became_peered":"2024-09-10T20:29:19.021239+0000","last_unstale":"2024-09-10T20:42:12.159575+0000","last_undegraded":"2024-09-10T20:42:12.159575+0000","last_fullsized":"2024-09-10T20:42:12.159575+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20736","last_scrub_stamp":"2024-09-10T20:42:12.159425+0000","last_deep_scrub":"51'20736","last_deep_scrub_stamp":"2024-09-10T20:42:12.159425+0000","last_clean_scrub_stamp":"2024-09-10T20:42:12.159425+0000","objects_scrubbed":0,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:12:00.852148+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17807,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":126,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:42:10.581618+0000","last_change":"2024-09-10T20:42:10.581618+0000","last_active":"2024-09-10T20:42:10.581618+0000","last_peered":"2024-09-10T20:42:10.581618+0000","last_clean":"2024-09-10T20:42:10.581618+0000","last_became_active":"2024-09-10T20:30:05.315842+0000","last_became_peered":"2024-09-10T20:30:05.315842+0000","last_unstale":"2024-09-10T20:42:10.581618+0000","last_undegraded":"2024-09-10T20:42:10.581618+0000","last_fullsized":"2024-09-10T20:42:10.581618+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:42:10.581560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:42:10.581560+0000","last_clean_scrub_stamp":"2024-09-10T20:42:10.581560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:35:12.343246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047971299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20439","reported_seq":22159,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:42:15.666130+0000","last_change":"2024-09-10T20:42:15.666130+0000","last_active":"2024-09-10T20:42:15.666130+0000","last_peered":"2024-09-10T20:42:15.666130+0000","last_clean":"2024-09-10T20:42:15.666130+0000","last_became_active":"2024-09-10T20:39:38.241889+0000","last_became_peered":"2024-09-10T20:39:38.241889+0000","last_unstale":"2024-09-10T20:42:15.666130+0000","last_undegraded":"2024-09-10T20:42:15.666130+0000","last_fullsized":"2024-09-10T20:42:15.666130+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20439","last_scrub_stamp":"2024-09-10T20:42:15.665997+0000","last_deep_scrub":"51'20439","last_deep_scrub_stamp":"2024-09-10T20:42:15.665997+0000","last_clean_scrub_stamp":"2024-09-10T20:42:15.665997+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:42.515074+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":150,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:42:11.262308+0000","last_change":"2024-09-10T20:42:11.262308+0000","last_active":"2024-09-10T20:42:11.262308+0000","last_peered":"2024-09-10T20:42:11.262308+0000","last_clean":"2024-09-10T20:42:11.262308+0000","last_became_active":"2024-09-10T20:29:20.019106+0000","last_became_peered":"2024-09-10T20:29:20.019106+0000","last_unstale":"2024-09-10T20:42:11.262308+0000","last_undegraded":"2024-09-10T20:42:11.262308+0000","last_fullsized":"2024-09-10T20:42:11.262308+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:42:11.262241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:42:11.262241+0000","last_clean_scrub_stamp":"2024-09-10T20:42:11.262241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:16.126213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043557699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20730","reported_seq":28145,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:42:11.180109+0000","last_change":"2024-09-10T20:42:11.180109+0000","last_active":"2024-09-10T20:42:11.180109+0000","last_peered":"2024-09-10T20:42:11.180109+0000","last_clean":"2024-09-10T20:42:11.180109+0000","last_became_active":"2024-09-10T20:40:39.465239+0000","last_became_peered":"2024-09-10T20:40:39.465239+0000","last_unstale":"2024-09-10T20:42:11.180109+0000","last_undegraded":"2024-09-10T20:42:11.180109+0000","last_fullsized":"2024-09-10T20:42:11.180109+0000","mapping_epoch":58,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20730","last_scrub_stamp":"2024-09-10T20:42:11.180009+0000","last_deep_scrub":"51'20730","last_deep_scrub_stamp":"2024-09-10T20:42:11.180009+0000","last_clean_scrub_stamp":"2024-09-10T20:42:11.180009+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:38.667028+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"67'3","reported_seq":158,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:42:10.183252+0000","last_change":"2024-09-10T20:42:10.183252+0000","last_active":"2024-09-10T20:42:10.183252+0000","last_peered":"2024-09-10T20:42:10.183252+0000","last_clean":"2024-09-10T20:42:10.183252+0000","last_became_active":"2024-09-10T20:30:05.412495+0000","last_became_peered":"2024-09-10T20:30:05.412495+0000","last_unstale":"2024-09-10T20:42:10.183252+0000","last_undegraded":"2024-09-10T20:42:10.183252+0000","last_fullsized":"2024-09-10T20:42:10.183252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'3","last_scrub_stamp":"2024-09-10T20:42:10.183195+0000","last_deep_scrub":"67'3","last_deep_scrub_stamp":"2024-09-10T20:42:10.183195+0000","last_clean_scrub_stamp":"2024-09-10T20:42:10.183195+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:12:51.300819+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0011665810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'20565","reported_seq":28028,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:42:13.214457+0000","last_change":"2024-09-10T20:42:13.214457+0000","last_active":"2024-09-10T20:42:13.214457+0000","last_peered":"2024-09-10T20:42:13.214457+0000","last_clean":"2024-09-10T20:42:13.214457+0000","last_became_active":"2024-09-10T20:40:17.419236+0000","last_became_peered":"2024-09-10T20:40:17.419236+0000","last_unstale":"2024-09-10T20:42:13.214457+0000","last_undegraded":"2024-09-10T20:42:13.214457+0000","last_fullsized":"2024-09-10T20:42:13.214457+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20565","last_scrub_stamp":"2024-09-10T20:42:13.214366+0000","last_deep_scrub":"51'20565","last_deep_scrub_stamp":"2024-09-10T20:42:13.214366+0000","last_clean_scrub_stamp":"2024-09-10T20:42:13.214366+0000","objects_scrubbed":0,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:14:20.376404+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":32452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"67'2","reported_seq":136,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:42:11.603507+0000","last_change":"2024-09-10T20:42:11.603507+0000","last_active":"2024-09-10T20:42:11.603507+0000","last_peered":"2024-09-10T20:42:11.603507+0000","last_clean":"2024-09-10T20:42:11.603507+0000","last_became_active":"2024-09-10T20:30:05.380764+0000","last_became_peered":"2024-09-10T20:30:05.380764+0000","last_unstale":"2024-09-10T20:42:11.603507+0000","last_undegraded":"2024-09-10T20:42:11.603507+0000","last_fullsized":"2024-09-10T20:42:11.603507+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'2","last_scrub_stamp":"2024-09-10T20:42:11.603442+0000","last_deep_scrub":"67'2","last_deep_scrub_stamp":"2024-09-10T20:42:11.603442+0000","last_clean_scrub_stamp":"2024-09-10T20:42:11.603442+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:45:37.077284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050075499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'20664","reported_seq":28081,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:42:14.171098+0000","last_change":"2024-09-10T20:42:14.171098+0000","last_active":"2024-09-10T20:42:14.171098+0000","last_peered":"2024-09-10T20:42:14.171098+0000","last_clean":"2024-09-10T20:42:14.171098+0000","last_became_active":"2024-09-10T20:36:30.320656+0000","last_became_peered":"2024-09-10T20:36:30.320656+0000","last_unstale":"2024-09-10T20:42:14.171098+0000","last_undegraded":"2024-09-10T20:42:14.171098+0000","last_fullsized":"2024-09-10T20:42:14.171098+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20664","last_scrub_stamp":"2024-09-10T20:42:14.170971+0000","last_deep_scrub":"51'20664","last_deep_scrub_stamp":"2024-09-10T20:42:14.170971+0000","last_clean_scrub_stamp":"2024-09-10T20:42:14.170971+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:53:32.128043+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":8085504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'20830","reported_seq":21138,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:42:16.637784+0000","last_change":"2024-09-10T20:42:16.637784+0000","last_active":"2024-09-10T20:42:16.637784+0000","last_peered":"2024-09-10T20:42:16.637784+0000","last_clean":"2024-09-10T20:42:16.637784+0000","last_became_active":"2024-09-10T20:39:58.266400+0000","last_became_peered":"2024-09-10T20:39:58.266400+0000","last_unstale":"2024-09-10T20:42:16.637784+0000","last_undegraded":"2024-09-10T20:42:16.637784+0000","last_fullsized":"2024-09-10T20:42:16.637784+0000","mapping_epoch":54,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20830","last_scrub_stamp":"2024-09-10T20:42:16.637670+0000","last_deep_scrub":"51'20830","last_deep_scrub_stamp":"2024-09-10T20:42:16.637670+0000","last_clean_scrub_stamp":"2024-09-10T20:42:16.637670+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:45:05.337400+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17553,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3649,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":125,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:42:12.610684+0000","last_change":"2024-09-10T20:42:12.610684+0000","last_active":"2024-09-10T20:42:12.610684+0000","last_peered":"2024-09-10T20:42:12.610684+0000","last_clean":"2024-09-10T20:42:12.610684+0000","last_became_active":"2024-09-10T20:30:05.449574+0000","last_became_peered":"2024-09-10T20:30:05.449574+0000","last_unstale":"2024-09-10T20:42:12.610684+0000","last_undegraded":"2024-09-10T20:42:12.610684+0000","last_fullsized":"2024-09-10T20:42:12.610684+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:42:12.610591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:42:12.610591+0000","last_clean_scrub_stamp":"2024-09-10T20:42:12.610591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:20:00.663465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071858000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20584","reported_seq":27859,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:42:20.560673+0000","last_change":"2024-09-10T20:42:20.560673+0000","last_active":"2024-09-10T20:42:20.560673+0000","last_peered":"2024-09-10T20:42:20.560673+0000","last_clean":"2024-09-10T20:42:20.560673+0000","last_became_active":"2024-09-10T20:38:50.656321+0000","last_became_peered":"2024-09-10T20:38:50.656321+0000","last_unstale":"2024-09-10T20:42:20.560673+0000","last_undegraded":"2024-09-10T20:42:20.560673+0000","last_fullsized":"2024-09-10T20:42:20.560673+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20584","last_scrub_stamp":"2024-09-10T20:42:20.560571+0000","last_deep_scrub":"51'20584","last_deep_scrub_stamp":"2024-09-10T20:42:20.560571+0000","last_clean_scrub_stamp":"2024-09-10T20:42:20.560571+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:57:39.380969+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17278,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":150,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:42:12.251428+0000","last_change":"2024-09-10T20:42:12.251428+0000","last_active":"2024-09-10T20:42:12.251428+0000","last_peered":"2024-09-10T20:42:12.251428+0000","last_clean":"2024-09-10T20:42:12.251428+0000","last_became_active":"2024-09-10T20:29:20.020590+0000","last_became_peered":"2024-09-10T20:29:20.020590+0000","last_unstale":"2024-09-10T20:42:12.251428+0000","last_undegraded":"2024-09-10T20:42:12.251428+0000","last_fullsized":"2024-09-10T20:42:12.251428+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:42:12.251343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:42:12.251343+0000","last_clean_scrub_stamp":"2024-09-10T20:42:12.251343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:13:07.332528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410455,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20986","reported_seq":21273,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:42:18.611656+0000","last_change":"2024-09-10T20:42:18.611656+0000","last_active":"2024-09-10T20:42:18.611656+0000","last_peered":"2024-09-10T20:42:18.611656+0000","last_clean":"2024-09-10T20:42:18.611656+0000","last_became_active":"2024-09-10T20:41:36.842440+0000","last_became_peered":"2024-09-10T20:41:36.842440+0000","last_unstale":"2024-09-10T20:42:18.611656+0000","last_undegraded":"2024-09-10T20:42:18.611656+0000","last_fullsized":"2024-09-10T20:42:18.611656+0000","mapping_epoch":64,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20986","last_scrub_stamp":"2024-09-10T20:42:18.611527+0000","last_deep_scrub":"51'20986","last_deep_scrub_stamp":"2024-09-10T20:42:18.611527+0000","last_clean_scrub_stamp":"2024-09-10T20:42:18.611527+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:43:16.357747+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":29288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3715,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":125,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:42:17.620905+0000","last_change":"2024-09-10T20:42:17.620905+0000","last_active":"2024-09-10T20:42:17.620905+0000","last_peered":"2024-09-10T20:42:17.620905+0000","last_clean":"2024-09-10T20:42:17.620905+0000","last_became_active":"2024-09-10T20:30:05.444943+0000","last_became_peered":"2024-09-10T20:30:05.444943+0000","last_unstale":"2024-09-10T20:42:17.620905+0000","last_undegraded":"2024-09-10T20:42:17.620905+0000","last_fullsized":"2024-09-10T20:42:17.620905+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:42:17.620821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:42:17.620821+0000","last_clean_scrub_stamp":"2024-09-10T20:42:17.620821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:57.264413+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00066886300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":130,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:42:13.653370+0000","last_change":"2024-09-10T20:42:13.653370+0000","last_active":"2024-09-10T20:42:13.653370+0000","last_peered":"2024-09-10T20:42:13.653370+0000","last_clean":"2024-09-10T20:42:13.653370+0000","last_became_active":"2024-09-10T20:30:05.314040+0000","last_became_peered":"2024-09-10T20:30:05.314040+0000","last_unstale":"2024-09-10T20:42:13.653370+0000","last_undegraded":"2024-09-10T20:42:13.653370+0000","last_fullsized":"2024-09-10T20:42:13.653370+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:42:13.653284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:42:13.653284+0000","last_clean_scrub_stamp":"2024-09-10T20:42:13.653284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:09:19.304434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081087800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"67'20517","reported_seq":27690,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T20:42:14.661679+0000","last_change":"2024-09-10T20:42:14.661679+0000","last_active":"2024-09-10T20:42:14.661679+0000","last_peered":"2024-09-10T20:42:14.661679+0000","last_clean":"2024-09-10T20:42:14.661679+0000","last_became_active":"2024-09-10T20:34:27.082993+0000","last_became_peered":"2024-09-10T20:34:27.082993+0000","last_unstale":"2024-09-10T20:42:14.661679+0000","last_undegraded":"2024-09-10T20:42:14.661679+0000","last_fullsized":"2024-09-10T20:42:14.661679+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'20517","last_scrub_stamp":"2024-09-10T20:42:14.661567+0000","last_deep_scrub":"67'20517","last_deep_scrub_stamp":"2024-09-10T20:42:14.661567+0000","last_clean_scrub_stamp":"2024-09-10T20:42:14.661567+0000","objects_scrubbed":1,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:00.369274+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3471,"num_bytes_recovered":14204928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":478113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39530,"num_bytes_recovered":37052416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":30068736,"data_stored":30061510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160719,"ondisk_log_size":160719,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182567,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443128,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93928712,"statfs":{"total":96636764160,"available":96183001088,"internally_reserved":0,"allocated":565248,"data_stored":214155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986339,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395020,"kb_used_data":30140,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93976820,"statfs":{"total":96636764160,"available":96232263680,"internally_reserved":0,"allocated":30863360,"data_stored":30519957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659285,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497336,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874504,"statfs":{"total":96636764160,"available":96127492096,"internally_reserved":0,"allocated":565248,"data_stored":214155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":30060544,"data_stored":30060544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:42:31.244 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-10T20:42:31.546 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-10T20:42:31.546 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-10T20:42:31.547 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-10T20:42:31.547 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T20:42:36.894 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-10T20:42:37.648 INFO:tasks.ceph.osd.0:Stopped 2024-09-10T20:42:37.648 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-10T20:42:37.648 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T20:42:37.723 INFO:tasks.ceph.osd.1:Stopped 2024-09-10T20:42:37.723 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-10T20:42:37.724 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T20:42:42.297 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-10T20:42:42.297 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-09-10T20:42:43.825 INFO:tasks.ceph.osd.2:Stopped 2024-09-10T20:42:43.826 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-10T20:42:43.826 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-10T20:42:43.826 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T20:42:43.857 INFO:tasks.ceph.mgr.x:Stopped 2024-09-10T20:42:43.857 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-10T20:42:43.857 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-10T20:42:43.857 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T20:42:43.912 INFO:tasks.ceph.mon.a:Stopped 2024-09-10T20:42:43.912 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-10T20:42:43.912 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T20:42:43.966 INFO:tasks.ceph.mon.b:Stopped 2024-09-10T20:42:43.966 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-10T20:42:43.966 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T20:42:44.020 INFO:tasks.ceph.mon.c:Stopped 2024-09-10T20:42:44.021 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-10T20:42:44.021 DEBUG:teuthology.orchestra.run.smithi131:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v '\(MON_DOWN\)' | head -n 1 2024-09-10T20:42:44.118 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi131.front.sepia.ceph.com 2024-09-10T20:42:44.119 DEBUG:teuthology.orchestra.run.smithi131:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-10T20:42:44.834 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi131.front.sepia.ceph.com 2024-09-10T20:42:44.834 DEBUG:teuthology.orchestra.run.smithi131:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-10T20:42:44.966 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi131.front.sepia.ceph.com 2024-09-10T20:42:44.966 DEBUG:teuthology.orchestra.run.smithi131:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-10T20:42:45.043 INFO:tasks.ceph:Archiving mon data... 2024-09-10T20:42:45.044 DEBUG:teuthology.misc:Transferring archived files from smithi131:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/skanta-2024-09-10_15:30:27-rados-wip-bharath7-testing-2024-09-10-1409-distro-default-smithi/7899571/data/mon.a.tgz 2024-09-10T20:42:45.044 DEBUG:teuthology.orchestra.run.smithi131:> mktemp 2024-09-10T20:42:45.060 INFO:teuthology.orchestra.run.smithi131.stdout:/tmp/tmp.qpcNDjXd8f 2024-09-10T20:42:45.060 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.qpcNDjXd8f 2024-09-10T20:42:45.195 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 0666 /tmp/tmp.qpcNDjXd8f 2024-09-10T20:42:45.302 DEBUG:teuthology.orchestra.remote:smithi131:/tmp/tmp.qpcNDjXd8f is 329KB 2024-09-10T20:42:45.398 DEBUG:teuthology.orchestra.run.smithi131:> rm -fr /tmp/tmp.qpcNDjXd8f 2024-09-10T20:42:45.413 DEBUG:teuthology.misc:Transferring archived files from smithi131:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/skanta-2024-09-10_15:30:27-rados-wip-bharath7-testing-2024-09-10-1409-distro-default-smithi/7899571/data/mon.b.tgz 2024-09-10T20:42:45.413 DEBUG:teuthology.orchestra.run.smithi131:> mktemp 2024-09-10T20:42:45.467 INFO:teuthology.orchestra.run.smithi131.stdout:/tmp/tmp.mDHFnRGwg2 2024-09-10T20:42:45.468 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.mDHFnRGwg2 2024-09-10T20:42:45.592 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 0666 /tmp/tmp.mDHFnRGwg2 2024-09-10T20:42:45.675 DEBUG:teuthology.orchestra.remote:smithi131:/tmp/tmp.mDHFnRGwg2 is 341KB 2024-09-10T20:42:45.747 DEBUG:teuthology.orchestra.run.smithi131:> rm -fr /tmp/tmp.mDHFnRGwg2 2024-09-10T20:42:45.762 DEBUG:teuthology.misc:Transferring archived files from smithi131:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/skanta-2024-09-10_15:30:27-rados-wip-bharath7-testing-2024-09-10-1409-distro-default-smithi/7899571/data/mon.c.tgz 2024-09-10T20:42:45.762 DEBUG:teuthology.orchestra.run.smithi131:> mktemp 2024-09-10T20:42:45.816 INFO:teuthology.orchestra.run.smithi131.stdout:/tmp/tmp.DU15zgWKqO 2024-09-10T20:42:45.817 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.DU15zgWKqO 2024-09-10T20:42:45.940 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 0666 /tmp/tmp.DU15zgWKqO 2024-09-10T20:42:46.024 DEBUG:teuthology.orchestra.remote:smithi131:/tmp/tmp.DU15zgWKqO is 341KB 2024-09-10T20:42:46.118 DEBUG:teuthology.orchestra.run.smithi131:> rm -fr /tmp/tmp.DU15zgWKqO 2024-09-10T20:42:46.133 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-10T20:42:46.133 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-10T20:42:46.261 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-10T20:42:46.261 INFO:tasks.ceph:Archiving crash dumps... 2024-09-10T20:42:46.263 DEBUG:teuthology.misc:Transferring archived files from smithi131:/var/lib/ceph/crash to /home/teuthworker/archive/skanta-2024-09-10_15:30:27-rados-wip-bharath7-testing-2024-09-10-1409-distro-default-smithi/7899571/remote/smithi131/crash 2024-09-10T20:42:46.263 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-10T20:42:46.325 INFO:tasks.ceph:Compressing logs... 2024-09-10T20:42:46.325 DEBUG:teuthology.orchestra.run.smithi131:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T20:42:46.413 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64638.log 2024-09-10T20:42:46.413 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-10T20:42:46.413 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49331.log 2024-09-10T20:42:46.413 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64638.log.gz 2024-09-10T20:42:46.413 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57227.log 2024-09-10T20:42:46.413 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35683.log 2024-09-10T20:42:46.414 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49331.log.gz 2024-09-10T20:42:46.414 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70059.log 2024-09-10T20:42:46.414 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57227.log.gz 2024-09-10T20:42:46.414 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45426.log 2024-09-10T20:42:46.414 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35683.log.gz 2024-09-10T20:42:46.414 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.70059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67833.log 2024-09-10T20:42:46.414 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70059.log.gz 2024-09-10T20:42:46.414 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53993.log 2024-09-10T20:42:46.414 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45426.log.gz 2024-09-10T20:42:46.414 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38050.log 2024-09-10T20:42:46.414 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67833.log.gz 2024-09-10T20:42:46.414 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50003.log 2024-09-10T20:42:46.414 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53993.log.gz 2024-09-10T20:42:46.414 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47735.log 2024-09-10T20:42:46.415 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38050.log.gz 2024-09-10T20:42:46.415 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35389.log 2024-09-10T20:42:46.415 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50003.log.gz 2024-09-10T20:42:46.415 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68967.log 2024-09-10T20:42:46.415 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47735.log.gz 2024-09-10T20:42:46.415 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38346.log 2024-09-10T20:42:46.415 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35389.log.gz 2024-09-10T20:42:46.415 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43297.log 2024-09-10T20:42:46.415 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68967.log.gz 2024-09-10T20:42:46.415 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37421.log 2024-09-10T20:42:46.415 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38346.log.gz 2024-09-10T20:42:46.415 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60226.log 2024-09-10T20:42:46.415 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43297.log.gz 2024-09-10T20:42:46.415 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55925.log 2024-09-10T20:42:46.416 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37421.log.gz 2024-09-10T20:42:46.416 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59554.log 2024-09-10T20:42:46.416 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60226.log.gz 2024-09-10T20:42:46.416 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65900.log 2024-09-10T20:42:46.416 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63966.log 2024-09-10T20:42:46.416 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48155.log 2024-09-10T20:42:46.416 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70269.log 2024-09-10T20:42:46.416 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57859.log 2024-09-10T20:42:46.416 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59764.log 2024-09-10T20:42:46.416 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63797.log 2024-09-10T20:42:46.416 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70101.log 2024-09-10T20:42:46.416 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46475.log 2024-09-10T20:42:46.416 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49373.log 2024-09-10T20:42:46.416 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44871.log 2024-09-10T20:42:46.417 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53699.log 2024-09-10T20:42:46.417 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39244.log 2024-09-10T20:42:46.417 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70395.log 2024-09-10T20:42:46.417 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35955.log 2024-09-10T20:42:46.417 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45389.log 2024-09-10T20:42:46.417 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54665.log 2024-09-10T20:42:46.417 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49499.log 2024-09-10T20:42:46.417 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51767.log 2024-09-10T20:42:46.417 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52691.log 2024-09-10T20:42:46.417 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38578.log 2024-09-10T20:42:46.417 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63587.log 2024-09-10T20:42:46.417 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31984.log 2024-09-10T20:42:46.417 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61024.log 2024-09-10T20:42:46.418 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38467.log 2024-09-10T20:42:46.418 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36187.log 2024-09-10T20:42:46.418 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36820.log 2024-09-10T20:42:46.418 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35597.log 2024-09-10T20:42:46.418 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67749.log 2024-09-10T20:42:46.418 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38652.log 2024-09-10T20:42:46.418 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37902.log 2024-09-10T20:42:46.418 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70017.log 2024-09-10T20:42:46.418 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61234.log 2024-09-10T20:42:46.682 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57607.log 2024-09-10T20:42:46.682 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53237.log 2024-09-10T20:42:46.682 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51053.log 2024-09-10T20:42:46.682 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67917.log 2024-09-10T20:42:46.682 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57985.log 2024-09-10T20:42:46.682 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55757.log 2024-09-10T20:42:46.682 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58291.log 2024-09-10T20:42:46.683 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66825.log 2024-09-10T20:42:46.683 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56933.log 2024-09-10T20:42:46.683 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54749.log 2024-09-10T20:42:46.683 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51137.log 2024-09-10T20:42:46.683 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58207.log 2024-09-10T20:42:46.683 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67371.log 2024-09-10T20:42:46.683 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55925.log: /var/log/ceph/ceph-client.admin.59554.log: /var/log/ceph/ceph-client.admin.65900.log: /var/log/ceph/ceph-client.admin.63966.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55925.log.gz 2024-09-10T20:42:46.683 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48155.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59554.log.gz 2024-09-10T20:42:46.683 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65900.log.gz 2024-09-10T20:42:46.683 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.58207.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63966.log.gz 2024-09-10T20:42:46.683 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52439.log 2024-09-10T20:42:46.683 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.70269.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58207.log.gz 2024-09-10T20:42:46.683 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57859.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48155.log.gz 2024-09-10T20:42:46.683 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59764.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70269.log.gz 2024-09-10T20:42:46.683 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% 0.0%/var/log/ceph/ceph-client.admin.63797.log: -- replaced with /var/log/ceph/ceph-client.admin.59764.log.gz 2024-09-10T20:42:46.684 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.70101.log: -- replaced with /var/log/ceph/ceph-client.admin.57859.log.gz 2024-09-10T20:42:46.684 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46475.log: /var/log/ceph/ceph-client.admin.67371.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63797.log.gz 2024-09-10T20:42:46.684 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47105.log 2024-09-10T20:42:46.684 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49373.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46475.log.gz 2024-09-10T20:42:46.684 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67371.log.gz 2024-09-10T20:42:46.684 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44871.log: -- replaced with /var/log/ceph/ceph-client.admin.70101.log.gz 2024-09-10T20:42:46.684 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53699.log: 0.0%/var/log/ceph/ceph-client.admin.39244.log: -- replaced with /var/log/ceph/ceph-client.admin.49373.log.gz 2024-09-10T20:42:46.684 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.70395.log: /var/log/ceph/ceph-client.admin.35955.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53699.log.gz 2024-09-10T20:42:46.684 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39244.log.gz 2024-09-10T20:42:46.684 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52439.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35955.log.gz -- replaced with /var/log/ceph/ceph-client.admin.70395.log.gz 2024-09-10T20:42:46.684 INFO:teuthology.orchestra.run.smithi131.stderr: 2024-09-10T20:42:46.684 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65522.log 2024-09-10T20:42:46.684 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0%/var/log/ceph/ceph-client.admin.45389.log: -- replaced with /var/log/ceph/ceph-client.admin.52439.log.gz 2024-09-10T20:42:46.684 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54665.log: /var/log/ceph/ceph-client.admin.49499.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45389.log.gz 2024-09-10T20:42:46.684 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54665.log.gz 2024-09-10T20:42:46.685 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49499.log.gz 2024-09-10T20:42:46.685 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44871.log.gz 2024-09-10T20:42:46.685 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47105.log: /var/log/ceph/ceph-client.admin.51767.log: /var/log/ceph/ceph-client.admin.52691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69597.log 2024-09-10T20:42:46.685 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38578.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47105.log.gz 2024-09-10T20:42:46.685 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51767.log.gz 2024-09-10T20:42:46.685 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52691.log.gz 2024-09-10T20:42:46.685 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38578.log.gz 2024-09-10T20:42:46.685 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.63587.log: /var/log/ceph/ceph-client.admin.31984.log: /var/log/ceph/ceph-client.admin.61024.log: /var/log/ceph/ceph-client.admin.38467.log: /var/log/ceph/ceph-client.admin.36187.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63587.log.gz 2024-09-10T20:42:46.685 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31984.log.gz 2024-09-10T20:42:46.685 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65522.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61024.log.gz -- replaced with /var/log/ceph/ceph-client.admin.38467.log.gz 2024-09-10T20:42:46.685 INFO:teuthology.orchestra.run.smithi131.stderr: 2024-09-10T20:42:46.685 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39429.log 2024-09-10T20:42:46.685 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65522.log.gz 2024-09-10T20:42:46.685 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36187.log.gz 2024-09-10T20:42:46.686 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36820.log: /var/log/ceph/ceph-client.admin.35597.log: /var/log/ceph/ceph-client.admin.67749.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36820.log.gz 2024-09-10T20:42:46.686 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38652.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35597.log.gz 2024-09-10T20:42:46.686 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67749.log.gz/var/log/ceph/ceph-client.admin.37902.log: 2024-09-10T20:42:46.686 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.70017.log: /var/log/ceph/ceph-client.admin.69597.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37902.log.gz 2024-09-10T20:42:46.686 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61234.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70017.log.gz 2024-09-10T20:42:46.686 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57607.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69597.log.gz 2024-09-10T20:42:46.686 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53237.log: -- replaced with /var/log/ceph/ceph-client.admin.38652.log.gzgzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.36927.log 2024-09-10T20:42:46.686 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61234.log.gz 2024-09-10T20:42:46.686 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57607.log.gz 2024-09-10T20:42:46.686 INFO:teuthology.orchestra.run.smithi131.stderr: 2024-09-10T20:42:46.686 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0%/var/log/ceph/ceph-client.admin.51053.log: -- replaced with /var/log/ceph/ceph-client.admin.53237.log.gz 2024-09-10T20:42:46.686 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67917.log: /var/log/ceph/ceph-client.admin.57985.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51053.log.gz 2024-09-10T20:42:46.687 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67917.log.gz 2024-09-10T20:42:46.687 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57985.log.gz 2024-09-10T20:42:46.687 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39429.log: /var/log/ceph/ceph-client.admin.55757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39170.log 2024-09-10T20:42:46.687 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.58291.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39429.log.gz 2024-09-10T20:42:46.687 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58291.log.gz 2024-09-10T20:42:46.687 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55757.log.gz 2024-09-10T20:42:46.687 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.66825.log: /var/log/ceph/ceph-client.admin.56933.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66825.log.gz 2024-09-10T20:42:46.687 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54749.log: /var/log/ceph/ceph-client.admin.51137.log: /var/log/ceph/ceph-client.admin.36927.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54749.log.gz 2024-09-10T20:42:46.687 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56933.log.gz 2024-09-10T20:42:46.687 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51137.log.gz 2024-09-10T20:42:46.687 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51347.log 2024-09-10T20:42:46.687 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36927.log.gz 2024-09-10T20:42:46.687 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65228.log 2024-09-10T20:42:46.687 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39170.log.gz 2024-09-10T20:42:46.687 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59176.log 2024-09-10T20:42:46.688 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51347.log.gz 2024-09-10T20:42:46.688 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42461.log 2024-09-10T20:42:46.688 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65228.log.gz 2024-09-10T20:42:46.688 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43423.log 2024-09-10T20:42:46.688 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59176.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59176.log.gz 2024-09-10T20:42:46.688 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38444.log 2024-09-10T20:42:46.688 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42461.log.gz 2024-09-10T20:42:46.688 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67707.log 2024-09-10T20:42:46.688 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43423.log.gz 2024-09-10T20:42:46.688 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48575.log 2024-09-10T20:42:46.688 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38444.log.gz 2024-09-10T20:42:46.688 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56051.log 2024-09-10T20:42:46.688 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67707.log.gz 2024-09-10T20:42:46.688 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40745.log 2024-09-10T20:42:46.689 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48575.log.gz 2024-09-10T20:42:46.689 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62957.log 2024-09-10T20:42:46.689 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56051.log.gz 2024-09-10T20:42:46.689 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60016.log 2024-09-10T20:42:46.689 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40745.log.gz 2024-09-10T20:42:46.689 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.62957.log: -- /var/log/ceph/ceph-client.admin.49583.log 2024-09-10T20:42:46.689 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62957.log.gz 2024-09-10T20:42:46.689 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53825.log 2024-09-10T20:42:46.689 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60016.log.gz 2024-09-10T20:42:46.689 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37458.log 2024-09-10T20:42:46.690 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49583.log.gz 2024-09-10T20:42:46.690 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54833.log 2024-09-10T20:42:46.690 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53825.log.gz 2024-09-10T20:42:46.690 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37458.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37458.log.gz 2024-09-10T20:42:46.690 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42607.log 2024-09-10T20:42:46.690 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54791.log 2024-09-10T20:42:46.690 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54833.log.gz 2024-09-10T20:42:46.690 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52061.log 2024-09-10T20:42:46.690 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42607.log.gz 2024-09-10T20:42:46.690 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60646.log 2024-09-10T20:42:46.690 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54791.log.gz 2024-09-10T20:42:46.691 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53279.log 2024-09-10T20:42:46.691 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52061.log.gz 2024-09-10T20:42:46.691 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56093.log 2024-09-10T20:42:46.691 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60646.log.gz 2024-09-10T20:42:46.691 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35257.log 2024-09-10T20:42:46.691 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53279.log.gz 2024-09-10T20:42:46.691 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56093.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56093.log.gz 2024-09-10T20:42:46.691 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62747.log 2024-09-10T20:42:46.691 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45278.log 2024-09-10T20:42:46.691 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35257.log.gz 2024-09-10T20:42:46.691 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.62747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50465.log 2024-09-10T20:42:46.691 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62747.log.gz 2024-09-10T20:42:46.691 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43255.log 2024-09-10T20:42:46.692 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45278.log.gz 2024-09-10T20:42:46.692 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67287.log 2024-09-10T20:42:46.692 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50465.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50465.log.gz 2024-09-10T20:42:46.692 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67959.log 2024-09-10T20:42:46.692 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43255.log.gz 2024-09-10T20:42:46.692 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67455.log 2024-09-10T20:42:46.692 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67287.log.gz 2024-09-10T20:42:46.692 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53027.log 2024-09-10T20:42:46.692 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67959.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67959.log.gz 2024-09-10T20:42:46.692 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67455.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67455.log.gz 2024-09-10T20:42:46.692 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44389.log 2024-09-10T20:42:46.692 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55799.log 2024-09-10T20:42:46.692 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53027.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53027.log.gz 2024-09-10T20:42:46.692 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61612.log 2024-09-10T20:42:46.692 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44389.log.gz 2024-09-10T20:42:46.692 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55799.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55799.log.gz 2024-09-10T20:42:46.693 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61276.log 2024-09-10T20:42:46.693 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47147.log 2024-09-10T20:42:46.693 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61612.log.gz 2024-09-10T20:42:46.693 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51179.log 2024-09-10T20:42:46.693 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61276.log.gz 2024-09-10T20:42:46.693 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66026.log 2024-09-10T20:42:46.693 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47147.log.gz 2024-09-10T20:42:46.693 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62453.log 2024-09-10T20:42:46.693 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51179.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51179.log.gz 2024-09-10T20:42:46.693 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.66026.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-10T20:42:46.693 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66026.log.gz 2024-09-10T20:42:46.693 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.62453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55547.log 2024-09-10T20:42:46.693 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62453.log.gz 2024-09-10T20:42:46.693 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38948.log 2024-09-10T20:42:46.693 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.55547.log: /var/log/ceph/ceph-client.admin.63629.log 2024-09-10T20:42:46.694 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55547.log.gz 2024-09-10T20:42:46.694 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62075.log 2024-09-10T20:42:46.694 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38948.log.gz 2024-09-10T20:42:46.694 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.63629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42691.log 2024-09-10T20:42:46.694 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63629.log.gz 2024-09-10T20:42:46.694 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.62075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51641.log 2024-09-10T20:42:46.694 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62075.log.gz 2024-09-10T20:42:46.694 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42691.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.42691.log.gz -- 2024-09-10T20:42:46.694 INFO:teuthology.orchestra.run.smithi131.stderr: /var/log/ceph/ceph-client.admin.55505.log 2024-09-10T20:42:46.694 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39392.log 2024-09-10T20:42:46.694 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51641.log.gz 2024-09-10T20:42:46.694 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66741.log 2024-09-10T20:42:46.694 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55505.log.gz 2024-09-10T20:42:46.694 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35313.log 2024-09-10T20:42:46.695 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39392.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39392.log.gz 2024-09-10T20:42:46.695 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.66741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69135.log 2024-09-10T20:42:46.695 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66741.log.gz 2024-09-10T20:42:46.695 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43171.log 2024-09-10T20:42:46.695 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35313.log.gz 2024-09-10T20:42:46.695 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47651.log 2024-09-10T20:42:46.695 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69135.log.gz 2024-09-10T20:42:46.695 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44011.log 2024-09-10T20:42:46.695 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43171.log.gz 2024-09-10T20:42:46.695 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58333.log 2024-09-10T20:42:46.695 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47651.log.gz 2024-09-10T20:42:46.695 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65732.log 2024-09-10T20:42:46.695 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44011.log.gz 2024-09-10T20:42:46.696 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.58333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45241.log 2024-09-10T20:42:46.696 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58333.log.gz 2024-09-10T20:42:46.696 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64428.log 2024-09-10T20:42:46.696 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65732.log.gz 2024-09-10T20:42:46.696 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53363.log 2024-09-10T20:42:46.696 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45241.log.gz 2024-09-10T20:42:46.696 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43045.log 2024-09-10T20:42:46.696 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64428.log.gz 2024-09-10T20:42:46.696 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62243.log 2024-09-10T20:42:46.696 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53363.log.gz 2024-09-10T20:42:46.696 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57901.log 2024-09-10T20:42:46.696 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43045.log.gz 2024-09-10T20:42:46.697 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.62243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50801.log 2024-09-10T20:42:46.697 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62243.log.gz 2024-09-10T20:42:46.697 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52313.log 2024-09-10T20:42:46.697 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57901.log.gz 2024-09-10T20:42:46.697 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65984.log 2024-09-10T20:42:46.697 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50801.log.gz 2024-09-10T20:42:46.697 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42419.log 2024-09-10T20:42:46.697 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52313.log.gz 2024-09-10T20:42:46.697 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66278.log 2024-09-10T20:42:46.697 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65984.log.gz 2024-09-10T20:42:46.697 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42419.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.42419.log.gz -5 2024-09-10T20:42:46.698 INFO:teuthology.orchestra.run.smithi131.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48701.log 2024-09-10T20:42:46.698 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.66278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69555.log 2024-09-10T20:42:46.698 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66278.log.gz 2024-09-10T20:42:46.698 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66531.log 2024-09-10T20:42:46.698 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48701.log.gz 2024-09-10T20:42:46.698 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44325.log 2024-09-10T20:42:46.698 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69555.log.gz 2024-09-10T20:42:46.698 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.66531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51809.log 2024-09-10T20:42:46.698 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66531.log.gz 2024-09-10T20:42:46.698 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44325.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.53069.log 2024-09-10T20:42:46.698 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44325.log.gz 2024-09-10T20:42:46.698 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37569.log 2024-09-10T20:42:46.698 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51809.log.gz 2024-09-10T20:42:46.698 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44515.log 2024-09-10T20:42:46.699 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53069.log.gz 2024-09-10T20:42:46.699 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57354.log 2024-09-10T20:42:46.699 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37569.log.gz 2024-09-10T20:42:46.699 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39207.log 2024-09-10T20:42:46.699 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44515.log.gz 2024-09-10T20:42:46.699 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62915.log 2024-09-10T20:42:46.699 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57354.log.gz 2024-09-10T20:42:46.699 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61360.log 2024-09-10T20:42:46.699 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39207.log.gz 2024-09-10T20:42:46.699 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.62915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38272.log 2024-09-10T20:42:46.700 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62915.log.gz 2024-09-10T20:42:46.700 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61360.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61360.log.gz 2024-09-10T20:42:46.700 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46769.log 2024-09-10T20:42:46.700 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54959.log 2024-09-10T20:42:46.700 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38272.log.gz 2024-09-10T20:42:46.700 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44538.log 2024-09-10T20:42:46.700 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46769.log.gz 2024-09-10T20:42:46.700 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66320.log 2024-09-10T20:42:46.700 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54959.log.gz 2024-09-10T20:42:46.700 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58501.log 2024-09-10T20:42:46.700 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44538.log.gz 2024-09-10T20:42:46.700 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.66320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44723.log 2024-09-10T20:42:46.700 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66320.log.gz 2024-09-10T20:42:46.700 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.58501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42019.log 2024-09-10T20:42:46.701 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58501.log.gz 2024-09-10T20:42:46.701 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61486.log 2024-09-10T20:42:46.701 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44723.log.gz 2024-09-10T20:42:46.701 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42019.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.42019.log.gz --verbose 2024-09-10T20:42:46.701 INFO:teuthology.orchestra.run.smithi131.stderr: -- /var/log/ceph/ceph-client.admin.49037.log 2024-09-10T20:42:46.701 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62789.log 2024-09-10T20:42:46.702 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61486.log.gz 2024-09-10T20:42:46.702 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49037.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.49037.log.gz --verbose 2024-09-10T20:42:46.702 INFO:teuthology.orchestra.run.smithi131.stderr: -- /var/log/ceph/ceph-client.admin.60604.log 2024-09-10T20:42:46.702 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.62789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52145.log 2024-09-10T20:42:46.702 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62789.log.gz 2024-09-10T20:42:46.702 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65186.log 2024-09-10T20:42:46.703 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60604.log.gz 2024-09-10T20:42:46.703 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59806.log 2024-09-10T20:42:46.703 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52145.log.gz 2024-09-10T20:42:46.703 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66110.log 2024-09-10T20:42:46.703 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65186.log.gz 2024-09-10T20:42:46.703 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47567.log 2024-09-10T20:42:46.704 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59806.log.gz 2024-09-10T20:42:46.704 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.66110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52985.log 2024-09-10T20:42:46.704 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66110.log.gz 2024-09-10T20:42:46.704 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55841.log 2024-09-10T20:42:46.704 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47567.log.gz 2024-09-10T20:42:46.704 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52985.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.52985.log.gz -- 2024-09-10T20:42:46.705 INFO:teuthology.orchestra.run.smithi131.stderr: /var/log/ceph/ceph-client.admin.56387.log 2024-09-10T20:42:46.705 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57523.log 2024-09-10T20:42:46.705 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55841.log.gz 2024-09-10T20:42:46.705 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50675.log 2024-09-10T20:42:46.705 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56387.log.gz 2024-09-10T20:42:46.705 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36862.log 2024-09-10T20:42:46.706 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57523.log.gz 2024-09-10T20:42:46.706 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48995.log 2024-09-10T20:42:46.706 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50675.log.gz 2024-09-10T20:42:46.706 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57143.log 0.0% 2024-09-10T20:42:46.706 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36862.log.gz 2024-09-10T20:42:46.706 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65354.log 2024-09-10T20:42:46.707 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48995.log.gz 2024-09-10T20:42:46.707 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61654.log 2024-09-10T20:42:46.707 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57143.log.gz 2024-09-10T20:42:46.707 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65354.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27549.log 2024-09-10T20:42:46.707 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65354.log.gz 2024-09-10T20:42:46.708 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61738.log 2024-09-10T20:42:46.708 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61654.log.gz 2024-09-10T20:42:46.708 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph.tmp-client.admin.27549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48323.log 2024-09-10T20:42:46.708 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27549.log.gz 2024-09-10T20:42:46.708 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.61738.log: -- /var/log/ceph/ceph-client.admin.46685.log 2024-09-10T20:42:46.708 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61738.log.gz 2024-09-10T20:42:46.709 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58711.log 2024-09-10T20:42:46.709 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48323.log.gz 2024-09-10T20:42:46.709 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47231.log 2024-09-10T20:42:46.709 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46685.log.gz 2024-09-10T20:42:46.709 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.58711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50969.log 2024-09-10T20:42:46.709 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58711.log.gz 2024-09-10T20:42:46.710 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58459.log 2024-09-10T20:42:46.710 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46853.log 2024-09-10T20:42:46.710 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50969.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50969.log.gz 2024-09-10T20:42:46.710 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.58459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53951.log 2024-09-10T20:42:46.710 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58459.log.gz 2024-09-10T20:42:46.710 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37532.log 2024-09-10T20:42:46.711 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46853.log.gz 2024-09-10T20:42:46.711 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59092.log 2024-09-10T20:42:46.711 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53951.log.gz 2024-09-10T20:42:46.711 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38911.log 2024-09-10T20:42:46.711 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37532.log.gz 2024-09-10T20:42:46.711 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59092.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59092.log.gz 2024-09-10T20:42:46.712 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40577.log 2024-09-10T20:42:46.712 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42103.log 2024-09-10T20:42:46.712 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38911.log.gz 2024-09-10T20:42:46.712 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68715.log 2024-09-10T20:42:46.712 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40577.log.gz 2024-09-10T20:42:46.713 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39059.log 2024-09-10T20:42:46.713 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42103.log.gz 2024-09-10T20:42:46.713 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32160.log 2024-09-10T20:42:46.713 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68715.log.gz 2024-09-10T20:42:46.713 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43129.log 2024-09-10T20:42:46.713 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39059.log.gz 2024-09-10T20:42:46.714 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.32160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59218.log 2024-09-10T20:42:46.714 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32160.log.gz 2024-09-10T20:42:46.714 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43549.log 2024-09-10T20:42:46.714 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43129.log.gz 2024-09-10T20:42:46.714 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40576.log 2024-09-10T20:42:46.714 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59218.log.gz 2024-09-10T20:42:46.714 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47189.log 2024-09-10T20:42:46.715 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43549.log.gz 2024-09-10T20:42:46.715 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63041.log 2024-09-10T20:42:46.715 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40576.log.gz 2024-09-10T20:42:46.715 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65942.log 2024-09-10T20:42:46.715 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47189.log.gz 2024-09-10T20:42:46.715 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.63041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59302.log 2024-09-10T20:42:46.715 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63041.log.gz 2024-09-10T20:42:46.715 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70143.log 2024-09-10T20:42:46.716 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65942.log.gz 2024-09-10T20:42:46.716 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64848.log 2024-09-10T20:42:46.716 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59302.log.gz 2024-09-10T20:42:46.716 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.70143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67035.log 2024-09-10T20:42:46.716 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70143.log.gz 2024-09-10T20:42:46.716 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61108.log 2024-09-10T20:42:46.716 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64848.log.gz 2024-09-10T20:42:46.716 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67035.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67035.log.gz 2024-09-10T20:42:46.716 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66867.log 2024-09-10T20:42:46.716 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37754.log 2024-09-10T20:42:46.716 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61108.log.gz 2024-09-10T20:42:46.717 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.66867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62032.log 2024-09-10T20:42:46.717 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66867.log.gz 2024-09-10T20:42:46.717 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68337.log 2024-09-10T20:42:46.717 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37754.log.gz 2024-09-10T20:42:46.717 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.62032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52019.log 2024-09-10T20:42:46.717 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62032.log.gz 2024-09-10T20:42:46.717 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37976.log 2024-09-10T20:42:46.717 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68337.log.gz 2024-09-10T20:42:46.717 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50129.log 2024-09-10T20:42:46.717 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52019.log.gz 2024-09-10T20:42:46.717 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37976.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37976.log.gz 2024-09-10T20:42:46.717 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61570.log 2024-09-10T20:42:46.717 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49709.log 2024-09-10T20:42:46.718 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50129.log.gz 2024-09-10T20:42:46.718 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69891.log 2024-09-10T20:42:46.718 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61570.log.gz 2024-09-10T20:42:46.718 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63713.log 2024-09-10T20:42:46.718 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49709.log.gz 2024-09-10T20:42:46.718 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55127.log/var/log/ceph/ceph-client.admin.69891.log: 2024-09-10T20:42:46.718 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69891.log.gz 2024-09-10T20:42:46.718 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.63713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46895.log 2024-09-10T20:42:46.718 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63713.log.gz 2024-09-10T20:42:46.718 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48239.log 2024-09-10T20:42:46.718 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55127.log.gz 2024-09-10T20:42:46.718 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58921.log 2024-09-10T20:42:46.718 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46895.log.gz 2024-09-10T20:42:46.719 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43927.log 2024-09-10T20:42:46.719 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48239.log.gz 2024-09-10T20:42:46.719 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.58921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54245.log 2024-09-10T20:42:46.719 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58921.log.gz 2024-09-10T20:42:46.719 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43927.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43927.log.gz 2024-09-10T20:42:46.719 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53615.log 2024-09-10T20:42:46.719 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47021.log 2024-09-10T20:42:46.719 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54245.log.gz 2024-09-10T20:42:46.719 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50339.log 2024-09-10T20:42:46.719 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53615.log.gz 2024-09-10T20:42:46.719 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45556.log 2024-09-10T20:42:46.719 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47021.log.gz 2024-09-10T20:42:46.719 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39022.log 2024-09-10T20:42:46.719 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50339.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50339.log.gz 2024-09-10T20:42:46.720 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42523.log 2024-09-10T20:42:46.720 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45556.log.gz 2024-09-10T20:42:46.720 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60940.log 2024-09-10T20:42:46.720 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39022.log.gz 2024-09-10T20:42:46.720 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42523.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42523.log.gz 2024-09-10T20:42:46.720 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52523.log 2024-09-10T20:42:46.720 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38198.log 2024-09-10T20:42:46.720 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60940.log.gz 2024-09-10T20:42:46.720 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44221.log 2024-09-10T20:42:46.720 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52523.log.gz 2024-09-10T20:42:46.720 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63461.log 2024-09-10T20:42:46.720 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38198.log.gz 2024-09-10T20:42:46.720 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44221.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44221.log.gz 2024-09-10T20:42:46.720 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62159.log 2024-09-10T20:42:46.721 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.63461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42919.log 2024-09-10T20:42:46.721 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63461.log.gz 2024-09-10T20:42:46.721 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.62159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63839.log 2024-09-10T20:42:46.721 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62159.log.gz 2024-09-10T20:42:46.721 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36565.log 2024-09-10T20:42:46.721 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42919.log.gz 2024-09-10T20:42:46.721 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.63839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59974.log 2024-09-10T20:42:46.721 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63839.log.gz 2024-09-10T20:42:46.721 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36565.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.36565.log.gz 2024-09-10T20:42:46.721 INFO:teuthology.orchestra.run.smithi131.stderr: /var/log/ceph/ceph-client.admin.51893.log 2024-09-10T20:42:46.721 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69639.log 2024-09-10T20:42:46.721 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59974.log.gz 2024-09-10T20:42:46.721 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52901.log 2024-09-10T20:42:46.722 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51893.log.gz 2024-09-10T20:42:46.722 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49835.log 2024-09-10T20:42:46.722 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69639.log.gz 2024-09-10T20:42:46.722 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54413.log 2024-09-10T20:42:46.722 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52901.log.gz 2024-09-10T20:42:46.722 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66488.log 2024-09-10T20:42:46.722 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49835.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49835.log.gz 2024-09-10T20:42:46.722 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60814.log 2024-09-10T20:42:46.722 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54413.log.gz 2024-09-10T20:42:46.722 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.66488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60184.log 2024-09-10T20:42:46.722 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66488.log.gz 2024-09-10T20:42:46.722 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46979.log 2024-09-10T20:42:46.722 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60814.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60814.log.gz 2024-09-10T20:42:46.722 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66909.log 2024-09-10T20:42:46.723 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60184.log.gz 2024-09-10T20:42:46.723 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65270.log 2024-09-10T20:42:46.723 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46979.log.gz 2024-09-10T20:42:46.723 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37255.log 2024-09-10T20:42:46.723 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.66909.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66909.log.gz 2024-09-10T20:42:46.723 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48197.log 2024-09-10T20:42:46.723 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65270.log.gz 2024-09-10T20:42:46.723 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38541.log 2024-09-10T20:42:46.723 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37255.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37255.log.gz 2024-09-10T20:42:46.723 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40660.log 2024-09-10T20:42:46.723 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48197.log.gz 2024-09-10T20:42:46.723 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38541.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.38541.log.gz 2024-09-10T20:42:46.723 INFO:teuthology.orchestra.run.smithi131.stderr: /var/log/ceph/ceph-client.admin.52733.log 2024-09-10T20:42:46.724 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67539.log 2024-09-10T20:42:46.724 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40660.log.gz 2024-09-10T20:42:46.724 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55883.log 2024-09-10T20:42:46.724 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52733.log.gz 2024-09-10T20:42:46.724 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54707.log 2024-09-10T20:42:46.724 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67539.log.gz 2024-09-10T20:42:46.724 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66657.log 2024-09-10T20:42:46.724 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55883.log.gz 2024-09-10T20:42:46.724 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43381.log 2024-09-10T20:42:46.724 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54707.log.gz 2024-09-10T20:42:46.724 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.66657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53783.log 2024-09-10T20:42:46.724 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66657.log.gz 2024-09-10T20:42:46.724 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35985.log 2024-09-10T20:42:46.725 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43381.log.gz 2024-09-10T20:42:46.725 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60436.log 2024-09-10T20:42:46.725 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53783.log.gz 2024-09-10T20:42:46.725 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68463.log 2024-09-10T20:42:46.725 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35985.log.gz 2024-09-10T20:42:46.725 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47609.log 2024-09-10T20:42:46.725 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60436.log.gz 2024-09-10T20:42:46.725 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42102.log 2024-09-10T20:42:46.725 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68463.log.gz 2024-09-10T20:42:46.725 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65438.log 2024-09-10T20:42:46.725 INFO:teuthology.orchestra.run.smithi131.stderr: 26.9% -- replaced with /var/log/ceph/ceph-client.admin.47231.log.gz 2024-09-10T20:42:46.725 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47609.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47609.log.gz 2024-09-10T20:42:46.726 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60562.log 2024-09-10T20:42:46.726 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42102.log.gz 2024-09-10T20:42:46.726 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65438.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-10T20:42:46.726 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65438.log.gz 2024-09-10T20:42:46.726 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68799.log 2024-09-10T20:42:46.726 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60562.log.gz 2024-09-10T20:42:46.726 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58417.log 2024-09-10T20:42:46.726 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49919.log 2024-09-10T20:42:46.726 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68799.log.gz 2024-09-10T20:42:46.726 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.58417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51221.log 2024-09-10T20:42:46.726 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58417.log.gz 2024-09-10T20:42:46.726 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32359.log 2024-09-10T20:42:46.726 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49919.log.gz 2024-09-10T20:42:46.726 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50927.log 2024-09-10T20:42:46.726 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51221.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51221.log.gz 2024-09-10T20:42:46.726 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.32359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68925.log 2024-09-10T20:42:46.727 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32359.log.gz 2024-09-10T20:42:46.727 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65016.log 2024-09-10T20:42:46.727 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50927.log.gz 2024-09-10T20:42:46.727 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58375.log 2024-09-10T20:42:46.727 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68925.log.gz 2024-09-10T20:42:46.727 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59470.log 2024-09-10T20:42:46.727 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65016.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65016.log.gz 2024-09-10T20:42:46.727 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.58375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67245.log 2024-09-10T20:42:46.727 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58375.log.gz 2024-09-10T20:42:46.727 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67077.log 2024-09-10T20:42:46.727 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59470.log.gz 2024-09-10T20:42:46.727 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44053.log 2024-09-10T20:42:46.727 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67245.log.gz 2024-09-10T20:42:46.727 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36478.log 2024-09-10T20:42:46.728 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67077.log.gz 2024-09-10T20:42:46.728 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35722.log 2024-09-10T20:42:46.728 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44053.log.gz 2024-09-10T20:42:46.728 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50255.log 2024-09-10T20:42:46.728 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36478.log.gz 2024-09-10T20:42:46.728 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38689.log 2024-09-10T20:42:46.728 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35722.log.gz 2024-09-10T20:42:46.728 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51599.log 2024-09-10T20:42:46.728 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50255.log.gz 2024-09-10T20:42:46.728 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42835.log 2024-09-10T20:42:46.728 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38689.log.gz 2024-09-10T20:42:46.728 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54119.log 2024-09-10T20:42:46.728 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51599.log.gz 2024-09-10T20:42:46.728 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52859.log 2024-09-10T20:42:46.729 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42835.log.gz 2024-09-10T20:42:46.729 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54581.log 2024-09-10T20:42:46.729 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54119.log.gz 2024-09-10T20:42:46.729 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43759.log 2024-09-10T20:42:46.729 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52859.log.gz 2024-09-10T20:42:46.729 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62495.log 2024-09-10T20:42:46.729 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54581.log.gz 2024-09-10T20:42:46.729 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47861.log 2024-09-10T20:42:46.729 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43759.log.gz 2024-09-10T20:42:46.729 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.62495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37361.log 2024-09-10T20:42:46.729 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62495.log.gz 2024-09-10T20:42:46.729 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49415.log 2024-09-10T20:42:46.729 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47861.log.gz 2024-09-10T20:42:46.730 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64344.log 2024-09-10T20:42:46.730 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37361.log.gz 2024-09-10T20:42:46.730 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47315.log 2024-09-10T20:42:46.730 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49415.log.gz 2024-09-10T20:42:46.730 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36271.log 2024-09-10T20:42:46.730 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64344.log.gz 2024-09-10T20:42:46.730 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65690.log 2024-09-10T20:42:46.730 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47315.log.gz 2024-09-10T20:42:46.730 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57269.log 2024-09-10T20:42:46.730 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36271.log.gz 2024-09-10T20:42:46.730 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55967.log 2024-09-10T20:42:46.730 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65690.log.gz 2024-09-10T20:42:46.730 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35935.log 2024-09-10T20:42:46.730 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57269.log.gz 2024-09-10T20:42:46.731 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58963.log 2024-09-10T20:42:46.731 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55967.log.gz 2024-09-10T20:42:46.731 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63671.log 2024-09-10T20:42:46.731 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35935.log.gz 2024-09-10T20:42:46.731 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.58963.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.68295.log -- replaced with /var/log/ceph/ceph-client.admin.58963.log.gz 2024-09-10T20:42:46.731 INFO:teuthology.orchestra.run.smithi131.stderr: 2024-09-10T20:42:46.731 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.63671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61528.log 2024-09-10T20:42:46.731 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63671.log.gz 2024-09-10T20:42:46.731 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68295.log: gzip 0.0% -5 --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.68295.log.gz /var/log/ceph/ceph-client.admin.42961.log 2024-09-10T20:42:46.731 INFO:teuthology.orchestra.run.smithi131.stderr: 2024-09-10T20:42:46.731 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48869.log 2024-09-10T20:42:46.732 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61528.log.gz 2024-09-10T20:42:46.732 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38985.log 2024-09-10T20:42:46.732 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42961.log.gz 2024-09-10T20:42:46.732 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60730.log 2024-09-10T20:42:46.732 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48869.log.gz 2024-09-10T20:42:46.732 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58669.log 2024-09-10T20:42:46.732 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38985.log.gz 2024-09-10T20:42:46.732 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36351.log 2024-09-10T20:42:46.732 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60730.log.gz 2024-09-10T20:42:46.732 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.58669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49163.log 2024-09-10T20:42:46.732 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58669.log.gz 2024-09-10T20:42:46.732 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38161.log 2024-09-10T20:42:46.732 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36351.log.gz 2024-09-10T20:42:46.732 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42606.log 2024-09-10T20:42:46.733 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49163.log.gz 2024-09-10T20:42:46.733 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52649.log 2024-09-10T20:42:46.733 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38161.log.gz 2024-09-10T20:42:46.733 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48827.log 2024-09-10T20:42:46.733 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42606.log.gz 2024-09-10T20:42:46.733 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48449.log 2024-09-10T20:42:46.733 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52649.log.gz 2024-09-10T20:42:46.733 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44612.log 2024-09-10T20:42:46.733 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48827.log.gz 2024-09-10T20:42:46.733 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61696.log 2024-09-10T20:42:46.734 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48449.log.gz 2024-09-10T20:42:46.734 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44612.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.44612.log.gz /var/log/ceph/ceph-client.admin.59722.log 2024-09-10T20:42:46.734 INFO:teuthology.orchestra.run.smithi131.stderr: 2024-09-10T20:42:46.734 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64176.log 2024-09-10T20:42:46.734 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61696.log.gz 2024-09-10T20:42:46.734 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38087.log 2024-09-10T20:42:46.734 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59722.log.gz 2024-09-10T20:42:46.734 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56009.log 2024-09-10T20:42:46.734 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64176.log.gz 2024-09-10T20:42:46.734 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53741.log 2024-09-10T20:42:46.734 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38087.log.gz 2024-09-10T20:42:46.734 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52817.log 2024-09-10T20:42:46.735 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56009.log.gz 2024-09-10T20:42:46.735 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69933.log 2024-09-10T20:42:46.735 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53741.log.gz 2024-09-10T20:42:46.735 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56555.log 2024-09-10T20:42:46.735 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52817.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52817.log.gz 2024-09-10T20:42:46.735 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43801.log 2024-09-10T20:42:46.735 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69933.log.gz 2024-09-10T20:42:46.735 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61192.log 2024-09-10T20:42:46.735 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56555.log.gz 2024-09-10T20:42:46.735 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35595.log 2024-09-10T20:42:46.735 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43801.log.gz 2024-09-10T20:42:46.735 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61192.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61192.log.gz 2024-09-10T20:42:46.735 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56177.log 2024-09-10T20:42:46.735 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54287.log 2024-09-10T20:42:46.736 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35595.log.gz 2024-09-10T20:42:46.736 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57691.log 2024-09-10T20:42:46.736 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56177.log.gz 2024-09-10T20:42:46.736 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56723.log 2024-09-10T20:42:46.736 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54287.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54287.log.gz 2024-09-10T20:42:46.736 INFO:teuthology.orchestra.run.smithi131.stderr:gzip/var/log/ceph/ceph-client.admin.57691.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.37002.log 2024-09-10T20:42:46.736 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57691.log.gz 2024-09-10T20:42:46.736 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42522.log 2024-09-10T20:42:46.736 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56723.log.gz 2024-09-10T20:42:46.736 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64596.log 2024-09-10T20:42:46.736 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37002.log.gz 2024-09-10T20:42:46.736 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61948.log 2024-09-10T20:42:46.736 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42522.log.gz 2024-09-10T20:42:46.737 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63209.log 2024-09-10T20:42:46.737 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64596.log.gz 2024-09-10T20:42:46.737 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66783.log 2024-09-10T20:42:46.737 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61948.log.gz 2024-09-10T20:42:46.737 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.63209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50885.log 2024-09-10T20:42:46.737 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63209.log.gz 2024-09-10T20:42:46.737 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.66783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48911.log 2024-09-10T20:42:46.737 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66783.log.gz 2024-09-10T20:42:46.737 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50885.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.40851.log -- replaced with /var/log/ceph/ceph-client.admin.50885.log.gz 2024-09-10T20:42:46.737 INFO:teuthology.orchestra.run.smithi131.stderr: 2024-09-10T20:42:46.737 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52271.log 2024-09-10T20:42:46.737 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48911.log.gz 2024-09-10T20:42:46.737 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40851.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.40851.log.gz --verbose 2024-09-10T20:42:46.737 INFO:teuthology.orchestra.run.smithi131.stderr: -- /var/log/ceph/ceph-client.admin.39318.log 2024-09-10T20:42:46.738 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59512.log 2024-09-10T20:42:46.738 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52271.log.gz 2024-09-10T20:42:46.738 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38726.log 2024-09-10T20:42:46.738 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39318.log.gz 2024-09-10T20:42:46.738 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60268.log 2024-09-10T20:42:46.738 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59512.log.gz 2024-09-10T20:42:46.738 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37003.log 2024-09-10T20:42:46.738 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38726.log.gz 2024-09-10T20:42:46.738 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42018.log 2024-09-10T20:42:46.738 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60268.log.gz 2024-09-10T20:42:46.738 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69093.log 2024-09-10T20:42:46.738 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37003.log.gz 2024-09-10T20:42:46.738 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45352.log 2024-09-10T20:42:46.738 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42018.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42018.log.gz 2024-09-10T20:42:46.739 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57101.log 2024-09-10T20:42:46.739 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69093.log.gz 2024-09-10T20:42:46.739 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66615.log 2024-09-10T20:42:46.739 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45352.log.gz 2024-09-10T20:42:46.739 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36479.log 2024-09-10T20:42:46.739 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57101.log.gz 2024-09-10T20:42:46.739 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.66615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60520.log 2024-09-10T20:42:46.739 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66615.log.gz 2024-09-10T20:42:46.739 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58123.log 2024-09-10T20:42:46.739 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36479.log.gz 2024-09-10T20:42:46.739 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54161.log 2024-09-10T20:42:46.739 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60520.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60520.log.gz 2024-09-10T20:42:46.739 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.58123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69009.log 2024-09-10T20:42:46.740 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58123.log.gz 2024-09-10T20:42:46.740 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62873.log 2024-09-10T20:42:46.740 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54161.log.gz 2024-09-10T20:42:46.740 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67623.log 2024-09-10T20:42:46.740 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69009.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69009.log.gz 2024-09-10T20:42:46.740 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.62873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54035.log 2024-09-10T20:42:46.740 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62873.log.gz 2024-09-10T20:42:46.740 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48743.log 2024-09-10T20:42:46.740 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67623.log.gz 2024-09-10T20:42:46.740 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51389.log 2024-09-10T20:42:46.740 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54035.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54035.log.gz 2024-09-10T20:42:46.740 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61444.log 2024-09-10T20:42:46.740 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48743.log.gz 2024-09-10T20:42:46.740 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62537.log 2024-09-10T20:42:46.741 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51389.log.gz 2024-09-10T20:42:46.741 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52397.log 2024-09-10T20:42:46.741 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61444.log.gz 2024-09-10T20:42:46.741 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.62537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50171.log 2024-09-10T20:42:46.741 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62537.log.gz 2024-09-10T20:42:46.741 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54455.log 2024-09-10T20:42:46.741 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52397.log.gz 2024-09-10T20:42:46.741 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log/var/log/ceph/ceph-client.admin.50171.log: 2024-09-10T20:42:46.741 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50171.log.gz 2024-09-10T20:42:46.741 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59680.log 2024-09-10T20:42:46.741 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54455.log.gz 2024-09-10T20:42:46.742 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61906.log 2024-09-10T20:42:46.742 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35473.log 2024-09-10T20:42:46.742 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59680.log.gz 2024-09-10T20:42:46.742 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41957.log 2024-09-10T20:42:46.742 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61906.log.gz 2024-09-10T20:42:46.742 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69513.log 2024-09-10T20:42:46.742 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35473.log.gz 2024-09-10T20:42:46.742 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46937.log 2024-09-10T20:42:46.742 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41957.log.gz 2024-09-10T20:42:46.742 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49667.log 2024-09-10T20:42:46.742 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69513.log.gz 2024-09-10T20:42:46.742 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69975.log/var/log/ceph/ceph-client.admin.46937.log: 2024-09-10T20:42:46.742 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46937.log.gz 2024-09-10T20:42:46.742 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.49667.log: -- /var/log/ceph/ceph-client.admin.64764.log 2024-09-10T20:42:46.742 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49667.log.gz 2024-09-10T20:42:46.743 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69723.log 2024-09-10T20:42:46.743 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69975.log.gz 2024-09-10T20:42:46.743 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59050.log 2024-09-10T20:42:46.743 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64764.log.gz 2024-09-10T20:42:46.743 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66194.log 2024-09-10T20:42:46.743 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69723.log.gz 2024-09-10T20:42:46.743 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66068.log 2024-09-10T20:42:46.743 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59050.log.gz 2024-09-10T20:42:46.743 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.66194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48953.log 2024-09-10T20:42:46.743 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66194.log.gz 2024-09-10T20:42:46.743 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.66068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51095.log 2024-09-10T20:42:46.743 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66068.log.gz 2024-09-10T20:42:46.743 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43969.log 2024-09-10T20:42:46.743 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48953.log.gz 2024-09-10T20:42:46.744 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55631.log 2024-09-10T20:42:46.744 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51095.log.gz 2024-09-10T20:42:46.744 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69345.log 2024-09-10T20:42:46.744 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43969.log.gz 2024-09-10T20:42:46.744 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52607.log 2024-09-10T20:42:46.744 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55631.log.gz 2024-09-10T20:42:46.744 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51683.log 2024-09-10T20:42:46.744 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69345.log.gz 2024-09-10T20:42:46.744 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69681.log 2024-09-10T20:42:46.744 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52607.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52607.log.gz 2024-09-10T20:42:46.744 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42271.log 2024-09-10T20:42:46.744 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51683.log.gz 2024-09-10T20:42:46.744 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47441.log 2024-09-10T20:42:46.744 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69681.log.gz 2024-09-10T20:42:46.745 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54917.log 2024-09-10T20:42:46.745 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42271.log.gz 2024-09-10T20:42:46.745 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60856.log 2024-09-10T20:42:46.745 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47441.log.gz 2024-09-10T20:42:46.745 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37717.log 2024-09-10T20:42:46.745 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54917.log.gz 2024-09-10T20:42:46.745 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42270.log 2024-09-10T20:42:46.745 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60856.log.gz 2024-09-10T20:42:46.745 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35809.log 2024-09-10T20:42:46.745 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37717.log.gz 2024-09-10T20:42:46.745 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38402.log 2024-09-10T20:42:46.745 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42270.log.gz 2024-09-10T20:42:46.745 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40493.log 2024-09-10T20:42:46.745 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35809.log.gz 2024-09-10T20:42:46.746 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38402.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38402.log.gz 2024-09-10T20:42:46.746 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70185.log 2024-09-10T20:42:46.746 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63125.log 2024-09-10T20:42:46.746 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40493.log.gz 2024-09-10T20:42:46.746 INFO:teuthology.orchestra.run.smithi131.stderr: 96.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-10T20:42:46.746 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.70185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61864.log 2024-09-10T20:42:46.746 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70185.log.gz 2024-09-10T20:42:46.746 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.63125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64470.log 2024-09-10T20:42:46.746 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63125.log.gz 2024-09-10T20:42:46.746 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61864.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61864.log.gz 2024-09-10T20:42:46.746 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56639.log 2024-09-10T20:42:46.746 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62285.log 2024-09-10T20:42:46.746 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64470.log.gz 2024-09-10T20:42:46.747 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64050.log 2024-09-10T20:42:46.747 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56639.log.gz 2024-09-10T20:42:46.747 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.62285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40492.log 2024-09-10T20:42:46.747 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62285.log.gz 2024-09-10T20:42:46.747 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53531.log 2024-09-10T20:42:46.747 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64050.log.gz 2024-09-10T20:42:46.747 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40492.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40492.log.gz 2024-09-10T20:42:46.747 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57775.log 2024-09-10T20:42:46.747 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63881.log 2024-09-10T20:42:46.747 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53531.log.gz 2024-09-10T20:42:46.747 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50045.log 2024-09-10T20:42:46.747 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57775.log.gz 2024-09-10T20:42:46.747 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.63881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44797.log 2024-09-10T20:42:46.748 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63881.log.gz 2024-09-10T20:42:46.748 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57649.log 2024-09-10T20:42:46.748 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50045.log.gz 2024-09-10T20:42:46.748 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68673.log 2024-09-10T20:42:46.748 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44797.log.gz 2024-09-10T20:42:46.748 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53867.log 2024-09-10T20:42:46.748 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57649.log.gz 2024-09-10T20:42:46.748 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63167.log 2024-09-10T20:42:46.748 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68673.log.gz 2024-09-10T20:42:46.748 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57185.log 2024-09-10T20:42:46.748 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53867.log.gz 2024-09-10T20:42:46.748 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.63167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36229.log 2024-09-10T20:42:46.748 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63167.log.gz 2024-09-10T20:42:46.749 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45093.log 2024-09-10T20:42:46.749 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57185.log.gz 2024-09-10T20:42:46.749 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55421.log 2024-09-10T20:42:46.749 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36229.log.gz 2024-09-10T20:42:46.749 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57439.log 2024-09-10T20:42:46.749 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45093.log.gz 2024-09-10T20:42:46.749 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67665.log 2024-09-10T20:42:46.749 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55421.log.gz 2024-09-10T20:42:46.749 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55589.log 2024-09-10T20:42:46.749 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57439.log.gz 2024-09-10T20:42:46.749 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37828.log 2024-09-10T20:42:46.749 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67665.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67665.log.gz 2024-09-10T20:42:46.749 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66573.log 2024-09-10T20:42:46.750 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55589.log.gz 2024-09-10T20:42:46.750 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48029.log 2024-09-10T20:42:46.750 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37828.log.gz 2024-09-10T20:42:46.750 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.66573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56849.log 2024-09-10T20:42:46.750 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66573.log.gz 2024-09-10T20:42:46.750 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62579.log 2024-09-10T20:42:46.750 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48029.log.gz 2024-09-10T20:42:46.750 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38309.log 2024-09-10T20:42:46.750 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56849.log.gz 2024-09-10T20:42:46.750 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.62579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40389.log 2024-09-10T20:42:46.750 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62579.log.gz 2024-09-10T20:42:46.750 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59428.log 2024-09-10T20:42:46.750 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38309.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38309.log.gz 2024-09-10T20:42:46.750 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36775.log 2024-09-10T20:42:46.750 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40389.log.gz 2024-09-10T20:42:46.751 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56345.log 2024-09-10T20:42:46.751 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59428.log.gz 2024-09-10T20:42:46.751 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66152.log 2024-09-10T20:42:46.751 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36775.log.gz 2024-09-10T20:42:46.751 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59386.log 2024-09-10T20:42:46.751 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56345.log.gz 2024-09-10T20:42:46.751 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.66152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44305.log 2024-09-10T20:42:46.751 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66152.log.gz 2024-09-10T20:42:46.751 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51473.log 2024-09-10T20:42:46.751 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59386.log.gz 2024-09-10T20:42:46.751 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53447.log 2024-09-10T20:42:46.751 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44305.log.gz 2024-09-10T20:42:46.751 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51431.log 2024-09-10T20:42:46.751 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51473.log.gz 2024-09-10T20:42:46.752 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40661.log 2024-09-10T20:42:46.752 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53447.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53447.log.gz 2024-09-10T20:42:46.752 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48281.log 2024-09-10T20:42:46.752 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51431.log.gz 2024-09-10T20:42:46.752 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40661.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40661.log.gz 2024-09-10T20:42:46.752 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61150.log 2024-09-10T20:42:46.752 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51263.log 2024-09-10T20:42:46.752 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48281.log.gz 2024-09-10T20:42:46.752 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61150.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61150.log.gz 2024-09-10T20:42:46.752 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47819.log 2024-09-10T20:42:46.752 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47777.log 2024-09-10T20:42:46.752 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51263.log.gz 2024-09-10T20:42:46.752 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47819.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47819.log.gz 2024-09-10T20:42:46.752 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62999.log 2024-09-10T20:42:46.753 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61780.log 2024-09-10T20:42:46.753 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47777.log.gz 2024-09-10T20:42:46.753 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.62999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47483.log 2024-09-10T20:42:46.753 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62999.log.gz 2024-09-10T20:42:46.753 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46601.log 2024-09-10T20:42:46.753 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61780.log.gz 2024-09-10T20:42:46.753 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47483.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-10T20:42:46.753 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47483.log.gz 2024-09-10T20:42:46.753 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46601.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46601.log.gz 2024-09-10T20:42:46.753 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45500.log 2024-09-10T20:42:46.753 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50423.log 2024-09-10T20:42:46.753 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57059.log 2024-09-10T20:42:46.753 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45500.log.gz 2024-09-10T20:42:46.754 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59890.log 2024-09-10T20:42:46.754 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50423.log.gz 2024-09-10T20:42:46.754 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35212.log 2024-09-10T20:42:46.754 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57059.log.gz 2024-09-10T20:42:46.754 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43003.log 0.0% 2024-09-10T20:42:46.754 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59890.log.gz 2024-09-10T20:42:46.754 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65648.log 2024-09-10T20:42:46.754 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35212.log.gz 2024-09-10T20:42:46.754 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53321.log 2024-09-10T20:42:46.754 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43003.log.gz 2024-09-10T20:42:46.754 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60688.log 2024-09-10T20:42:46.754 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65648.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65648.log.gz 2024-09-10T20:42:46.754 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52775.log 2024-09-10T20:42:46.754 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53321.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53321.log.gz 2024-09-10T20:42:46.755 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57817.log 2024-09-10T20:42:46.755 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60688.log.gz 2024-09-10T20:42:46.755 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68211.log 2024-09-10T20:42:46.755 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52775.log.gz 2024-09-10T20:42:46.755 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69051.log 2024-09-10T20:42:46.755 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57817.log.gz 2024-09-10T20:42:46.755 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45463.log 2024-09-10T20:42:46.755 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68211.log.gz 2024-09-10T20:42:46.755 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62201.log 2024-09-10T20:42:46.755 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69051.log.gz 2024-09-10T20:42:46.755 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65480.log 2024-09-10T20:42:46.755 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45463.log.gz 2024-09-10T20:42:46.755 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.62201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53573.log 2024-09-10T20:42:46.755 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62201.log.gz 2024-09-10T20:42:46.755 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64092.log 2024-09-10T20:42:46.756 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65480.log.gz 2024-09-10T20:42:46.756 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32458.log 2024-09-10T20:42:46.756 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53573.log.gz 2024-09-10T20:42:46.756 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59260.log 2024-09-10T20:42:46.756 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64092.log.gz 2024-09-10T20:42:46.756 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.32458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69471.log 2024-09-10T20:42:46.756 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32458.log.gz 2024-09-10T20:42:46.756 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44834.log 2024-09-10T20:42:46.756 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59260.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59260.log.gz 2024-09-10T20:42:46.756 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68001.log 2024-09-10T20:42:46.756 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69471.log.gz 2024-09-10T20:42:46.756 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62705.log 2024-09-10T20:42:46.756 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44834.log.gz 2024-09-10T20:42:46.756 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68001.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68001.log.gz 2024-09-10T20:42:46.757 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59005.log 2024-09-10T20:42:46.757 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.62705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43675.log 2024-09-10T20:42:46.757 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62705.log.gz 2024-09-10T20:42:46.757 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56219.log 2024-09-10T20:42:46.757 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59005.log.gz 2024-09-10T20:42:46.757 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43675.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.43675.log.gz 2024-09-10T20:42:46.757 INFO:teuthology.orchestra.run.smithi131.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.54077.log 2024-09-10T20:42:46.757 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64134.log 2024-09-10T20:42:46.757 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56219.log.gz 2024-09-10T20:42:46.757 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68253.log 2024-09-10T20:42:46.757 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54077.log.gz 2024-09-10T20:42:46.757 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49541.log 2024-09-10T20:42:46.758 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64134.log.gz 2024-09-10T20:42:46.758 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68169.log 2024-09-10T20:42:46.758 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68253.log.gz 2024-09-10T20:42:46.758 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58081.log 2024-09-10T20:42:46.758 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49541.log.gz 2024-09-10T20:42:46.758 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32319.log 2024-09-10T20:42:46.758 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68169.log.gz 2024-09-10T20:42:46.758 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.58081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58585.log 2024-09-10T20:42:46.758 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58081.log.gz 2024-09-10T20:42:46.758 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.32319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44156.log 2024-09-10T20:42:46.759 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32319.log.gz 2024-09-10T20:42:46.759 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.58585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45167.log 2024-09-10T20:42:46.759 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58585.log.gz 2024-09-10T20:42:46.759 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.44156.log: /var/log/ceph/ceph-client.admin.42690.log 2024-09-10T20:42:46.759 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44156.log.gz 2024-09-10T20:42:46.759 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50717.log 2024-09-10T20:42:46.759 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45167.log.gz 2024-09-10T20:42:46.759 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42690.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42690.log.gzgzip 2024-09-10T20:42:46.759 INFO:teuthology.orchestra.run.smithi131.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47987.log 2024-09-10T20:42:46.759 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36477.log 2024-09-10T20:42:46.759 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50717.log.gz 2024-09-10T20:42:46.759 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63251.log 2024-09-10T20:42:46.759 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47987.log.gz 2024-09-10T20:42:46.759 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64806.log 2024-09-10T20:42:46.760 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36477.log.gz 2024-09-10T20:42:46.760 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.63251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37087.log 2024-09-10T20:42:46.760 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63251.log.gz 2024-09-10T20:42:46.760 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36904.log 2024-09-10T20:42:46.760 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64806.log.gz 2024-09-10T20:42:46.760 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67161.log 2024-09-10T20:42:46.760 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37087.log.gz 2024-09-10T20:42:46.760 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55169.log 2024-09-10T20:42:46.760 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36904.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36904.log.gz 2024-09-10T20:42:46.760 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63293.log 2024-09-10T20:42:46.760 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67161.log.gz 2024-09-10T20:42:46.760 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55169.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55169.log.gz 2024-09-10T20:42:46.760 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49961.log 2024-09-10T20:42:46.761 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.63293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40977.log 2024-09-10T20:42:46.761 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63293.log.gz 2024-09-10T20:42:46.761 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60772.log 2024-09-10T20:42:46.761 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49961.log.gz 2024-09-10T20:42:46.761 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55673.log 2024-09-10T20:42:46.761 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40977.log.gz 2024-09-10T20:42:46.761 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68043.log 2024-09-10T20:42:46.761 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60772.log.gz 2024-09-10T20:42:46.761 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55463.log 2024-09-10T20:42:46.761 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55673.log.gz 2024-09-10T20:42:46.761 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67875.log 2024-09-10T20:42:46.761 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68043.log.gz 2024-09-10T20:42:46.761 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55463.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-10T20:42:46.762 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55463.log.gz 2024-09-10T20:42:46.762 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37680.log 2024-09-10T20:42:46.762 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67875.log.gz 2024-09-10T20:42:46.762 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51977.log 2024-09-10T20:42:46.762 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59596.log 2024-09-10T20:42:46.762 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37680.log.gz 2024-09-10T20:42:46.762 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38800.log 2024-09-10T20:42:46.762 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51977.log.gz 2024-09-10T20:42:46.762 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38124.log 2024-09-10T20:42:46.762 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59596.log.gz 2024-09-10T20:42:46.762 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63377.log 2024-09-10T20:42:46.762 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38800.log.gz 2024-09-10T20:42:46.762 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66236.log 2024-09-10T20:42:46.762 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38124.log.gz 2024-09-10T20:42:46.763 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.63377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60982.log 2024-09-10T20:42:46.763 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63377.log.gz 2024-09-10T20:42:46.763 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.66236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35338.log 2024-09-10T20:42:46.763 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66236.log.gz 2024-09-10T20:42:46.763 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60982.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.60982.log.gz /var/log/ceph/ceph-client.admin.58165.log 2024-09-10T20:42:46.763 INFO:teuthology.orchestra.run.smithi131.stderr: 2024-09-10T20:42:46.763 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35338.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.35338.log.gz --verbose 2024-09-10T20:42:46.763 INFO:teuthology.orchestra.run.smithi131.stderr: -- /var/log/ceph/ceph-client.admin.47357.log 2024-09-10T20:42:46.763 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.58165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40809.log 2024-09-10T20:42:46.763 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58165.log.gz 2024-09-10T20:42:46.763 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67791.log 2024-09-10T20:42:46.764 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47357.log.gz 2024-09-10T20:42:46.764 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49625.log 2024-09-10T20:42:46.764 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40809.log.gz 2024-09-10T20:42:46.764 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48785.log 2024-09-10T20:42:46.764 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67791.log.gz 2024-09-10T20:42:46.764 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53489.log 2024-09-10T20:42:46.764 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49625.log.gz 2024-09-10T20:42:46.764 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57311.log 2024-09-10T20:42:46.764 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48785.log.gz 2024-09-10T20:42:46.764 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53489.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.53489.log.gz 2024-09-10T20:42:46.764 INFO:teuthology.orchestra.run.smithi131.stderr: /var/log/ceph/ceph-client.admin.36352.log 2024-09-10T20:42:46.764 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54875.log 2024-09-10T20:42:46.764 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57311.log.gz 2024-09-10T20:42:46.764 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51725.log 2024-09-10T20:42:46.764 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36352.log.gz 2024-09-10T20:42:46.765 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51011.log 2024-09-10T20:42:46.765 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54875.log.gz 2024-09-10T20:42:46.765 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49877.log 2024-09-10T20:42:46.765 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51725.log.gz 2024-09-10T20:42:46.765 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65564.log 2024-09-10T20:42:46.765 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51011.log.gz 2024-09-10T20:42:46.765 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43633.log 2024-09-10T20:42:46.765 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49877.log.gz 2024-09-10T20:42:46.765 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69303.log 2024-09-10T20:42:46.765 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65564.log.gz 2024-09-10T20:42:46.765 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57565.log 2024-09-10T20:42:46.765 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43633.log.gz 2024-09-10T20:42:46.765 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36733.log/var/log/ceph/ceph-client.admin.69303.log: 2024-09-10T20:42:46.765 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69303.log.gz 2024-09-10T20:42:46.766 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37791.log 2024-09-10T20:42:46.766 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57565.log.gz 2024-09-10T20:42:46.766 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36733.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36733.log.gz 2024-09-10T20:42:46.766 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61066.log 2024-09-10T20:42:46.766 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55253.log 2024-09-10T20:42:46.766 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37791.log.gz 2024-09-10T20:42:46.766 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50843.log 2024-09-10T20:42:46.766 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61066.log.gz 2024-09-10T20:42:46.766 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39096.log 2024-09-10T20:42:46.766 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55253.log.gz 2024-09-10T20:42:46.766 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50843.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50843.log.gz 2024-09-10T20:42:46.766 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43213.log 2024-09-10T20:42:46.766 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56975.log 2024-09-10T20:42:46.766 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39096.log.gz 2024-09-10T20:42:46.767 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43213.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.43213.log.gz 2024-09-10T20:42:46.767 INFO:teuthology.orchestra.run.smithi131.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.53153.log 2024-09-10T20:42:46.767 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42755.log 2024-09-10T20:42:46.767 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56975.log.gz 2024-09-10T20:42:46.767 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57481.log 2024-09-10T20:42:46.767 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53153.log.gz 2024-09-10T20:42:46.767 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35869.log 2024-09-10T20:42:46.767 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42755.log.gz 2024-09-10T20:42:46.767 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66362.log 2024-09-10T20:42:46.767 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57481.log.gz 2024-09-10T20:42:46.767 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52355.log 2024-09-10T20:42:46.767 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35869.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35869.log.gz 2024-09-10T20:42:46.767 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.66362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44095.log 0.0% 2024-09-10T20:42:46.767 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.66362.log.gz 2024-09-10T20:42:46.768 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53909.log 2024-09-10T20:42:46.768 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52355.log.gz 2024-09-10T20:42:46.768 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42877.log 2024-09-10T20:42:46.768 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44095.log.gz 2024-09-10T20:42:46.768 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35515.log 2024-09-10T20:42:46.768 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53909.log.gz 2024-09-10T20:42:46.768 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54371.log 2024-09-10T20:42:46.768 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42877.log.gz 2024-09-10T20:42:46.768 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38013.log 2024-09-10T20:42:46.768 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35515.log.gz 2024-09-10T20:42:46.768 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55001.log 2024-09-10T20:42:46.768 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54371.log.gz 2024-09-10T20:42:46.768 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44473.log 2024-09-10T20:42:46.769 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38013.log.gz 2024-09-10T20:42:46.769 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64008.log 2024-09-10T20:42:46.769 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55001.log.gz 2024-09-10T20:42:46.769 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68505.log 2024-09-10T20:42:46.769 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44473.log.gz 2024-09-10T20:42:46.769 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38615.log 2024-09-10T20:42:46.769 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64008.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64008.log.gz 2024-09-10T20:42:46.769 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62831.log 2024-09-10T20:42:46.769 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68505.log.gz 2024-09-10T20:42:46.769 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38615.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-10T20:42:46.769 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38615.log.gz 2024-09-10T20:42:46.769 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.62831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70227.log 2024-09-10T20:42:46.769 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62831.log.gz 2024-09-10T20:42:46.769 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54329.log 2024-09-10T20:42:46.770 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.70227.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70227.log.gz -5 2024-09-10T20:42:46.770 INFO:teuthology.orchestra.run.smithi131.stderr: --verbose -- /var/log/ceph/ceph-client.admin.62621.log 2024-09-10T20:42:46.770 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66951.log 2024-09-10T20:42:46.770 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54329.log.gz 2024-09-10T20:42:46.770 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.62621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68547.log 2024-09-10T20:42:46.770 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62621.log.gz 2024-09-10T20:42:46.770 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70435.log 2024-09-10T20:42:46.770 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.66951.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66951.log.gz 2024-09-10T20:42:46.770 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47525.log 2024-09-10T20:42:46.770 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68547.log.gz 2024-09-10T20:42:46.770 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.70435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50213.log 2024-09-10T20:42:46.770 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70435.log.gz 2024-09-10T20:42:46.770 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55211.log 2024-09-10T20:42:46.770 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47525.log.gz 2024-09-10T20:42:46.771 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38763.log 2024-09-10T20:42:46.771 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50213.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50213.log.gz 2024-09-10T20:42:46.771 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58837.log 2024-09-10T20:42:46.771 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55211.log.gz 2024-09-10T20:42:46.771 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38763.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38763.log.gz 2024-09-10T20:42:46.771 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35431.log 2024-09-10T20:42:46.771 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.58837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37643.log 2024-09-10T20:42:46.771 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58837.log.gz 2024-09-10T20:42:46.771 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35431.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35431.log.gz -5 2024-09-10T20:42:46.771 INFO:teuthology.orchestra.run.smithi131.stderr: --verbose -- /var/log/ceph/ceph-client.admin.55715.log 2024-09-10T20:42:46.771 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38235.log 2024-09-10T20:42:46.771 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37643.log.gz 2024-09-10T20:42:46.771 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36145.log 2024-09-10T20:42:46.771 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55715.log.gz 2024-09-10T20:42:46.771 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38874.log 0.0% 2024-09-10T20:42:46.772 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38235.log.gz 2024-09-10T20:42:46.772 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36145.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.36145.log.gz 2024-09-10T20:42:46.772 INFO:teuthology.orchestra.run.smithi131.stderr: /var/log/ceph/ceph-client.admin.32527.log 2024-09-10T20:42:46.772 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47273.log 2024-09-10T20:42:46.772 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38874.log.gz 2024-09-10T20:42:46.772 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.32527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65058.log 2024-09-10T20:42:46.772 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32527.log.gz 2024-09-10T20:42:46.772 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65396.log 2024-09-10T20:42:46.772 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47273.log.gz 2024-09-10T20:42:46.772 INFO:teuthology.orchestra.run.smithi131.stderr: 95.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-10T20:42:46.772 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51305.log 2024-09-10T20:42:46.772 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65058.log.gz 2024-09-10T20:42:46.772 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55379.log 2024-09-10T20:42:46.772 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65396.log.gz 2024-09-10T20:42:46.773 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59134.log 2024-09-10T20:42:46.773 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51305.log.gz 2024-09-10T20:42:46.773 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67581.log 2024-09-10T20:42:46.773 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55379.log.gz 2024-09-10T20:42:46.773 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59134.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59134.log.gz 2024-09-10T20:42:46.773 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36604.log 2024-09-10T20:42:46.773 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56891.log 2024-09-10T20:42:46.773 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67581.log.gz 2024-09-10T20:42:46.773 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69807.log 2024-09-10T20:42:46.773 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36604.log.gz 2024-09-10T20:42:46.773 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60394.log 2024-09-10T20:42:46.773 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56891.log.gz 2024-09-10T20:42:46.773 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69807.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69807.log.gz -5 2024-09-10T20:42:46.773 INFO:teuthology.orchestra.run.smithi131.stderr: --verbose -- /var/log/ceph/ceph-client.admin.40431.log 2024-09-10T20:42:46.774 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66993.log 2024-09-10T20:42:46.774 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60394.log.gz 2024-09-10T20:42:46.774 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63335.log 2024-09-10T20:42:46.774 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40431.log.gz 2024-09-10T20:42:46.774 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.66993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37086.log 2024-09-10T20:42:46.774 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66993.log.gz 2024-09-10T20:42:46.774 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.63335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56135.log 2024-09-10T20:42:46.774 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63335.log.gz 2024-09-10T20:42:46.774 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42778.log 2024-09-10T20:42:46.774 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37086.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37086.log.gz 2024-09-10T20:42:46.774 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56135.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-10T20:42:46.774 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56135.log.gz 2024-09-10T20:42:46.774 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64932.log 2024-09-10T20:42:46.774 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51515.log 2024-09-10T20:42:46.775 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46811.log 2024-09-10T20:42:46.775 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64932.log.gz 2024-09-10T20:42:46.775 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64302.log 2024-09-10T20:42:46.775 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51515.log.gz 2024-09-10T20:42:46.775 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41915.log 2024-09-10T20:42:46.775 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46811.log.gz 2024-09-10T20:42:46.775 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47063.log 2024-09-10T20:42:46.775 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64302.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64302.log.gz 2024-09-10T20:42:46.775 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37606.log 2024-09-10T20:42:46.775 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41915.log.gz 2024-09-10T20:42:46.775 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65858.log 2024-09-10T20:42:46.775 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47063.log.gz 2024-09-10T20:42:46.775 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44649.log 2024-09-10T20:42:46.776 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37606.log.gz 2024-09-10T20:42:46.776 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65858.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65858.log.gz 2024-09-10T20:42:46.776 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69177.log 2024-09-10T20:42:46.776 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42186.log 2024-09-10T20:42:46.776 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44649.log.gz 2024-09-10T20:42:46.776 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64890.log 2024-09-10T20:42:46.776 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69177.log.gz 2024-09-10T20:42:46.776 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50297.log 2024-09-10T20:42:46.776 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42186.log.gz 2024-09-10T20:42:46.776 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64890.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64890.log.gz 2024-09-10T20:42:46.776 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67329.log 2024-09-10T20:42:46.776 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60058.log 2024-09-10T20:42:46.776 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50297.log.gz 2024-09-10T20:42:46.776 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68127.log 2024-09-10T20:42:46.777 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67329.log.gz 2024-09-10T20:42:46.777 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56807.log 2024-09-10T20:42:46.777 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60058.log.gz 2024-09-10T20:42:46.777 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62411.log 2024-09-10T20:42:46.777 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68127.log.gz 2024-09-10T20:42:46.777 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56807.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56807.log.gz 2024-09-10T20:42:46.777 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61990.log 2024-09-10T20:42:46.777 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.62411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40893.log 2024-09-10T20:42:46.777 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62411.log.gz 2024-09-10T20:42:46.777 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61990.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61990.log.gz 2024-09-10T20:42:46.777 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64386.log 2024-09-10T20:42:46.777 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43591.log 2024-09-10T20:42:46.777 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40893.log.gz 2024-09-10T20:42:46.777 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64386.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64386.log.gz 2024-09-10T20:42:46.778 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35723.log 2024-09-10T20:42:46.778 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64680.log 2024-09-10T20:42:46.778 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43591.log.gz 2024-09-10T20:42:46.778 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58039.log 2024-09-10T20:42:46.778 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35723.log.gz 2024-09-10T20:42:46.778 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65143.log 2024-09-10T20:42:46.778 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64680.log.gz 2024-09-10T20:42:46.778 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.58039.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58039.log.gz 2024-09-10T20:42:46.778 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42187.log 2024-09-10T20:42:46.778 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35829.log 2024-09-10T20:42:46.778 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65143.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65143.log.gz 2024-09-10T20:42:46.778 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60100.log 2024-09-10T20:42:46.779 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42187.log.gz 2024-09-10T20:42:46.779 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64260.log 2024-09-10T20:42:46.779 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35829.log.gz 2024-09-10T20:42:46.779 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62117.log 2024-09-10T20:42:46.779 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60100.log.gz 2024-09-10T20:42:46.779 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48407.log 2024-09-10T20:42:46.779 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64260.log.gz 2024-09-10T20:42:46.779 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.62117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44908.log 2024-09-10T20:42:46.779 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62117.log.gz 2024-09-10T20:42:46.779 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45019.log 2024-09-10T20:42:46.779 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48407.log.gz 2024-09-10T20:42:46.779 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54623.log 2024-09-10T20:42:46.779 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44908.log.gz 2024-09-10T20:42:46.779 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57943.log 2024-09-10T20:42:46.779 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45019.log.gz 2024-09-10T20:42:46.780 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56471.log 2024-09-10T20:42:46.780 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54623.log.gz 2024-09-10T20:42:46.780 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44982.log 2024-09-10T20:42:46.780 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57943.log.gz 2024-09-10T20:42:46.780 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56303.log 2024-09-10T20:42:46.780 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56471.log.gz 2024-09-10T20:42:46.780 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43339.log 2024-09-10T20:42:46.780 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44982.log.gz 2024-09-10T20:42:46.780 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58249.log 2024-09-10T20:42:46.780 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56303.log.gz 2024-09-10T20:42:46.781 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63755.log 2024-09-10T20:42:46.781 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43339.log.gz 2024-09-10T20:42:46.781 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.58249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51935.log 2024-09-10T20:42:46.781 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58249.log.gz 2024-09-10T20:42:46.781 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.63755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68589.log 2024-09-10T20:42:46.781 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63755.log.gz 2024-09-10T20:42:46.781 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51935.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51935.log.gz 2024-09-10T20:42:46.781 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67119.log 2024-09-10T20:42:46.781 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68085.log 2024-09-10T20:42:46.781 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68589.log.gz 2024-09-10T20:42:46.781 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56681.log 2024-09-10T20:42:46.781 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67119.log.gz 2024-09-10T20:42:46.781 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63419.log 2024-09-10T20:42:46.782 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68085.log.gz 2024-09-10T20:42:46.782 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45315.log 2024-09-10T20:42:46.782 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56681.log.gz 2024-09-10T20:42:46.782 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.63419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37384.log 2024-09-10T20:42:46.782 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63419.log.gz 2024-09-10T20:42:46.782 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63545.log 2024-09-10T20:42:46.782 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45315.log.gz 2024-09-10T20:42:46.782 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53195.log 2024-09-10T20:42:46.782 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37384.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37384.log.gz 2024-09-10T20:42:46.782 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.63545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52187.log 2024-09-10T20:42:46.782 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63545.log.gz 2024-09-10T20:42:46.782 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57397.log 2024-09-10T20:42:46.782 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53195.log.gz 2024-09-10T20:42:46.783 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50381.log 2024-09-10T20:42:46.783 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52187.log.gz 2024-09-10T20:42:46.783 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62663.log 2024-09-10T20:42:46.783 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57397.log.gz 2024-09-10T20:42:46.783 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67203.log 2024-09-10T20:42:46.783 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50381.log.gz 2024-09-10T20:42:46.783 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.62663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46517.log 2024-09-10T20:42:46.783 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62663.log.gz 2024-09-10T20:42:46.783 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62327.log 2024-09-10T20:42:46.783 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67203.log.gz 2024-09-10T20:42:46.783 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60898.log 2024-09-10T20:42:46.783 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46517.log.gz 2024-09-10T20:42:46.783 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.62327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55085.log 2024-09-10T20:42:46.784 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62327.log.gz 2024-09-10T20:42:46.784 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42335.log 2024-09-10T20:42:46.784 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60898.log.gz 2024-09-10T20:42:46.784 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40744.log 2024-09-10T20:42:46.784 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55085.log.gz 2024-09-10T20:42:46.784 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67413.log 2024-09-10T20:42:46.784 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42335.log.gz 2024-09-10T20:42:46.784 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37171.log 2024-09-10T20:42:46.784 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40744.log.gz 2024-09-10T20:42:46.784 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63924.log 2024-09-10T20:42:46.784 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67413.log.gz 2024-09-10T20:42:46.784 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37171.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.37171.log.gz /var/log/ceph/ceph-client.admin.44241.log 2024-09-10T20:42:46.785 INFO:teuthology.orchestra.run.smithi131.stderr: 2024-09-10T20:42:46.785 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.63924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50549.log 2024-09-10T20:42:46.785 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63924.log.gz 2024-09-10T20:42:46.785 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43843.log 2024-09-10T20:42:46.785 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44241.log.gz 2024-09-10T20:42:46.785 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36061.log 2024-09-10T20:42:46.785 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50549.log.gz 2024-09-10T20:42:46.785 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43087.log 2024-09-10T20:42:46.785 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43843.log.gz 2024-09-10T20:42:46.785 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55337.log 2024-09-10T20:42:46.785 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36061.log.gz 2024-09-10T20:42:46.785 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68379.log 2024-09-10T20:42:46.785 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43087.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43087.log.gz 2024-09-10T20:42:46.786 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69261.log 2024-09-10T20:42:46.786 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55337.log.gz 2024-09-10T20:42:46.786 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68379.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68379.log.gz 2024-09-10T20:42:46.786 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-10T20:42:46.786 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43885.log 2024-09-10T20:42:46.786 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69261.log.gz 2024-09-10T20:42:46.786 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36353.log 2024-09-10T20:42:46.786 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64554.log 2024-09-10T20:42:46.786 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43885.log.gz 2024-09-10T20:42:46.786 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39355.log 2024-09-10T20:42:46.787 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36353.log.gz 2024-09-10T20:42:46.787 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49079.log 2024-09-10T20:42:46.787 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64554.log.gz 2024-09-10T20:42:46.787 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56429.log 2024-09-10T20:42:46.787 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39355.log.gz 2024-09-10T20:42:46.787 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38504.log 2024-09-10T20:42:46.787 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49079.log.gz 2024-09-10T20:42:46.787 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53657.log 2024-09-10T20:42:46.787 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56429.log.gz 2024-09-10T20:42:46.787 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46643.log 2024-09-10T20:42:46.787 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38504.log.gz 2024-09-10T20:42:46.787 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32279.log 2024-09-10T20:42:46.787 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53657.log.gz 2024-09-10T20:42:46.787 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55295.log 2024-09-10T20:42:46.788 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46643.log.gz 2024-09-10T20:42:46.788 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.32279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64722.log 2024-09-10T20:42:46.788 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32279.log.gz 2024-09-10T20:42:46.788 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61318.log 2024-09-10T20:42:46.788 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55295.log.gz 2024-09-10T20:42:46.788 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36691.log 2024-09-10T20:42:46.788 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64722.log.gz 2024-09-10T20:42:46.788 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61318.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.61318.log.gz --verbose 2024-09-10T20:42:46.788 INFO:teuthology.orchestra.run.smithi131.stderr: -- /var/log/ceph/ceph-client.admin.52481.log 2024-09-10T20:42:46.788 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36103.log 2024-09-10T20:42:46.788 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36691.log.gz 2024-09-10T20:42:46.788 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69429.log 2024-09-10T20:42:46.788 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52481.log.gz 2024-09-10T20:42:46.789 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37170.log 2024-09-10T20:42:46.789 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36103.log.gz 2024-09-10T20:42:46.789 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66404.log 2024-09-10T20:42:46.789 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69429.log.gz 2024-09-10T20:42:46.789 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60142.log 2024-09-10T20:42:46.789 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37170.log.gz 2024-09-10T20:42:46.789 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.66404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63503.log 2024-09-10T20:42:46.789 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66404.log.gz 2024-09-10T20:42:46.789 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51851.log 2024-09-10T20:42:46.789 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60142.log.gz 2024-09-10T20:42:46.789 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.63503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52565.log 2024-09-10T20:42:46.789 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63503.log.gz 2024-09-10T20:42:46.790 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53405.log 2024-09-10T20:42:46.790 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51851.log.gz 2024-09-10T20:42:46.790 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45056.log 2024-09-10T20:42:46.790 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52565.log.gz 2024-09-10T20:42:46.790 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50507.log 2024-09-10T20:42:46.790 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53405.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53405.log.gz 2024-09-10T20:42:46.790 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48491.log 2024-09-10T20:42:46.790 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45056.log.gz 2024-09-10T20:42:46.790 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48659.log 2024-09-10T20:42:46.790 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50507.log.gz 2024-09-10T20:42:46.790 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56261.log 2024-09-10T20:42:46.790 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48491.log.gz 2024-09-10T20:42:46.790 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48659.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.48659.log.gz 2024-09-10T20:42:46.791 INFO:teuthology.orchestra.run.smithi131.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.35596.log 2024-09-10T20:42:46.791 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50087.log 2024-09-10T20:42:46.791 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56261.log.gz 2024-09-10T20:42:46.791 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35596.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.35596.log.gz -5 2024-09-10T20:42:46.791 INFO:teuthology.orchestra.run.smithi131.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50633.log 2024-09-10T20:42:46.791 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68883.log 2024-09-10T20:42:46.791 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50087.log.gz 2024-09-10T20:42:46.791 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57017.log 2024-09-10T20:42:46.791 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50633.log.gz 2024-09-10T20:42:46.791 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65101.log 2024-09-10T20:42:46.791 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68883.log.gz 2024-09-10T20:42:46.791 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52943.log 2024-09-10T20:42:46.791 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57017.log.gz 2024-09-10T20:42:46.792 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60352.log 2024-09-10T20:42:46.792 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65101.log.gz 2024-09-10T20:42:46.792 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42377.log 2024-09-10T20:42:46.792 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52943.log.gz 2024-09-10T20:42:46.792 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48365.log 2024-09-10T20:42:46.792 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60352.log.gz 2024-09-10T20:42:46.792 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44760.log 2024-09-10T20:42:46.792 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42377.log.gz 2024-09-10T20:42:46.792 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54539.log 2024-09-10T20:42:46.792 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48365.log.gz 2024-09-10T20:42:46.792 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44760.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.44157.log 2024-09-10T20:42:46.792 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44760.log.gz 2024-09-10T20:42:46.792 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37254.log 2024-09-10T20:42:46.792 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54539.log.gz 2024-09-10T20:42:46.793 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58627.log 2024-09-10T20:42:46.793 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44157.log.gz 2024-09-10T20:42:46.793 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70353.log 2024-09-10T20:42:46.793 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37254.log.gz 2024-09-10T20:42:46.793 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.58627.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58627.log.gz 2024-09-10T20:42:46.793 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64218.log 2024-09-10T20:42:46.793 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.70353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67497.log 2024-09-10T20:42:46.793 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70353.log.gz 2024-09-10T20:42:46.793 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60478.log 2024-09-10T20:42:46.793 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64218.log.gz 2024-09-10T20:42:46.793 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56513.log 2024-09-10T20:42:46.793 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67497.log.gz 2024-09-10T20:42:46.794 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56765.log 2024-09-10T20:42:46.794 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60478.log.gz 2024-09-10T20:42:46.794 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70311.log 2024-09-10T20:42:46.794 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56513.log.gz 2024-09-10T20:42:46.794 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56765.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56765.log.gz 2024-09-10T20:42:46.794 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49205.log 2024-09-10T20:42:46.794 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.70311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44575.log 2024-09-10T20:42:46.794 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70311.log.gz 2024-09-10T20:42:46.794 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58543.log 2024-09-10T20:42:46.794 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49205.log.gz 2024-09-10T20:42:46.794 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60310.log 2024-09-10T20:42:46.794 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44575.log.gz 2024-09-10T20:42:46.795 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.58543.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58543.log.gz 2024-09-10T20:42:46.795 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61402.log 2024-09-10T20:42:46.795 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48617.log 2024-09-10T20:42:46.795 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60310.log.gz 2024-09-10T20:42:46.795 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52229.log 2024-09-10T20:42:46.795 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61402.log.gz 2024-09-10T20:42:46.795 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37865.log 2024-09-10T20:42:46.795 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48617.log.gz 2024-09-10T20:42:46.795 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43507.log 2024-09-10T20:42:46.795 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52229.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52229.log.gz 2024-09-10T20:42:46.795 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64974.log 2024-09-10T20:42:46.795 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37865.log.gz 2024-09-10T20:42:46.796 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38837.log 2024-09-10T20:42:46.796 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43507.log.gz 2024-09-10T20:42:46.796 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64974.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64974.log.gz 2024-09-10T20:42:46.796 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69765.log 2024-09-10T20:42:46.796 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59848.log 2024-09-10T20:42:46.796 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38837.log.gz 2024-09-10T20:42:46.796 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68631.log 2024-09-10T20:42:46.796 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69765.log.gz 2024-09-10T20:42:46.796 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49751.log 2024-09-10T20:42:46.796 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59848.log.gz 2024-09-10T20:42:46.796 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68631.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68631.log.gz 2024-09-10T20:42:46.796 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56597.log 2024-09-10T20:42:46.796 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65606.log 2024-09-10T20:42:46.797 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49751.log.gz 2024-09-10T20:42:46.797 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54497.log 2024-09-10T20:42:46.797 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56597.log.gz 2024-09-10T20:42:46.797 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62369.log 2024-09-10T20:42:46.797 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65606.log.gz 2024-09-10T20:42:46.797 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54497.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54497.log.gzgzip 2024-09-10T20:42:46.797 INFO:teuthology.orchestra.run.smithi131.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.59932.log 2024-09-10T20:42:46.797 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.62369.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.62369.log.gz 2024-09-10T20:42:46.797 INFO:teuthology.orchestra.run.smithi131.stderr: /var/log/ceph/ceph-client.admin.68757.log 2024-09-10T20:42:46.797 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69219.log 2024-09-10T20:42:46.797 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59932.log.gz 2024-09-10T20:42:46.798 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44945.log 2024-09-10T20:42:46.798 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68757.log.gz 2024-09-10T20:42:46.798 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50759.log 2024-09-10T20:42:46.798 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69219.log.gz 2024-09-10T20:42:46.798 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48071.log 2024-09-10T20:42:46.798 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44945.log.gz 2024-09-10T20:42:46.798 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46727.log 2024-09-10T20:42:46.798 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50759.log.gz 2024-09-10T20:42:46.798 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48071.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48071.log.gz 2024-09-10T20:42:46.798 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43465.log 2024-09-10T20:42:46.798 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58753.log 2024-09-10T20:42:46.798 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46727.log.gz 2024-09-10T20:42:46.798 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37495.log 2024-09-10T20:42:46.799 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43465.log.gz 2024-09-10T20:42:46.799 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.58753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69849.log 2024-09-10T20:42:46.799 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58753.log.gz 2024-09-10T20:42:46.799 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37495.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.37495.log.gz 2024-09-10T20:42:46.799 INFO:teuthology.orchestra.run.smithi131.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.68841.log 2024-09-10T20:42:46.799 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47399.log 2024-09-10T20:42:46.799 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69849.log.gz 2024-09-10T20:42:46.799 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43717.log 2024-09-10T20:42:46.799 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68841.log.gz 2024-09-10T20:42:46.799 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54203.log 2024-09-10T20:42:46.799 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47399.log.gz 2024-09-10T20:42:46.799 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66699.log 2024-09-10T20:42:46.799 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43717.log.gz 2024-09-10T20:42:46.799 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50591.log 2024-09-10T20:42:46.800 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54203.log.gz 2024-09-10T20:42:46.800 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.66699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49457.log 2024-09-10T20:42:46.800 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66699.log.gz 2024-09-10T20:42:46.800 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68421.log 2024-09-10T20:42:46.800 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50591.log.gz 2024-09-10T20:42:46.800 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49457.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49457.log.gz -5 2024-09-10T20:42:46.800 INFO:teuthology.orchestra.run.smithi131.stderr: --verbose -- /var/log/ceph/ceph-client.admin.59344.log 2024-09-10T20:42:46.800 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68421.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68421.log.gz 2024-09-10T20:42:46.800 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59638.log 2024-09-10T20:42:46.800 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37939.log 2024-09-10T20:42:46.801 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59344.log.gz 2024-09-10T20:42:46.801 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49793.log 2024-09-10T20:42:46.801 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59638.log.gz 2024-09-10T20:42:46.801 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39133.log 2024-09-10T20:42:46.801 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37939.log.gz 2024-09-10T20:42:46.801 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48533.log 2024-09-10T20:42:46.801 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49793.log.gz 2024-09-10T20:42:46.801 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44431.log 2024-09-10T20:42:46.801 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39133.log.gz 2024-09-10T20:42:46.801 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69387.log 2024-09-10T20:42:46.802 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48533.log.gz 2024-09-10T20:42:46.802 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47945.log 2024-09-10T20:42:46.802 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44431.log.gz 2024-09-10T20:42:46.802 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37319.log 2024-09-10T20:42:46.802 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69387.log.gz 2024-09-10T20:42:46.802 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51557.log 2024-09-10T20:42:46.802 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47945.log.gz 2024-09-10T20:42:46.802 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36605.log 2024-09-10T20:42:46.802 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37319.log.gz 2024-09-10T20:42:46.802 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66446.log 2024-09-10T20:42:46.802 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51557.log.gz 2024-09-10T20:42:46.802 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47903.log 2024-09-10T20:42:46.802 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36605.log.gz 2024-09-10T20:42:46.803 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.66446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64512.log 2024-09-10T20:42:46.803 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66446.log.gz 2024-09-10T20:42:46.803 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47903.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.47903.log.gz /var/log/ceph/ceph-client.admin.61822.log 2024-09-10T20:42:46.803 INFO:teuthology.orchestra.run.smithi131.stderr: 2024-09-10T20:42:46.803 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46559.log 2024-09-10T20:42:46.803 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64512.log.gz 2024-09-10T20:42:46.803 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65816.log 2024-09-10T20:42:46.803 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61822.log.gz 2024-09-10T20:42:46.803 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.46559.log: -- /var/log/ceph/ceph-client.admin.58795.log 2024-09-10T20:42:46.803 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46559.log.gz 2024-09-10T20:42:46.803 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53111.log 2024-09-10T20:42:46.803 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65816.log.gz 2024-09-10T20:42:46.803 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.58795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40935.log 2024-09-10T20:42:46.803 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58795.log.gz 2024-09-10T20:42:46.803 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58879.log 2024-09-10T20:42:46.804 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53111.log.gz 2024-09-10T20:42:46.804 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49247.log 0.0% 2024-09-10T20:42:46.804 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40935.log.gz 2024-09-10T20:42:46.804 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.58879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48113.log 2024-09-10T20:42:46.804 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58879.log.gz 2024-09-10T20:42:46.804 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65312.log 2024-09-10T20:42:46.804 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49247.log.gz 2024-09-10T20:42:46.804 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45204.log 2024-09-10T20:42:46.805 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48113.log.gz 2024-09-10T20:42:46.805 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63083.log 2024-09-10T20:42:46.805 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65312.log.gz 2024-09-10T20:42:46.805 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45204.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45204.log.gz 2024-09-10T20:42:46.805 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55043.log 2024-09-10T20:42:46.805 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.63083.log: /var/log/ceph/ceph-client.admin.39281.log 2024-09-10T20:42:46.806 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63083.log.gz 2024-09-10T20:42:46.806 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57733.log 2024-09-10T20:42:46.806 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55043.log.gz 2024-09-10T20:42:46.806 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49289.log 2024-09-10T20:42:46.806 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39281.log.gz 2024-09-10T20:42:46.806 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49121.log 2024-09-10T20:42:46.806 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57733.log.gz 2024-09-10T20:42:46.807 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52103.log 2024-09-10T20:42:46.807 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49289.log.gz 2024-09-10T20:42:46.807 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47693.log 2024-09-10T20:42:46.807 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49121.log.gz 2024-09-10T20:42:46.807 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52103.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.44686.log 2024-09-10T20:42:46.807 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52103.log.gz 2024-09-10T20:42:46.807 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45130.log 2024-09-10T20:42:46.808 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47693.log.gz 2024-09-10T20:42:46.808 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65774.log 2024-09-10T20:42:46.808 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44686.log.gz 2024-09-10T20:42:46.808 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45130.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45130.log.gz 2024-09-10T20:42:46.808 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65774.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65774.log.gz 2024-09-10T20:42:46.808 INFO:teuthology.orchestra.run.smithi131.stderr: 96.4% -- replaced with /var/log/ceph/ceph-client.admin.42778.log.gz 2024-09-10T20:42:47.298 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-10T20:42:47.763 INFO:teuthology.orchestra.run.smithi131.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-10T20:42:48.414 INFO:teuthology.orchestra.run.smithi131.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-10T20:42:48.522 INFO:teuthology.orchestra.run.smithi131.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-10T20:42:49.122 INFO:teuthology.orchestra.run.smithi131.stderr: 89.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-10T20:45:54.464 INFO:teuthology.orchestra.run.smithi131.stderr: 91.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-10T20:47:55.085 INFO:teuthology.orchestra.run.smithi131.stderr: 90.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-10T20:49:01.596 INFO:teuthology.orchestra.run.smithi131.stderr: 91.1% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-10T20:49:01.598 INFO:teuthology.orchestra.run.smithi131.stderr: 2024-09-10T20:49:01.598 INFO:teuthology.orchestra.run.smithi131.stderr:real 6m15.217s 2024-09-10T20:49:01.598 INFO:teuthology.orchestra.run.smithi131.stderr:user 5m10.801s 2024-09-10T20:49:01.598 INFO:teuthology.orchestra.run.smithi131.stderr:sys 0m12.033s 2024-09-10T20:49:01.598 INFO:tasks.ceph:Archiving logs... 2024-09-10T20:49:01.599 DEBUG:teuthology.misc:Transferring archived files from smithi131:/var/log/ceph to /home/teuthworker/archive/skanta-2024-09-10_15:30:27-rados-wip-bharath7-testing-2024-09-10-1409-distro-default-smithi/7899571/remote/smithi131/log 2024-09-10T20:49:01.600 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T20:50:00.386 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-10T20:50:00.397 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-10T20:50:00.397 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T20:50:00.466 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T20:50:00.474 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T20:50:00.474 DEBUG:teuthology.orchestra.run.smithi131:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T20:50:00.489 INFO:teuthology.orchestra.run.smithi131.stderr:bash: line 1: ntpq: command not found 2024-09-10T20:50:00.539 INFO:teuthology.orchestra.run.smithi131.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T20:50:00.539 INFO:teuthology.orchestra.run.smithi131.stdout:=============================================================================== 2024-09-10T20:50:00.539 INFO:teuthology.orchestra.run.smithi131.stdout:^* hv01.front.sepia.ceph.com 3 6 377 35 -398us[ -449us] +/- 35ms 2024-09-10T20:50:00.540 INFO:teuthology.orchestra.run.smithi131.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 37 +2153us[+2101us] +/- 71ms 2024-09-10T20:50:00.540 INFO:teuthology.orchestra.run.smithi131.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 40 -1275us[-1327us] +/- 65ms 2024-09-10T20:50:00.540 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T20:50:00.540 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T20:50:00.549 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T20:50:00.550 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T20:50:00.559 DEBUG:teuthology.orchestra.run.smithi131:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T20:50:00.733 DEBUG:teuthology.orchestra.run.smithi131:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T20:50:00.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:50:00.762 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T20:50:00.771 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T20:50:00.780 INFO:teuthology.task.internal:Duration was 1685.425812 seconds 2024-09-10T20:50:00.780 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T20:50:00.788 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T20:50:00.788 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T20:50:00.866 INFO:teuthology.orchestra.run.smithi131.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T20:50:00.926 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T20:50:00.926 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi131.front.sepia.ceph.com 2024-09-10T20:50:00.926 DEBUG:teuthology.orchestra.run.smithi131:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T20:50:00.946 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T20:50:00.946 DEBUG:teuthology.orchestra.run.smithi131:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T20:50:01.013 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T20:50:01.013 DEBUG:teuthology.orchestra.run.smithi131:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T20:50:01.193 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T20:50:01.203 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T20:50:01.203 DEBUG:teuthology.orchestra.run.smithi131:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T20:50:01.243 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T20:50:01.253 DEBUG:teuthology.orchestra.run.smithi131:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T20:50:01.321 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern = core 2024-09-10T20:50:01.341 DEBUG:teuthology.orchestra.run.smithi131:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T20:50:01.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:50:01.397 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T20:50:01.407 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T20:50:01.408 DEBUG:teuthology.misc:Transferring archived files from smithi131:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/skanta-2024-09-10_15:30:27-rados-wip-bharath7-testing-2024-09-10-1409-distro-default-smithi/7899571/remote/smithi131 2024-09-10T20:50:01.408 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T20:50:01.477 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T20:50:01.477 DEBUG:teuthology.orchestra.run.smithi131:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T20:50:01.525 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T20:50:01.539 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T20:50:01.539 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T20:50:01.550 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T20:50:01.550 DEBUG:teuthology.orchestra.run.smithi131:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T20:50:01.580 INFO:teuthology.orchestra.run.smithi131.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 20:50 /home/ubuntu/cephtest 2024-09-10T20:50:01.582 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T20:50:01.592 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T20:50:01.616 INFO:teuthology.run:Summary data: description: rados/singleton/{all/lost-unfound-delete mon_election/connectivity msgr-failures/few msgr/async objectstore/{bluestore-options/write$/{write_v2} bluestore/bluestore-comp-zstd} rados supported-random-distro$/{centos_latest}} duration: 1685.4258117675781 flavor: default owner: scheduled_skanta@teuthology success: true 2024-09-10T20:50:01.617 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T20:50:01.718 INFO:teuthology.run:pass